Communication and control between netwoked devices have been around for ages. But what's special now is that we can allow these devices to make decisions. A server can act like a central brain, doing complex computations and issue commands to these devices. Sometimes, when the processor in a device is powerful enough, machine learning code can be run locally on the device or can be co-ordinated with the server for complex arrangements. The creativity is yours! In this section, we will look at how to program and network different devices and augment them with ML capabilities

1) Programming the peripherals.

Microcontrollers are used for performing tasks that are more specific. Depending on the input, some processing needs to be done and output is delivered. For eg, using ultrasonic sensor to send a message to server whenever it detects some object in front of it. We will focus on low-power microcontrollers from AVR and ESP families.

Emerging System-on-chip boards with higher computing capabilities at lower cost have paved way for processing complex data like audio, video. If the board provides for General Purpose Input Output (GPIO) pins, different microcontrollers can be combined to form complex devices. For eg, a device which sends data to a server from it's microphone after some initial processing, in addition to some data from ultrasonic sensor. If the microprocessor is powerful enough, light weight versions of machine learning code can be run on-device. We will focus on a family of boards from RaspberryPI and NVIDIA Jetson.

Our smart phones already have a myraid of built-in sensors. Moreover, cameras and microphones from our phone, when linked to a high performace server, can make use of bleeding-edge, compute intensive machine learning models, to build a variety of new-age applications. For eg, stream from your android camera to a server which then runs Multi-obejct segmentation and tracting algorithm, and uses its results to control other sensors.

2) Networking.

Networking devices through a router have become the standard today. But there are also many other ways to network devices - radio-frequency chips, bluetooth. I will be briefly touching on these topics as well. For connections through a router, several networking protocols have been developed to make things standardized and easier. Networking protocols are categorized for these two kinds of connections,



a) Peer-to-peer connection:

When one device directly communicates with another device without passing through a server This is useful for real-time streaming applications For eg, using a motion sensor to precisely control a servo motor position

Protocols: Sockets, WebRTC (requires initial handshake with a server)

b) Connections through server Devices send data to the server which manages the flow of information. This is used when some complex processing has to be carried out in the server intermediate to the flow of data.

Protocols: WebSockets (Full-duplex TCP), HTTP (Half-duplex TCP), MQTT (wrapper for publish/subscribe model)


Go back Modules