tinyML Talks: Running and Managing Fleets of Single Board Computers at Scale

"Running and Managing Fleets of Single Board Computers at Scale"

Seth Clark
Co-founder & Head of Product
Modzy

The increase of compute power available on single board computers (SBCs) has opened the door to a whole new class of ML-powered applications that can run on the likes of a Raspberry Pi or a Jetson Nano. In this talk, we will explore the different architectures for running ML models on large fleets of SBCs. We will start by discussing the benefits and challenges of running ML at the edge, including reducing latency, improving privacy, and reducing bandwidth requirements. We will then delve into different architectures for running ML on fleets of these devices and explore the advantages and potential pitfalls of implementing each. Finally, we’ll walk through several different demos to contextualize how these architectures can be used to power real-world ML enabled solutions, including NLP, computer vision, and sensor data monitoring.