Demo: An Experimental Platform for AI Model Partitioning on Resource-constrained Devices

Partitioning Artificial Intelligence (AI) models such as Deep Neural Networks (DNNs) or Transformer-based Architectures is essential for minimizing latency in resource-constrained edge computing environments, which is critical for applications such as real-time video analytics, autonomous vehicles, smart IoT systems, and most importantly, on-device Large Language Models (LLMs)¹. This paper presents a demonstration of an experimental platform showcased for DNN partitioning for the inference stage, which comprises a WiFi network of Raspberry Pi 4 devices. The platform supports research on optimizing inference in distributed setups, focusing on performance and resource usage and it can support several architectures, ranging from simple and deep NNs, to more advanced transformer-based architectures.