EdgeFM Accelerator: Towards a faster, cheaper & scalable FM inference

Foundation Models are increasingly being used for solving real life problems. Researchers and industries are gradually evolving to optimize FM to be used on a larger scale with low latency and higher performance. Using Edge devices to run FM is becoming a highly favorable option to reduce overall cost of inference at scale. FM inference at Edge also comes with a varied option of data input, data pre-processing, post-processing, and running ML models compatible on the edge hardware. Romil and Fabian, along with a team have developed EdgeFM Accelerator which provides a highly configurable pipeline for running inference on FM at edge. The pipeline uses a variety of open-source dependencies to carry out the data processing and ML inference tasks. The session focuses on the architecture of the accelerator, functioning of the micro-services within the system and integration with AWS IoT services.