Session

Making a wearable live caption display using Azure Cognitive Services and Ably Realtime

Live captioning of speech to text has so many useful applications and Cognitive Services makes it fast and easy to build captioning into your applications. Together with Ably Realtime, it is possible to make wearable devices which can display what you’re saying, in real time. Wearable live captions!

Inspired by the frustration that my mother, who is hard of hearing, feels at being unable to lip read people who are wearing masks, combined with the muffled sound a mask creates, I built a wearable live caption display into a mask. It shows the text of what I’m saying, in real-time so that she can continue a conversation with me easily, while maintaining eye contact.

The talk will go over how to program an LED display, how to use Azure Cognitive Services to transform speech into text and how to send data in real time from ACS to the display using MQTT.

Jo Franchetti

Lead Developer Advocate at Ably

London, United Kingdom

View Speaker Profile