Jo Franchetti

Information & Communications Technology

Media & Information

Transports & Logistics

JavaScript Realtime communication Front End Development Accessability Unreal Engine Web APIs IoT

London, England, United Kingdom

Making a wearable live caption display using Azure Cognitive Services and Ably Realtime

Live captioning of speech to text has so many useful applications and Cognitive Services makes it fast and easy to build captioning into your applications. Together with Ably Realtime, it is possible to make wearable devices which can display what you’re saying, in real time. Wearable live captions!

Inspired by the frustration that my mother, who is hard of hearing, feels at being unable to lip read people who are wearing masks, combined with the muffled sound a mask creates, I built a wearable live caption display into a mask. It shows the text of what I’m saying, in real-time so that she can continue a conversation with me easily, while maintaining eye contact.

The talk will go over how to program an LED display, how to use Azure Cognitive Services to transform speech into text and how to send data in real time from ACS to the display using MQTT.


Jo Franchetti

Developer Advocate at Ably

Jo is a developer advocate at Ably. She is passionate about Realtime Data, PWAs and great CSS. She’s got 10 years experience as a front end developer and has worked in various parts of the tech industry from startups, agencies, charities to large organisations. She is also a mentor and organiser at codebar.io where she is able to act on her passion not only for teaching good use of the web but also for improving the diversity and inclusivity of the tech industry.

Jo's full speaker profile