Session

Wearable Live Captions (making mask wearing more accessible for those who are hard of hearing)

During the pandemic we've all been feeling pretty isolated, and we've all been doing our best and wearing masks. But what if everyone wearing masks cuts off your ability to converse?

My dear mum relies on lip reading and clear sounds to understand what people are saying. But I, of course want her to stay safe. So the thought occurred. Can I make a live captioning display fit into a mask so that she can read what I'm saying as I'm speaking?

This talk will cover how to build a wearable LED display, how to use Microsoft Azure Cognitive Services to convert speech to text, using Ably's MQTT broker to send messages to a microprocessor and some good old JavaScript and C++.

Live captioning of speech to text has so many useful applications and Cognitive Services makes it fast and easy to build captioning into your applications. Together with Ably Realtime, it is possible to make wearable devices which can display what you’re saying, in real time. Wearable Live Captions!

I'll bring the mask to the talk and it will be livecaptioning what I say as the talk goes on.

Jo Franchetti

Lead Developer Advocate at Ably

London, United Kingdom

View Speaker Profile