This week in Cannes, France, IBM Watson Captioning was honored in the Advanced TV Innovation of the Year category at the Content Innovation Awards 2018. The award highlights technology that has improved the TV experience, overcome challenges to the market or enabled viewers in new ways. In this respect, IBM had recently launched IBM Watson Captioning Live, which debuted in April of 2018 to offer live intelligent closed captions for broadcast networks.
The solution, which aids in automated caption generation, helps solve some of the most critical issues facing live content creators today in providing captions. This includes both the ability to scale caption generation and in regards to accuracy, all through infusing the technology with artificial intelligence. This article details what is IBM Watson Captioning and some background on the Content Innovation Awards. For even more information on this technology, also download our white paper Captioning Goes Cognitive: A New Approach to an Old Challenge.
The need and challenges around captions
20% of Americans report some degree of hearing loss. For this audience, captions provide a clear bridge to understanding video content. Unfortunately, captioning has historically been difficult or very costly, especially for live content. Now while there have been automated approaches for some time, they have traditionally not been adequate for professional use. Struggles include any dialogue that wasn’t spoken very clearly, terminology that might be new and context for similar sounding phrases or to make a judgement call when something was slurred.
IBM has been working to improve this process, though, and give broadcasters something that can be used that addresses previous shortcomings through the introduction of IBM Watson Captioning.
About IBM Watson Captioning Live
IBM has enhanced automatic captioning through infusing artificial intelligence (AI) from IBM Watson into the caption process. This still involves using speech-to-text, taking audio and transferring it to a machine readable format in the form of text. However, through the AI, this process is enhanced with increased accuracy by being able to train the technology, making it better equipped to deliver accurate captions after the initial attempt. This is crucial for live captioning where there is no oppurtinity to edit generated captions after the fact: they need to be accurate.
To accommodate this, the technology can be taught, making it more capable of generating accurate captions of live content.
AI training for captions
There are several different ways to train the AI in order to boost accuracy for the captions it needs to generate. One is through the expansion of the available vocabulary. This includes teaching it new names, along with their proper spellings. When committed to fully, this can create a solution that can process market-specific terminology so that the solution can accurately identify local landmarks and events along with local personalities. Furthermore, terms can be edited or removed at any time. For example, if there was a company called “Azzure” that changed its name to “Azzur” that could quickly be edited and reflected in future generated captions. Learn more about AI vocabulary training.
Another aspect is the ability to provide context through building out a corpus, a collection of texts. Now a corpus is a library of samples for the AI to learn from, such as transcripts from past programming. This allows the AI to make more informed decisions when it goes to generate the automated captions. One angle this particularly helps with is similar sounding words, known as homophones, or phrases. For example, the terms “jeans” and “genes” sound identical when spoken, although mean very different things. Context can help in relating which is which, like “He went to buy jeans from the store” is fairly straight forward. Sometimes it’s not so clear, though, like “He got the jeans/genes from his mom”. In that passage, either word could be correct. As a result, it will need to look at the greater context, but also could lean on the corpus to help decide. If more content is directed toward fashion, jeans might be a better assumption as the correct course. Alternatively, if content is more scientific in nature and does talk about genetics, genes might be a better assumption.
The final aspect to improve the accuracy is the ability to schedule and load a script for live content. This can be invaluable for context and is something readily available for news and other type of programming that has something structured ahead of time. It should also be noted that the script isn’t used verbatim to generate captions. This is important as, for example, if someone goes “off topic” or “off script”, it will not ‘break’ the caption process. Instead, the artificial intelligence uses the script as a frame of reference, aiding in contextual decisions.
About the Content Innovation Awards
Launched in 2015, the Content Innovation Awards honor innovations for the global TV industry in the realms of content providers, distributors and technology companies. The awards are hosted by Digital TV Europe and Television Business International and have, since their launch, been held in Cannes, France.
The 2018 ceremony saw 24 awards given out, including a lifetime achievement award. A full list of winners can be seen on the awards highlights page. This year also marks the second time that IBM Watson Media has been honored at the ceremony. The first time was last year, in 2017, when Watson Media received the award for Best OTT TV Technology of the Year at the Content Innovation Awards 2017.
Summary
IBM has introduced an AI enhanced solution to tackle some of the biggest challenges facing broadcasters in providing live closed captions with their content. That technology has rapidly advanced, having this year launched a live capable version for broadcasters. As a result, more viewers are given access to content, while networks benefit from the ability to scale captioning options, all of which has landed IBM in this opportune situation to be honored by the Content Innovation Awards for their work.
To learn more about how Watson Captioning can deliver accurate captions through training, register for our Auto Closed Captions and AI Training webinar.