What can modern streaming apps do using AI?
1. Scene Localization (Cultural Adaptation Beyond Language)
The first thing I would like to see is the localization of scenes. For example, let’s say there is an act happening and there is a scene behind it. If it is written in Spanish, Tamil, Hindi, or any other language, even those scenes or the images behind the actors should be converted and localized. It’s not just about language translation; it’s also about understanding the context of the local audience. For instance, in India, there are multiple languages, and it’s not just about language; there are also cultural differences. So, when there is a cultural difference, the scene should be adapted to that culture. That’s the first aspect I would like to see, which I call localization.
2. Subtitles and Captions (Hyper-Personalized & In-Movie Language Translation)
Then, the second aspect is subtitles and captions. Currently, there are plenty of languages; I’m assuming there are over a hundred languages, and your streaming partner might not provide everything. Depending on your choice, you should be able to select the right street language. I don’t want to have all the available languages. There should be a settings option where I can choose which languages I want the subtitles to be in. These subtitles shouldn’t just come from the spoken language; there are also new languages introduced even within the movie itself. For example, take Baahubali. There’s a language used by Kalakeya. In that case, they embed that language and translate it into English or Tamil, depending on where it is, but it can be extra work. Even that language, which is ingrained as part of the movie itself, should be translated into the local language. Moreover, sometimes it’s not just about captioning; even the voice itself, if it can be auto-translated, would be great.
3. Content Rating and Filtering (Dynamic Content Adjustment)
The third thing I would like to see is content rating. Let’s say, for example, I’m watching a PG-13 movie and I want to enjoy the same movie with my kid. Currently, the only options available are to let your kid watch it or tell them, “Hey, don’t listen to this,” or “Close your eyes,” something along those lines. If I have to watch a PG-13 movie but want to eliminate any abusive content, adult language, or similar issues, I should be able to select what I don’t want to be shown. There should be a trimmed version or some way to adjust it. It could be a short story. For instance, if a fight scene lasts five minutes and isn’t kid-friendly, I would like to see a kid-friendly version that merely depicts what it means without a lengthy fight sequence.
4. Scene Summarization
Then, the fourth aspect is scene summarization. What I mean by scene summarization is that sometimes you don’t understand what is really happening in the movie when you miss the 5 minutes of the movie when you step outside while your entire family is enjoying the movie. If I could just have a summary of what has happened so far, that would be really great.
5. Voice-Controlled Scene Navigation
Finally, the fifth aspect I want to see is when I want to navigate to a scene, I don’t want a seek bar. Instead, I should have the capability to use my voice remote to say, “Hey, this is the scene; this is the situation in this movie. Can you take me to that particular moment?” That’s the fifth thing I would like to see.
Below is a quick depiction of how these features fit into the big ecosystem of AI in modern streaming apps.
Share your thoughts on areas that can be improved using AI in different domains.
Happy learning!
Shankar's Blog 