Interactive cinematic experiences will mainstream AI films.
In this post, I range from talking about AI, plastic surgery in Tokyo, tech stacks, Danny Thomas, Mayberry, Gomer Pyle, and the Marvel universe. Somehow, it makes sense.
The tech stack for creating films with AI is maturing. Problems of visual drift in a character’s appearance (e.g., facial details, wardrobe) are diminishing, and the path to consistency is coming into place within a few more release cycles of AI-based video models. (Some existing models already do it quite well but only over short durations.)
AI films open the door, some will say the flood gates, to many more people creating cinematic stories. I’m not going to address the protest about AI slop since human-generated slop dominates entertainment already.
Anyone arguing that AI lacks creativity, and only spits out the average of what AI has been train on, hasn’t spent hours everyday for the last few years striving to understand AI and working with the tools. The advances are staggering, though still not recognized by the widespread public. We’re still in the early stages.
I love cinema. Movies have formed a vital role in my life: in helping face sorrow after the death of my father; in recognizing patterns of behavior in characters that resemble myself and those around me; and simply enjoying a chill night on the sofa watching a no-brainer flick.
I’m looking beyond movies as linear experiences roughly two-hours long, and looking towards cinematic stories that we could not experience with AI.
Plastic Beauty
Recently, I watched the Japanese series Plastic Beauty. As my usual routine, I avoided knowing almost anything about the series before watching the show. I prefer to experience a story as it unfolds. Your expectations about a show changes if you’ve not read the blurb about the plot beforehand.
The first episode, in this series about obsession with plastic surgery, highlights the character Yoshiko Tanaka , a voice actress aspiring to be an idol. But she was rudely told that she would never make it with her face. Yoshiko story centers the first episode and the plot opens up with two doctors with different views about plastic surgery. After episode two, Yoshiko disappears from the story as the drama between the two surgeons take hold of the main plot. (The story about the tension within the plastic surgery clinic wasn’t all that interesting to me.)
But, what happened to Yoshiko? (Played by actress Hana Amano.) Initially, the first two episodes gave me the expectation that the series was to be her story. Perhaps I partly thought that since one of the thumbnails on Netflix featured her.
(It’s not the first show I’ve seen on Netflix with a misleading thumbnail.)
I wish I had the option within the Netflix app to branch the show after episode two and follow Yoshiko’s story.
But how would that work?
We’re entering an era with AI where streaming platforms, in 2028, will be able to offer these branching capabilities (for an upgraded subscription price).
2028: barely more than a year away. I wanted to make a prediction and attach a date. The barriers are becoming less about AI. Business strategy and content licensing are bigger constraints.
Let’s take a high-level look at the components required.
The app, the standards, the engine.
Protocols sit in the middle so platforms and generation engines can be built by different companies. IP rights underpin everything.
The viewer
Depending upon your perspective within industry, you might refer to the viewer as subscriber, user, audience. Obviously, the viewer is the heart of the experience. In the scenario I’m describing here, I’m only referring to the viewer as having the ability to branch a storyline by choosing a character and follow that character’s POV.
The UI is the easiest part, though it will get more complex as more decision points and options are added. For our example, we will keep to a single decision point of changing the episodes to be the story of a secondary character.
Platform capabilities
The platform, not just the frontend app, must be able to support capabilities that are featured. These features may become a differentiator among platforms. Netflix and Amazon Prime may seek competitive advantages in which advanced features are offered for branching media.
Platform capabilities at this level of the system are about software engineering and not AI. There’s a lot of state management that would need handling for every user. Platforms already do that through user profiles, watchlists, tracking progress in a watched video, personalized recommendation algorithms, session management, etc.
Membership tiers via paid subscriptions, and video purchases, are core tracking choices per user that decide what the viewer can and cannot watch. Branching media would be a paid upgrade that some viewers would choose and others not. Branching would be an extension of the server-side functions that platforms already run. Think of the streaming platform as the playback, authorization, and delivery tier.
Does the demand for this type of service justify the costs in developing these systems? Traditional live action production costs prohibit this follow-a-secondary-character scenario entirely. Would this branching capability justify an upgraded subscription price? Would the revenue of the upgraded subscription provide adequate return for the costs of licensing and developing these branched episodes?
It’s not a far leap to imagine how production companies can utilize AI to generate alternate versions of episodes.
Protocols
Streaming platforms do not need to control all the layers for supporting branching media. Other companies, like studios or AI providers, could manage the engine layer. Protocols will be needed for interoperability between the generative engine layer and the platforms. There are likely tremendous business opportunities emerging for the engine layer, which is context engineering on a large scale.
I wouldn’t be surprised if Netflix, Amazon, Disney+, or Google develop their own server-side approaches and present those as the de facto standards for branching media platforms.
In teaching undergraduates, I always was surprised that hardly any student knew about interoperability standards. The IETF and W3C have been standardizing protocols for decades. Those standards, along with de facto industry standards, enable everything we do online.
To facilitate branching media, platforms do not need to invent protocols from scratch. Existing protocols for delivery audio/video over the Internet already handle many aspects. For example, streaming platforms do not deliver the entire movie at one-time while you’re watching it. There’s a reason why it’s called streaming.
Standards make for a process of delivering online video that does not rely on proprietary techniques that each vendor must figure out for themselves. But the companies must engineer their systems and applications to support the standards. Professionals who have a deep understanding of standards and the process of evolving and writing standards remain a good career track.
At least three core categories of protocols and standards are required:
Narrative graph and divergence
Stream packaging and dynamic manifests
Versioning
Other standards also would need to be adopted, such as authentication and content authenticity. But those are not unique to branching media.
AI film tech stack
The making of AI films are receiving a lot of attention with companies like Runway dedicated to AI filmmaking while others, like ElevenLabs, are offering tools for easing the integration of audio and video. In this article, I’m not going to say much about the AI film tech stack since it’s broadly covered elsewhere. But I will be writing about it in detail in future posts.
Context engineering for storyworlds
The context layer is where I’m putting most of my work these days. A storyworld that supported optional episodes following different characters needs a lot of information. Plastic Beauty has 2 main characters and 22 supporting characters.
Compute
In our example, if someone has requested the Yoshiko storyline before, are those Yoshiko episodes cached on the system for the next person choosing that storyline? That’s the likely approach. In our scenario where the viewer’s only decision point is to choose to follow a supporting character, then all episodes could be pre-generated. At some point, as these systems evolve, perhaps each Yoshiko storyline could be unique for each viewer. But that requires another level of engineering.
Content and IP rights.
Content is the huge differentiator now for streaming platforms. Clearing IP rights for branching media will be an endeavor unto itself for existing content. But original content is likely to be developed with branching media in mind.
But, why?
Spin-offs have long been a staple of TV production. On the February 15, 1960 episode of The Danny Thomas Show (previously known as Make Room for Daddy), the character of Andy Griffith was introduced; the episode was titled Danny Meets Andy Griffith.
Over the next 8 years, 249 episodes of The Andy Griffith show aired on CBS. My childhood memories have many afternoons watching reruns about the show set in Mayberry, though I much preferred the Jim Nabors spin-off Gomer Pyle – USMC.
A modern variation of the spin-off concept is the Marvel franchise, an interconnected storyworld engine. The Marvel universe is a better model for what we’ll likely see emerge with series built on a cohesive storyworld where you can follow various characters. Not all storyworlds need be superhero fiction or fantasy.
Perfect Beauty isn’t the perfect example that could form the basis of a vast storyworld. But its Tokyo settings, characters, and their dilemmas project the possibilities that could emerge. And I think we all could imagine the possibilities of storyworlds originating from Japan. Or, maybe someone will imagine a return to Mayberry where you can choose to follow the antics of whichever character you like in a small North Carolina town.