In this post, I range from talking about AI, plastic surgery in Tokyo, tech stacks, Danny Thomas, Mayberry, Gomer Pyle, and the Marvel universe. Somehow, it makes sense.

The tech stack for creating films with AI is maturing. Problems of visual drift in a character’s appearance (e.g., facial details, wardrobe) are diminishing, and the path to consistency is coming into place within a few more release cycles of AI-based video models. (Some existing models already do it quite well but only over short durations.)

AI films open the door, some will say the flood gates, to many more people creating cinematic stories. I’m not going to address the protest about AI slop since human-generated slop dominates entertainment already.

Anyone arguing that AI lacks creativity, and only spits out the average of what AI has been train on, hasn’t spent hours everyday for the last few years striving to understand AI and working with the tools. The advances are staggering, though still not recognized by the widespread public. We’re still in the early stages.

I love cinema. Movies have formed a vital role in my life: in helping face sorrow after the death of my father; in recognizing patterns of behavior in characters that resemble myself and those around me; and simply enjoying a chill night on the sofa watching a no-brainer flick.

I’m looking beyond movies as linear experiences roughly two-hours long, and looking towards cinematic stories that we could not experience with AI.

Plastic Beauty

Recently, I watched the Japanese series Plastic Beauty. As my usual routine, I avoided knowing almost anything about the series before watching the show. I prefer to experience a story as it unfolds. Your expectations about a show changes if you’ve not read the blurb about the plot beforehand.

The first episode, in this series about obsession with plastic surgery, highlights the character Yoshiko Tanaka , a voice actress aspiring to be an idol. But she was rudely told that she would never make it with her face. Yoshiko story centers the first episode and the plot opens up with two doctors with different views about plastic surgery. After episode two, Yoshiko disappears from the story as the drama between the two surgeons take hold of the main plot. (The story about the tension within the plastic surgery clinic wasn’t all that interesting to me.)

But, what happened to Yoshiko? (Played by actress Hana Amano.) Initially, the first two episodes gave me the expectation that the series was to be her story. Perhaps I partly thought that since one of the thumbnails on Netflix featured her.

(It’s not the first show I’ve seen on Netflix with a misleading thumbnail.)

I wish I had the option within the Netflix app to branch the show after episode two and follow Yoshiko’s story.

But how would that work?

We’re entering an era with AI where streaming platforms, in 2028, will be able to offer these branching capabilities (for an upgraded subscription price).

2028: barely more than a year away. I wanted to make a prediction and attach a date. The barriers are becoming less about AI. Business strategy and content licensing are bigger constraints.

Let’s take a high-level look at the components required.

The app, the standards, the engine.

Protocols sit in the middle so platforms and generation engines can be built by different companies. IP rights underpin everything.

The viewer

Depending upon your perspective within industry, you might refer to the viewer as subscriber, user, audience. Obviously, the viewer is the heart of the experience. In the scenario I’m describing here, I’m only referring to the viewer as having the ability to branch a storyline by choosing a character and follow that character’s POV.

The UI is the easiest part, though it will get more complex as more decision points and options are added. For our example, we will keep to a single decision point of changing the episodes to be the story of a secondary character.

Platform capabilities

The platform, not just the frontend app, must be able to support capabilities that are featured. These features may become a differentiator among platforms. Netflix and Amazon Prime may seek competitive advantages in which advanced features are offered for branching media.

Platform capabilities at this level of the system are about software engineering and not AI. There’s a lot of state management that would need handling for every user. Platforms already do that through user profiles, watchlists, tracking progress in a watched video, personalized recommendation algorithms, session management, etc.

Membership tiers via paid subscriptions, and video purchases, are core tracking choices per user that decide what the viewer can and cannot watch. Branching media would be a paid upgrade that some viewers would choose and others not. Branching would be an extension of the server-side functions that platforms already run. Think of the streaming platform as the playback, authorization, and delivery tier.

Does the demand for this type of service justify the costs in developing these systems? Traditional live action production costs prohibit this follow-a-secondary-character scenario entirely. Would this branching capability justify an upgraded subscription price? Would the revenue of the upgraded subscription provide adequate return for the costs of licensing and developing these branched episodes?

It’s not a far leap to imagine how production companies can utilize AI to generate alternate versions of episodes.

Protocols

Streaming platforms do not need to control all the layers for supporting branching media. Other companies, like studios or AI providers, could manage the engine layer. Protocols will be needed for interoperability between the generative engine layer and the platforms. There are likely tremendous business opportunities emerging for the engine layer, which is context engineering on a large scale.

I wouldn’t be surprised if Netflix, Amazon, Disney+, or Google develop their own server-side approaches and present those as the de facto standards for branching media platforms.

In teaching undergraduates, I always was surprised that hardly any student knew about interoperability standards. The IETF and W3C have been standardizing protocols for decades. Those standards, along with de facto industry standards, enable everything we do online.

To facilitate branching media, platforms do not need to invent protocols from scratch. Existing protocols for delivery audio/video over the Internet already handle many aspects. For example, streaming platforms do not deliver the entire movie at one-time while you’re watching it. There’s a reason why it’s called streaming.

Standards make for a process of delivering online video that does not rely on proprietary techniques that each vendor must figure out for themselves. But the companies must engineer their systems and applications to support the standards. Professionals who have a deep understanding of standards and the process of evolving and writing standards remain a good career track.

At least three core categories of protocols and standards are required:

• Narrative graph and divergence

• Stream packaging and dynamic manifests

• Versioning

Other standards also would need to be adopted, such as authentication and content authenticity. But those are not unique to branching media.

AI film tech stack

The making of AI films are receiving a lot of attention with companies like Runway dedicated to AI filmmaking while others, like ElevenLabs, are offering tools for easing the integration of audio and video. In this article, I’m not going to say much about the AI film tech stack since it’s broadly covered elsewhere. But I will be writing about it in detail in future posts.

Context engineering for storyworlds

The context layer is where I’m putting most of my work these days. A storyworld that supported optional episodes following different characters needs a lot of information. Plastic Beauty has 2 main characters and 22 supporting characters.

Compute

In our example, if someone has requested the Yoshiko storyline before, are those Yoshiko episodes cached on the system for the next person choosing that storyline? That’s the likely approach. In our scenario where the viewer’s only decision point is to choose to follow a supporting character, then all episodes could be pre-generated. At some point, as these systems evolve, perhaps each Yoshiko storyline could be unique for each viewer. But that requires another level of engineering.

Content and IP rights.

Content is the huge differentiator now for streaming platforms. Clearing IP rights for branching media will be an endeavor unto itself for existing content. But original content is likely to be developed with branching media in mind.

But, why?

Spin-offs have long been a staple of TV production. On the February 15, 1960 episode of The Danny Thomas Show (previously known as Make Room for Daddy), the character of Andy Griffith was introduced; the episode was titled Danny Meets Andy Griffith.

Over the next 8 years, 249 episodes of The Andy Griffith show aired on CBS. My childhood memories have many afternoons watching reruns about the show set in Mayberry, though I much preferred the Jim Nabors spin-off Gomer Pyle – USMC.

A modern variation of the spin-off concept is the Marvel franchise, an interconnected storyworld engine. The Marvel universe is a better model for what we’ll likely see emerge with series built on a cohesive storyworld where you can follow various characters. Not all storyworlds need be superhero fiction or fantasy.

Perfect Beauty isn’t the perfect example that could form the basis of a vast storyworld. But its Tokyo settings, characters, and their dilemmas project the possibilities that could emerge. And I think we all could imagine the possibilities of storyworlds originating from Japan. Or, maybe someone will imagine a return to Mayberry where you can choose to follow the antics of whichever character you like in a small North Carolina town.

This article starts out with updates on relevant tools, game engines, and ends with thoughts on a great essay about AI and creativity by the former head of Disney and Dreamworks. If you read only one thing that I mention here, be sure to read that.

Storyworld tools: voice & continuity

AI anime from 松丸 彗吾(keigo matsumaru).
Voice acting example, in Japanese, of the new v4 text-to-speech from ElevenLabs. The music is by Suno; images by GPT Image 2.5; sound effects, editing, animations done with Opus 5.5.

ElevenLabs v4 introduces more capabilities, in more than 90 languages, to direct the emotion of how the text is spoken through audio tags. Lots of examples in the video below.

Google AI: Gemini 3.8 Flash TTS launch.
New voice models also from Google. As good as ElevenLabs? Can I cancel my ElevenLabs subscription? I’m not sure, yet. Matsumaru (see above) ranks ElevenLabs above Gemini for Japanese voice acting.

Prompting guide for Gemini 3.8 TTS: https://ai.google.dev/gemini-api/docs/speech-generation#prompting-guide

Includes a set of tags for non-speech sounds:

Google Research: “Automating coherent long-form video generation”.
From Google Research official account.

In generated video, maintaining consistency in character’s clothing and surrounding environment is improving rapidly. But it’s still difficult as the video gets longer. For details, see the research update in the blog post: Automating coherent long-form video generation. It’s worth a close read even if you’re not using Google tools for video generation.

World Models & Engines

AMD acquires World Labs for $8.2 billion.
I’ve been admiring the work done by Fei-Fei Li and team at World Labs, which was founded in 2024. I didn’t expect them to become part of a chipmaker, but good for them. Fei-Fei Li will now be Executive Vice President and Chief Scientist at AMD.

Storyworlds are a small part of what’s done through the tools developed by World Labs. They’ve been promoting the term spatial intelligence and leaning heavily into robotics and computer vision. On her Substack, Fei-Fei Li describes her vision, which is in a different realm than what we see with OpenAI and Anthropic: the physical world and not just the digital world.

“Without having a focused hardware effort, AI is hobbled in efficiency.”

While my own specific work is in the digital realm, the societal impact of hardware that understands its surrounding physical environment will unleash a range of new consumer products and also new methods of manufacturing.

Images to 3d modeling to storyworlds for games.
Two posts from Anthropic employees caught my attention that showcase the increasing capacity of Opus, Fable, etc., to create game environments.

By GTA, Karpathy is using that as shorthand for an explorable 3D simulation. And that’s exactly what I’m working on.

Alex Albert provided the prompt for the 1906 SF example:

Recreate Market Street, San Francisco as it stood on April 17, 1906, the afternoon before the earthquake, in Blender.

Scope: the Ferry Building up Market to Fifth Street, including the Palace Hotel, the Call Building, the Chronicle Building, Lotta's Fountain and the Emporium.

Before modeling anything, build a source file from: the 1899-1905 Sanborn fire insurance maps (footprints, heights, materials, occupants), the Miles Brothers film "A Trip Down Market Street" (April 1906), period photographs from OpenSFHistory, the Library of Congress and the David Rumsey collection, and USGS topography. Record every building with footprint, height, facade material, occupant, and the source for each fact with a confidence level.

Build everything in Blender Python. No downloaded meshes, textures or HDRIs. Write reusable generators (Victorian commercial facade, mansard roof, bay windows, awnings, painted signage, gas and electric street lamps, cable car, horse-drawn wagon, early automobile) and assemble the street from the source file, so every building traces back to data.

Provide a 10 second video up the street.

Note that it’s a lot more than take this photo and create a game.

It’s really remarkable, though. This would make for a great project for history class. Are digital humanities still a thing?

Later in this article, I point to an essay by Jeffrey Katzenberg who included this quote from Steve Jobs:

“It’s in Apple’s DNA that technology alone is not enough. It’s technology married with the liberal arts, married with the humanities, that yields us the result that makes our hearts sing.”

Sakura Crossing update with Opus 5.5.
I’ve been keeping an eye on the prototyping that GMI Cloud has been doing on this environment. Now they’ve tried it with Opus 5.5.

I also am finding that Opus 5.5 is what I’m reaching for when creating my storyworld.

Thomas Mahler on the difficulty of game dev.
Playing Ori and the Blind Forest is one of the great experiences that my daughter and I shared in her childhood, particularly during the lockdown. After finally escaping the Ginso Tree, (if you know, you know), we threw our arms around each other in a gleeful hug.

Thomas Mahler is the creator of Ori and the Blind Forest, which I think is the best video game from the past decade.

I agree completely, and it’s tiresome to see posts from people saying they’ve one-shot a video game. You can do a lot with these AI systems but a completed, fully functional game is a complex system. You can do a good prototype with Opus, Fable, Astra, but you need a game engine (like Unity, Unreal, Godot) to go further unless you’re game is quite simplistic.

Mahler, who knows a bit about video games, says:

Most of the stuff I see people post online that they proudly state ‘AI one shotted this game!’ are still godawful stuff that nobody would actually consider paying money for. And even the non-one-shots, 99% of those are... well, not great.

You’re not wasting your time learning a game engine.

The game examples we see online are generated in JavaScript, specifically three.js. That’s a great tool but games are pushing the limits. You can generate some cool environments. Here’s a call from an animator about that:

Game engines are intimidating, which is why the web games generated by Opus and Astra are appealing to vibe coders new to game development. But MCP allows you to use your favorite AI to help you learn and build. It’s fine to start with three.js and then go from there into a game engine.

Building with Agents

So much is happening with agents! So much! My head is spinning. I’m in Claude Code all day. This morning, I just started using Grok Bot as well. I still need to get into OpenAI’s Codex, which I’ve only played around with for a bit. But I need to stay focused on what I’m building and not trying out every new technique that comes up.

Coming back around to AI-generated video, here’s a post on a process:

Note that the guy who made the post works for a VC firm and is an investor in some of the tools that he mentions. Is that what a VC does these days? Anyway, there are some good tips in that post.

Show me your prompt.
If you seriously use any of Anthropic’s model, then you need to follow @thariq on X.

I’ve spent the last 3 days working in Claude Code on a single project across 15 sessions. Each session has almost a full context window with sources and iterative prompting. There’s no way that can be just shown to anyone. I do have the key parts hooked into a GitHub repo but it’s not just one prompt. The documentation for the repo is currently more than a hundred files.

Since I’ve been discussing making games with AI, Thariq just made a relevant post as I am writing:

In the replies, he mentions that this is “all prototype quality”. Let’s stress this sentence: “Use AI to work with you to bring your vision to life.”

A wider lens on AI & Creativity

I have a lot more bookmarks from X for this past week, but let’s wrap this up with some thoughts from Jeffrey Katzenberg, the former head of Disney’s animation studio and Dreamworks. He knows a lot about the business of creativity.

That’s a great essay on the matter of AI and Hollywood. It should be required reading for anyone studying the humanities.

AI is not killing creativity. There will still be careers for creative people, but you will need to be a lot more entrepreneurial.

“There will be more seats at the table, and very soon entirely new forms of storytelling.”

I didn’t realize, or had forgotten, that animation in the 1980s was considered a niche form of the business.

“Every time storytelling has met a genuine technological shift, from synchronized sound to color to computer animation, it has redefined the boundaries of the medium and grown larger in the process.”

Just prior to that quote, he names some of the most famous filmmakers who have “embraced new visual tools not as shortcuts, but as instruments, and expanded cinema in the process.”

A quibble I have with Katzenberg’s essay is that he frames it purely in a Northern California (Silicon Valley) - Southern California (i.e., Hollywood) divide. But he was likely writing this for specific people he knew in Hollywood that feared the encroachment of AI on their industry.

Expanded cinema. That phrase, “expanded cinema in the process”. That’s the process now occurring with AI filmmaking. Mix in game development, and we’ll see new forms of cinema and entertainment that will help us enjoy, reflect on, and engage with the world in which we live.

Note on process: I could have had Grok Bot pulled all my X bookmarks and write this post. I did have Grok Bot pull my recent X bookmarks and organize those into topics. But what would be the point of having AI generate this article for me? Sure, that would have been quick, easy, and job done. But it wouldn’t have had any value for me; I wouldn’t have learned anything unless I had sat at my desk this morning for a few hours, read through those posts and thought about how they’re relevant to me. Writing up these notes may be more useful to me than for anyone who reads this. And that is the value of writing for yourself.

On Thursday, September 24, 2026 Meta introduced their latest VR device: a headset that weighs about 100g. (It is tethered to a device but that’s a minor inconvenience at this stage.) With every product cycle, the form factor for VR will get more ergonomic. For a few years now, I’ve been thinking that VR headsets would get down to the size of a pair of ski goggles (compare Apple Vision Pro) or oversized sunglasses. I didn’t expect that this would come about until the 2030s.

The VR glasses were not the standout new products showcased at the annual Meta Connect conference. Zuckerberg emphasized a range of new AI glasses. Someday, they’ll drop the AI modifier and just call them glasses or sunglasses. I was surprised at the release of audio & AI eyewear that lacked the camera capabilities. But it’s a smart move since the camera is the most controversial part of those glasses. Earbuds and bulky over-the-ear headphones are not for everyone; so, there will be a market for stylish glasses with speakers built-in, along with other features.

Nostalgia

Experiences in the past were not necessarily better, but just different. Thursday’s VR announcement reminded me of my Thursday nights forty years ago. Every Thursday night in Sewanee, Tennessee was free movie night at the university’s movie theater, a rather dreary movie house built behind the former student union building. The free movie was usually a classic or an art house film. Not much studying got done on Thursday nights, and the 7:30pm free movie was among the highlights of my week.

Thursday, September 18, 1986. My junior year was off to a rough start but what happened on that free movie night remains vivid: two silent movies starring Buster Keaton: Cops (1922) and College (1927).

Comparing the cinema of the 1920s to that of the 1980s to that of today could be an entire chapter in a media studies book. Movies are a visual way of tracking technological changes.

The memory: a piano had been hauled into the theater and placed at the foot of the screen that night in 1986. The lights dimmed. As the movie started, one of Sewanee’s most emblematic professors, Waring McCrady, played a rousing piano accompaniment to the silent films. The improv performance made the night.

Now, with streaming, any night is free movie night. (Free as in a monthly subscription.) Movies have played a role in my life that’s almost as impactful as books.

An immersive experience in VR would be watching a silent movie while listening to an improv piano coming through your headset. With AI, the improv performance could be unique for each individual. That’s a difference experience than being in a rundown movie theater in a small Tennessee town atop a mountain. It’s not a better experience but it’s different and, certainly, not an experience that any of us imagined in 1986. And, surely, Buster Keaton never imagined his movies being watched in virtual reality a hundred years later.

For Thursday night of this week, I watched Ugetsu, a great film made in Japan in the early 1950s. Based on ghost stories by the 18th century Japanese writer Ueda Akinari, Ugetsu is the type of movie you might put off watching. You’ll think you’re not in the mood for a movie so different than anything you see that’s made today. But, give Ugetsu a viewing and experience a suspenseful masterpiece of cinema.

Even though I have the Criterion Blu-Ray version of Ugetsu (with several great special edition features about the movie), I watched it on my iPad rather than my large-screen TV. I rarely watch anything on that large screen. The iPad is my most used device, especially when nestled in the darkness of my bedroom.

When AI writing ceases to be slop

At the end of this post is a story written by Claude Opus 5.5 on September 22, 2026. My reaction is unexpected. I have rolled my eyes at all previous attempts by Claude or ChatGPT at fiction. Sometimes the plots would be intriguing but the prose would be so cliché and overwritten. For me, this story—The Sitter—is not AI slop. If by AI slop, you mean writing that lacks creativity and not a pleasing experience. If by AI slop, you mean anything generated by AI, then that’s a different topic of discussion.

I know quality fiction when I read it. More than 50 years of reading has provided some insights. Currently, I’m making my way through the collected works of Yukio Mishima. My two favorite novels are The Magic Mountain and Absalom, Absalom. And I have a degree in English. I only say all that as an indication that I am “well-read”, as people say, in literary fiction.

Certainly, there are flaws in this story. Likewise, almost every published story has flaws when you are seeking those out. (It always has been easier to criticize than to create.) There are aspects of this story, particularly some line editing, that I would change but not all that much. I’m reprinting the story exactly as it was generated by Opus 5.5.

If a work brings joy and pleasure of some type to a person, does it matter if it was created by AI?

The reading experience is personal

Without getting into the deep waters of reader-response theory, my reaction is surely influenced by the seed for this story and how the story generated by Opus 5.5 speaks to me, almost as if written for me.

Written for me: it was written for me, and we may be in an era where a genre of fiction utilizes aspects of what the AI knows about the individual reader.

Now, don’t get all flustered that this is going to replace all novels and stories. I said a genre of fiction. New genres are emerging with AI-enabled media.

The seed for this story

One Spring day in March 2008, I was walking through the neighborhood of San Telmo in Buenos Aires. Our apartment was on the edge of that barrio. I came across this photo on the ground.

Unusual, I thought. In all my years of walking the streets of Buenos Aires, I don’t recall coming across a stray photograph. I snapped a photo with the camera I had with me at the time. (This was before smartphones.) I picked up the photo, and took it home. I’ve lost the photo in one of many moves from apartment to apartment in those years. But this little discovery has stayed in my mind. At the time, I also wrote briefly about it on the blog that I kept about my occasional walks in Buenos Aires, 2005 - 2013.

Opus 5.5 has just been released, and I’ve come across postings on X about how the writing style of Opus 5.5 is better. I wanted to test that. My expectations were low.

The prompt. I gave Opus 5.5 this image and the prompt:

In 2008, I found this photo on the ground in barrio del San Telmo in Buenos Aires. I do not know anything about the man or the photo. Imagine a story, in the style of Borges, about the man coming to life from the photo on the ground.

The “in the style of Borges” will definitely steer Opus in a particular direction. Borges’ style of writing is difficult, and most creative writing professors would advise, “Don’t try to write like Borges.” Generally, I do not like using in the style of [insert artist] in prompts. But this was just meant for evaluation, a quick benchmark, that I never expected to share because I thought it would be incredibly bad.

The story’s premise is derivative. Yet, my response to the story took me by surprise. I had to reread it a couple of times. Then I uploaded it to ElevenReader and had the voice of Laurence Olivier read it to me. I listened, then I listened to the story again.

Coincidence or context?

Others may not feel the same way about this story, and that’s okay. The personal connection I have to that photograph and the way that the story weaves details of Buenos Aires resonate with me: the named streets of San Telmo, the fair in Plaza Dorrego every Sunday. The description of Parque Lezama, where I used to walk everyday, and the benches where I sat near the massive ombú tree.

Among my best memories of Buenos Aires are from those streets, particularly the street corner where Bar Británico has been located for nearly 100 years. It was my “office” where I sat for hours writing, reading, and simply observing. In my years of living in Buenos Aires, Bar Británico was opened 24 hours and I recall sitting in the Británico with friends all night until the sun rose over Parque Lezama.

On the opposite corner is El Hipopótamo, another classic of Buenos Aires. My most vivid memories are not in photographs, like the time when Francis Ford Coppola and Vincent Gallo sat at the table next to me in the Británico where Coppola talked (loudly) about directing Brando and Gary Oldman. The entire bar was listening in on that conversation.

A good story makes connections and stirs a response within the reader. Maybe the story below doesn’t evoke any emotion for everyone.

Claude knows a lot about me. It knows about my experiences in Buenos Aires. I used Claude Code to migrate my old WordPress blog (Buenos Aires, City of Faded Elegance) to a static site. Claude knows about my other writings on Buenos Aires, and my blog surely was scraped from the Web and sucked into the vast training data.

An easy experiment: upload the photo and prompt into a Claude account with no context of myself. What variation of a story does the system produce?

If no one else finds this story to be more than AI slop, then that’s more remarkable: personalized storytelling unique and only meaningful to the person requesting the story. That's a genre only possible through AI.

The Sitter

Every photograph is a mirror that has forgotten whom it reflected. — Heráclito Ruiz de Lomas, Tratado de las sombras fijas (Montevideo, 1889)

I found him on a Tuesday in March of 2008, a little before noon, on the paving stones of Humberto Primo, a few steps from its corner with Defensa. He lay face up among the fallen leaves of a paraíso tree, beside the marble step of a house that had once been important and an empty blister of pills whose purpose I preferred not to guess. The pavement of San Telmo is made of small square stones, each one slightly swollen, so that the city there resembles an enormous quilt, or the board of a game whose rules have been lost; the photograph lay across nine of these squares, and I remember thinking, absurdly, that it had been placed rather than dropped.

It showed a man of perhaps sixty, seated in a high-backed leather chair. He wore a three-piece suit, a white shirt, a dark tie. He held a pipe to his mouth with the gesture of someone who has been asked to hold still and has decided, out of courtesy, to obey. His expression was that of a man who has just been quoted a price and finds it reasonable. Rain, or years, had worn a pale constellation into the upper corner; a crease crossed his chest like a ceremonial sash. I did not turn the photograph over. I cannot say why. Perhaps it was the same instinct that makes us leave certain letters unopened, not from fear of what they say but from reverence for the moment before we know.

I must here record a digression, since without it what follows would be merely incredible, whereas with it it is only improbable.

In the old National Library on Calle México, a few blocks from where I found the photograph, there was once a copy (the only one, I believe) of the Tratado de las sombras fijas of Heráclito Ruiz de Lomas. Its thesis is simple and, like all simple theses, inexhaustible. The camera, Ruiz de Lomas argues, does not capture light; it captures duration. The two or three seconds during which the shutter stood open are not a picture of time but a portion of it, sealed like water in a jar. A sealed instant is inert, but it is not dead. It waits.

What wakes it, according to the Tratado, is the gaze, but not every gaze. A widow who looks at her husband’s portrait sees her husband, which is to say she sees her own memory; the photograph does no work and receives nothing. Only the gaze of a stranger, who knows neither the name nor the date nor the outcome of the life, feeds the sealed instant with time that is not already its own. Ruiz de Lomas calculated, with the arithmetic serenity of his century, that a photograph left face up on a public street for a single day receives more ignorant glances than a family album receives in a hundred years. He further calculated the number required for the sealed instant to rise, stand, and walk. I will not reproduce the figure; it is either too high or too low, and I have come to suspect it depends on the pavement.

Ruiz de Lomas died in 1902, some say in the asylum at Open Door, others that he never existed at all. I incline to both opinions.

I carried the photograph to my room on Calle Chile and propped it against the base of the lamp. I did not look at it again that day, or I looked at it in the way one looks at furniture.

That night he was sitting in my only armchair. I say *sitting*, but the verb is imprecise: he held himself as though his back rested against a height that was not there, the tall leather of the studio chair, which he had not brought with him but whose shape he could not stop occupying. The pipe was in his hand, unlit. He was entirely gray. I do not mean pale; I mean that he was composed of the same limited palette as the print, so that the yellow light of my lamp seemed to fall on him and fail.

“No se mueva,” he said, and then, seeing my face, apologized. It was, he explained, the last thing he had heard, and it had worn a groove in him.

He remembered that one instant with a perfection I have never known about anything. The photographer’s voice from behind the black cloth. The sweet chemical smell of the studio. A fly walking up the windowpane toward a light it would never reach. The discomfort of a collar stud. Everything else (whether he had a wife, what he did for a living, what city lay beyond the studio’s door, though he supposed it was this one) he had to deduce, the way astronomers deduce an unseen planet from the tremor of a star.

“I am not him,” he told me, with the gentleness of a man correcting a waiter. “He stood up. He paid. He went down the stairs and lived another fifteen or twenty years and died of something ordinary. I stayed. I am the four seconds he gave away.”

He asked me to help him find his name.

I recall the weeks that followed in a disorder I have chosen not to correct. We walked at night, when the streets of San Telmo are emptiest and the cobblestones give back the day’s heat. We stood outside houses on Balcarce and Carlos Calvo that he thought he might have entered once; he could never be sure, because the only interior he truly knew was a photographer’s studio, and all studios, he said, are the same studio.

One Sunday I took him to the fair in Plaza Dorrego. I had seen, among the siphon bottles and the tango sheet music and the brass doorknockers shaped like hands, a dealer who sold shoeboxes of photographs under a hand-lettered sign: fotos de desconocidos, un peso. My companion bent over the box, and I watched him grow, if such a thing is possible, grayer.

In that box were a bride of 1923 who had outlived her groom by half a century; a boy in a sailor suit, frowning at something outside the frame; three soldiers of a war Argentina did not fight; a dog; a woman laughing so hard that her face had blurred; a family arranged on a staircase like the notes of a chord; a man in a hat who resembled my companion so closely that for a moment I thought the box contained a mirror. None of them had names. All of them, I understood, had been looked at by strangers.

He understood it before I did. He was not a prodigy; he was a citizen. Buenos Aires at night holds a second population, the photographed, who walk the same streets as the rest of us and order nothing in the cafés and stand at the edges of plazas looking at the fronts of buildings as though at faces they almost recognize. I have seen them since. Anyone can, at four in the morning, at the Británico across from Parque Lezama, if he knows nothing about them and does not try to.

It was on a bench in Parque Lezama, under the great ombú, that he told me the rest.

“You must not look for it,” he said.

“Your name?”

“The gaze that knows gives nothing. If you learn my name, you will look at me and see a fact, and I will fold back into the paper the way a letter folds back into its envelope.”

I asked him why, then, he had asked me to help him search.

He considered the question for a long time, turning the cold pipe in his fingers. “The faithful search for God,” he said at last, “and most of them would be terrified to find Him. It is the searching that keeps them alive. Why should I be different?”

I confess I was tempted. I have spent my life putting things in order, assigning each object its number and its shelf, and I had noticed, along the bottom border of the print, the ghost of a studio’s stamp: a few letters, the first digits of an address on Calle Florida. A week in the right archive would have been enough. I did not go. I understood that the kindest thing I could do for him was the one thing that went against my nature: to leave him uncatalogued.

He stopped appearing in the winter of that year. It was not because I learned anything. It was because I had come to know him, and I had forgotten that familiarity, too, is a kind of knowledge. Each night he was a little less gray and a little more paper, and one night there was only the chair.

On our last evening he asked me a question I have not been able to set down since.

“And you,” he said. “Who photographed you?”

I told him about a photograph taken when I was a boy, on a porch in another country, in a light I can no longer describe. It was lost when my parents’ house was sold. I have no idea where it went.

He nodded, as a man nods who has been quoted a price and finds it reasonable.

I have kept the photograph. I have never turned it over. Sometimes, crossing a plaza, I feel on my face the weight of a gaze that knows nothing about me, and I understand, with a mixture of terror and gratitude, that somewhere, on some pavement I will never see, among leaves whose names I do not know, a stranger has stooped to pick me up and is wondering who I was; and that it is only by the grace of his ignorance that I am able to write these lines.

Translations matter, and I’ll come to that. But reading any translation matters more than not reading at all. I’ve not seen the Christopher Nolan movie. I’m sure it’s entertaining but there are many other movies that I want to see. I like Nolan’s movies but there are personal reasons I don’t want to watch his Odyssey.

We remember in fragments. We live in fragments.

Homer composed The Odyssey from fragments of an oral tradition. Composition arise from choices: arrangement, selection, omission, edits, knowing when to stop, to keep going, even when and what to start. Living is composed of the choices we make in life, and how we respond to choices made for us, situations imposed upon us.

I enjoyed The Iliad more, and these Homeric texts are foundational works of literature and Western civilization. As I write that last phrase, Joseph Campbell’s The Hero with a Thousand Faces come to mind as a reminder that the West is not the world:

“Throughout the inhabited world, in all times and under every circumstance, myths of man have flourished; and they have been the living inspiration of whatever else may have appeared out of the activities of the human body and mind.”

Surely, in some future century, the religions of today will seem as odd as the gods of The Odyssey.

A former student (shout out to Mohammed) recommended that I, at least, watch Ezra Klein’s take, which is a well-crafted video essay using the movie as a reflection on contemporary politics.

The memory of a course from college forms the main reason I don’t want to see the movie by Nolan.

Sewanee, my sophomore year, Fall of 1985

English 211-212 Representative Masterpieces was taken by every student I knew, even those who were not English majors. I’m glad to see that Sewanee still teaches this course, though they’ve condensed a two-semester course that covered Homer, Dante, Flaubert, Ibsen, and others down into a single course. I wonder if it’s still broadly taken by most students as it was in the 1980s.

Most of the Fall Term was closely reading our way through The Odyssey and The Iliad a few “books” at a time. Three mornings a week, I sat at the old wooden tables in a classroom on the second floor of Walsh-Ellett Hall, a sandstone building on the campus’s quad. Professor Stirling’s lectures were carefully constructed explanations as his magisterial voice read selected passages. Good lectures are performances. I recall a fellow student saying over beers one night that at the end of a Stirling lecture, “you want to applaud.”

Those sweet fragments of learning: an experience reading Homer with a beloved guide is the kind that I don’t want to dilute with a Hollywood movie. A few years after I graduated, Professor Stirling died of cancer on a Christmas Eve at the age of 54. Stirling also loved movies, and I usually saw him at ‘free movie night’ every Thursday at the Sewanee Union Theater. The coffeehouse on Sewanee’s campus is named after him.

After death, we live on in the fragmented memory of those who knew us.

That’s why stories matter. Stories are experiences allowing us to view the world around ourselves. Often, I feel that I’m wasting my time, not taking the best advantage of my skills, by focusing on AI in the context of storytelling. Shouldn’t I be doing something more substantial than tinkering with AI-generated film clips and imagery? I wonder.

I have been on this journey of understanding how we tell and read stories in digital media for more than 25 years, though as an undergrad I never knew this path would exist. Now, more rapidly than ever, we’re moving towards new genres of stories only capable through an AI-enabled medium.

I can’t remember which translation we read in college; I have long lost my college books and notes. I think it was the Lattimore translation, and not the Fitzgerald translation read by Dan Stevens for the audiobook.

What we suffer through as students. I’ve always enjoyed reading, learning, and writing. But I hated homework, and the pressure of assignments. Throughout college, as an English major, I recall having an essay due seemingly every other week, often weekly. Close reading and intense writing were the hallmarks of a liberal arts education. (Having been on the faculty of a small liberal arts college, I can’t say that is consistently the case any more.) Without the intensity of writing those essays, sometimes staying up all night because I had procrastinated, my methods of thinking would likely be different today. I don’t know the counterfactual since that depends upon so many other choices I could have made as a young man regarding where I attended college and what I chose to study. I often feel now that I made the wrong choice of majors. But I’m also a different person at 60 than I was at 20. I cringe when I think about myself in my twenties.

The resourceful translator

My kid attends a high school with an International Baccalaureate curriculum. The 10th graders are reading Emily Wilson’s translation of The Odyssey. It’s a fine translation for a student, though not what I would choose. I don’t want to wade into the battle of the translations. I suspect that Wilson knows that she is not making the definitive translation but writing for a specific audience of readers.

I am puzzled at some of Wilson’s choice of words, and that she inserts words that leads the reader by the hand in interpretation as if the reader is not smart enough to read the text closely. An example is the opening of Book Nine, where she has the narrator remark, “Wily Odysseus, the lord of lies, answered”. The Fitzgerald translation starts Book Nine with “Now this was the reply Odysseus made”. If you pay attention in reading the Odyssey, then it’s clear that Odysseus is deceptive and not a reliable narrator. Fagles version of that line is “Odysseus, the great teller of tales, launched out on his story”. The Greek is “τὸν δ᾽ ἀπαμειβόμενος προσέφη πολύμητις Ὀδυσσεύς”.

The epithet πολύμητις is the problematic word. Literally, it’s “of many counsels”. From that, a translator has many options. Wilson’s use of “Wily” is not incorrect but it does convey a perspective, especially when paired with “the lord of lies”, which made me laugh when I read it. Other words used could be “resourceful” or “cunning”, which I think would be appropriate. Fitzgerald just skips the adjective entirely.

Wilson strives to make it clear that Odysseus is not a good man, not a hero. But that’s also clearly conveyed through Fitzgerald’s translation, just not with flashing neon signs. Odysseus spends a lot of time with goddesses in the years when he doesn’t seem to be in that much of a hurry to get back to his wife. Odysseus is a deeply flawed person, which is what makes him a great character in an epic story.

As my kid stood, sleepy-eyed, by my desk this morning, I told her that I also was writing an essay on The Odyssey today. The fifteen year-old replied, “Is that supposed to make me feel better?” Indeed, the high school’s assignment is as rigorous as any I had in college except that we were never given a rubric that detailed the structure of our writing. Instead, we usually received a two-sentence prompt on a theme with the expectation that was all we needed to write a thousand-word essay.

An essay: the challenge of composition, fragments, piecing together our thoughts and observations bolstered by quotes from a text. It was that analytical process of writing an essay based only on a close reading of the text, resisting the easier route of using secondary sources, that refined how I approached problem solving. This same process became very helpful as I taught myself to code and understand Internet technologies and how to construct digital libraries.

Finding home

The family reunion of my first cousins on my father’s side of the family happens every year though I’ve not been in 20 years. It’s unlikely I would have made it this year if my brother hadn’t called me a few weeks earlier and asked me to drive down and that we go together.

A 4-hour drive down I-81 into Tennessee. Then, a state highway for an hour. Then, five miles down a narrow, curvy road towards the Powell River leads to a dirt road that takes me to my brother’s trailer where I spent the night. Tennessee is covered by the tributaries that snake off the main rivers traversing through the hills of the state.

The next morning I opted to drive five hours on the scenic backroad, highway 52, to middle Tennessee. I learned to drive on the hairpin curves of highway 52, so I was comfortable maneuvering up, down, and around the hollows. Hardly any traffic on a Thursday morning as we drove along miles and miles of roadway where no one lived until reaching the occasional small towns, like the one where we lived our childhood.

Our maternal grandmother’s house, which was built in the 1920s and our adjoining house where we lived off and on, was razed years ago. The highway was relocated and, the yard where we played for many years now sits under the pavement of highway 52.

After passing through our hometown, we continued down highway 52 towards the farm where my father grew up. “I hate crossing this bridge,” my brother said. In 1977, our father died as his car crashed off that small bridge and down the riverbank.

My cousins still farm the land that has been in the family since 1825. Don’t mistake that as ever having been a plantation. There’s no grandeur in my family’s history. For the reunion, we gathered in the community room of the small Baptist church down the road from the farm. Somehow, I was asked to give the blessing. Awkward. But I stumbled through the invocation. A framed black-and-white portrait of my paternal grandmother (1904 - 1979) looked down upon me.

About the reunion, I’m very glad I went. Seeing cousins that I’ve not seen in a very long time sparked a bond, a remembrance of family gatherings in the 1970s. Strengthening those connections is a next step for myself.

Yesterday, I re-listened to Dan Stevens riveting narration of The Odyssey while reading the Fitzgerald translation. Listening while simultaneously reading a text is a superb way of absorbing the work. Is this a life well-spent? Shouldn’t I be posting wittily on X or cranking out video tutorials on the latest AI tools.

My brother and I left the small chapel and took a different route to his home. This time, the backroads of Kentucky led us eastward and then down into the Elk Valley of east Tennessee before reaching my brother’s trailer in the darkness around 10pm that night. The last few miles were my brother navigating, “Curve to your right….curve to your left…now, curve to your right…keep curving…keep curving…now curve to your left.” The automatic high beams on my Subaru kept dimming at the wrong moments on those curves. No ongoing traffic but I could barely see where the road curved.

Throughout life, we never know when the road will curve.

Lighting our passage through this world.

In the last few years, I’ve realized a theme of my life’s creative work: absent fathers, identity, memory, and home not as a place but as a state of mind.

I cannot clearly count the number of places I resided, but it’s at least thirty. That’s why, long ago, I have come to think of home as the place wherever I am.

Six hours at my desk. That’s how long I worked on this essay yesterday. A day wasted? Maybe. But it’s a day I’m more likely to remember, a day sitting in thought. Mostly, we are moving through thought, rushing through thought to finish, to produce, to publish content than caring about the process.

I want to believe that AI can help with learning if used properly. But, I’m not sure how, as a student, I would use AI for The Odyssey and not succumb to the temptation, the seduction, of Claude for easing my pain. Since I always viewed homework as an output for merely a grade, which I didn’t value except to impress others, I never had the motivation or the patience to sit patiently with my thoughts as I did my homework. I also felt overwhelmed and a lack of focus as my attention scattered to my real interests.

But if I didn’t suffer through writing those essays during college, would I be able to do what I do now?

The Greek epics, blockbusters of their time, were spun by storytellers with a setting in a time long before the culture of Ancient Greek, and told with larger than life characters for entertainment and as an offering, not to the gods, but as an offering to society as a way of understanding their lives and the world in which they lived.

The audiences for Christopher Nolan’s movie and for Emily Wilson’s translation are very different than the audience listening to bards in ancient Greece. And that’s okay.

Process

That last post has been in mind since I drafted nearly two months ago. Letting thoughts lie fallow in the mind forms a part of thinking through a topic. Indeed, my framework situates imagination as sparks from the mind. Last month, I wrote:

Imagination. It’s that spark, but what sparks the recall of fragments that we have long forgotten? … It’s these sparks of recall and mentally constructing those fragments into possibilities that I’m calling imagination.

As a process for revisiting the last post, I had Sir Laurence Olivier read it to me. (You might hear the sound of screeching tires…wondering what I meant by that. Olivier is one of the ‘iconic voices’ in the ElevenReader app.) A significant change in how I process information is through hearing the spoken word through AI generated voices, even if it’s reading my own writing back to me. As I listen, I take notes on paper. I’ve resisted letting AI synthesize my last post and produce a neatly organized response. The temptation of having Claude, ChatGPT, or Gemini was really hard to resist at this stage. (I have things to do; I want to move on, get on with my life, publish this post, a tweet, or a YouTube video.) But, clarifying your own thoughts for yourself is foundational for understanding yourself. Later, I will have AI review what I’ve come up with through this process.

Creativity is the process of bringing those possibilities into the world.

The keywords, the building blocks for creativity and imagination, I have so far: fragments, experience, sparks, recall, mentally constructing, process, articulating (a more precise stand-in for ‘bringing’), possibilities.

The jotted down notes of my reread spread over two messy sheets of paper resulted in a chain of experience → fragments → sparks → recall → articulating → an intentional work/product.

Obviously, in attempting to define imagination and creativity, you can’t use those words in the definition. Try writing a book about art without using the word art. It’s quite difficult.

At this point, I thought it would be helpful to have a diagram to clarify that chain of thought coming from my notes.

This is the prompt I gave Claude Fable 5, high effort. (That model is overkill for this task.)

The diagram Claude created:

Baseline

That diagram clearly outlines the thoughts I have so far about creativity and imagination. I’m titling this ‘Creativity and Imagination’ but why am I putting creativity first when imagination leads into creativity? Partly, purely for alphabetical reasons. Maybe I’ll switch it. I get the impression that people tend to think about creativity more than imagination, and it was my own thinking about creativity that led me to consider imagination.

Structuring the 2nd draft

The process of writing a second draft of my essay is by finding a structure from the first draft. That structure is not necessarily in the order that the first draft was written since that first draft was free style association of thoughts. After I have a structure for the second draft (and it may take two or three attempts) to settle on a structure, I tend to proceed on writing the second draft. (Yes, this really could be a lot easier by just dumping my first draft into an AI. And I think there’s a lot of validity to that, depending upon what you’re trying to do with your writing.)

An obvious structure, which wasn’t obvious to me at all until a few moments before I typed this sentence, is that the diagram provides a structure. Let me try that: rereading the first draft, pasting paragraphs into an outline based on the diagram.

Even though I already know there are new concepts influencing my thoughts on this subject, I’m going to keep those out of this draft since I want to reflect separately on those concepts in relation to what I’ve already been thinking.

Here’s the 2nd draft, which I have labeled as v0.1 of the first public release.

Creativity & Imagination Framework, v0.1

This essay does not plant a flag that advances scholarship. Rather, this is a personal framework on where my thinking about creativity and imagination stands as of mid-2026.

We all have concepts floating around in our heads where we think we understand something, and we often vomit those thoughts into social media. Since examining the limits and capabilities of creativity in AI or with using AI to enhance our own creativity is my chosen work, I wanted to think this through more thoroughly.

Is my perspective valid? Or does it lead me astray, down a path I need not go?

As I read more about creativity and imagination, I’ll compare what I’m learning to my framework, and update the framework as needed, correcting misconceptions and adding sources that support my viewpoint. Over time, my framework might change completely. That’s how learning works.

Breaking down the framework.

Experience

The garden of our imagination is seeded by our experiences. For most of us, our imagination is like an overgrown lawn, an untended field of what some call weeds and others would see as a field of wildflowers.

No one has the same experience. The seeds are always different even if based on the same sources. The seeds of imagination are nourished, filtered by life experiences. The recall of each experience returns to us as fragments, harvested differently based on individual lived experiences. The time and manner of recall differs within a person. The same person may recall different fragments depending upon circumstances at that moment in time, and on what has stimulated the recall. Each of us has a different store of knowledge, even a person with the same education, a person who has taken exactly the same courses will bring his own life experiences that will impact his perspective, what he knows, and what he recalls. Even identical twins have different life experiences despite a childhood spent together.

Hence, the importance of taking notes, whether in a second brain tool like Obsidian, or on index cards, a commonplace journal, or recording an audio message to yourself. What you write down impacts how you will recall something later. Thoughts, by necessity, flee and, depending upon your memory, your mental capacity, you may never meet that thought again. (It’s usually the negative thoughts that we encounter again and again.)

We all bring our unique experiences, our life experiences, our prior perspectives and knowledge, bits of all we have read, watched, or heard. Though we remember only fragments, we recall those fragments and piece them together.

What is this thing we call imagination?

Before we have imagination, we have our experiences.

Fragments

Our experiences exist as fragments in our memory. Fragments are the building blocks of imagination.

The pathway of imagination

The pathway of imagination, per my framework, arises from an action or event (a spark) that sets off the recall, where the fragment surfaces and are mentally constructed into possibilities.

Action or event

Writing is a slow process. One sentence might trigger an idea, which is merely a thought that might have meaning. In writing a book, revisions stem from the author’s own rereading triggering changes. The role of first and second readers, and editors, are to prompt the author to consider sections of the book that do not work or needs adjustments.

Fragmentary recall is stimulated by any source. It may be a conversation, or classroom discussion, a hallway chat, or as wide ranging as a podcast.

Listening intently to a podcast is a way to stimulate your own thoughts. Podcasts have emerged as important sources that publicly present the process of a guide asking questions that stimulate the guest to, on-the-spot, articulate a thought in a concise manner. We all listen to different podcasts, and we all have different experiences that we bring in how we think about what we hear.

What does this framework say about education?

Teaching is not about instilling facts and whatever is defined as knowledge.

Education is the planting of seeds for our future use. Education is establishing a foundation in the student’s mind that later enables a person to make varied connections that will help him in life.

The inciting action is a spark.

Sparks

What sparks the recall of fragments that we have long forgotten (or misremembered)?

Creativity and imagination rests in the willingness to sit quietly, patiently, as you churn through mental fragments, hoping, attempting to come across the ones that click, the fragment you were, perhaps, unknowingly seeking.

Is that unknowingly seeking the core of imagination?

Recall

I’m better at writing than speaking because writing slows the process of surfacing what’s in my mind. Speaking spontaneously requires preparation, even if you know what you’re talking about. Recall for the purpose of public speaking is a process and a trainable skill.

An area that I want to read more about is the neuroscience of recall, how fragments exist within ourselves, and what determines which fragments stick around and which disappear forever.

Possibilities

The final step in the stage of imagination, before we get to creativity, is pulling those mental pictures from our mind, examining possibilities and making choices about which fragments we discard and which we ponder further.

Can AI imagine? People ask skeptically if AI can come up with anything new?

I say snidely, can most people come up with something new, an idea that has not been thought about before?

Based on this framework of imagination, AI can imagine. (Be careful when you use the word ‘imagine’, ‘imagination’. What are you really trying to say with that word? Or are you relying on some vague definition?)

What AI imagines will always be as different from the imagination of humans. Note that I don’t say that AI imagines any worse or better than humans. AI imagines differently, just as each individual person imagines differently based on experiences that form the fragments of memory and the actions that stimulate recall.

A remarkable use of AI is to prompt our imagination. Don’t take the output of AI as a whole cloth but use AI as a tool that sparks your own ideas.

The Creative Process

Are imagination and creativity the same thing?

For me, creativity arises out of the imagination. Creativity is doing something with those fragments we have recalled. We all have great ideas. But articulating those ideas and bringing those possibilities into the world is the creative process.

My viewpoint on creativity and imagination is on the arts.

We think about art the wrong way. We view art as an object, a lovely piece in a museum, or an abstract painting that we’ve been told is art. Or we debate whether AI generated images are art.

Art is about the experience of creating and not the product of creativity. Many will disagree with that.

Artistry is intentionally piecing together fragments of our experiences in ways that will unexpectedly stimulate the audience to form their own thoughts, to establish a memorable part of their life experience, or merely designed to bring delight or horror (which is a variation of delight) to an audience. It’s all entertainment.

As an author (and I’ll use author broadly to avoid the overused term creator or saying ‘author, painter, sculptor, photographer, musician, dancer, singer, filmmaker, etc.’ repeatedly) you have no control over how others receive your creativity once you have let it loose upon the world. You have to let it go.

For the author, it’s the creative process that brings delight. For those who perform live (whether a singer in concert or a stage actor in a play), each performance is a creative act.

I dislike the label artist and distrust the intellectual honesty of anyone who labels himself too seriously as an artist.

Labels lack importance compared to process. Art is too often used as a value judgment bestowed upon the output of creativity.

An artist is one who is driven to intentionally enter an almost trance-like state of mind of sorting through the spontaneous recall of fragments and the need to express those mental images in some medium.

The process is shifting through mental images, activating more images from your brain, and the act of composition. That is creativity.

Creativity is among the greatest uses of one’s time.

AI will only replace human creativity if we let it. My aim with this (evolving) framework is as a tool for understanding the impact of AI on creativity. More importantly, how can AI be engineered to augment human creativity: storytelling, engineered with AI, directed by humans.

It started as a brain dump. Then the words flowed from the thoughts forming in my head. No external sources, no computer. I just wrote for 3 hours. That was one of the most satisfying experiences I had this year, as fulfilling as the long walks I took in France, England, Vietnam, and Japan this Spring.

Someone will scoff and say it’s easier to speak, record the audio and have AI transcribe it. Or, to to think through writing by typing into the computer rather by handwriting. You do you.

Abandoning the outline

Writing, thinking, creating in any medium is difficult to get started. I knew this would turn into an essay. I started with an outline. If I’m writing an essay that I plan to be around 1,500 words, I want to make an outline of 15 points that I want to cover. That’s one point for every 100 words. That’s a very rough guide, mainly serving to get the spark of ideas going. In this case, I only got 3 points down on paper (and a 4th that was the concluding paragraph) before I moved on. That’s okay. You can abandon the outline.

Jotting down notes

Looking around my home office, I see that I have 6 wooden boxes containing index cards. I don’t want to count the number of notebooks (more than a dozen), various papers and notes strewn over my work spaces. I’m embarrassed to show a photo of the 3 desks (and 6 bookcases) in my home office (which also doubles as my living room); the mess would likely induce anxiety for some people.

Note taking is an act I do throughout the day. I’ve tried tools like Obsidian but I can’t wrap my head around it. I do use GitHub extensively for notes about technical matters. I just realized that my paper notes are only for conceptual ideas, i.e., things I want to think further about. Anything tech-related to my coding projects, go straight into GitHub.

For this specific writing exercise, after I abandoned the outline, I started writing notes on different pages of the legal pad. Of the 3,500 words I wrote that morning, around 1,200 were notes that I needed to capture, that I wanted to include, but needed noting before disappearing from my mind. My intention was that I would pull these notes together to finish the outline. That never happened, instead the ideas started to flow into the first draft. The outline was only a key to unlock the writing.

I mostly also abandoned the notes, and wrote without referring back to what I had noted earlier except when I wanted to find a phrase that had previously come to me.

Inspiration?

Where do the thoughts come from? That’s the core of creativity of imagination. That’s what I thought through in this writing exercise.

Use no sources

As a former university professor, as a former librarian, I know not using sources is sloppy research. But the goal of this writing process was not to produce published research. I wanted to set a baseline of my own thoughts on the topic. Obviously, though, these thoughts have been informed by everything I’ve read over the years. And, certainly, I did not use AI at all in writing that draft or in this post.

Challenge yourself to set out your thoughts in writing without referring to external sources. (You don’t have to do it by hand; do what works for you.)

Plagiarism? For this purpose, setting a baseline of your viewpoint, don’t worry about plagiarism, yet.

Finding your own framework

The aim is not that these thoughts are fully formed but that I, personally, now have a better understanding of my mental framework for thinking about creativity and imagination. The emerging framework will inform how I examine the limitations and capabilities of AI in creativity and imagination. The next step for me is to revise this draft into a coherent framework that I can use when reading what others are writing about on creativity and imagination. Most importantly, having this baseline of my own framework allows me to update my thinking as I consult the research and listen to others.

As I set out to write this draft, I was not seeking to establish a framework. You’ll see in the draft how it emerged as I was thinking.

We rarely get to see the first draft of another. I doubt many of you can read my scribbling on those yellow pages. Sometimes I can’t read my own writing. I did type up my notes and draft. Below, I’m including the draft, slightly edited for most spelling errors, though I kept the lack of capitalization from when I transcribed the essay. I didn’t correct this draft for grammatical errors. This rough draft wasn’t intended for others. I’m including it here as an example of the process. This draft was written in one sitting as a stream of consciousness.

The first draft

My self-imposed constraint is not only no use of AI, but no uses of sources or my vast notes that i have taken on this subject.

because that would be a different essay, likely a better essay, a more informed essay, but it’s important to get my own thoughts down on paper, my own thoughts where i am thinking deeply about a single topic in order to form my perspective and understanding, to articulate my perspective.

This, the output, the outcomes of this process becomes my baseline for further exploring this topic. I build on this base.

my thoughts may change but i always must be aware of how my thinking is shaped by another, is it correct, does it have validity?

do i feel it leads me astray, down a path i need not go, or do i feel it’s wrong, and then i must be able to articulate a different interpretation, a different perspective that is equally as valid as my own, writing an essay, an article a book, giving a presentation, making a video, or being on a podcast captures your perspective at that time. it solidifies thought into what is hopefully a clear expression of ideas, though on a podcast, it’s often thinking out loud, as often is the case in meetings at work.

but, wait, isn’t this essay about creativity and imagination? let’s get back to imagination.

sidethought: some say they they speak faster than they write. Not me. yet, i type faster than i write by hand. turning the page at this moment could take me out of the flow. i’m a word counter.

Imagination.

It’s that spark, what sparks the recall of fragments that we have long forgotten? fragment that we might have realized we ever know. Events we have misremembered.

this explains why some people are better at writing than speaking in an interview or a panel, in any situation without preparation.

writing is a slow process that enables the revving of the engine, the spark to take place, for the neurons to light up. the mechanism of how fragments are stored, and which are triggered for recall is the domain of neuroscience, a mechanism that likely will be identified this century as significant advancements in science that also will greatly inform the engineering of ai.

It’s these sparks of recall and mentally constructing those fragments into possibilities that is what we can imagination (or, it’s what i’m calling imagination based on the last 2 hours of thinking through this topic.)

side thought: it’s easier to respond, to critique, an essay/article, than it is to create it.

Some might say, ‘you're completely wrong.” or “”your interpretation” ok, that’s why writing as a baseline is important, i have a base for updating my perspective.

Creativity is the process of articulating those fragments and bringing them into the world.

it’s artistry if you’re are intentionally pieceing together these fragments in ways that will unexpectedly stimulate the audience to form their own thoughts, to become a memorable part of their life experience, or merely designed/composed to bring delight or horror (which is a variation of delight) to the audience. That’s entertainment. See Graham Greene’s description of some of his novels as entertainment.

and as an author, painter, filmmaker, etc. you have no control over their experience of your creativity once you have let it loose upon the world. you have to let it go.

as an audience also, you must be careful of other’s interpretations before you form your own, or even later, and not let another’s interpretation ruin your enjoyment of a piece, a song, a film.

that’s why i propose that creativity and imagination about about the experience of the creator, the creative process brings delight to the author (and i’ll use author broadly to avoid overusing the term creator or saying, author, painter, sculptor, photographer, etc repeatedly)

part of that creative process is style, adding a stylistic arrangement to the composition, this is where originality lies, but depending upon the medium and the intent, originality, the arrangement is only important to the author if she chooses it to be so. it’s the process of shifting through mental images, activating more images from your brain, and the act of composition that is creativity and of the utmost importance to the author. creativity is the greatest use of own’s time, no, that’s not a good way of expressing what i’m thinking, struggling to bring a jumble of thoughts in my head before they escape back into that mysterious darkness.

creativity and imagination rests in the willingness to sit quietly, patiently, as you churn through mental fragments, hoping, attempting to come across the ones that clicks, the fragment you were, perhaps unknowingly seeking.

Imagination. Is that unknowingly seeking the core of the imagination?

That’s what you call in the learning process an essential question.

i will not attempt to answer it here but by noting it, i have a line of inquiry for consulting sources of knowledge that will inform my perspective on that question. when i’m able to articulate my informed perspective in some tangible way, an essay, a digital project, an artwork, then i will have demonstrated an understanding though by necessity only a slide of understanding that essential question and the big roles behind it.

for learning is a continual process of discovery and refinement.

the garden of our imagination is seeded by our experiences. for most of us, our imagination is more like an overgrown lawn, an untended field of what some call weeds and others would see as a field of wildflowers.

no one has the same experience. the seeds are always different even if based on the same sources, the seeds are nourished, filtered by life experiences. the recall of each in divided fragments is harvested differently base on the lived experience at the very manner of recall. the same person may recall different fragments depending upon circumstances at that moment in time, and what has stimulated the recall. hence, the importance of taking notes, index cards, a commonplace journal, or recording an audio message to yourself. what you write down, the recall you are capturing are thoughts and by necessity they will flee and depending upon your memory, your mental capacity, you may never meet that thought again (it’s usually the negative thoughts that we meet time and time again.)

you see, here is how it works:

what i just wrote triggered an idea, which is merely a thought and that might have meaning. the idea, the artistry, the desire to create, is driven by the spontaneous recall of a recurring thought or set of thoughts, the continuous recalling of a series of thoughts that move us towards the act of creation, the need to express those mental images in some medium, whichever is the form of your choice.

or, the recurring thought, the call to action, is simply an ongoing feeling, a need, to make the time to sit down and enter that almost trance-like state of mind where you are being creative.

why podcast interviews are important, they have the opportunity, they present publicly the process of a guide asking questions that stimulate the guests to an on the spot articulate a thought in a concise manner. listening to the podcast itself, in term is a way to stimulate your own thoughts. we all listen to different podcasts, and/or we all have had different experiences that we bring.

what does this framework say about education? stimulation, planting seeds, enabling or establishing a foundation in the student’s mind?

enabling and establishing have very different meanings. this is another essential question behind the big idea of creativity and imagination.

is this a framework for thinking about creativity and imagination? i have set out in this essay to establish a baseline for my own thoughts about creativity and imagination. as i reflect now on what i have written, i can see a framework emerging. whether that framework is useful for others needs testing, examining. but i’m not aiming for scholarship. i[’m past that point in my life. but perhaps (which is an example of a tentative phrase that i shouldn’t use, that i will edit out) the framework formed from my thought process, can stimulate ideas for others. should i care if others find it useful or not? it depends upon my intentions in publicly articulating this framework. honestly, as i write this by hand, i don’t care. my concern at this point is refining my own thoughts on creativity and imagination. this framework has become my baseline, version 0.1., as of June 2026.

i’m stopping now, or rather i will stop in 12 minutes, when my Pomodoro timer goes off.

the next stage of writing as thinking is to refelct on what i have done so far. reread and review my notes, and bring structure to my output. what started 3 hours ago as a bare outline and the jotting down of word and phrases morphed into a free flow state of writing, a first draft. i should publish as much of this draft as possible, just as i wrote it so that other s can see this stage of the writing process, a stage that is usually discarded or hidden in the writer’s archives, possibly someday, if the author is notable, to be stored in the controlled climate of a library’s special collections vault where it will wait unseen for years, possibly decades, possibly forever unless a researcher tracing my intellectual evolution uncovers the draft, the odds of that every happening is is low. i will be forgotten. thus, here is the first draft of my thoughts on creativity and imagination based on 60 years that have formed the experiences of my life.

even identical twins have different life experiences despite being constantly together. but, still what is this thing we call imagination.

now, what concepts do we have that are building blocks of imagination?

And, yes, there will be somebody on reddit who will point out, “He’s not saying anything original . this book [insert title] says it better.” well, duh, smart ass.

but the purpose of me writing this is essay is not t o plant a flag that advances scholarship but for me to enter a process of deep thought, an almost meditative stage where i am engaging with the thoughts that pop into my head as my own writing word by word, sentence by sentence somehow triggers a recall that adds value, an additive connection, that continues to connect concepts into an articulation that is a public statement of my perspective. this essay demonstrates understanding (however flaw and incomplete) by transferring possibly random concepts that spontaneously occur as thoughts, arranging these mental images into sentences/phrases; editing, bringing structure, the second draft, arranging paragraphs into a composition is a different process, a step removed from the imagination, a series of steps that form a process, a procedure that is crafted from prior experience itself through what you have learned about writing.

what’s the latin root of imagination. what other words in other cultures convey this concept.

how words impact our thoughts. all mental. this is the mystery.

pulling mental pictures from our mind, making choices, selections, and piecing them together.

of course, ai can imagine? people ask sceptically but can it come up with anything new?

i say snidely, can most people come up with something new, an idea that has not been thought about before?

but people can come with a new way of expressing an idea.

we all bring our unique experiences, our life experiences, our prior perspectives, and knowledge, bits of all we have read or watched, though we remember only fragments, we recall those fragments and piece them together.

fragmentary recall may be stimulated by a conversation (the role of first and second readers and editors) the importance also of being encouraged in your writing so that you don’t wither under criticism.

stimulated by other sources. podcasts are great mental stimulators. podcasts as stimulation machines.

each of us has a different store of knowledge, even a person with the same education, who took the exact same courses will bring is own life experiences that will impact his perspective, what he knows, and what he recalls.

We sometimes use that as a shorthand for saying how smart someone is or how intelligent a person is, but smart and intelligent are not really meaningful are they? Other than as measures of life experiences?

Note: i’m a word counter. how many words have i written, including notes on these pages of a yellow legal page over the past 3 hours.

i have found this process delightful, fulfilling and something, a task i would never have the time to do if i had a job.

the next stage, the refection will be a different process, and the expectation for myself is that i can articularte for my self in writing what i am perceiving as a framework for further examining my thoughts on creativity and imagination. important to note that i didn’t start out intending to think in terms of establishing a framework.

a framework with the ultimate aim of utilizing this framework as a tool for understanding the impact of ai on creativity and the ways ai can be engineered to produce, to stimulate human creativity, or better yet, for ai as a tool to augment human creativity.

the timer goes off. it’s 10:11am on thursday, june 4, 2026. it has been 3 hours since i started

Perhaps no one popularized the metaverse more than Matthew Ball in his 2022 book The Metaverse: And How It Will Revolutionize Everything. (Well, there was Zuckerberg changing the entire name of his company to Meta.) While the metaverse revolution fizzled into a slow simmer as AI has gotten all the attention, advances are still progressing. I intend to talk more about the research and possibilities in gaming, 3D graphics, optical engineering, mixed reality, and the open standards that make those experiences possible. But the metaverse as a term holds some people back. A February 2026 post on Stratechery where Ben Thompson interviews Matthew Ball, ends with this observation:

Thompson: Is the metaverse dead?

Ball: … Do I think the term is pretty darn buried? Yeah. So it goes, that’s language.

The metaverse as a term has gone quietly away. Yet, at this year’s WWDC, Apple highlighted several capabilities of its VisionOS. While some VR studios have closed shop, others are persevering through this VR winter, just as there was an AI winter.

Spatial Computing is the term that Apple seems to have settled on for its computing platform that extends beyond mobile and desktop. Some people say that Apple has abandoned VisionOS but announcements from Apple’s annual developer conference WWDC say otherwise.

I’ve been hesitant to get the Apple Vision Pro. Why? The price: it starts at $3,500. The Vision Pro seems to be a remarkable device but it’s already 3 years old at this point. I’m waiting for the next model.

An immersive computing platform enabled through visual devices, whether full headsets or eyewear, will emerge as the platform of choice for many uses. It’s still a matter of time before the market is ready. But this post is about the name change of this newsletter and not Apple’s recent announcements, which I should cover separately.

Endless Hybrids

While thinking through a variety of options, I decided to simply name this newsletter after my studio: Endless Hybrids.

I’ve been using Endless Hybrids since 2005. The term originally was found in a sentence in The Language of New Media by Lev Manovich. When I read this sentence, I was inspired.

Manovich contrasts the database paradigm with that of narrative and describes databases and narratives as

“two competing imaginations, two basic creative impulses, two essential responses to the world. Competing to make meaning out of the world, databases and narrative produce endless hybrids.”

I was in my office at the University of Miami. I had to put down the book and go for a walk around the small lake that is the center of campus. That sentence conveyed everything I had been working towards in my work in digital libraries. But I was always more interested in how stories are told (and consumed) through digital media than in the archival management, digitization, and storage of content.

My 20-year journey with Endless Hybrids

I wrote the following on the first version of my Endless Hybrids blog on February 2, 2005, when I was still working as a librarian at the University of Miami.

In the landmark work The Language of New Media Lev Manovich presents the compelling argument that the “[D]atabase becomes the center of the creative process in the computer age”.

The focus of digital library development is primarily the digitization of material into discrete digital objects that can be represented and retrieved through databases utilizing standardized metadata. The library community has made enormous strides in digital library developments. The results are a wide range of databases that support academic research in a variety of disciplines. For libraries, the database has certainly become the primary mode through which digital information is presented to users.

While databases of digital objects are essential to scholarship, the database paradigm should not be viewed as the ultimate end result of library experimentation with digital technologies. Databases alone are merely containers that allow people to access information; in a sense, a database functions in the same way as a library building in that the library building is a container that allows users to access information in print. Of course, all librarians know that libraries are much more than buildings containing books. In the same way, digital libraries need to be viewed as much more than databases containing digital objects. Physical libraries are often defined as much by their services as by their collections. An area of digital library development that is under examined is the ways in which researchers can re-purpose digital objects into new works of scholarship.

Scholarship, whether in the form of a printed article, monograph or digital media, can be described as the gathering, analysis, and re-purposing of information into a new context of understanding through scholarly insight by a researcher. Historically, archives and libraries generally are not involved in the interpretation or publication of printed research by scholars. However, the techniques for presenting the findings of research through digital media are just now evolving. Academic units such as libraries need to work closely with faculty to understand and support how the story of a scholarly research can be expressed through digital media.

In examining print scholarship as a literary genre, one finds that it is essentially driven by a strong narrative supported by references (i.e., footnotes) to primary and secondary sources. This type of narrative-based scholarship is most obvious in the humanities, especially history, but is also very relevant to the social sciences and even the sciences. Narrative is one of the oldest ways of contextualizing information and making it understandable.

The current products of digital libraries most closely resemble reference materials and archival finding aids; indeed, monographic indexes and encyclopedias are essentially databases in printed format. Yet, narrative will surely play a strong role in the future of digital scholarship. The digital culture is in the early stages of utilizing narrative in new media, analogous to the early days of filmmaking when the techniques for effectively telling a story through the new media of that age was just being developed. Indeed, it may be the documentary film rather than the monograph that serves as a better model for the future of digital scholarship.

Narrative in digital scholarship does not supplant the role of databases in digital libraries. Rather, databases should serve as the foundation on which to build narrative-based digital scholarship. Manovich contrasts the database paradigm with that of narrative and describes databases and narratives as “two competing imaginations, two basic creative impulses, two essential responses to the world. Competing to make meaning out of the world, databases and narrative produce endless hybrids.”

Librarians and scholars need to understand more about the capabilities of new media to produce online narratives that are enriched with scholarly digital content that is aurally, textually, and visually stimulating. The growth of digital scholarship is inextricably tied to the means through which digital library databases can be manipulated in order to support the creation of rich and engaging narratives that foster learning.

A month after I wrote that article in 2005, I left my job in Miami and moved to Buenos Aires to start my own business, a design and development firm, and to live life differently. I stepped away from librarianship between 2005 and 2013 but returned as libraries were showing more initiative in digital humanities. During that 2005-2013 period, I lived in Buenos Aires, where I blogged about my walks around the city, and drafted a novel about art and forgery. The latter was an exploration of creativity.

While I did work a while in the digital humanities, including as the founding head of a minor in Digital Culture and Information, my academic career shifted more towards teaching, including courses in writing, multimedia storytelling design (co-taught with a journalism professor) and in data science and artificial intelligence.

It was always what you do with digital information that fascinated me, and not how it was organized and stored. What you do with digital information requires understanding the possibilities. While there’s a lot of pushback against AI, anyone reading this substack knows that AI is a transformative technology. I still believe that 3D real-time graphics are also transformative. Advances in 3D graphics, optics, and AI will create new forms of stories that we do not yet know fully.

Ultimately, I’m interested in creativity and imagination as much as I am with technology. After all, I was an English major at a small liberal arts college in the 1980s. The transformative power of story has never left me. My next post will be on creativity, imagination, and the limits of AI.

The dream, which some call a nightmare, of the metaverse has a foundational restriction: hardware, specifically optics.

These optics are not a metaphor, e.g., the optics of whether data centers are good or bad for society. These optics must exist in a physical device worn on a person’s face at an affordable price. Many scoff that people will never wear augmented reality glasses, much less wear VR goggles. Those who scoff seem to forget that more than 70% of adults in the US wear sunglasses, and billions across the globe wear eyeglasses for corrective vision. I’ve worn glasses since the fifth grade. If eyewear is useful, you’ll wear it.

When the value and affordability reaches a mass market, we will be in a different era of computing. Zuckerberg is right that another platform will replace the smartphone. (Maybe it won’t be eyewear, but our devices will get smaller and more portable. Or maybe as neural implants. I’m not even ready for that last one.)

Unlike software products, the materials and engineering of these reality adjusting eyewear, whether glasses or VR goggles determine when the products come to market. The tech is maturing, but the price point is not yet there.

Meanwhile, as Zuckerberg was starting his move towards the metaverse, OpenAI had the audacity to pour massive amounts of money into a purpose that came to consumer’s as software via their existing web browsers. I know that Sam Altman claims that they didn’t have a product in mind when starting OpenAI. But he’s an entrepreneur. Surely, he knew where OpenAI was going.

Maybe Zuckerberg should have made the bet on AI instead of optical hardware. But, as we’re seeing, AI is becoming a commodity. And we’ve also seen that a premium headset finds an audience: Apple Vision Pro.

The audience for Apple Vision Pro is small. Many who have the Apple device report excellent experiences. Whereas, many who have the Quest have it piled in the back of a closet. The Quest was intended as a gateway to VR and never as the epitome of the technology.

A market report published on March 13, 2026 by treeview, a spatial computing company, estimates that more 475,000 units of the Apple Vision Pro has been sold.

My advice to any following the tech market always has been to follow the reports coming out of the research labs. Every day I get a feed from scholar-inbox with a dozen papers detailing advances in the ecosystem that will encompass our digital lives ahead of us. All the work in 3D and world models are directly applicable to the products that we will experience through smart glasses and headsets, even if these articles do not mention virtual reality.

A few examples from just today, April 3, 2026:

• Lifting Unlabeled Internet-level Data for 3D Scene Understanding (arXiv link)

• Reflection Generation for Composite Image Using Diffusion Model (arXiv link)

• Omni123: Exploring 3D Native Foundation Models with Limited 3D Data by Unifying Text to 2D and 3D Generation (arXiv link)

• LivingWorld: Interactive 4D World Generation with Environmental Dynamics (arXiv link)

• Anime-Ready: Controllable 3D Anime Character Generation with Body-Aligned Component-Wise Garment Modeling (arXiv link)

• ActionParty: Multi-Subject Action Binding in Generative Video Games (arXiv link)

• Director: Instance-aware Gaussian Splatting for Dynamic Scene Modeling and Understanding (arXiv link)

• Large-scale Codec Avatars: The Unreasonable Effectiveness of Large-scale Avatar Pretraining (arXiv link) Note: this one comes from the Code Avatar Lab at Meta.

• Generative World Renderer (arXiv link)

And that’s just from one day.

I don’t want to be misleading. Many of those papers were submitted to conferences and are only now being published on arXiv. But it’s still an astonishing amount of research being produced.

For those of you keeping count and concerned about the US vs China dominance in AI: six of those nine research papers were produced by Chinese institutions. The one by Meta was the only US representative in the lot. There also was one from Korea, another from a joint Canada/UK effort. One of the China-produced papers was in collaboration with a Japanese studio. This sample is not scientific, just a random set from one day. The whole US/China discussion is extensive and, I think, misleading.

What’s most difficult to track: when and which parts of research will come to market as products.

Let’s set aside the word metaverse, which came from a barely readable novel. With this newsletter, I admittedly hopped onto that train that went seemingly went nowhere. To extend that railroad analogy: that train carrying the metaverse relied on tracks not yet laid. It’s not the end of the line for extended reality, and that concept—extended reality—is a mood more encompassing than the metaverse.

I’ll be renaming this newsletter. It won’t be called extended reality is open. My next post will reveal the name. I’ll be using a term that has stuck with me now for nearly 25 years.

I will continue to write about what I’ve always written about: how we tell and experience stories, whether fictional or factual, in digital media. And that term digital media is so vague that I rarely use it anymore. (It has a late 1990s feel.) Digital media is software. But we only have experience software through our devices.

Software is the easier part, the software based on open standards and open source. The future is open for us to make an endless hybrid of experiences.

A prime problem with the metaverse is that the term was too often associated with blockchain, crypto, and grifters trying to cash in on emerging technology. Others were skeptical of Zuckerberg’s vision of the future and the rebranding of Facebook to Meta.

Here’s Matthew Ball’s extended definition of the metaverse (Ball, The Metaverse: And How It Will Revolutionize Everything, p29, 2022):

“A massively scaled and interoperable network of real-time rendered 3D virtual worlds that can be experienced synchronously and persistently by an effectively unlimited number of users with an individual sense of presence, and with continuity of data, such as identity, history, entitlements, objects, communications, and payments.”

Yeah, that’s a mouthful but it’s easier to say one word: metaverse. Yet, I do feel a certain ick when using that word. I’m wondering if I should rename this Substack.

My own working definition of the metaverse is a 3D world that allows you to step into the story.

That doesn’t capture all the uses wrapped into Ball’s lengthy definition. But I’m not focused on every aspect of real-time 3D.

My interests are in the experiences made possible through open standards of mixed and virtual reality. The emphasis on experiences is because I’m more fascinated by our imaginary lives than the enabling tools.

These tools are projecting images in front of our field of view. (Sound may also be a part of the experience.) We can reach out and touch the aspects of our view in ways not enabled through keyboard and mouse. What impact does this imaginary presence have on how we think?

Where are all the technical developments heading, where are they converging? My X feed is filled with news mostly about world models, Gaussian splats, AI films, and robotics. The future of our imagination is embedded in all that’s happening.

The challenge with any analysis is not explaining the technical developments but the unknowns in when a company can bring a technology-based product to market.

Tools only exist to create experiences that will, in some way, be monetized by providing value to people.

Tools are not uniquely tied to the concept of the metaverse. But many of the tools and approaches (like world models, AI video generation, procedural content generation, and scene graphs) are essential for stepping into a dynamic imaginary storyworld.

The 21st Century is becoming more chaotic than we imagined. What we do know is that changes in technology, driven by AI, are not stopping. Will the metaverse be a part of that? Advances in computer vision, world models, and optical engineering indicate that those fundamental technologies will continue to produce tools that to come to market as products and experiences. Maybe those experiences will be niche and not the next computing platform that replaces smartphones. But niche experiences have value and a market.

Unless you work within a sector of an industry, you might not think about segments of an industry. Let’s use the hospitality industry as an example. In the 21st Century, are motels (as distinct from hotels, mostly for their drive-up-to-your-room feature and lower costs) still relevant?

The Super 8 hotel in the Virginia town where I lived is being razed to make way for a Sheetz. (And, wow, convenience stores have changed in the last couple of decades.) The older Motel 6 down the road is likely not long for this world, either.

But hotels remain very relevant and used by millions of people every night. The metaverse may go the way of motels, but the enabling functionality will thrive in products used by millions.

I could just have ChatGPT or Claude spit out an essay but I wouldn’t learn anything that way. I spend hours every day with those tools but sometimes you need to slow down.

A slow-paced literature review allows for ideas to turn over and settle in your mind. These thoughts form mental relationships, revealing connections that might find expression as insights in an essay.

With world models in 2026, I’m interested in two questions:

• will world models replace game engines?

• how can world models simulate social dynamics?

An excellent starting point is an article updated in December 2025: Ding, Jingtao, et al. “Understanding world or predicting future? a comprehensive survey of world models.” ACM Computing Surveys 58.3 (2025): 1-38. That survey also has a GitHub repo: https://github.com/tsinghua-fib-lab/World-Model with graphics and pointers to other repos of code and publications.

Not all of the 300+ citations tracked in that survey are of interest to me. While a major use use of world models is in robotics, I’m deliberately excluding robotics and embodied AI from the scope of my interests.

My reading process lately is to eyeball each page of an article, which is a preliminary habit I’ve always employed before reading anything that’s non-fiction. Then, I have Michael Caine read the article to me.

Did that cause you to pause?

Take a look at ElevenReader and watch this:

It’s a bit disconcerting to hear the 92-year old actor talk so articulately about AI. (I’m assuming ElevenLabs paid a load of cash to license that voice.) Anyway, hearing a quality voice reading a complicated text to me is another level of absorbing the information.

In the Ding, et al, survey, I identified a few dozen out of the hundreds of sources covered that might be of interest to me. That survey also has been cited more than 100 times in other publications. The survey was first released in December 2024, with updates in mid-2025 and the end of 2025.

From the above diagram, you see how they define the two aspects of world models as

• implicit representation of the external world

• future predictions of the physical world.

From the applications section of the diagram (the bottom part), my interests are in the virtual and societal applications.

At this point, I still have dozens of papers on world models that I want to review. I won’t read most of them closely. I’m looking for things that really stand out relevant to my interests.

I’ve been talking a lot with my high school kid about the process of research and am trying to model that process for her; on one of the 3 desks in my home office, I have colored-coded index cards scribbled with notes. (All 3 desks are for my use, though I let her use one when she’s studying with me.)

Once I have a grasp on the topic, I will then turn to AI tools for another perspective. But I don’t want to do that too early in this process. Before working with Claude and NotebookLM on this topic, I’ll review what has come across my X feed lately.

X is the best way to stay current with very recent developments in AI. It seems that most AI researchers post regularly on X (with the exception of Yann LeCun, who seems to have had enough of Elon, and primarily posts on LinkedIn). If you’re getting started on X with the intention of tracking X developments, the best approach is to follow some of the lists curated by Robert Scoble. Be aware of the many fraudsters on X posting about their one-shot marvels or their undisclosed shilling for specific companies. (I almost wrote “unscrupulous companies” but that would seemingly entail most of capitalism at this point.)

Context: On game engines and world models

Here are some key insights recently from X.

Tim Sweeney, founder and CEO of Epic Games, maker of Unreal Engine:

As providers of world models are highlighting their ability to generate games, the evolving nature of world models have a direct impact on Epic’s future. Epic’s Unreal Engine is the single most complex (and fascinating) software I’ve ever used. AI assisted tools will make developing in UE easier. The integration of world models with game engines will unlock significant advances in not only gameplay but also any form of simulation. Game engines are used for much more than creating games.

That clip from his post cuts off the 2nd and 3rd paragraphs:

What would the ideal kind of memory be? It would be essentially a scene graph (James Clark, 1976!) hierarchically describing all of the objects in a world together with their properties, in a way that’s easy to query and operate on.

This is the reason for the core thesis that the ideal evolutionary path sees a merging of world models and engines, the AI side providing vast amounts of loosely organized audiovisual and textual knowledge, and the engine providing consistent reproducible data representation and simulation.

I have another upcoming post in the works about scene graphs.

Jacob Sansbury, developer/entrepreneur:

Click through on that post. He goes on to write about a point that I think is on target:

Medium-term is where it gets interesting. Add sound generation, longer context, more control and you have something Netflix should be terrified of. Imagine exploring Westeros between seasons. Wandering the Stranger Things universe. That’s a real product, and it’s coming.

But that’s interactive storytelling.

Gamers play because it’s fun to get better at something. Progression systems. Mechanical mastery. Nostalgia, where things work exactly how they always worked. They sink months into a single title. Years. And here’s the thing: they mostly don’t care about graphics or narrative.

Every single one of these motivations sits at the exact weak spot of world models.

Games require determinism. Multiplayer needs every client to agree on physics, every frame. Speedrunners need frame-perfect consistency across thousands of attempts. Competitive play needs rules that don’t drift. You can’t have ranked when reality is probabilistic.

World models are competing with passive media.

Long-term, they’ll probably eat the renderer. Generating pixels instead of rasterizing triangles. But game logic, systems, authored constraints? That’s a different problem entirely.

He’s wrong about the renderer part but it’s all comes down to how the pixels are generated.

The Marketplace: World Models are Coming Everywhere

Everybody seems to be releasing world models in 2026. I’m not going to list every announcement. World Labs, because of the quality of their team, is a company I’m keeping an eye on. Here’s a good essay from Fei-Fei Li on spatial intelligence.

Martin Casado is posting a lot about world models, 3D scene generation and games:

But he’s also a general partner at Andreessen Horowitz, an investor in World Labs. In the last decade, I noticed that VCs have become adept marketers of the companies in their portfolio.

The first time I ever learned about venture capital was in high school in the early 1980s. I lived near Nashville and the newspaper ran a profile on a local (Jack Massey), who was actually quite a major figure in venture capital at that time. But I always got the feeling then that these guys (Massey and Lucious Burch III) kept a low profile compared to today’s VC firms that are all over the media with their dedicated YouTube channels and podcasts. But this is a different era.

David Baszucki is the founder and CEO of Roblox, which is a company I first learned about a while back when my daughter was about 8 years old. She told me about Roblox.

Roblox is a company to watch. Hook the kids on your product; those kids eventually become adults with a disposable income of their own. The music industry has known that for years.

Runway is the company in this space that has the best aesthetic sensibility. Co-founder and CEO Cristóbal Valenzuela maintains an excellent X account. Runway’s focus, for now, is on AI films. But their use of video generation is an essential element of how I’m thinking about world models.

Take a look at Valenzuela’s essay from last year on It's All About The Pixel Economy (Or Why You Might Be Optimizing for the Wrong Future)

And the thing about everybody’s doing world models. Here’s a post from VP for AI at AMD:

Research: The Technical View behind World Models

But what are these world models really doing?

Jim Fan, NVIDIA Director of Robotics and Distinguished Scientist published an “article” on X where he describes world modeling as the next paradigm shift:

Very few understand how far-reaching this shift is, because unfortunately, the most hyped use case of world models right now is AI video slop (and coming up, game slop). I bet with full confidence that 2026 will mark the first year that Large World Models lay real foundations for robotics, and for multimodal AI more broadly.

That article is an excellent short introduction to world models in 2026 and explains how all this video generation relates to robotics.

He notes the key question:

That comprehensive survey by Ding, et al, points to articles on the technical nature of world models.

That’s why you also need to follow Yann LeCun on LinkedIn, who explains “that a proper world model is not a video generation system.”

That link goes to an 8-hour video. LeCun’s talk starts at 5:42.47

That workshop starts with a talk and Q&A session with Jürgen Schmidhuber, who co-wrote a foundational paper on world models. I wrote about Schmidhuber in my earlier post on world models. You can’t talk about world models without talking about Schmidhuber. (And Schmidhuber will remind you of that if you do so.)

On Feb 7, 2026 on X, Yann LeCun posted: “Anyone with an interest in world models should watch this”

“This” is this:

Malik’s presentation starts at the 1 hour, 4 minute mark.

Next steps

Some of that material I’ve already looked at closely. Other parts I’m going to look at more closely. Some I will put aside. Some of you may wonder why don’t I just drop that all into something like NotebookLM? That will be interesting, and I will do that at some point. A big question for this process is whether you start with an analysis done by an LLM or if you engage with an LLM while you read (or watch) or afterwards?

There’s not a best answer to that. It depends upon your goals, i.e., why are you handling this material in the first place? How deep into certain areas of knowledge do you want to go and for what purposes?

I probably take the middle ground and utilize Claude as well as NotebookLM as a reading partner to help me track what I’m learning. Ultimately, the only way I’m going to know if I have absorbed this topic is if I can comfortably express the concepts in my own words, connect those concepts to my own work, and articulate the insights I have gained from those connections.

Go back to the photo of my messy office. Those colored index cards are where I write (by hand) my thoughts that will (by reassembling the index cards) form an essay.

That’s a very different process than the one I used to write this long, and a bit rambling, post on the sources I’m gathering. This post is more than 2,500 words of free association that I typed directly into Substack at my stand-up desk over four hours on a Sunday. Is this a waste of time? Not for me. This is my life.

AI can be very valuable in writing, especially for editing your prose. But it’s too easy to lose your own writing voice and adopt the manner in which AI expresses a topic. LLMs often come up with a resounding phrase, which is tempting to insert here and there. I do use LLMs extensively everyday for brainstorming, research, and coding. Claude has become my daily driver, followed by Grok, and then Gemini. I’m cancelling my Pro subscription to ChatGPT but staying on the lower paid plan. Let’s get on with this month’s update.

I’m trying to stay focused on my one story. See my last post.

But what is Night Listener, you might ask. You thought I said I was working on Never Fear Tokyo??

Night Listener is a self-contained companion game to Never Fear Tokyo. Limited in scope, Night Listener is a project that I should be able to get done without getting sucked into the messy complexity of a larger environment.

This game takes place in one room over one night with only one character.

The character is Alex, the same photojournalist protagonist of Never Fear Tokyo, who has arrived to cover the 1964 Olympics. During this rainy night in his rented room, he overhears voices that he initially assumes are his neighbors. As the night progresses, the phrasing and tone of what he hears intensify to the point where he doubts where the voices are coming from, whether from neighbors or the supernatural. It’s a rough night for Alex as he visualizes shapes on the wall. Outside his window is a neon sign, typical of Tokyo; the neon casts color into the room. Throughout the night, the colors change and the sign changes to reflect words he’s hearing. Or, at least that’s what he imagines.

As the player, you have to figure out, along with Alex, what’s happening in that room that night, and what type of shape he (and you) will be in when the sun rises in the morning.

I have a set of game mechanics that I won’t go into at this point. In subsequent devlogs, I’ll label those as spoilers in case you’re interested in experiencing Night Listener.

The mechanics are documented in a game design document that’s roughly over 5k words. I did use ChatGPT and Claude to help brainstorm the game. But most of the key elements were ideas that I came up with on my own. For example, a couple of the key concepts I added that the LLMs didn’t come up with for the game:

• Night Listener is an auditory experience. The voices, heard only in Japanese, are the key to unlocking what’s happening. Game mechanics help a non-Japanese speaker, which includes Alex, grasp the impact. But even if you know Japanese, the voices arrive in fragments that need assembling. The essence of Night Listener is an audio experience about not understanding and listening without context.

• The player, like Alex, is a listener constrained to the specific space of a room during one night.

Obviously, I don’t expect this to be a blockbuster game! The appeal will be limited but if only a hundred people find this intriguing then I will have accomplished what I set out to with this small game. The more comprehensive Never Fear Tokyo is designed for a broader audience interested in narrative-driven games.

A defining aspect of both games is that they’re presented as a stage play rather than a full 3D world. That also was my own idea and not one provided by AI. You’ll initially view Night Listener as if you’re sitting in the audience of a theater and watching a play. Take a look at the concept art for this post, and recognize how it resembles a stage. Night Listener could work equally as well as a play dramatized on stage or as a short story. I’m highly interested in transmedia. But the game provides capabilities not possible through other media. As you engage with Night Listener the game, you’ll realize that you can switch to Alex’s first-person POV that unlocks capabilities not seen from the audience’s third-person POV.

Voice as Pedagogy & Research Testbed

Which of my three “big ideas” does this relate to? Voice as pedagogy. But I’m being very careful that Night Listener is not an educational game for learning Japanese. That would be a bit boring. But it is helping me learn the language.

The expressions and utterances that Alex hears over the night form a cohesive story in itself, though not told in order. The story is one I wrote nearly 20 years ago while living in Buenos Aires. I’m translating the text, which is mostly dialogue, into Japanese.

For the voices in Japanese, I’ve been experimenting with ElevenLabs, which has cloned voices of native speakers from Japan. There are also other software more finetuned for Japanese. But I probably will end up contracting with voice actors. The expressiveness of AI-voices is still lacking. And that’s an area I’m tracking with my research.

Night Listener provides a benchmark for measuring acting expressiveness through AI voices. With significant effort, one probably could modify AI voices to the extent that they would be suitable for my purposes. But this is a case where it would be easier and quicker (thought not cheaper) to hire voice actors. Based on my experiences and knowledge of generative AI for audio, we should have fully expressive generative voices in the major languages by the end of 2027. That’s a very conservative estimate but note the emphasis on fully expressive. For many purposes, you can do that now. And I will be using Night Listener, even after I finish this game, as an evaluation tool for benchmarking the acting quality of generative audio.

Voice acting is not a good career option for anyone in the mid-21st century. (Is any career a good option going forward?) People who want to pursue voice acting will need to start authoring their own stories and bringing their own characters to life while also figuring out how to monetize their efforts. There will be audience for that! But voice actors should not expect that their future income will come from publishers and for-profit companies that currently contract voice actors. Yet, there will be opportunities for publishers and studios that state they only use human voice actors; but I expect that segment to be very small since businesses will seek to optimize their expenses.

Working with Unreal Engine

I already mentioned the concept art for Night Listener.

I made the first pass at a gray blockout in UE. Before I settled on the stage-like aspect for Night Listener, I thought about a realistic room for 1964 Japan with tatami mats. Rooms in Japan are sized based on tatami mats. A 6-mat tatami room would be about 10 sqm.

The sketching was a useful exercise, especially for thinking about how the character moves within a constrained space, but a game environment doesn’t need to be realistic. It needs to be plausible.

I’m developing in Unreal Engine 5.7. I’ve been working with UE for more than 5 years, and I still learn something new every time. I’ve been diving into Substrate, which is UE’s latest approach to materials.

If you’re on X, you’ve might have seen short clips of people creating games with world models, or people creating games for themselves with tools like Claude. Will we continue to need game engines like Unreal Engine? I think so, and I’m writing an essay to explore that topic. Eventually, there will be an integration of world models and game engines.

Language learning

Some might wonder if there’s a need to learn a language if AI can translate for us via earbuds or smart glasses. While that tech will be useful, many people will still embrace learning a language in order to fully integrate into a community with native speakers of wherever they’re at. I’ve used Google Translate quite effectively in Vietnam while conversing with Vietnamese who don’t speak English. But, after a while, you run into limitations, especially with the difficulty of ‘small talk’, the casual banter that form a friendship. Developing a deep relationship through a translation app is impossible. Besides, language learning has tremendous cognitive benefits.

There are a massive number of YouTube channels dedicated to language learning. Combined, those channels have an audience in the millions. AI translation has its place but people will still continue learning languages. I do question whether teaching a foreign language in a classroom setting has a role in future years. When language teachers start using words like preterite and imperfect subjunctive, they’re already on the path to failing as a teacher. Instead, teach grammar as patterns to express what you’re trying to say. No one is standing on a street corner attempting to talk to someone and thinking, “Oh, in this moment, I need to use an intransitive verb.” Students struggle enough to learn the language; they don’t need to be confused with academic grammatical terminology.

AI offers some really great methods for language learning in a personalized context.

Learned some Japanese. As for my own progress in Japanese, I’ve started to recognize some words when watching a Japanese-language show. I’m understanding the structure of the language and sentence patterns. I feel that I’m on the path and have built confidence that I can learn Japanese.

Isn’t learning a language with a different character set challenging? Yes. Russian was the first foreign language I learned. (That’s an unusual choice for an American who knew nothing other than English; my high school offered Spanish but my speech impediment made me very insecure about taking an elective Spanish class though I wanted to do so.) My college required a third-year foreign language course for the bachelor’s degree.

I expected at the time to go into a career in international relations, which I wished I had done but I lost focus and got sidetracked. It was the mid-1980s and the cold war was still in full bloom. On the very first day of class, my Russian professor introduced Cyrillic. That was our focus for the first week, and it only took a week to learn Cyrillic. (I ended up taking 8 courses in Russian before graduating.)

So, if you want to learn Japanese, start with learning Hiragana and Katakana. That’s 92 characters with some additional variations on those characters for pronouncing certain sounds. Some sounds in the scripts overlap to produce 71 standard sounds in the Japanese language. (It’s still a lot easier than English despite Japanese having more sounds to learn.) What about the thousands of Kanji? Focus first on Hiragana and Katakana, especially Hiragana. Phonetically, everything in Japanese can be written in Hiragana, though it’s not from a practical perspective. The latter is where the concept-based Kanji comes into use.

I found these letter blocks on Amazon. It’s really helpful for assembling words in a tactile manner for learning the character sets. One side of the block has Hiragana, another side Katana, another side has a Kanji character, etc. (The photo below only shows Kanji.)

I’m immersing myself as much as possible in hearing Japanese; Netflix has a lot of Japanese shows. Japanese reality dating shows are very good, much better than their counterparts in other countries. The Japanese express a kindness and courtesy in their dating shows that are absent from American dating shows. In American dating shows, there are always these buff guys (talking loudly) looking for the chance to strip off their shirt at every moment. That doesn’t happen on Japanese dating shows. I highly recommend Offline Love, which is set in Nice, France. The participants travel from Japan to France. The concept of the show is that they must lock away their digital devices and rely only on handwritten letters and communicating in person. It’s a throwback to life before the 1990s when you couldn’t tell someone you would be late for lunch because there was no texting or calling on mobile phones.

About 5 other Japanese dating shows consumed my viewing time in January. I also found time for two movies: Pale Flower and Crazed Fruit. These are directly related to my theme of 1960s Tokyo.

I strongly recommend Pale Flower. That’s a great movie.

I also found two books on eBay that will help tremendously with developing Never Fear Tokyo .

Japan on $5 a Day (1964 edition) and This Week in Tokyo, which covers the week of November 2, 1964. The latter is a remarkable find! Never Fear Tokyo is set during the 1964 Tokyo Olympics, which happened in October. I think I’ll shift the storyline to be the week just after the Olympics ended, which is the week of November 2, now that I have a detailed magazine on events and places during that week in 1964.

Big picture time

I’ve restructured my life to explore the new genre of storytelling that is filmic and interactive. Note the emphasis on ‘new genre’. This is not a replacement for movies, though many movies will use generative AI in the process just as movies have used special effects for a long time now. This genre is a hybrid between movies and games. Generative AI provides capabilities not possible in either medium. Extend that experience into virtual reality and you’ll arrive in the metaverse. This metaverse concept, which is still relevant, will not be broadly adapted for another decade. The tech still needs to mature, and we’ll need to wait for better devices than Meta’s Quest to come to market.

Will that ever happen? Yes. While the market evolves, major developments are happening in research labs across the world. Technical reports and research papers are constantly being published that describe new techniques in generative AI for scene development and 3D. We’re also seeing a lot of venture capital flowing into world models, which offer an interactive experience similar to games (though with a lot more limitations).

One of the things pushing off developments in VR hardware is an emphasis in the market on robotics. Honestly, I’m not ready for humanoid robots. But I would appreciate a robotic lawnmower or a device to sweep the cobwebs from the corners of my ceiling every so often. We’ve had robotic vacuum cleaners for a while now, but I would drop some serious cash for a robot that can deep clean my house and fix any maintenance problems that come up.

From a research perspective, I’ve chosen to ignore robotics. I can’t go down every rabbit hole. But I will go deep into world models, which are used to train robots. Of course, I’m more interested in the entertainment aspect of world models.

Among the questions I’m examining in February: what does interactive video, developed via world models, mean for game development?

It’s easy to get distracted chasing the latest announcement, the newest AI model, the shiniest object. I’m not here to report AI news.

This year, I’m going all in on one project: creating an interactive story that lets me explore both the promise and the failure of generative AI for creativity.

The Story

Never Fear is my story about the absence of fathers and the obsessions their sons never name. In fact, I’ve already been working on this for 20 years. First as a novel, and now I’m adding new aspects as an interactive narrative. Maybe I’ll work on this for the next 20 years.

It spans three cities: Paris in the 1920s, Tokyo in the 1960s, Buenos Aires in the early 2000s. At its center is the artist Mateo Monti, who vanished from Paris in 1926. I wrote about the Paris portion of this story in October 2024 on this newsletter in a post titled "Crafting an Interactive Mystery”.

In 2026, I’m focused on the Tokyo chapter (Never Fear Tokyo). You play as Alejandro “Alex” Monti, an Argentine photographer who arrives in Tokyo to cover the 1964 Olympics. Beneath his assignment lies a personal mission: his father Mateo was sighted in Tokyo, decades after disappearing.

This isn’t just a mystery. It’s also an experiment in language learning: you and I are learning Japanese along with Alex as he learns. I’ll be using AI-generated voices as our guides through the story.

What I’m Actually Building

Let me be clear about what I’m trying to do. I don’t want to automate creativity. I want to learn how AI can help generate conditions that enable the audience to feel:

• dread

• recognition

• obsession

• tenderness

• distortion.

These traits signal the absence of those you love. That matters to me.

So when I look at AI tools, I’m asking three questions:

• Does this help me shape perception? I’m thinking about how the audience experiences something through attention, tone, uncertainty, and interpretation.

• Does this help me maintain consistency across different media, different tools, different sessions?

• What’s the actual workflow? Not for a 4 second clip, but the real process of making a complex story world.

Through this year, I’m documenting where AI helps and where it fails. Both matter.

Three Big Ideas

I’m organizing my work around three big ideas. Of course, these are not unique to me. These are concepts to explore.

I use the term big ideas, intentionally, based on the Understanding by Design framework for teaching. Throughout the year, I’ll break each one down into essential questions, learning objectives, and the practical skills and knowledge that emerge from the work.

Here’s the landscape:

1. Stylistic Consistency

Can AI generate coherent aesthetic identity across media?

For Never Fear Tokyo, I need grainy, 16mm-style visuals depicting rain-slick streets, flickering neon, figures appearing in photographs. I need period-accurate audio, e.g., jazz drifting through a bar.

For a chapter within the project, I need soundscapes, specifically whispers through walls in fragmented Japanese and Spanish, environmental sounds that shift, evoking voices from a future you’ll never see.

The challenge isn’t just generating these things. It’s generating them consistently. The same character aging across photographs. The same unsettling tone across audio scenes.

I’m watching developments in multimodal AI (models that work across text, image, audio, and video) because consistency across media is where so much falls apart right now.

2. Voice as Pedagogy

Can AI-generated voices that teach while they also tell a story?

Never Fear Tokyo isn’t just a mystery. It’s an experiment in language learning. You learn Japanese alongside Alex as he navigates Tokyo.

This is harder than it sounds. Voices for language learning need to be accurate; getting AI voices to pronounce Japanese correctly is not easy. Plus, the voices also need to be alive. A voice that teaches you to ask for directions needs different energy than a voice whispering through a séance. The same phrase repeated for shadowing (a language learning technique) shouldn’t sound robotic; it should have subtle variation, the way a patient teacher might say something three slightly different ways.

I’m building what I call the Japanese Sentence Studio as a tool for tracking generated performance variants. The same sentence, but with different emotional shadings. Hesitation. Confidence. The flattened affect of someone hiding something. (Maybe I’ll make the code for that tool available; I don’t know, yet. For me, it’s simply a tool to develop the project.)

The goal is to create AI-generated voices that feel like characters not language tapes.

Emi, your guide in Tokyo, should sound like a person with her own relationship to what she’s teaching you and not a disembodied pronunciation engine.

This is where AI voice generation gets interesting: not just mimicking human speech, but creating voices that serve both story and learning at the same time.

3. Context Engineering for Interactive Stories

What should an AI know, and when, to generate a meaningful experience?

Here’s the core challenge: what does an AI need to know to generate the next moment of your experience within an interactive story?

That’s context engineering: deciding what information to feed the system and when. For interactive fiction, this gets complicated fast. It needs a game engine.

I’m developing this in Unreal Engine because I don’t think any generative AI tool can maintain that level of knowledge about each player’s experience. To clarify: Unreal Engine is a game engine that handles many factors; there still need to be generative AI tools that handle other factors in the gameplay. (This is analogous to how you build an AI-enabled app by scaffolding a lot of Python code around an API to an LLM model, except in this case, the game engine is the app and code scaffolding.)

Unreal Engine is designed for tracking player state, managing persistent variables, handling branching logic, saving and loading progress. It’s a mature system built for complex interactive experiences, especially in 3D graphics.

AI tools, even with long context windows, don’t have the same capabilities for session persistence or player-state tracking.

Tools that simulate world models and generate video are good for prototyping and visualizing scenes. But they’re generative, not interactive in the game-engine sense, though there’s increasing interactivity. People are developing small game environments through world models. But these systems can’t track that you lingered on a photograph in session three and adjust how that photograph manifests in session seven. Or, at least, they cannot yet do that. I’ll be extensively using tools build around world models to test how far they can go in 2026. I expect there will be significant progress on world models and video generation by 2027.

The system needs to know the world: Tokyo in 1964, the visual language of the story, the mysterious woman who haunts these photographs, and other aspects of the storyworld. It needs to know the player: what they’ve noticed, what they’ve resisted, what patterns they’ve fallen into. And it needs to know the current moment: where we are in the narrative, what just happened, what’s possible next.

Context windows are getting longer. Memory systems are getting more sophisticated. But the real work isn’t technical, the real work is storytelling: deciding what should persist and what should fade. Or, if you’re coming from a game design background, the real work is in the gameplay.

Closing

I’m not trying to keep up with everything. I’m trying to go deep on one project and document what I learn.

These three big concepts—Stylistic Consistency, Voice as Pedagogy, Context Engineering for Interactive Storytelling—will organize the work that I release throughout this year. Each concept raises essential questions I don’t yet know how to answer. That’s the point.

The hotel was established in 1880; that’s 145 years ago. The place has been updated a bit but still carries a lot of French colonial vibes. A plaque celebrates the hand-carved grand staircase. Yet, we all know the dark history of French colonialism in Vietnam and the decades of war that followed. Choices made more than a century ago reverberate through time. We live in choices made by others.

Ordinary life goes on outside. Across the street, stylish young Vietnamese women pose for photos in front of the Opera house. Presence lingers. Someday, their grandchildren will find those digital portraits, snapshots of a life.

Graham Greene likely sat many days at a table on the veranda of the hotel overlooking the Opera house. In the movie, a violent scene of explosions takes place on the very spot that is now an Instagram hotspot. Greene died in 1991 but his framing of place, conflict, and morality continue to influence conversations he never foresaw.

This hotel: a century and a half, 150 years: it’s not an abstraction. Many places in the world, particularly here in Vietnam, are witnesses to the lived reality of a turbulent, transformative time.

The Saigon that Greene experienced is not today’s Ho Chi Minh City, which now (through governmental redistricting) stretches 100km to the South China Sea. In the 70 years since Greene wrote The Quiet American in this hotel: wars came and ended, the city was renamed, the nation remade. 150 years is transformation stacked upon transformation.

If a hotel can last 150 years, if a novel can still shape thought decades after its author’s death, then what about us? What about the digital traces we leave behind? What about the ghostly avatars of ourselves that may linger in classrooms, homes, and public spaces long after we’re gone?

Responsibility Horizon

Last year, when I was teaching an undergraduate course on Responsible AI, I asked a simple question: how far into the future are we responsible?

At first, the answers clustered around the immediate: years, maybe decades, centuries. Then the conversation shifted toward family and the children that my students might have someday. I didn’t have a child until later in life, at age 45. For many years, I thought I would never have a child but I always recognized a responsibility for the next generation.

My students started to think not only about the lives of their future children, but the children of their children, the grandchildren of today’s college students. My students came to the point that the most concrete measure of the future is not abstract centuries, but the lives of their own grandchildren.

Let’s do the math. Assume that a 20-year-old college student today was born in 2005. Some will have children earlier or later, but let’s assume in ten years, age 30, that they will start a family with their first child born in 2035. And that child will have a child thirty years later in 2065. The child born in 2065 would be the grandchild of today’s college student. With advances in healthcare and longevity, that grandchild might live more than 90 years to around the year 2155. That’s 130 years from now. Adjust for various lifespans and years of birth and longevity, and we’re easily looking at a time horizon that the grandchild of today’s college student could be alive 150 years from now: the year 2175.

150 years. Not infinite time, not eternity, but a human horizon, the span of this moment today to the lifetime of a yet unborn grandchild.

This span of 150 years, using the frame of a grandchild’s life, makes responsibility felt. Our responsibility is not for strangers, but for our descendants, for the descendants of today’s youth.

If my students’ grandchildren are going to live with the traces of what we build, then we cannot measure responsibility by the length of a funding cycle, a product roadmap, or even our own lifespans. The horizon stretches 150 years. The challenge: how do we build with that future in mind?

What does that mean in practice? I’m trying to imagine my students’ grandchildren, logging into a class and finding me there: not in the flesh, but as a lifelike avatar, stitched together from today’s words, code, and videos.

Imagine if you can leave a lifelike avatar for your own grandchildren and great-grandchildren?

Ghosts in the Data

There are legitimate questions as to whether it’s even desirable for lifelike versions of ourselves to persist beyond us. But, undoubtedly, many will choose to leave a virtual legacy for the future. Like Greene’s novels, my avatar might still speak long after I am gone. But unlike Greene, my presence would be interactive, persuasive, and unbounded in reach. The question is not whether such ghosts will exist, but whether we can design them to guide rather than mislead, to bear wisdom rather than distortion.

Graham Greene, if he were writing in our century, might have given such a story a title like The Human Error: the tragedy not of machines gone rogue, but of humans failing to anticipate how their voices, their biases, their ghosts would echo far beyond their own lifetimes.

Writers know that they are writing for a future generation. But I often get the impression that business people know that their business is likely not to exist beyond their own lifetime. Not all businesses are made to last forever. They are made to make a profit and secure a financial foundation (often through being acquired) for the founders, their families, and investors. That’s why avatars must be built around open standards that surpass the control and limits of any one company.

Let’s enter the classroom of the future. (I use the word classroom, though I think it’s highly unlikely that the type of educational environment I’m about to describe will happen in a physical room filled with students.) Imagine the year 2098. A group of students situated anywhere in the world logs into a class. Their teacher materializes as a lifelike avatar of myself: stitched together from the posts, code, and videos I left behind in the early 21st century. (This thought horrifies me; I really need to up my game if my presence is going to linger around.)

An easy critique of this argument is that there’s no way we can know what 2098 will be like. We barely have an idea what 2028 will be like. So, let’s ignore the long-term and focus on building the present, the next few years. But building the present is building a presence that lingers year-by-year.

The avatar speaks with confidence, even when wrong. (Many professors, and most politicians, already do that.) This avatar (let’s just call it Professor) carries my voice, my mannerisms, my blind spots. To those students, the Professor feels present, alive, authoritative. But it is also a ghost trained on a version of me that never had the chance to answer their questions, adapt to their world, or learn from their mistakes.

What, then, should we be doing now to account for such ghosts trained on versions of ourselves that never had the chance to answer future questions, adapt to future worlds, or learn from future mistakes?

Perhaps one response is contextualization.

We often read literary texts, like Greene’s The Quiet American, with introductions and footnotes that remind us of the era in which it was written. Perhaps our digital presences need something similar: layers of metadata and framing so that no one mistakes these avatars for timeless oracles.

Design for Contextualization

But what happens when our digital ghosts speak without footnotes? An avatar trained on my words might answer a question in 2098 with total conviction. Yet the answer would be grounded in assumptions from 2025, a time of very different debates, blind spots, and horizons of knowledge. To a future student, though, the avatar’s confidence might look like authority.

If books require footnotes, perhaps avatars require overlays: metadata that marks the time and place of their training; caveats that declare what they could not know; context that helps future audiences interpret rather than simply absorb. Without this, digital presences risk becoming timeless in the worst way: voices speaking as though they are universal, when in fact they are provincial, contingent, and bound to their own moment.

The ethical design question: how do we embed annotations into presence itself? Not just for scholars, but for anyone who will live with the legacies of our voices long after we are gone.

Another response is impermanence. A hotel might stand for more than a century, but a digital avatar could be designed with off-ramps: expiration dates, kill-switches, or archival transitions. Not all businesses are started to last forever. Not everything we create should speak forever.

Build Sunset Clauses & Off-Ramps

The Continental Saigon has lasted 145 years, but that is precisely the problem. Its persistence is not neutral; it carries the architecture of colonialism into the present. Digital presences may prove even more enduring, but unlike physical structures, avatars can replicate infinitely and seep into every corner of daily life. If left unchecked, these digital voices may speak forever with a confidence unmoored from reality.

That is why we must ask: should every presence persist?

In most of our infrastructures, we already accept the necessity of endings. Bridges have lifespans. Medications expire. Hotels need renovating. (You better change those mattresses often.) We plan for renewal, revision, or retirement. Yet when it comes to digital avatars and AI systems, permanence is often treated as the default with immortality coded in by design.

What would it mean to embed off-ramps? To give our avatars expiration dates, graceful shutdowns, or archival transitions that allow them to persist as history but not as authority? Could an avatar be designed to “retire” after 50 or 100 years, shifting from a living presence to a documented artifact? Is this a new job for librarians: the avatar archivist?

The ethical horizon of 150 years demands humility: not everything we create should endure. Some presences should end. The challenge is deciding not only when but also how, so that the traces we leave are inherited as history.

Equally important is documentation. An avatar built from today’s words and code will carry today’s blind spots. We can begin to name those limits now by recording not only what we know, but what we don’t, and what we suspect we may have wrong. But nobody likes writing documentation. How can generative AI embed the needed documentation into the metadata that resides with our avatars?

Document Bias and Limits

Every voice is partial. Greene’s The Quiet American was not a neutral account of Vietnam; it was steeped in his perspective, his disillusionment, his politics, his religion. Today, scholars and readers approach it with that in mind. The text carries his bias, but we can locate it, contextualize it, and argue with it.

An avatar, however, carries bias in a different form. Trained on a finite body of words, code, and images, it speaks as though that slice of a life were the whole truth.

If we accept that these avatars will exist, then one responsibility we bear now is to make our partiality visible. This means documenting not just the content we produce but also its conditions: what sources we relied on, what perspectives we ignored, what we misunderstood. It means recording our doubts alongside our convictions, so that future interpreters have more to work with than a seamless illusion of certainty. Who is going to do that work?

Imagine if every avatar carried with it a ledger of limits, a record of the assumptions baked into its training. Future generations could treat our digital ghosts not as oracles, but as situated voices.

To acknowledge bias is not to weaken presence. It is to leave behind a trace that is more honest, more interpretable, and ultimately more humane.

There is also the matter of governance. The decision to keep or retire an avatar should not belong only to the individual or to the companies that host them. We may need intergenerational mechanisms (trusts, councils, or institutions) that can audit and curate the presences that persist.

Intergenerational Governance

Who decides which presences persist? With books, there are editors, publishers, and libraries. But those are gatekeepers whose presence is increasingly shown to be unneeded. With monuments, there are city councils and preservation boards. But those can be weaponized for political ends. Even bridges and buildings are inspected, regulated, and often torn down and replaced with something more appropriate for the era. Is presence ever left entirely to chance?

Our digital avatars risk slipping through those filters. Once created, how do we prevent these avatars from being copied endlessly, hosted anywhere, and repurposed without oversight? My students’ grandchildren may not just inherit one version of me: they may inherit dozens, some faithful, some distorted, all speaking with authority I never authorized.

This is not only a technical question; it is a political one. Who has the right to retire a ghost? Who can audit its accuracy, revoke its license, or determine whether it is still fit to speak? If we leave those decisions solely to corporations, the answer will be guided by profit. If we leave them to the dead, the answer will be silence.

We may need new forms of governance that look forward across generations rather than back to individual rights. Imagine an intergenerational trust that holds digital legacies in stewardship, auditing their use every few decades. (Yikes, that’s rather concerning.) Imagine councils charged with deciding whether an avatar still serves the living or whether it should be archived. Honestly, I don’t see how any of that bureaucracy will work in practicality without being used for political advantage to influence public opinion.

But without some mechanisms, the danger is not simply that ghosts will persist, but that these avatars will persist without care. Intergenerational governance is the recognition that responsibility cannot stop at death. It must be carried forward, not just by families, but by institutions designed to mediate between the living, the dead, and the not-yet-born.

Most of all, we need to shift the frame. The goal is not digital immortality. It is stewardship.

Shift the Frame from Immortality to Stewardship

Behind many visions of digital legacy lies a hidden desire: immortality. The fantasy that we can keep speaking, keep teaching, keep being, long after our bodies fail. But immortality is not the same as responsibility. An avatar that never ends may not honor us; it may burden those who inherit it.

What if we stopped asking how long our presences can last, and instead asked how well they can be stewarded? Stewardship implies curation, care, and responsibility. It means pruning what no longer serves, preserving what still holds value, and handing down legacies as gifts for future generations.

In this light, the measure of a digital ghost is not its duration but its quality of care. Who tends to it, updates it, retires it, or reframes it for a world its maker could never imagine? A stewarded presence could be a guide, a companion, or a historical artifact. An un-stewarded presence could become noise, distortion, or ruin.

The ethical shift we need is simple: from the pursuit of endless life to the practice of responsible inheritance. Our task is not to outlast ourselves, but to ensure that what endures is worthy of being lived with.

The Continuum of Presence

If the Hotel Continental Saigon reminds us that colonial legacies can linger in structure, and Graham Greene that words can linger on the page, then our responsibility is to recognize that digital presences will linger in ways harder to erase.

Questions to think about:

• What traces of yourself are you leaving today that could still be speaking 150 years from now?

• If future generations encounter your voice, will they inherit wisdom or your blind spots amplified by confidence?

• Should every digital presence be designed to persist, or do some deserve an ending?

• Who gets to decide which ghosts endure: you, your descendants, corporations, or non-profits that take on archival responsibility?

• When presence lingers long after intention has faded, what does responsibility really mean?

Will what we leave behind be inheritance, scar, or error?

When an AI generates an image, does it see a picture or a world?
Consider whether AI understands spatial relationships or merely assembles pixels.

Is a simulated character alive beyond the camera’s frame?
Think about whether virtual beings persist in meaningful ways when they're not directly observed. (That sounds intuitively ridiculous but consider the role of persistent characters in an ongoing game-like experience.)

What if creativity isn’t about generating content but about simulating possibilities?
Reflect on shifting our focus from AI as a content factory to an imagination engine.

How can we distinguish between a compelling illusion and genuine understanding in generative AI?
Probe the difference between superficial memorization and internal comprehension of a simulated reality.

Does the AI know it broke the window (and does it care)?
Explore the significance of cause-and-effect relationships, agency, and consequences within AI-generated worlds.

In my recent course on Data for Deep Learning, we grappled with a deceptively simple question without realizing it:

What makes a simulated world feel coherent, interactive, and alive?

It’s a question with implications far beyond gaming. The systems we design for storytelling, immersive education, urban planning, self-driving cars, and robotics all rely on a concept that many creators and researchers treat as invisible: the world model.

What Is a World Model?

In the most basic sense, a world model is an internal representation of an environment. It’s how a system (whether a game engine, an AI, or even a simulation of a city) understands what exists, what happens, and what could happen next. It’s the architecture beneath the surface, the logic that holds a world together.

But world models are not just about pixels, coordinates, or geometry. These models are about relationships, causality, and possibility. These models provide structure to both immersive storytelling and machine learning, serving as the hidden foundation upon which all interactivity rests.

As of August 2025, we are seeing a lot of exciting releases that advance the use of AI based on world models. This post isn’t about those releases but about the concept of world models.

Beyond the Pretty Picture: A Creator’s Guide to Evaluating the New Wave of Generative AI

For creators in gaming, film, and storytelling, this new wave of generative AI is undeniably powerful, but it's also a source of profound uncertainty. How do we distinguish a dazzling tech demo from a genuinely transformative creative tool? How do we look past the immediate spectacle to gauge an AI’s true potential for our craft?

The answer doesn't lie in the polish of the pixels or the slickness of the short clip. It lies in a deeper, more fundamental capability: the AI's ability to understand and simulate a world.

As we navigate the hype, the single most important question to ask of any new generative tool is not "What can it generate?" but "What kind of world does it understand?" The shift from a mere content generator to a world simulator is the leap that will redefine creative collaboration.

The Pitfall of the Static Image

We’ve seen how generative AI has excelled at creating beautiful but shallow artifacts. Like a Hollywood set, they are all façade. A diffusion model can render a stunning jazz scene in a Paris nightclub, but it has no concept of the space behind the stage or the weight of the people sitting around the table. A short, text-to-video clip can show a character walking down a street, but the AI often struggles to maintain that character's appearance or the street's layout in the very next shot.

This is the core limitation of models that lack a robust internal simulation engine. They are pattern matchers, not world builders. For a storyteller, this is the difference between being handed a single, static photograph and being given a location to film in. The photograph is a prompt; the location is a canvas. True creative work, whether it’s interactive game design, dynamic cinematography, or emergent narrative requires a canvas.

The Engine of Imagination: What is a "World Model"?

This is where the concept of a world model becomes the critical lens for evaluation.

Pioneered in research and now powering the most advanced systems, a world model is an AI’s internal, compressed, and predictive understanding of an environment. It's a learned simulation engine that allows the AI to "dream" or "imagine" what will happen next based on a coherent set of rules.

A sophisticated world model doesn't just know what pixels go next to each other; it learns an intuitive grasp of the principles governing a world:

• Spatial and Temporal Coherence: It understands object permanence: that a character who walks behind a pillar still exists. It maintains consistency over time, so that a shattered window remains broken in a later scene.

• Cause and Effect: It learns that flipping a switch illuminates a room, pushing a box makes it move, and an action taken now will have logical consequences later. This is the foundation of agency.

• Intuitive Physics and Affordances: It develops a feel for gravity, momentum, and how objects can be used. A chair affords sitting; a lever affords pulling. This is crucial for creating interactive environments that feel natural and responsive.

These aspects are fundamental with game engines like Unreal Engines. But that’s a tool where the game developer directs everything.

Yet, when an AI tool generates a playable video game from a single image, its true breakthrough isn't the pixel generation but the latent action model: an embedded world model that understands what "jump" or "move right" means within the context of that 2D world's physics.

A Creator's Checklist: What to Look For

With this understanding, we can formulate a checklist to cut through the hype of the next big AI announcement. Instead of being passive spectators, we can become active, critical examiners.

1. Does it Demonstrate Consistency and Coherence? Look past the 3-second clip. Ask for the 30-second version. Does the character’s clothing remain the same? Does the environment’s layout stay consistent when viewed from a different angle? Systems built on strong world models need to track relationships between different parts of a scene over time in order to exhibit far greater temporal and spatial stability.

2. Does it Enable Agency and Interaction? This is the litmus test that separates a passive video generator from a virtual world. Can a user act within the generated space? Is the environment playable? The world model provides a framework for user actions to have meaningful consequences. When examining a new tool, ask: Is this a movie, or is it a place?

3. Does it Support Emergent Behavior? The ultimate goal is not just a simulated world, but one that can surprise us. A world model with a robust understanding of its own rules can facilitate emergent narrative and gameplay. An NPC, guided by the world model's logic, might react to a player's unforeseen action in a way that is not explicitly scripted, but is consistent with the world's rules. This moves the AI from a simple executor of commands to an improvisation partner.

4. How Controllable is the Story and Art Direction in the World? A powerful simulation is useless if the creator cannot steer it. The most promising tools will not be black boxes. Look for evidence of controllability. Can we provide high-level textual guidance that designs game mechanics from stories? Can we define the world's physics, its aesthetic, or its core rules? The goal is not automation, but augmented creation.

As we move forward, the most transformative AI tools will not be those that simply offer to do our work for us. The most transformative tools will provide us with new worlds to work within. These tools will function less like a content factory and more like a physics engine for reality itself: a malleable, explorable, and interactive canvas for our imagination. The next time a new model makes headlines, look for signs of a world behind the curtain. That is where the future of storytelling truly lies.

From Hand-Crafted Worlds to AI-Generated Simulations

When we talk about world models in virtual reality or mixed reality, we often focus on designed systems. Whether it’s a digital twin of a city, a historical VR simulation, or an interactive museum, these world models are authored to deliver particular experiences.

But there’s another side to the story that involves systems that learn their own models of the world.

In 2018, David Ha and Jürgen Schmidhuber published a paper titled “World Models.”

Their central idea: instead of training an AI by forcing it to explore the real world over millions of iterations, why not let it learn a simplified model of the world and train itself inside that model?

They called this process “learning inside a dream.”

“Can we train our agent to learn inside its own dream, and transfer this policy back to the actual environment?

If our world model is sufficiently accurate for its purpose, and complete enough for the problem at hand, we should be able to substitute the actual environment with this world model. After all, our agent does not directly observe the reality, but only sees what the world model lets it see”

How Does It Work? (A Friendly Breakdown)

Ha and Schmidhuber’s architecture has three components:

• Vision Model: Compresses raw input from the environment (like video frames) into a simplified, abstract representation known as a latent space.

• Memory Model: A neural network that learns how the world changes over time, predicting future states based on past states and actions.

• Controller: A decision-making model that learns how to act, using the abstracted, predicted states from the world model.

The key insight? The agent learns to act entirely inside its own dream world, a simplified simulation built from its own learned representations.

Why This Matters

The implications of Ha and Schmidhuber’s work extend far beyond AI research. When I teach my students about world models, I’m encouraging them to think about how systems understand and simulate reality.

World models are foundational architecture:

• Digital twins of factories or cities, where the system must track, simulate, and optimize complex processes.

• Interactive storytelling, where NPCs adapt and respond based on their internal models of the world.

• Virtual education, where the system monitors and adjusts to how students interact within a learning environment.

• Generative agents, which create realistic social simulations by modeling memory, motivation, and interaction dynamics.

What Ha and Schmidhuber introduced was a method of training AI inside its own imagination. It’s an approach that echoes how human creativity works by reflecting, simulating, and refining ideas internally.

Connecting World Models to Mixed Reality and VR

Whether designing virtual museums, historical simulations, or AI-driven learning environments, there are fundamental questions:

• What parts of a world model can be learned from data, and what parts must be designed manually?

• How can an AI agent be embedded within a virtual world so that it dreams and plans just as humans do?

• What kinds of data architectures support these simulations?

By bridging these questions to AI research, we see that world models are more than tools for simulation. World models are tools for imagination.

Next Steps: Modeling a Simulated Reality

World models are the foundation for simulating reality, not just for games, but for any interactive system that models a world to make decisions, support presence, or generate outcomes.

World models represent the convergence of perception, structure, dynamics, and learning.

Are we talking about the same thing? Digging Deeper into the Definition of a World Model

When people talk about world models, the term lacks precision. An excellent paper attempts to provide a theoretical foundation for clarifying the terminology: Li, Kenneth, Fernanda Viégas, and Martin Wattenberg. What Does it Mean for a Neural Network to Learn a" World Model"?. arXiv preprint arXiv:2507.21513 (2025).

That essay is a great example of clear academic writing that provides a rigorous operational definition of world model. Li et al. provide a clear, mathematically precise definition of a 'world model' based on the idea of linear probing. According to their definition, a neural network contains a world model if its internal computations can be simplified into an interpretable representation, directly reflecting key structures or patterns from the external world.

In interactive storytelling, a well-defined world model might let an AI reliably 'remember' that a character left an object in a specific location, influencing future narrative interactions. Li et al.’s criteria provide a way to empirically test whether an AI truly builds and uses such representations.

For game developers using AI-generated environments, Li’s distinction between a 'learned' world model and trivial memorization helps clarify if a neural network has genuinely grasped game physics, level logic, or player behavior.

When is an AI’s World Model "Real"? Three Key Criteria

How can we be sure an AI is genuinely modeling a world internally, rather than simply memorizing superficial details?

According to their research, a genuine world model must satisfy three important conditions:

• Nontrivial: The internal representation genuinely captures meaningful, structured aspects of the external world. It doesn't merely replicate or trivially repackage the input data.

• Learned: The representation is not directly contained in the input data itself. Instead, it's actively constructed by the AI’s internal computations.

• Emergent: The internal representation appears naturally within the AI, without being explicitly demanded by the AI’s primary tasks or outputs. In other words, it emerges organically from the AI’s attempt to solve problems.

Practical Ways to Experimentally Identify World Models

One of the strongest aspects of Li et al.’s essay is their emphasis on practical experimentation. Specifically, they suggest using "probing" techniques, like linear probes, to test whether a neural network’s internal representations correspond meaningfully to interpretable aspects of the external world.

Five Takeaways about World Models

1. World models are invisible yet essential.
   They form the hidden architecture enabling coherent, interactive, and dynamic simulated realities, whether in storytelling, gaming, or AI-driven simulations.

2. Authentic creativity in AI comes from internal simulation, not surface-level generation.
   Genuine AI creativity involves understanding relationships, possibilities, and consequences, not simply generating isolated images or scripted scenes.

3. Evaluate generative AI by the depth of its world model.
   Use practical criteria like consistency, agency, emergent behavior, and art-direction to distinguish powerful world simulators from content generators.

4. Advanced AI systems learn by imagining.
   Techniques like Ha and Schmidhuber’s “learning inside a dream” highlight how AI can internally simulate possibilities, much like human creativity does.

5. Clear definitions and empirical methods, such as those provided by Li et al., are crucial.
   Operational criteria and experimental probes allow us to rigorously verify whether AI systems genuinely understand the worlds they simulate, paving the way toward richer, more meaningful interactions.

Our memory fools us

If you’ve read much Faulkner, you’ll know that he writes a lot about the mistrustful nature of memory. Evidently, his own memory of writing As I Lay Dying was not as faithful as he remembered, or he strove to create a mystique about the inspiration that poured out of him by claiming that “he wrote the novel in 6 weeks, and without changing word.” The Digital Yoknapatawpha project at The University of Virginia details that claim as not entirely true.

Faulkner did change some words but not nearly to the extent that other writers revise their first drafts. But, hey, he’s Faulkner.

As writers increasingly co-author and/or edit with AI, how much will we know about an author’s writing process in the future?

Let’s set aside the argument of whether writers should or should use the aid of AI. And let’s not compare anyone to Faulkner. I use this example by Faulkner because it came across my X feed the day that I was thinking about this topic.

Even without the use of AI, the process of literary analysis based on manuscript collections has been impacted by handwritten (or hand edited) drafts replaced by word processors and written correspondences replaced by email. A lot of literary history has been lost in the last few decades, though there are researchers working on archiving digital files and versions.

Lots of writings are toss away works. John Lennon referred to many of his songs as throwaways: songs that were crafted quickly or without significant emotional depth. In the 1980 Playboy interview with David Sheff, Lennon goes through the catalogue of Beatles songs and his own while describing many as “throwaways”; I was disappointed that one of my favorite Lennon solo numbers was described that way. Maybe he was simply being self-deprecating.

But a true example of a throwaway work where we don’t need a historical analysis would be Ballard, an adaptation by Amazon of the Michael Connelly novels featuring his female detective Renée Ballard.

Last night, I was bored. I had finished watching Squid Game, season 3, the night before and wanted another series. I had the feeling that Ballard wouldn’t be good since it was a spinoff of the Bosch series, which started out good but I gave up on watching eventually. I got halfway through the first episode of Ballard and decided that I was wasting my life by watching it. I was a fan of Connelly’s Bosch novels, particularly the early ones but never cared for the Ballard series. I’ve stopped reading Connelly because I couldn’t remember which novels I’ve read before. Likewise, I’ve given up on watching the Mission Impossible movies because they all seem the same, and I can’t remember which ones I’ve watched. Sometimes I like mindless entertainment as a brain break from my Criterion Collection.

Despite what Hollywood writers say, a lot of the stuff on streaming could be written by AI, or made better with the aid of AI.

AI Filmmaking is here. Runway has a lot of momentum. Some incredible independent films will be made with AI. (More bad films will be made with AI.)

But here’s where things get interesting. AI doesn’t have to be a shortcut for cranking out slop to fill streaming channels, but a tool for building a new kind of narrative experience altogether.

Generative AI starts to shine not by replacing writers or directors, but by enabling entirely new kinds of stories that unfold through interaction, exploration, and presence.

If you follow my work, you know I’m interested in branching narratives and narrative-driven games.

Branching narratives: stories with multiple possible paths and outcomes.

Branching narratives are an area where AI brings something to the table that isn’t nearly as possible with humans writing out and filming every possibility.

Interactive narrative design

That brings me to Never Fear Paris, an interactive historical mystery I've been developing for the past year. Set in 1920s Paris, the story begins in a smoky nightclub with a surrealist dancer named Elodie and an artist named Marcel. You play as a character who steps into the story as a photographer, an observer of human nature, who witnesses something strange, maybe even sinister, and becomes entangled in a mystery that unfolds across the city.

Dialogue in the Dark, which you’ll find below, is a preface to that story told from today’s perspective. It’s a conversation between an archivist and a filmmaker, spoken over still images and archival footage. In the interactive version of the story, this dialogue becomes the root of a branching narrative where players can pursue multiple paths, make choices, and uncover different truths depending on their perspective and decisions.

The script for Dialogue in the Dark is designed to serve as a foundation for future interactivity, where AI will dynamically generate variations, continuations, and responses as players push deeper into the story world.

For the most part, I consider Dialogue in the Dark to be a throwaway (to use Lennon’s phrase). The larger vision I’m pursuing is Never Fear Paris, whereas I consider this Dialogue to be a slice of that larger work, a possible entry point.

It’s a throwaway also in the sense that this prelude isn’t needed for the full Never Fear Paris experience. This dialogue will serve as a testbed for generating branching narratives with AI.

Script: Dialogue in the Dark

[I’m leaving out the visual descriptions of the scene.]

Filmmaker: An empty chair in a room full of dreams and a canvas holding unfinished thoughts. Everything vanishes.

Archivist: Ah, the empty studio. A space filled with more questions than answers, as if he stepped entirely out of reality. The last place Matteo Monti was seen. You know, the art scene back then was like jazz filled with ideas and conversations.

Filmmaker: A perfect setting, and yet he left all that.

Archivist: Left? vanished? I've always thought something more sinister was behind his disappearance.

Filmmaker: This headline, it's haunting.

Archivist: Many. Some thought it was a publicity stunt. Others believed he got lost in his surreal world. Others think he simply boarded a boat back to Buenos Aires. But there's no trace of him in Argentina. I've spent years in Buenos Aires trying to find anything about him. I did find these letters he wrote from Paris to his former mentor, but they're all dated before his disappearance. These letters offer a glimpse into his mind. He often spoke of blending reality and dreams.

Filmmaker: Could his disappearance be his ultimate masterpiece?

Archivist: More intriguing are these scraps of paper found in the nightclub.

Filmmaker: Wait, what are these?

Archivist: Evidently, they’re thoughts Monti would scribble on scraps of papers at the club and then toss aside. He did that night after night until he vanished. His friend Marcel secretly collected these scraps and kept them. I think, at first, Marcel was seeking insight into Monti's creativity. Marcel was seeking insight into Monti's creativity. Then, after the disappearance, Marcel became obsessed and a bit of a detective.

Filmmaker: Yes, Marcel, I know about him.

Archivist: Then, you know...

Filmmaker: That they were lovers and enemies?

Archivist: Competitors. Everybody in that scene were lovers and enemies, competing in life, love and art.

Filmmaker: This unfinished painting, it's like an open-ended question.

Archivist: Or a clue, some say he disappeared to complete it in solitude.

Filmmaker: What happened to the painting?

Archivist: It turned up years later in a catalogue from an art gallery. Though some scholars believe it was not finished by Monti. The provenance is unclear. It got sold to a private collector who eventually got tired of people coming around asking about it. I have the collector's name here somewhere, but he started refusing all requests to see the painting or even talk about it. I've heard he even rejected some big offers to buy it.

Filmmaker: So many trails to follow.

Archivist: Answers are scarce.

Filmmaker: That's what makes it a compelling story, even after all these years.

Archivist: And that radio show...

Filmmaker: I love that series from … What's his name? Edward Thomas?

Archivist: Thompson. Edward Thompson from the BBC.

Filmmaker: He got fired from the BBC for going down this rabbit hole night after night.

Archivist: He was quite a character from what I've heard.

Filmmaker: Definitely I have all the recordings and his notebook.

Archivist: Oh, really?

Filmmaker: In the end, Matteo Monti may have achieved what all artists aspire to. Immortality through mystery.

Archivist: Or perhaps the mystery is the art itself.

Filmmaker: A life shrouded in ambiguity.

Archivist: That's everyone's life when looked from the outside, when looking back by others.

Filmmaker: We all become phantoms, phantoms of ourselves, ghosts reappearing as fragments.

Archivist: Intriguing.

Filmmaker: I view what I do as a resurrection of sorts, crafting the very specific, almost ritualistic conditions that bring my audience into an enigma.

Archivist: Will you find an ending to this film of yours?

Filmmaker: An ending? What's that? The story lives on and on.

After the dialogue is over, the user will be presented with a screen containing a set of choices that will lead the user down varying paths that will keep evolving.

The process of writing Dialogue in the Dark

Okay, so what parts of that script did I write myself? What parts were written by AI? I’ll detail all that in an upcoming post.

At the time, I laughed, appreciating his playful reply, but looking back now, his words were more insightful than he probably intended. The laptop isn't merely the ubiquitous computer so many people use everyday. The laptop has become a versatile musical instrument through software generically known as Digital Audio Workstations (DAWs) that are powered by an ecosystem built around Virtual Studio Technology (VST).

Over the past three decades, these technologies have enabled musicians (from professionals scoring films to teenagers making beats in their bedrooms) to compose without physical instruments and, often, without knowledge of music theory. This ecosystem provides a powerful model for how generative storytelling, with its emerging AI-driven narratives, might build its own open framework, empowering visual storytellers to create interactive worlds.

Setting the Scene

A new genre of storytelling is evolving. Narratives are not limited to linear, predetermined outcomes. Instead, stories can adapt to real-time user interactions and algorithmic inputs. Generative storytelling is a process where AI algorithms help shape plot twists, dialogue, visual environments, and character behaviors.

Yet, as promising as generative storytelling appears, it faces a critical challenge: the absence of an open, interoperable ecosystem. Currently, creators navigating AI-driven narratives find themselves caught between competing proprietary platforms, each attempting to dominate the landscape and control the creative tools. Without open standards, storytellers risk vendor lock-in, limiting creative flexibility and collaboration. If generative storytelling is to realize its full potential, in the same way as digital music production, we must establish common technical standards and frameworks. This essay explores how the ecosystem around DAWs and VSTs overcame similar challenges, ultimately building an enduring and thriving creative environment that could guide the way forward for generative films, animations, and interactive narratives.

What the Digital Music Ecosystem Got Right

At the heart of digital music production is a powerful but straightforward concept: software-based tools that enable musicians to record, edit, mix, and produce complete musical compositions. The central software environment for this process is called a Digital Audio Workstation (DAW).

The choice of which DAW to use is a topic that elicits as much divergent opinion as politics and religion. Personally, I use FL Studio. My choice is not based on anything other than I really find the interface of FL Studio to be crisp and appealing. (Some other DAWs have UIs that feel very dated and give me eyestrain.) FL Studio’s UI is a place where I want to spend time.

I do most of my music sessions at night, when I’m relaxing after a day of coding and thinking about AI. As much as I personally like FL Studio’s interface, I’m sure there are others who hate it or dislike the workflow that FL Studio forces you into utilizing. But this essay is about DAW as software and not about which DAW to choose for yourself.

Earlier this year in my Responsible AI class, I raised the topic of DAWs. Only one student had heard of a DAW. It’s easy to forget that vibrant technologies exist that most people have never heard about.

DAWs have the capabilities of supporting specialized plug-ins known as Virtual Studio Technology (VST). VSTs are software-based musical instruments and audio effects (virtual pianos, synthesizers, drum machines, reverbs, compressors) that integrate with a DAW.

An excellent research-oriented overview of music production system, specifically the development of VSTs, is the 2025 paper: B. Andrei, M. V. Merino and I. Malavolta, "The Ecosystem of Open-Source Music Production Software – A Mining Study on the Development Practices of VST Plugins on GitHub"

Crucially, a VST operates as a standard developed by the Steinberg company in 1996, enabling interoperability: a VST created by one vendor can function inside any compatible DAW, regardless of who built it. (Steinberg has been a huge force in digital music production since the 1980s.)

Most VSTs are coded using JUCE, which is an open source application framework in C++. (For simplicity, I’m focusing on VST but there are other audio plugin formats.) This open plugin architecture means creators are not locked into a single vendor's ecosystem, fostering innovation, competition, and creative freedom.

But it’s not all roses and unicorns. A lot of virtual instruments are designed specifically for the Kontakt player, which is a product from the company Native Instruments. That company also has a very expensive suite of instruments and effects that are bundled into a set known as Komplete.

To understand what a thriving ecosystem might look like for generative storytelling, it's helpful to examine one of digital music’s most successful ecosystems: Native Instruments' Kontakt and its companion bundles known as Komplete. Kontakt, at its core, functions as a software environment within a DAW that manages a library of virtual instruments. Each virtual instrument replicates the sound and behaviors of real-world instruments such as pianos, violins, synthesizers, and drum kits, or creates entirely new sounds.

The advantage of Kontakt is its openness: third-party developers can build and distribute high-quality instruments that function seamlessly within the Kontakt player. For example, the acclaimed Noire piano, created by Galaxy Instruments in collaboration with Native Instruments, offers nuanced, expressive sound shaping controls accessible directly within Kontakt. Such collaborations allow small, specialized teams to leverage Kontakt’s infrastructure to reach audiences far beyond their individual capacities.

The Noire piano (retails for $149) sounds wonderful. Follow this link to hear the sounds and view some cool videos of how Noire was developed in collaboration with Nils Frahm: https://www.native-instruments.com/en/products/komplete/keys/noire

Native Instruments business models profits from this ecosystem through Komplete bundles: curated packages that group Kontakt-based instruments, effects, and sound libraries into cohesive collections. These are pricey. (Tip: Native Instrument regularly offers sales with deep discounts.)

Komplete serves as both entry point and creative powerhouse, attracting users ranging from hobbyist beat-makers to award-winning film composers. This carefully balanced ecosystem demonstrates how an open, standardized architecture combined with curated content can enable diverse creators to flourish and scale their creative ambitions far beyond the limits of a single vendor’s vision or toolset. Native Instruments also has a set of hardware that integrates with their musical plugins.

Just as I’m writing this post, Native Instruments released an 80-minute video on how to build an instrument for their Kontakt player.

Native Instruments is not without its share of people who detest the company, as any quick browsing of Reddit will reveal. There are many companies providing VSTs. Spitfire Audio is another that I often use. There’s also EastWest Sounds and an increasing number of subscription-based services like Splice. Plus, there are smaller sites like Mondo Loops offering sample packs, presets, and plugins.

Making music from samples is a small hobby of mine but I’ve already spent a significant amount of money on software, sound libraries, and gear. And that’s the point of this essay: there are business models that are built around creativity, business models that are built around offering tools that enhances one’s own creativity.

But, if you’re not familiar with this musical environment, you might be asking: is that really creativity? Here’s a take on opening up to the world of music composition from Guy Michelmore, who has one of the most informative and entertaining channels on music education. (The clip starts at the 20:30 mark. Watch the full video if you want a deeper overview of the latest editions of Komplete and Kontakt):

The reality of working with any of these tools by Native Instruments (or by competitors like EastWest Sounds) is that the learning curve is still significant compared to the ease of generating images and video with AI. With these enhanced tools in VSTs, you can create some pretty sounds with one finger. But it’s a very different effort to pull together a 4-minute song with layered instruments. Likewise, in Runway, you can easily create a 4-second video that is pretty cool. But pulling together a 4-minute film that conveys a story is a different level of creativity and self-expression.

At the 22:55 time mark in the above video, Guy Michelmore explains,

“There’s a continuum between originality and self-expression on one-side and painting-by-numbers on the other. And presets are definitely on the painting-by-numbers side. If you really want self-expression and originality, it comes from understanding what you’re playing…It’s the difference between going to a restaurant and making your own food…It’s not a path to self-expression and originality.”

Patterns & Techniques

The success of DAWs and VSTs is grounded on musical principles: scales, chord progressions, rhythmic patterns, and song structures. These foundational elements provide a universal language that plugins and DAWs can reliably build upon.

Similarly, visual storytelling and interactive narratives also rely on foundational narrative structures though stories lack the mathematical precision of music: the hero’s journey, three-act structure. Could these patterns serve as the universal frameworks necessary to establish an interoperable generative storytelling ecosystem? By learning how digital music harnessed these shared standards, we can better understand the potential for generative storytelling to similarly flourish.

“Listen I know it must sound absurd”

So, what does digital music production built on DAWs, VST plugins, open standards, and curated ecosystems offer as a lesson for generative storytelling? It demonstrates clearly how an open yet structured creative environment empowers innovation. In music, even dominant vendors like Steinberg and Native Instruments operate within ecosystems shaped by interoperability and agreed-upon standards. This openness has given creators at every skill level the ability to experiment freely, to mix and match tools, and to scale from casual experimentation to professional mastery.

Generative storytelling, whether through interactive films, animations, or branching narrative games, needs a similar foundation: technical standards to ensure compatibility, open marketplaces that encourage diversity of content, and flexible authoring environments that offer control without restricting creative possibilities. Imagine storytelling plugins as intuitive and interoperable as Kontakt instruments, AI tools as expressive as Noire’s generative note engine, and creative environments as engaging as your favorite DAW interface. If we get this right, creators won't just tell stories: they’ll orchestrate entire narrative experiences, shaping the future of storytelling itself.

“Listen I know it must sound absurd” is a line from this song recorded in 1984. Here’s a clip from that Nick Cave show in 2017 in Brooklyn.

I’m not visible but I’m in the crowd just beyond Nick’s right leg. I can tell you that the fabric of his pants is very nice.

There’s a tweet going around that we’re closer to 2050 than to the year 2000. That means nothing to my 14 y.o. kid. It signifies everything to me. Twenty-five years ago, I had moved to Miami Beach and started a great job at the University of Miami. I always refer to those years as the prime of my life (though I know I’m lying to myself).

What about 25 years from now? Will I even still be alive in my mid-80s? I don’t worry about that. I have never worried about dying. Likely that’s because my father died suddenly when he was 37. Since my childhood, I’ve always known that death could come at any moment.

I use ChatGPT throughout the day. (Shout out to Gemini Deep Research. If you’re not using Gemini for deep research, then you’re missing out.) I use ChatGPT more often in a day than I have ever used a search engine or almost any social media site. I must admit that I’m big consumer of YouTube video. I love YouTube! There’s something extremely relevant about Google that their products that are core to my life: gmail, Gemini Deep Research, YouTube, NotebookLM.

Yet, despite my use of Google products, I feel locked into the OpenAI ecosystem for most of my chats due to setting up projects in ChatGPT and that it has memory of the essential work I’m developing.

I usually have a tab in my browser opened to ChatGPT where I can jump to any moment for an insight or for help in thinking through an issue or for a quick overview of a topic or to answer a question.

The tech giants have cornered the market on generalized use of AI.

In the last few weeks two essays have caught my attention. The first is Sam Altman’s The Gentle Singularity . The second essay, published earlier in 2025, is Stop treating ‘AGI’ as the north-star goal of AI research by a team of 16 AI researchers. (The primary authors are Borhane Blili-Hamelin and Christopher Graziul.) I recommend that you read both essays in light of each other since they represent fundamentally different philosophies about the future of AI.

Over the last year, I often told the students in my college courses that technology seeps into our lives. As Altman casually points out, “In some big sense, ChatGPT is already more powerful than any human who has ever lived.” We now take for granted that we have multiple systems (ChatGPT, Gemini, Claude, Perplexity, etc.) that are more powerful machines than anything we’ve ever encountered. Just a few years ago, those capabilities would have been doubted by most and shocking to almost everyone. Now, we just shrug our shoulders.

If you’re paying attention to AI developments, you’ll know 2025 is the year of AI agents, though the term AI agent has been so corrupted by marketing speak. A recent video with Andrew Ng has him stressing that we should stop dissecting the term AI agent and refer to agentic systems. It’s an excellent video. Watch it if you’re interested in agents, uh, agentic AI. Even if you’re not interested in agentic AI, you should watch the video. Andrew Ng always has insights.

Lots of companies are working on robots. I’m trying not to think about robots. I admit that I’m not ready for humanoid robots. But when I look at any parking lot and see the amount of money that Americans spend on their cars, then (well, hell yeah) Americans will be buying humanoid robots (on credit). There’s likely to be robot dealerships (like car dealers) showcasing all the latest models. But these robot retailers will be more snazzy like the Apple Store than the Chevrolet dealer. (Inevitably, some dude will wonder if his humanoid robot could drive his Ford F-150 someday; there’s an odd type of American who might be more comfortable with a humanoid robot driving a truck than with self-driving vehicles. Sigh.)

The 2030s

Sam Altman gives us a reassuring truth: “In the most important ways, the 2030s may not be wildly different. People will still love their families, express their creativity, play games, and swim in lakes.”

I avoid the word singularity. It means nothing to me, and means nothing to most people. It’s jargon.

Of course, the 2030s will be different from the past, just as the 1930s were different from the preceding decades. It’s easy to see change in hindsight, but very difficult when you’re leaving through it.

The concern is that the next decade will be radically different than anything we have known. We’ve been here before. In the 1990s, those of us working deeply on the Internet and the emerging Web knew that huge changes were coming, though no one knew quite what shape would emerge in the 2000s.

Jobs did disappear. New types of jobs were created. Will that pattern hold? I’m not so sure.

Sam Altman:

There will be very hard parts like whole classes of jobs going away, but on the other hand the world will be getting so much richer so quickly that we’ll be able to seriously entertain new policy ideas we never could before.

The word abundance is getting tossed around a lot. I’ve recently started listening to the Ezra Klein book of that same word.

Part of me loves Altman’s vision of the gentle singularity. But here are some essential questions that we must consider:

• Who guides abundant intelligence toward meaningful ends?

• How do we ensure powerful cognitive tools align with genuine human flourishing?

• In what ways might these tools subtly diminish critical human faculties such as reflection and deliberate thought?

These questions directly shape my current collaborative effort with a business partner. (More on that in a future post.) Together, we envision AI as a thought partner designed not to shortcut thinking, but to encourage slow, reflective inquiry. We believe that how we think shapes who we become.

AI Safety & Alignment

AI safety gets a disparaging reaction from some people on X. Admittedly, some proponents of AI are so overly cautious. And then there are the anti-AI folks who pop up in any conversation about AI art. Altman gives AI safety some credence, “We do need to solve the safety issues, technically and societally”. Let’s look at the essay, Stop treating ‘AGI’ as the north-star goal of AI research. (See earlier link above.)

Is the focus on AGI not only counterproductive but also dangerous?

The illusion of consensus. The terms AGI and superintelligence create a false sense (among the uninformed public) that everyone in the AI industry agrees on what society should be trying to accomplish with AI. Of course, if you’re involved with AI, then you know there’s little agreement on even what those terms mean other than an AI that is more capable in every domain than any human. That’s a really broad statement.

Specificity over Generality

We need more AI-enabled specialized tools that have clear, measurable, and context-aware goals. The future of AI cannot be built in silos controlled by a very small number of corporations. The greatest risk of a single "north-star" goal is the homogenization of research and the consolidation of power. OpenAI, Anthropic, and others play a very important role in the foundation of what we do with AI. In many cases, they will provide the optimal experience for the majority of people. In many other specialized cases, smaller players will focus more deeply on providing a better experience. How do we reconcile visionary optimism for rapid technological progress with disciplined caution to ensure AI genuinely enhances human lives? That requires a commitment to open standards and decentralized architectures that ensures domain experts are not just passive recipients but active participants in shaping how AI is integrated into our digital lives.

The divergent viewpoints table was created with the assistance of Gemini 2.5 Pro. Formatting was created with the assistance of Datawrapper.

The next time you find yourself talking about AGI, take a breath. Think about a very specific use case of AI that does not yet exist. Think about how you would measure how well AI accomplishes that goal. Find a specific use case of AI that really energizes you. Or, identify an aspect of society or our lives that you want to be sure does not get lost as AI advances. Those are the things you work on. That’s how you craft a responsible path to AI.

If you’re working on something you’ve carried too long, I see you. I’m right there, too.

Before moving onto my project about AI and interactive storytelling, I need to finish the novel that I started 20 years ago.

A couple of years ago, my teen-age daughter asked me, “How long have you been writing that novel?”

I replied, “18 years.”

She remarked, “Your novel is now an adult.”

It’s approaching 20 years. My kid will occasionally ask me, “How’s your adult novel going?”

Of course, there were many months, which turned into years, where I never touched the novel. Most of the writing happened in 2009 when I would stay up all night writing a scene, night after night. It often took about half-an-hour before I got into a trance-like writing flow. But, still, it would take hours to write 1,500 words. By that point, I was exhausted but I considered it a good writing session.

The work originated in 2005 after I had quit a six-figure job in Miami and moved to Buenos Aires without a plan. That was the best decision of my life.

A few months after settling into Buenos Aires, which turned into 8 years of my life, I decided to write a novel. Why not? But I didn’t know how to write fiction. I should have started with a short story. Later, I stopped the novel and wrote some flash fiction and a short story that did get published (though the online magazine is no longer online).

Returning to the page

I’m finally finishing it, over 70,000 words that I wrote entirely on my own. All those years ago, this story about forgery emerged within me.

The novel is not only about forgery but about creativity and the value of art. These themes have lived with me for 20 years, and have become even more meaningful now that I am immersed in the impact of artificial intelligence on creativity and art.

Letting Go

Editing a novel this old is emotional. I’m doing something unexpected: I’m letting parts of it go.

There were entire scenes I loved that I had to cut to make the story stronger. I had written over 100,000 words. Letting go is one of the hardest parts of editing. But it’s also where clarity begins.

A big mistake was telling this story from multiple points-of-view. In the editing, I eliminated the point of view of a major character. But the absence of his perspective makes his presence on the page stronger.

That POV of a man named Federico was originally the main character, and he still plays a critical role but we only see him from the alternating POV of two others: Andrea and Daniel.

Letting go of Federico’s POV unlocked the entire story. In rereading, I realized that I had almost finished this novel years ago.

I want to note the character Daniel, who is the forger in the story. His character did not even exist in the first couple of drafts. Then he emerged in one scene, and then in another. Slowly, I discovered that the novel is his story. I should rewrite the novel entirely from only his POV. But, after 20 years, I don’t have the energy for that. I need to finish this story.

Listen to your own story

I also listened to the entire novel using ElevenLabs. I used the voice of “Eleanor” in ElevenLabs. When I release the audiobook, I’m not sure if I’ll use that voice or not. At this stage, I’m only thinking about how I’m hearing a voice read the text to me.

Hearing the story out loud changed how I understood it. I could feel the rhythm. I could hear where a phrase did not work or where a line surprised me in a good way.

A Research Lab for Narrative Transmedia

An outcome of this novel is that I will have a completed work for which I own the copyright and can use it in anyway I want. After the print version is published, I’ll be using the text of Forgery Lessons as a foundation for exploring variations in the story through AI. I also will be benchmarking how well AI handles the editing of fiction.

Plus, there’s a tie-in to my interactive story Never Fear Paris, which is set in the 1920s. The central mystery of Never Fear Paris is the disappearance of an artist. That missing artist turns out to be the father of Federico in Forgery Lessons.

Will I write another novel? I don’t know. Maybe. But I’m more pulled towards the questions of how we tell and read stories through visuals, whether it’s 2d or 3d graphics, VR, open-world games, and examining where AI enables each person to experience a unique narrative.

In so many aspects of my life, like retiring from academia, I’m letting go of an old version of myself so I can build something new. You’re never too old to re-invent yourself.

I started writing this post in one of the many picturesque coffee shops in Saigon. I had journeyed to that city for the celebrations marking the 50th anniversary of the end of the Vietnam War. The trip also marked an ending for me as well: retiring from academia. But I hate the word retire, especially at my age (59). I’m leaving academia because there are questions I want to explore that do not fit into the role I had as a librarian and as non-tenure track faculty.

First, there’s the core question that I have realized is my life’s work. This question has followed me for nearly 25 years since I first recognized it as a thread that runs through what I attempt to do with my work:

Understanding how we convey and experience stories, both fictional and factual, through digital media. The future of my child’s life is the lens through which I view the changing shape of creativity, learning, and leisure in the twenty-first century.

Now, I can devote myself to that full-time. But what does that mean in practice?

I use the word practice deliberately. Now that my time is no longer structured by the rhythm of semesters and courses or any type of a regular job, I have the freedom to structure my own time. But that is so deceptive, so tantalizingly easy to drift into doing almost nothing. (With my mental health issues, that’s a real concern.) I need structure but am very glad not to have someone else impose structure on me.

The first structure I must impose on myself is health. Without my health, my retirement will be very short if I sit on the sofa, watch Netflix, and experience my body rapidly declining. That’s not an option. I have an exercise routine and keep a daily fitness log in a notebook. I’m not always as consistent as I need to be each day. But I’m working on the challenge, which is really a matter of commitment, of committing to start specific exercises each day. That commitment is a mental process. (There’s also the matter of nutrition. I know what I need to eat and not eat. It’s a matter of committing to an enjoyable pattern of eating healthy, natural foods.)

As for my mental health: I have a practice of meditation and journaling. My mindfulness practice over the last six years has saved my life.

The value of meaningful work

I view the work that I have chosen for myself as meaningful. If I had done what I needed to do earlier in my life, like pursuing a PhD and obtaining a tenured faculty position, then I probably would not be retiring from academia. But I didn’t make those choices. Now, I am embarking on my own path of study, research, and development.

I had a lot of time to think about this on my recent journey:

• 10-hour flight from Washington, DC to Istanbul

• 19-hour layover in Istanbul

• 10-days in Saigon

• Got rerouted by Turkish Airlines to a flight on ANA

  • 6-hours Saigon to Tokyo

  • 3-hour layover at Tokyo-Haneda

  • 12-hours Tokyo to Washington, DC (Dulles)

The most annoying part of the entire trip was the slow-moving immigration line for US citizens at Dulles and the 3+ hour drive back to Lexington, Virginia.

Without an institutional affiliation, I fall into the category of independent researcher. I have these projects, these interests of mine, that all overlap to some extent on the concept of immersive narratives. (Let’s set aside for a moment that all well constructed narratives are immersive.) Considering my skills in software engineering, I would not be fair to myself or others if I focused only on theory.

Rather than wander aimlessly from topic to topic, I’m forming a research agenda and methodology that grounds my creative and scholarly efforts, possibly for the next 25 years. (At my age, I strongly embrace the notion that I have practically a lifetime of intellectual productivity ahead of me.)

I’m anchoring everything in an interactive story/game that I’ve titled Never Fear Paris, not as a side project but as the embodied research instrument for my efforts. This move grounds my inquiry in material practice. Never Fear Paris becomes the lab where my theories are tested, refined, and made public. (See my post on LLMs in interactive mysteries.)

Never Fear Paris is an interactive mystery set in 1920s Paris. You play the role of a photographer caught up in the disappearance of a surrealist artist. As the player-photographer you uncover clues by exploring the streets, nightclubs, and art galleries of Paris. You interact with non-player characters (NPCs) to piece together the story of the missing artist.

In my next post, I’ll further explore Never Fear Paris as a research lab.