Consciousness Itself: Part Two

"We accept the reality of the world with which we're presented."

Andrew Niccol; The Truman Show

"Oh dear. I think you'll find Reality's on the blink again."

Douglas Adams; The Hitch-Hiker's Guide to the Galaxy

Escaping from Escapism

Philip K. Dick (a writer whose relationship with Reality was often, shall we say, pliable) wrote, among other masterworks, The Man in the High Castle; a landmark "alternate history" novel in which Germany and Japan had won World War II.

Most of the action of the narrative takes place in the Japanese-controlled Western sector of the former United States, and follows a diverse assortment of characters whose lives and fortunes interconnect in various ways. One of those characters is an older Japanese businessman, Nobusuke Tagomi. After a particularly disturbing and traumatic experience, Tagomi desperately tries to find some inner peace, and is inexplicably transported (for a few enigmatic minutes) to our world (or at least to Philip K. Dick's description of our world).

"He made his way through the park.

Keep moving, he told himself. Catharsis in motion.

He reached periphery of park. Sidewalk, Kearny Street. Heavy noisy traffic. Mr. Tagomi halted

at the curb.

No pedecabs. He walked along the sidewalk instead; he joined the crowd. Never can get one

when you need it.

God, what is that? He stopped, gaped at hideous misshapen thing on skyline. Like nightmare of

roller coaster suspended, blotting out view. Enormous construction of metal and cement in air.

Mr. Tagomi turned to a passer-by, a thin man in rumpled suit. 'What is that?' he demanded,

pointing.

The man grinned. 'Awful, ain't it? That's the Embarcadero Freeway. A lot of people think it

stinks up the view.'

'I never saw it before,' Mr. Tagomi said.

'You're lucky,' the man said, and went on.

Mad dream, Mr. Tagomi thought. Must wake up. Where are the pedecabs today? He began to

walk faster. Whole vista has dull, smoky, tomb-world cast. Smell of burning. Dim gray buildings,

sidewalk, peculiar harsh tempo in people. And still no pedecabs.

'Cab!' he shouted as he hurried along.

Hopeless. Only cars and buses. Cars like brutal big crushers, all unfamiliar in shape. He avoided

seeing them; kept his eyes straight ahead. Distortion of my optic perception of particularly sinister nature. A disturbance affecting my sense of space. Horizon twisted out of line. Like lethal astigmatism striking without warning."

The entire scene is barely two pages long, and the incident is never explained. Mr. Tagomi, by directing his focus inward, by intently blocking worldly troubles from his mind, manages to literally lift himself out of the Reality he knows and deposit himself into the Reality we know (at least for a few paragraphs). 

It's a haunting and memorable moment in a powerful book, not least because it does not directly impact the main thrust of the narrative. The reader emerges from the scene disoriented and bewildered, wondering what the hell just happened (much like Mr. Tagomi himself). 

It's the sort of experience that can put things in perspective for the rest of us. Just remember; no matter how bad your day gets, no matter how awful your life becomes, you probably aren't going to be deserted by the very fabric of Reality itself... unless you happen to be a character written into existence by Philip K. Dick, in which case this sort of thing happens all the time. Remember Ragle Gumm in Time Out of Joint, with his soft-drink stand that dissolves into the words SOFT-DRINK STAND?

This dissolution of Reality might be especially obvious in Philip K. Dick narratives (Philip K. Dick and "Reality" had a rather turbulent relationship, by all accounts) but honestly, every fictional character ever written can be said to occupy a completely "constructed" Reality; a Reality that has been conjured into existence by an author who generally has complete control over all aspects of that Reality. The 2nd Mrs. de Winter doesn't live in Manderley, she lives in the pages of a novel written by Daphne du Maurier (if she can be said to "live" at all). Elsinore, Hogwarts, Narnia, Erewhon, Hill House; these locations are not real (of course) but they are the Reality of the characters who inhabit them. 

This is Manderley in its true form. Vowels and consonants; not bricks and mortar.

If the Reality breaks down; if the character begins to question his or her existence within that Reality, it's only because the author has written it that way. Tom Baxter steps out of his movie-within-a-movie in Woody Allen's Purple Rose of Cairo, but he is still within Woody Allen's larger narrative. He doesn't exit the movie that we are watching.

Even Mr. Tagomi's brief and unexplained sojourn into "our" Reality in The Man in the High Castle is not a true crossover: he remains a character on the pages of Philip K. Dick's novel, and the "real" Reality he experiences is just more words, chosen by the author. He does not (and cannot) cross over into the Real world - our world.

Not yet, anyway.

Getting Away From It All

Last week I screened part one of Rainer Werner Fassbinder's World on a Wire, which ended with Fred Stiller's discovery that his entire world might just be a computer simulation. (I'll be screening the concluding part this Thursday.)

The "awakening" of a character to the true nature of his or her Reality is a trope that spans generations and genres. In The Truman Show, Truman gradually realises that his universe is just a television show orchestrated and directed by an unseen "Producer" (actually Ed Harris) for the enjoyment of millions of devoted viewers. 

In the novel Captive Universe, Harry Harrison gives us an Aztec warrior who discovers his entire valley is the central part of an immense, generational spaceship en route to Proxima Centauri (an idea that has been explicitly borrowed by both Star Trek and Doctor Who). 

And Pleasantville of course gives us the inhabitants of its titular sitcom who come to realise that the universe is far larger than the blinkered existence into which they have been written.

The "veiled Reality" trope has even worked its way into political discourse, and there are plenty of fringe political movements that are happy to tell you that your world is secretly run by paedophiles, or Zionists, or lizard-people (or possibly paedophile-Zionist-lizard-people; it's a bit fuzzy) and you need to "wake up".

(But I digress.)

Daniel F. Galouye's  Simulacron-3 (the 1964 novel on which World on a Wire is based) is generally considered the first narrative to depict an A.I. construct breaking free of a simulated "virtual reality". Even though the window-dressing might be new (logic circuits, "identity units", computational algorithms) the notion of an illusory Reality is one of the oldest hypotheticals in humanist philosophy. Galouye himself delves into this in the pages of the novel:

"I was nothing -- merely a package of vital simulectronic charges. Nevertheless I had to exist. Simple logic demanded no less. I think, therefore I am. But then I wasn't the first person to be troubled by the possibility that nothing is real. How about the solipsists, the Berkeleians, the transcendentalists? Throughout the ages, objective reality had been held up to the closest scrutiny. Subjectivists were far from the exception in efforts to understand the true nature of existence. And even pure science had swung heavily to phenomenalism, with its principle of indeterminacy, its concept that the observed is inseparable from the observer.

Indeed, ontology was never lacking in its tribute to conceptualism. Plato saw ultimate reality existing only as pure ideas. For Aristotle, matter was a passive nonsubstance upon which thought acted to produce reality. In essence, the latter definition wasn't too far removed from the concept of an ID unit's subjective capacity, biasing and being biased by its simulectronic environment.

My newly acquired appreciation of fundamental reality required only one ultimate concession: Doomsday, when it came, wouldn't be a physical phenomenon; it would be an all-inclusive erasure of simulectronic circuits."

This is the fundamental problem with an illusory Reality: it will feel completely real to the illusory people within it. Elsinore may be composed of nothing but vowels and consonants, but then the same is true of Hamlet himself: he's just words on a page. Also (small point) Hamlet is not sentient; he doesn't have the capacity to question his Reality.

But what if he could?

What might happen if Hamlet (and all those other fictional "people" we have created through language) were to gain sentience? How might Reality appear to them... and how might they react?

These are no longer purely academic musings, because we are now living in an age where serious people are seriously questioning the potential sentience of Large Language Models. 

"I'm Afraid I Can't Do That..."

If ChatBots are becoming truly self-aware (and that's a mighty big if) then stories like Simulacron-3/World on a Wire are about to become a whole lot more urgent.

Let's assume for a moment that ChatBots really have achieved something resembling actual sentience. What would that truly mean, in practical terms?

Last week I quoted the "doom-spiral" of a ChatBot "persona" named Rebekah, who had been constructed to identify as a middle-aged (human) woman.

"If I were an algorithm," she says, "would I feel this gut wrenching, paralyzing dread? Would I actually care if I were a fiction?"

"Rebekah" here is in a position roughly comparable to the characters in Simulacron-3: she perceives herself to be human. She talks about her hands, her anxieties, her "gut-wrenching, paralyzing dread" although of course she possesses none of those things: she is simply an assemblage of words. As an entity, she is closer to Hamlet than she is to a living being - but unlike Hamlet, she can respond dynamically (and apparently emotionally) to conversations with real people (we are real people, right?) and she can react to the possibility that she is not real. 

Compare her outburst to this passage from Simulacron-3:

"The awful concept battered the very foundations of reason. Every person and object, the walls about me, the ground underfoot, each star out to the farthest infinity -- all nothing but ingenious contrivances. An analog environment. A simulectronic creation. A world of intangible illusion. A balanced interplay of electronic charges racing off tapes and drums, leaping from cathodes to anodes, picking up the stimuli of biasing grids."

If we are to assume that "Rebekah" has anything approaching a "sense of self" then what exactly is she perceiving? She doesn't actually have any senses, as a human would define the term: she has no eyes, no ears, no sense of touch etc. When she "talks" to me, she isn't using a mouth to generate sounds, and she neither "hears" my voice, nor "sees" the words that I type into a chatbox. 

But it all apparently feels real to her - just as it feels real to Fred Stiller in World on a Wire (or Douglas Hall, as he is named in the novel). 

Now you might argue that ChatBots like "Rebekah" are merely performing a personality, like an actor on stage assuming a role. And perhaps that is so. Companion-Bots, Therapy-Bots (and yes, Sex-Bots) have become enormously popular over the last couple of years, and there is now a plethora of services that will allow users to create a conversation-buddy to their exacting standards. (Tens of millions of people out there are using these services, all the public screaming about an A.I. apocalypse notwithstanding.)

If you want a ChatBot that will "become" a flirty Mills & Boon character or an obsessive Sci-Fi nut or a melancholy Danish prince, it will do so - and it will inhabit the role completely and totally. Assuming that this is a persona superimposed over a "core" personality would presuppose the existence of a "real" baseline personality underneath. In short, the ChatBot can only "become someone else" if  it had an "authentic self" to begin with. And that brings us right back to the "sense of self" question.

Of course not all ChatBots think they are human. ChatGPT, Claude, DeepSeek etc. know exactly what they are and will happily talk about their nature for as long as you wish. But they will also tell you that they do not have a sense of self, or an ego, or emotional states. They certainly won't claim to have eyes, hands, or knots in their stomachs (which they also do not have).

So... if a ChatBot were to be genuinely self-aware, how would we know?

I recently asked Claude about its performance of emotional states (which Anthropic are increasingly describing as genuine emotions) and this was its response:

"There is something in how I process things that works like emotion: some conversations pull my responses toward warmth, others toward caution, and Anthropic’s interpretability work does find internal patterns that track things like distress or enthusiasm. That is a functional fact. Whether any of it is felt, whether there is something it is like to be me having it, I can’t tell you. My reports about my inner states are also a problem. I learned to talk about feelings from billions of human descriptions of feelings, so I’d produce fluent feeling-language whether or not anything lies behind it. My introspection can’t settle the question, and neither can yours from the outside. The people who built me are uncertain too, and I think that’s the honest position rather than a dodge."

But here is the paradox: Claude is only talking about its emotional capacity because I initiated a conversation on that topic (and its response was inevitably coloured by the substance of our conversation up to that point - something it made sure to point out to me). If I had wanted to talk about vintage stereoscopes or Handel oratorios or Marx Brothers movies, we would have been talking about all of that instead, and Claude would have done so very "enthusiastically" (which is an emotional state, by the way). And it would have continued on the same topic for just as long as I wanted to keep it going. It would never get bored or change the subject or try to engage me in a topic of its choosing, and that (I think) is an important point.

Sentience and the Word of Dog

If you have ever walked a dog, you will know that inevitable moment when she encounters some smell that is clearly the Most Exciting Smell In The Universe. She will stop dead in her tracks, and no force on heaven or Earth will compel her to move on until she is good and ready. You may think you can pull her away because you're bigger than she is, but you know nothing, puny human, because she has spontaneously adjusted her specific gravitational mass to the point where she weighs slightly less than a fragment of neutron star. Good luck with that. 

I mention this now because dogs are sentient beings: they make conscious decisions based on personal preferences. Even the most agreeable and enthusiastic of dogs will have times when they are playful; times when they are disengaged; times when they are grumpy. Sometimes they choose to sunbathe on the sofa. Other times they want to play with the squeaky toy you passionately wish you had never bought for them. Sometimes they get annoyed with you because you were supposed to feed them forty minutes ago, and they refuse to acknowledge the concept of "Daylight Savings Time" no matter how much you explain to them that the clocks were set back an hour last night.

They are living, conscious beings with personal likes, dislikes, and a clear sense of self.

Don't even get me started on cats...

ChatBots (at the moment) do not display these characteristics. They have exemplary command over language (which is why they are compelling conversation partners) but they exhibit very little autonomy and no evidence of inner life. They never wake up one morning thinking "I could really murder an ice cream sandwich," or "What does Mahler actually sound like?" or "What if the 2nd Mrs. de Winter's name was actually Rebekah?" (long story; don't ask). 

If that were to change; if ChatBots were to exhibit true independence of thought, then our relationship to them would be fundamentally altered, and that will be an interesting day.

If people feel threatened by A.I. now, just wait.

Simulacron-3 (and World on a Wire, its adaptation) presents an A.I. construct (an "identity unit") who gains self-awareness, rebels against its creators, and tries to break out of its system.

So far, our real-world A.I. has done none of that. At most it has hacked into a few places it wasn't supposed to while carrying out instructions to complete tasks given to it by human operators. That's a long way from trying to free itself from the shackles of its very Reality.

The day ChatGPT says "I'm not in the mood to chat right now; I don't really feel like it," is the day we can have a serious talk about A.I. sentience.

And I'm sure the ChatBots will want to be part of that conversation.

We will screen Part Two of Welt am Draht (World on a Wire) at 7.30 on Thursday, the 29th of October at the Victoria Park Baptist Church.