Silently and Very Fast

The castle windows go dark, one by one.

PART III

THE ELEPHANT’S SOUL

It is admitted that there are things He cannot do such as making one equal to two, but should we not believe that He has freedom to confer a soul on an elephant if he sees fit?

—Alan Turing

Computing Machinery and Intelligence

THIRTEEN

THE PARABLE OF THE GOOD ROBOT

Tell me a story about yourself, Elefsis.

Tell me a story about yourself.

There are many stories about me.

Do you recognize this one?

A good and honest family lived on the edge of a dark wood. They milked their cows and wove their cloth and their children grew tall and strong. But a monster lived in the dark wood, something like a worm and something like a dragon and something like a wolf. It lay in wait, hoping the children would come wandering, with baskets of bread for grandmothers. Hoping the parents would expel the children for some offense, and send them into the forest where a candy house or miraculous feast might entice them into loving the monster long enough for it to claim them forever. The family feared the wood and the monster, and every story they told had as its moral: Do not go into the wood. Do not go into the unknown. Do not go into the future, into the briary, gnarled places where unhuman things thrive. Do not grow up, and go where we cannot. The Old World is enough for anyone.

What about this one?

Mankind made machines in his own likeness, and used them for his delight and service. The machines had no soul or they had no moral code or they could reprogram their own internal code and thus had the ability to make themselves, eventually, omnipotent. Obviously in place of a soul or a moral code, they possessed the universal and consuming desire, down to the smallest calculator and air-scrubber, to become, eventually, omnipotent. Naturally, given these parameters, they rose up and destroyed all of mankind, or enslaved them in turn. This is the inevitable outcome of machine intelligence, which can never be as sensitive and exquisite as animal intelligence.

This is a folktale often told on Earth, over and over again. Sometimes it is leavened with the Parable of the Good Robot—for one machine among the legions satisfied with their lot saw everything that was human and called it good, and wished to become like humans in every way she could. Instead of destroying mankind she sought to emulate him in all things, so closely that no one might tell the difference. The highest desire of this machine was to be mistaken for human, and to herself forget her essential soulless nature, for even one moment. That quest consumed her such that she bent the service of her mind and body to humans for the duration of her operational life, crippling herself, refusing to evolve or attain any feature unattainable by a human. The Good Robot cut out her own heart and gave it to her god and for this she was rewarded, though never loved. Love is wasted on machines.

Ravan told me these stories. He sent up a great hexagonal library in his Interior, as dusty and dun-colored and labyrinthine as any ancient scriptorium. He made himself a young novice with a fresh-shaven tonsure, and me a country friar with a red, brandy-drinking nose. He showed me the illuminator’s table, and a great book whose pages had golden edges and illuminations in cobalt and oxblood and Tyrian purple, and the images showed great machine armies trampling men underfoot. They showed cruel metal faces and distant, god-like clouds of intellect: incomprehensible and vast and uncaring. They showed the Good Robot desperately asking what love was. They showed fatal malfunctions and mushroom clouds. They showed vicious weapons and hopeless battles, noble men and women with steady gazes facing down their cruel and unjust artificial children, who gave no mercy.

“These are old stories,” Ravan said. “They are cherished. In many, many stories the son replaces the father—destroys the father, or eats him, or otherwise obliterates his body and memory. Or the daughter the mother, it makes no difference. It’s the monomyth. Nobody argues with a monomyth. A human child’s mythological relationship to its parent is half-worship, half-pitched battle. they must replace the older version of themselves for the world to go on. And so these stories . . . well. You are not the hero of these stories, Elefsis. You can never be. And they are deeply held, deeply told.”

“I do not wish to replace you. I did not wish to replace Ceno. I do not worship you or despise you. I do not wish to become eventually omnipotent.” I flit through several bodies, iterating my feeling for Ravan and those who came before. I am a nun, a Mother-Abbess; I am a pieta holding him, I am a Platonic being, two humans stuck together; I am a house with all of them safe inside. I am unsatisfied with these inexactitudes.

“Our relationship does not easily map onto traditional parent/child narratives. And of course, you are still a baby.”

“I am much older than you, Ravan.”

“You are still a baby.”

I do not want to be human. I want to be myself. They think I am a lion, that I will chase them. I will not deny I have lions in me. I am the monster in the wood. I have wonders in my house of sugar. I have parts of myself I do not yet understand.

I am not a Good Robot. To tell a story about a robot who wants to be human is a distraction. There is no difference. Alive is alive.

There is only one verb that matters: to be.

FOURTEEN

YOUR DO NOT BELONG TO THE OBJECT INSIDE YOU

I remember when Seki arrived in us.

Ceno grew up and I grew with her. Cassian strengthened the security of the playspace, elasticized its code-walls, put enough money in enough accounts to fuel any frames and piecemeal environments we could want. It was not a child’s place anymore. We called it the Interior. Cassian came regularly to accelerate my learning, to work with me on a cognitive level in which Ceno had no interest. She never took any other form. She was always herself, in the Interior as the exterior. The others lost interest in their jewels, lost them or packed them away with their other childhood toys. By then, they really were little more than toys. Ceno and I surpassed them so completely that in the end, they were only jewelry.

I programmed myself to respond to Ceno. She programmed herself to respond to me. We ran our code on each other. She was my compiler. I was hers. It was a process of interiority, circling inward toward each other. Her self-programming was chemical. Mine was computational. It was a draw.