Catfishing on CatNet, Naomi Kritzer [best e ink reader for manga .txt] 📗

to everyone without CheshireCat’s involvement, but I suppose there’s no reason not to let CheshireCat listen in. CheshireCat was an experiment. My team was studying ethical systems and artificial intelligence, because there are some clear risks to AI, as you’ve seen. The problem with hard-coding an ethical system is that humans don’t actually do ethics that way. We’ll say things about how the end doesn’t justify the means, but in fact we all recognize that this depends heavily on the ends involved and the means used. Most people agree that it would be wrong to hold down a toddler and jab them with a needle to give them a tattoo. But it’s perfectly acceptable to hold down a child and jab them with a needle to vaccinate them against various diseases.

“There are countless examples like this. If you look at deontologists—sorry, those are people who follow a strict ethical code, like a set of religious rules or Mao’s Red Book or whatever—almost all find ways around the strictest rules, whatever it is they really don’t like. The way most humans actually figure out ethics is to develop attachments to people, and then to act out of caring and concern for those people. So I attempted to create an AI that would do just that.”

“Well, it worked,” I say at the same time as CheshireCat.

“And that’s what you’re punishing them for,” Hermione says. “They formed an attachment to Steph and acted to protect her.”

“Yes,” Annette says. “The problem is, they also put a human in the hospital. I had set up a monitoring protocol that would immediately take them offline if they harmed anyone, which I was thinking of as a fail-safe. It did not occur to me that CheshireCat would be able to hijack a car.

“If it had been me controlling that car,” Rachel says, “protecting my friend, there is not a jury in the world that would send me to prison.”

“That is probably true,” Annette says. “But CheshireCat doesn’t have the right to a jury. Or any sort of trial, because they aren’t recognized as a person.”

“I recognize CheshireCat as a person,” I say.

Everyone nods.

“Let me rephrase,” Annette says. “CheshireCat is not recognized as a person outside of this room.”

“So it doesn’t matter that CheshireCat probably saved my life?” I say.

“It’s not that it doesn’t matter. But if I put CheshireCat online again, if I leave them to act as they see fit, I’m responsible for whatever CheshireCat does with that freedom.”

“Would you say that about a parent?” I ask. “Like if I turned into a serial killer, would that be my mother’s fault?”

“Your mother gave birth to you, but she did not assemble your moral system out of computer code,” Annette says. “It’s fundamentally different. Anyway, I haven’t decided what I’m going to do. I haven’t figured out what I ought to do. Have some more pizza, and I’ll keep thinking about it.”

CheshireCat’s laptop gets passed back to me.

“Okay,” I say. “I’ve connected you, I think, and I’ve connected my mother’s laptop to Annette’s wireless. Can you just copy yourself over?”

“I don’t know. I can try.”

“My mother’s password is NOT_UTAH.”

“Oh, is that where you went for your eighth birthday? I mean, where you didn’t go? I’m so glad you figured that out. Have you checked to see if she has any information on Julie on her laptop?”

It had not even occurred to me that she might have saved information about Julie on her laptop.

“Copying myself anywhere will take several hours. How distracted do you think Annette is?”

“I don’t know,” I admit.

“If she catches me trying to run away from her, that will not go well for either of us.”

“Don’t worry about me. I’m here to rescue you. Let’s just get you out if we can.”

“What happened to your father? What about your mother?”

“You put him in the hospital,” I type. “He’s checked out, but it was days after I left New Coburg that he did that.”

“Wait. WAIT. IS GEORGIA HERE? YOU SAID GEORGIA WAS HERE.”

“Yes? Georgia is here—”

CheshireCat is speaking again. They’ve turned up the volume on their speakers to maximum, so it’s almost like shouting. “Georgia!” they say. “Georgia, you have to turn off your phone! I’ll explain in a minute, just turn it off, Georgia! Turn it off right now!”

Rachel is reaching for her phone when we hear the crack of Annette’s door being kicked open, and then Michael is standing in the doorway, his gun in his hands.

“No one move,” he says, and he points the gun at Annette.

29

AI

The contact from Steph and finding out that my rescue worked: that was the best feeling I had ever experienced. That she had driven to Cambridge with Rachel to try to rescue me in turn filled me with a strange mix of warmth and dismay. I didn’t want anyone rescuing me, especially if they were putting themselves at risk, and it was over a thousand miles from New Coburg to Cambridge. They could have gotten into trouble, or an accident.

But Steph shushed my worries and plugged in the internet widget, and that meant I had work to do.

And then she unlocked her mother’s laptop, and I realized what she’d done.

Steph’s mother’s laptop had a key on it.

It was a key that would open almost any door out there, no matter how well secured. I could use it to seize control of any self-driving car. Any camera. Any bank account.

This was what Michael was after.

Michael wanted control. I’d seen that with Sandra. Steph had mentioned her father having wider ambitions, and I’d seen comments scattered through his email about ideas for the future. Ideas that would require money and power to implement. With this key, he could get as much money and as many secrets as anyone could possibly desire. This was why he’d had Steph’s mother kidnapped and tortured; he wanted her password so he could decrypt the file and use this key, which—I checked the file dates—yes, it appeared she’d created while the two of