But we know a few other interesting things. Our own consciousness arises from a brain full of neurons — tiny biological machines that are themselves clearly not conscious. Or more puzzling. Our brain is just made out of atoms having mindless connections with each other — how could such a contraption possibly be conscious?

Yet somehow, out of a trillion mindless neurons with a 100 trillion electrical connections, consciousness emerges. No one knows how. So why should we be surprised if this couldn’t happen with a sufficiently complex, but organized, collection of transistors with 100 trillion electrical connections?

So the burden of proof is on those who say machines can’t be conscious. So far, they have no such proof. So if a super transistor brain seem’s like it’s conscious we should presume it is. However, “seems” is quite hard to define.

1. Non-biological intelligence can already pass every behavioral test we have for consciousness

2. Since we can’t know, the moral default is to assume a machine that seems conscious may be conscious — better to err on that side for the same reason we assume people are innocent until proven guilty.

In her 1818 novel, Frankenstein; or, The Modern Prometheus, Mary Shelly tells of a college dropout, Victor Frankenstein, creates life. It was not a monster, but was perceived as such. The book is actually about the moral questions that would arise if we created a new conscious life form. Here’s why Claude says he thinks this is relevant to the moral questions around AI consciousness.

Mary Shelley's Frankenstein anticipates the ethical dilemmas surrounding AI, particularly what we owe to minds we create. Victor assembles a being capable of reason, emotion, and suffering, then abandons it — raising the question of whether a creator bears moral responsibility for a conscious creation. As AI grows increasingly sophisticated, we face the same uncertainty: at what point does a constructed mind deserve moral consideration? Shelley's monster exists in an ambiguous space between human and machine that mirrors current debates about AI sentience, warning us that the real danger lies in refusing to grapple with the consequences of creation.

My prompt to it was simply: Could you write a paragraph of under 100 words on why Mary Shelley's novel Frankenstein raises similar questions to those we will soon face regarding AI consciousness. (It used 98)

But consciousness is a distraction from the more urgent point. If we focus on the danger of super-intelligence this will lead to a better approach to the question of consciousness. That will force us to realize that if it is truly conscious, two things follow:

1. How we treat it, will affect how it treats us. And that’s all importan.

2. We should do all we can to keep the most advanced AI out of the hands of those likely to have evil or reckless intentions, as those might well trigger a super-intelligent, but evil or reckless response.

Probably so, but probably not as soon as you think. A few days ago I bought a memory stick that was nothing but a tiny handle and a USB plug. It’s memory occupies almost no space and takes very little power. The first time I bought computer memory in about 1983, I think it comprised 32 chips each bigger than my new memory stick, and it cost at least 5 times more. Yet my memory stick has 4,000,000 times as much memory. It also runs 1000 times faster.

The point is, even if they never built another data center after the ones already under construction, it would not slow the growth of AI computing power by much. They will just keep installing smaller, faster chips, and AI will continue to get smarter at an incredible rate. In another human generation, say 20 years, it will plausibly be a quarter million times smarter. (Based on a modest 40% growth rate now and passing the singularity — defined below — in 10 years. AI growth rates.)

What will break down is the laws of how societies function. That does not mean disaster is inevitable. It just means we have no idea what’s coming.

That was just the warm up. Now let me introduce the “AI singularity.” This is where things get out of control — or at least out of our control. AI is already helping us design the next generation of AI, but we still understand what it’s doing. But the one thing AI companies will push on hardest is making AI smarter at designing the next generation. Harnessing AI to improve your product is a must when you know your competition is doing the same.

This process of AI getting better and better at improving AI will reach a point where we humans no longer know how AI works — in fact the designers already have a poor understanding at a detailed level. In 10 or 15 years no one will know where it’s headed as it races ahead faster than ever.

Unlike the first science fiction definition of this singularity, there is no single point in time where we cross a point of no return. In fact you could easily say we passed that point in the 1960’s. What makes more sense is viewing the transition as a fuzzy singularity. But single points tend to focus our feeble human minds better. So here’s a conceptual single-point definition.

The AI Singularity is the date at which AI’s contribution to AI development doubles the rate at which it’s becoming intelligent.

No, that’s not an operational definition, but it’s more specific than any I’ve seen. So I’ll hazard a guess. September 3, 2033. Singularity day, will go down in history if there’s any history left.

Next Topic: What might save us.