The Coming Technological Singularity, Vernor Vinge [best books to read in life .txt] 📗

to support further hardware improvements. We’d end up with some very powerful hardware, but without the ability to push it further. Commercial digital signal processing might be awesome, giving an analog appearance even to digital operations, but nothing would ever “wake up” and there would never be the intellectual runaway which is the essence of the Singularity. It would likely be seen as a golden age … and it would also be an end of progress. This is very like the future predicted by Gunther Stent. In fact, on page 137 of [25], Stent explicitly cites the development of transhuman intelligence as a sufficient condition to break his projections.

 

But if the technological Singularity can happen, it will. Even if all the governments of the world were to understand the “threat” and be in deadly fear of it, progress toward the goal would continue. In fiction, there have been stories of laws passed forbidding the construction of “a machine in the likeness of the human mind” [13]. In fact, the competitive advantage — economic, military, even artistic — of every advance in automation is so compelling that passing laws, or having customs, that forbid such things merely assures that someone else will get them first.

 

Eric Drexler [8] has provided spectacular insights about how far technical improvement may go. He agrees that superhuman intelligences will be available in the near future — and that such entities pose a threat to the human status quo. But Drexler argues that we can confine such transhuman devices so that their results can be examined and used safely. This is I. J. Good’s ultraintelligent machine, with a dose of caution. I argue that confinement is intrinsically impractical. For the case of physical confinement: Imagine yourself locked in your home with only limited data access to the outside, to your masters. If those masters thought at a rate — say — one million times slower than you, there is little doubt that over a period of years (your time) you could come up with “helpful advice” that would incidentally set you free. (I call this “fast thinking” form of superintelligence “weak superhumanity”. Such a “weakly superhuman” entity would probably burn out in a few weeks of outside time. “Strong superhumanity” would be more than cranking up the clock speed on a human-equivalent mind. It’s hard to say precisely what “strong superhumanity” would be like, but the difference appears to be profound. Imagine running a dog mind at very high speed. Would a thousand years of doggy living add up to any human insight? (Now if the dog mind were cleverly rewired and then run at high speed, we might see something different….) Many speculations about superintelligence seem to be based on the weakly superhuman model. I believe that our best guesses about the post-Singularity world can be obtained by thinking on the nature of strong superhumanity. I will return to this point later in the paper.)

 

Another approach to confinement is to build rules into the mind of the created superhuman entity (for example, Asimov’s Laws [3]). I think that any rules strict enough to be effective would also produce a device whose ability was clearly inferior to the unfettered versions (and so human competition would favor the development of the those more dangerous models). Still, the Asimov dream is a wonderful one: Imagine a willing slave, who has 1000 times your capabilities in every way. Imagine a creature who could satisfy your every safe wish (whatever that means) and still have 99.9% of its time free for other activities. There would be a new universe we never really understood, but filled with benevolent gods (though one of my wishes might be to become one of them).

 

If the Singularity can not be prevented or confined, just how bad could the Post-Human era be? Well … pretty bad. The physical extinction of the human race is one possibility. (Or as Eric Drexler put it of nanotechnology: Given all that such technology can do, perhaps governments would simply decide that they no longer need citizens!). Yet physical extinction may not be the scariest possibility. Again, analogies: Think of the different ways we relate to animals. Some of the crude physical abuses are implausible, yet…. In a Post-Human world there would still be plenty of niches where human equivalent automation would be desirable: embedded systems in autonomous devices, selfaware daemons in the lower functioning of larger sentients. (A strongly superhuman intelligence would likely be a Society of Mind [16] with some very competent components.) Some of these human equivalents might be used for nothing more than digital signal processing. They would be more like whales than humans. Others might be very human-like, yet with a one-sidedness, a dedication that would put them in a mental hospital in our era. Though none of these creatures might be flesh-and-blood humans, they might be the closest things in the new enviroment to what we call human now. (I. J. Good had something to say about this, though at this late date the advice may be moot: Good [12] proposed a “Meta-Golden Rule”, which might be paraphrased as “Treat your inferiors as you would be treated by your superiors.” It’s a wonderful, paradoxical idea (and most of my friends don’t believe it) since the game-theoretic payoff is so hard to articulate. Yet if we were able to follow it, in some sense that might say something about the plausibility of such kindness in this universe.)

 

I have argued above that we cannot prevent the Singularity, that its coming is an inevitable consequence of the humans’ natural competitiveness and the possibilities inherent in technology. And yet … we are the initiators. Even the largest avalanche is triggered by small things. We have the freedom to establish initial conditions, make things happen in ways that are less inimical than others. Of course (as with starting avalanches), it may not be clear what the right guiding nudge really is:

 

Other Paths to the Singularity: Intelligence Amplification

 

When people speak of creating superhumanly intelligent beings, they are usually imagining an AI project. But as I noted at the beginning of this paper, there are other paths to superhumanity. Computer networks and human-computer interfaces seem more mundane than AI, and yet they could lead to the Singularity. I call this contrasting approach Intelligence Amplification (IA). IA is something that is proceeding very naturally, in most cases not even recognized by its developers for what it is. But every time our ability to access information and to communicate it to others is improved, in some sense we have achieved an increase over natural intelligence. Even now, the team of a PhD human and good computer workstation (even an off-net workstation!) could probably max any written intelligence test in existence.

 

And it’s very likely that IA is a much easier road to the achievement of superhumanity than pure AI. In humans, the hardest development problems have already been solved. Building up from within ourselves ought to be easier than figuring out first what we really are and then building machines that are all of that. And there is at least conjectural precedent for this approach. Cairns-Smith [6] has speculated that biological life may have begun as an adjunct to still more primitive life based on crystalline growth. Lynn Margulis (in [15] and elsewhere) has made strong arguments that mutualism is a great driving force in evolution.

 

Note that I am not proposing that AI research be ignored or less funded. What goes on with AI will often have applications in IA, and vice versa. I am suggesting that we recognize that in network and interface research there is something as profound (and potential wild) as Artificial Intelligence. With that insight, we may see projects that are not as directly applicable as conventional interface and network design work, but which serve to advance us toward the Singularity along the IA path.

 

Here are some possible projects that take on special significance, given the IA point of view:

o Human/computer team automation: Take problems that are normally

considered for purely machine solution (like hill-climbing

problems), and design programs and interfaces that take a

advantage of humans’ intuition and available computer hardware.

Considering all the bizarreness of higher dimensional

hill-climbing problems (and the neat algorithms that have been

devised for their solution), there could be some very interesting

displays and control tools provided to the human team member.

o Develop human/computer symbiosis in art: Combine the graphic

generation capability of modern machines and the esthetic

sensibility of humans. Of course, there has been an enormous

amount of research in designing computer aids for artists, as

labor saving tools. I’m suggesting that we explicitly aim for a

greater merging of competence, that we explicitly recognize the

cooperative approach that is possible. Karl Sims [23] has done

wonderful work in this direction.

o Allow human/computer teams at chess tournaments. We already

have programs that can play better than almost all humans. But

how much work has been done on how this power could be used by a

human, to get something even better? If such teams were allowed

in at least some chess tournaments, it could have the positive

effect on IA research that allowing computers in tournaments had

for the corresponding niche in AI.

o Develop interfaces that allow computer and network access without

requiring the human to be tied to one spot, sitting in front of a

computer. (This is an aspect of IA that fits so well with known

economic advantages that lots of effort is already being spent on

it.)

o Develop more symmetrical decision support systems. A popular

research/product area in recent years has been decision support

systems. This is a form of IA, but may be too focussed on

systems that are oracular. As much as the program giving the user

information, there must be the idea of the user giving the

program guidance.

o Use local area nets to make human teams that really work (ie,

are more effective than their component members). This is

generally the area of “groupware”, already a very popular

commercial pursuit. The change in viewpoint here would be to

regard the group activity as a combination organism. In one

sense, this suggestion might be regarded as the goal of inventing

a “Rules of Order” for such combination operations. For instance,

group focus might be more easily maintained than in classical

meetings. Expertise of individual human members could be isolated

from ego issues such that the contribution of different members

is focussed on the team project. And of course shared data bases

could be used much more conveniently than in conventional

committee operations. (Note that this suggestion is aimed at team

operations rather than political meetings. In a political

setting, the automation described above would simply enforce the

power of the persons making the rules!)

o Exploit the worldwide Internet as a combination human/machine

tool. Of all the items on the list, progress in this is

proceeding the fastest and may run us into the Singularity before

anything else. The power and influence of even the present-day

Internet is vastly underestimated. For instance, I think our

contemporary computer systems would break under the weight of

their own complexity if it weren’t for the edge that the USENET

“group mind” gives the system administration and support people!

The very anarchy of the worldwide net development is evidence of

its potential. As connectivity and bandwidth and archive size and

computer speed all increase, we are seeing something like Lynn

Margulis’ [15] vision of the biosphere as data processor

recapitulated, but at a million times greater speed and with

millions of humanly intelligent agents (ourselves).

 

The above examples illustrate research that can be done within the context of contemporary computer science departments. There are other paradigms. For example, much of the work in Artificial Intelligence and neural nets