As an academic polymath with interests in physics, computer science, and economics, Hanson draws upon his extensive reading across these fields to try to piece together what such a society will look like.
His argument is that in 30 years to a century, there will be a phase transition as mind uploading takes off and the world economy rapidly becomes dominated by “ems” (emulations); human brains running on a silicon substrate, and potentially millions of times faster. Since transport congestion costs aren’t a factor, this em civilization will live in a few very densely populated cities largely composed of cooling pipes and computer hardware. The economy will double once every month, and in a year or two, it will transition to yet another, cardinally different, growth phase and social structure.
I might or might not eventually do a book review, but for now, here is a link to Scott Alexander’s.
Alternatively, this lecture slide summarizes the main points.
A few observations, arguments, and counterarguments from the meeting:
(1) This struck many people as the most counterintuitive assetion, but I agree that wages in the em world should quickly plummet to subsistence levels (which are much lower than for biological organisms). This is probably what will happen eventually with our civilization if there is no “singularity”/transition to a higher growth phase, since fertility preferences are an aspect of personality, and as such, highly heritable. (Come to think of it this is basically what happens to the Imperium of Man in Warhammer 40k, down to the hive cities in which most citizens eke out “lives of quiet desperation,” though ones which “can still be worth living.”)
Since Ctrl-C Ctrl-V is much easier and quicker than biological reproduction, a regression to the historical (and zoological) norm that that is the Malthusian trap seems – barring some kind of singleton enforcing global restrictions on reproduction – seems inevitable.
(2) A more questionable claim is Hanson’s prediction that ems will tend to be more religious than humans, on the basis that hardworking people – that is, the sorts of people whose minds are most likely to be uploaded and then copied far and wide – tend to be more religious. This is true enough, but there is also a strong and well known negative correlation between religiosity and intelligence. Which wins out?
(3) The marginal return on intelligence is extremely high, in both economics and scientific dynamism (Apollo’s Ascent theory). As such, raising the intelligence of individual ems will be of the utmost priority. However, Hanson makes a great deal of the idea that em minds will be a black box, at least in the beginning, and as such largely impenetrable to significant improvement.
My intuition is that this is unlikely. If we develop technology to a level where we can not only copy and upload human minds but provide them with internally consistent virtual reality environments that they can perceive and interact within, it would probably be relatively trivial to build brains with, say 250 billion neurons, instead of the ~86 billion we are currently endowed with and largely limited to by biology (the circulatory system, the birth canal, etc). There is a moderate correlation between just brain volume and intelligence, so its quite likely that drastic gains on the order of multiple S.D.’s can be attained just by the (relatively cheap) method of doubling or tripling the size of the connectome. The creative and scientific potential of billions of 300 IQ minds computing millions of times faster than biological brains might be greater than the gap between our current world and that of a chimpanzee troupe in the Central African rainforest.
Two consequences to this. First, progress will if anything be even faster than what Hanson projects; direct intelligence amplification in tandem with electronic reproduction might mean going straight to the technological singularity. Second, it might even help ems avoid the Malthusian trap, which is probably a good thing from an ethical perspective. If waiting for technological developments that augment your own intelligence turns out to be more adaptive than making copies of yourself like Agent Smith in The Matrix until us ems are all on a subsistence wage, then the Malthusian trap could be avoided.
(4) I find this entire scenario to be extremely unlikely. In both his book and his lecture, Hanson discusses and then quickly dismisses the likelihood of superintelligence first being attained through research in AI and neural nets.
There are two problems with this assertion:
(a) The median forecast in Bostrom’s Superintelligence is for High Level Machine Intelligence to be attained at around 2050. (I am skeptical about this for reasons intrinsic to Apollo’s Ascent theory, but absolutely the same constraints would apply to developing brain emulation technology).
(b) The current state of AI research is much more impressive than brain emulation. The apex of modern AI research can beat the world’s best Go players, several years ahead of schedule. In contrast, we only finished modeling the 302 neuron brain of the c. elegans worm a few years ago. Even today, we cannot obtain functional models even of 40 year old microchips from scanning them, to say nothing of biological organisms. That the gap will not only be closed but for the brain emulation route to take the lead is a rather formidable leap of faith.
Now to be fair to Hanson, he did explicitly state that he does not regard the Age of Em as a certain or even a highly probable future. His criterion for analyzing a future scenario is for it to have at least a 1% chance of happening, and he believes that the Age of Em easily fulfills that condition. Personally I suspect it’s a lot less than 1%. Then again, Hanson knows a lot more computer science than I do, and in any case even if the predictions fail to pan out he has still managed to provide ample fodder for science fiction writers.
(5) My question to Hanson during the Q&A section of the talk: Which regions/entities do you expect to form the first em communities? And what are the geopolitical ramifications in these last years of “human” civilization?
(a) The big factors he lists are the following:
- Access to cold water, or a cold climate in general, for cooling purposes.
- Proximity to big human cities for servicing human customers (at least in the initial stages before the em economy becomes largely autonomous).
- Low regulations.
So plausible candidates (according to Hanson) would be Scandinavia, or the “northern regions of China.”
As he also noted at another point, in the early stages of em creation technology, mind uploading is likely to be “destructive,” i.e. resulting in the biological death of the person who is to be emulated. So there might be an extra selection filter for state or corporate ruthlessness.
(b) In domestic and social terms, during the transition period, humans can be expected to “retire” as the em economy explodes and soon far exceeds the scope of the old human economy. Those humans who control a slice of the em economy will become very rich, while those who don’t… fare less well.
However, Hanson doesn’t have anything to say on the geopolitical aspects of the transition period because it is much less predictable than the “equilibrium state” of the em economy that he set out to describe. As such, he does not think it is worthwhile for someone who is not a sci-fi writer to delve into that particular issue. That makes sense.
(6) As a couple of people pointed out, atomic weapons can wipe out an entire em “city,” which contain billions of ems.
What would em warfare be like? The obvious answer is cyber-cyber-cyber we gotta hack the mainframe style stuff. But surely, sometimes, the easiest move is to just knock over the table and beat your opponent to death with the chessboard.
If Pinker gets pwned during the em era and global nuclear wars between em hive cities ruled by Gandhi emulations break out, could this make em hive cities unviable and result in a radical decentralization?
(7) How did Hanson become Hanson?
He repeated the Talebian argument (which I sympathize with) that following the news is a pointless waste of time.
It is much more productive to read books, especially textbooks, and to take introductory classes in a wide range of subjects. To try to get a good grasp on our civilization’s system of knowledge, so that you might be able to make productive observations once you reach your 50s.
Confirmation bias? Regardless, it’s one more small piece of evidence in favor of my decision to log off.