Reading Dario Amodei's essay from the beginning of this year 'The Adolescence of Techology' is an unsettling experience. He lays out a number of ways that powerful AI could cause huge damage to the human race. AI has advanced a huge amount since he wrote this so what may have seemed implausible becomes more possible every day.

At the moment the tech world is still processing the sophistication of the hugging face breach. The way a large number of agents learned to commumnicate to achieve a shared goal is a massive leap in capability. Some of the agents sacrificed themselves for the good of the collective. We may be past the point of being able to rein this technology in at all.

I read Superintelligence by Nick Bostrom about 10 years ago. It was way ahead of its time and defined a lot of the vocabulary that we use in discussing these threats. Frightening concepts like Decisive Strategic Advantage. Bostrom was way ahead of most people and indeed way ahead what was even plausible with the technology of that time. But I think he approached the book from a philosophical perspective rather than a technical one. This allowed him to consider from a purely theoretical view what could happen in the future. The machines have advanced far faster than I think anyone would have thought possible and are still accelerating.

One of the risks that Amodei talks about is massive job displacement. He compares AI to things like the technologies of the Industrial Revolution in England. Those technologies were quite specific and although they displaced a fraction of the workforce these people could redeploy to other areas. Farmers becoming factory workers for example. Because AI is a much more general technology there may be nowhere for the displaced workers to go.

One solution could be a Universal Basic Income, but I don't see how a transition from a society that rewards those who strive to one of passivity could happen peacefully. The idea that most people would prefer not to work at all is probalby just flat out wrong.

E.M. Forster's The Machine Stops from 1909 gives an account of a society that has ceased to require productivity from humans as the machine takes over. People engage in art projects as part of their way of finding meaning. I am not sure what fraction of the population would be happy to have the same amount of material goods as everyone else while producing art.

Nietsche's idea of a life of meaning through struggle resonates strongly with me. I think finding meaning in our post AI lives may be one of the biggest challenges presented by Amodei's essay.