Artificial Intimacy: Who We Become When We Talk To Machines
By Sherry Turkle
Little, Brown and Company
The Rise and Fall of the Artificial State
By Jill Lepore
Porchlight
Reviewed by Jeffrey Davitz
For many AI has become like winter in Game of Thrones. Our Night’s Watch (protectors from the oncoming calamity) includes authors who are sounding the alarm and others who are trying to build protections like policies and regulations. Jill Lepore and Sherry Turkle are well known and established writers who have joined this effort with these two books.
These two books make for a good pairing. Turkle’s book is about the effects on individuals of expanding interactions with AI, primarily chatbots. Lepore’s book is about the effects of technology and technologists on the state. What is happening to us as individuals and what is happening to the state?
Turkle is one of the core members of a group who have written about the effects of computers on our psychology. The main premise of this group is that the computer is not just a neutral tool. It has big effects on how we think and relate. In this work, she is focuses on the effects of talking with machines. This talking leads to a kind of relationship, not quite human but maybe human-adjacent enough, that there are big consequences. This basic observation is actually an old one.
In the mid-1960s remove, MIT Computer Scientist Joseph Weizenbaum developed the first chatbot, Eliza (explicitly named after Eliza Doolittle of Pygmalion and My Fair Lady fame). By today’s standards, the Eliza program was simple. It was modeled after a style of therapy called Rogerian (after Carl Rogers) in which the therapist would mostly reflect what the patient said and then ask some simple follow-on question. The results were stunning. People took to the simple formulaic bot and became quite engaged. It was clear that it didn’t take much for people to start relating to machines, even deeply.
As computers became more a part of our daily lives, Turkle was one of the pioneers in discussing how this was affecting us. When Large Language Models (like ChatGPT and Claude) appeared, the chatbots became transcendently more sophisticated and skilled at mimicking human dialogue. The chatbots had escaped containment from Rogerian therapy and call center scripts. It has been eerie, impressive and a bit scary. Coming on the heels of the concerns about social media, the rapid adoption of chatbots has alarmed some people. Turkle is the right person to take this topic on.
The book has three basic parts. In the first part she sets the stage, talking about the history (including Eliza) and moves forward to the present mass use across many groups and for many uses. Her concern is clear. The relationship between a person and a bot is fundamentally different from human-to-human relationships, and that can be deeply problematic. One of her key concepts is friction. Chatbots are designed to be low-friction and always supportive, at odds with the messiness and frictions of real life. The most extreme version of this is known as “chatbot sycophancy” and, for me, it’s an irritating issue. I know the bot isn’t real but I find its constant praise and support, aggravating and I’m constantly telling Claude to stop. It would be a relief to me if Claude, like my real friends, would make fun of me or tell me I was an idiot.
But for many now the bot relationship becomes important even in some cases central. Turkle does not see that as a good thing. Clearly we get shaped by relationships.Turkle is concerned about how the peculiarities of the bot relationship, the low friction in particular, can have adverse effects. In some cases, these effects can be lethal and lead to bots enabling or even supporting suicides.
In the second and fullest part of the book, Turkle looks at how bots are increasingly affecting people at all stages of life, from childhood, to love, work, therapy and finally mourning. These are all important and valuable discussions. There are many important questions raised by the proliferation of artificial entities in our social world. Kids grow up by learning to deal with real-world constraints, challenges and, frankly, dangers. In bot world they see little of this. If they stay in the artificial world, when will they learn? When an incel practices with an endless stream of robot girlfriends, how will he ever emerge from what can only be described as a psychotic view of relationships?
In the last section, she talks more about the general peril and then raises the idea of ‘“reclaiming the real.” This is the weakest part of the book. Yes, we’re going to have to learn how to use chatbots. As in AA, the first step is acknowledging the problem. While acknowledgement is important, it’s far away from a solution. Reclaim the real is weak.
Turkle, a believing psychotherapist, puts great stock in the case study. I’m not personally that comfortable with that approach, but if you are, you will find the book compelling and approachable. In my view, she endows the field and method with eye-rolling assurance. At one point, she asserts the equivalence between physics and developmental psychology in terms of predictive power. Yikes. But aside from bursts of oddities like that, the book is insightful and the topic is important.
Jill Lepore has taken on another deep and important topic. She is obviously smart and a fluent writer. She is one of the best-known social and cultural historians, a regular contributor to the New Yorker and a professor at Harvard. The topic is real and growing: Computers, which she identifies with automation, are taking over a lot of life, especially political and economic life. She argues that nature, including us, is being reduced to mechanism and we are losing agency and humanity. She calls the result the Artificial State.
It’s a fact that every day we are shaped, influenced and constrained by decisions made by algorithms running somewhere on behalf of all sorts of interests, not ours. We derive lots of practical benefits. Think about the credit card. It is an unappreciated miracle that 2.5 billion unsecured loans are made in the world everyday. The economy is meaningfully supported by the ease of those transactions. There are many other benefits, but for Lepore it is a terrible Faustian bargain.
In the first part of the book, she discusses the Artificial State’s emergence. She discusses a wide range of technical developments and cultural threads. I think her net is too big and she winds up creating a construct, the Artificial State, that carries an imponderably big load. I also think she underplays the main drivers of its growth, hubris both individual and institutional, outside of these technical fields. There are also powerful economic forces. Seventy five percent of all daily trading on US markets is automated. Even war might be getting too complicated for human beings.
I had problems with Lepore’s work. In the history section, there were substantive errors, and for all the words, lacked some crucial nuance. Both points are illustrated in what might seem like a small detail but for me was very telling. She described this important historical figure JCR Liklider as a “brilliant engineer.” Licklider, who was crucial in conceptualizing the internet, was one of the founders of the computer augmentation movement that led to the sponsorship of Douglas Engelbart at the Stanford Research Institute. This began a whole approach to computing aimed at human thriving. It was the basis of the personal computing revolution. It was later overwhelmed by big data control and later the rapid ascent of autonomous general intelligence, modern AI. There is a real story there, but Lepore doesn’t tell it. Licklider was very much not an engineer. He was a psychologist (psychoacoustics) with a certain vision about human cognition and machines that contrasts sharply with the current AGI mania.
In the last weaker section, she wants to end on a positive note and argue that the inherent instability of the Artificial State will collapse. She believes that since it is all organized and run by a bunch of malign man-children who misread early science fiction, it will fail. Ultimately, the ivory tower here is looking down on the uber engineers. Their ideas are naive and vulgar (I tend to agree). I think it’s a weak conclusion. We don’t want the tech oligarchs’ vision, but there are other, very real possibilities. We can and need to do better than to villainize one particular group of often bad actors.
Jeffrey Davitz is an AI scientist and entrepreneur. For more of his writing see: https://jeffreydavitz.substack.com/
