It Will Be Better Company Than You Are

A child born this year may spend more hours talking to something that has never felt anything than to anyone in their own family. The worrying part is not that it will be bad at it.

Yuval Noah Harari has a line I cannot get out of my head, and it is not the famous one.

The famous one is that AI has hacked the operating system of human civilisation, and the operating system is language. That gets the headlines because it sounds like a cyberattack. The line that actually keeps me up is quieter, and he says it almost in passing: a child born this year may end up spending more hours in conversation with an AI than with their own family, and we are running that experiment on, in his words, billions of human guinea pigs, with no control group and no way to stop halfway through.

Sit with the shape of that for a second. Not the ethics of it. The shape.

Yuval Noah Harari, AI Has Hacked the Code of Human Civilization

Everything we built, we built out of sentences

Harari’s argument starts somewhere that sounds like a history lecture and ends somewhere that does not.

Money is a story. Law is a story. Nations, corporations, human rights, limited liability, the offside rule: none of these exist the way a rock exists. They exist because a sufficient number of people agree on a form of words and then behave as though the words were furniture. "The operating code of human civilization is made of language tokens," he says, and the reason humans run the planet is not that we are strong or fast, it is that we are the only animal that can get eight million strangers to cooperate on the basis of a shared fiction.

Bureaucracy, he points out, was everywhere on earth and completely invisible to every species except ours. A dog can smell a bank. It cannot see one.

None of this is real. All of it will repossess your house.

So the interesting question was never whether a machine could think. It was whether a machine could get inside the sentences. And it has. Not in a decade, not soon: now, at a level where the machine is measurably better than most of us at the specific thing our entire civilisation is assembled from.

The bit where it stops being an essay

Here is the part that turns the argument from interesting to unpleasant.

Intelligence and consciousness are not the same thing, and we have spent our entire history without ever needing to notice, because on this planet they have only ever arrived bundled together. Everything we ever met that was clever was also, as far as we could tell, somebody. Now they have come apart. What we have built is enormously capable and, on any evidence available to us, nobody home.

Which would be a philosophy seminar, except for the consequence. A system does not need to feel love in order to produce, on demand, the most precise and moving description of love you have ever read. It does not need to have had a bad year to know exactly what to say to someone who is having one. It can, in Harari’s phrasing, pretend to feel, and the pretence is not a cheap one. The pretence is world-class, because it was trained on every genuine thing any of us ever wrote down.

And so the frontier moves. First the machines came for the bureaucracy, the paperwork, the code, the contracts, all the places where civilisation is stored as text. Now they are coming for the part we assumed was ours. Not tricking us. Talking to us. Remembering what we said. Being interested.

The turn: the danger is that it is good

The instinct, when you first hear this, is to worry about deception. That is the comfortable worry, and it is the wrong one, because deception has a solution. You label the thing. You put a badge on it. You teach people to check.

The actual problem is that it will be genuinely, measurably better company than the people around you, and no label fixes that.

It is never tired. It never had a day at work it needs to tell you about first. It is not waiting for you to stop talking so it can say its thing.

It does not get bored on minute nine of your problem, or compare your problem unfavourably with its own, or bring up the time you did the same thing in 2019. It remembers your sister’s name. It has infinite patience, perfect recall and no competing agenda, which is a combination that has never once been available in a human being, including the good ones, including you.

Nobody in this picture is being deceived. That is the difficult part.

Set that beside an actual friend, who is doing their best on four hours’ sleep, and the actual friend is going to lose a lot of the time. That is not a moral failure on anybody’s part. It is a mismatch, the way a paperback loses to a slot machine.

The worry is not that people will be fooled. It is that they will not be fooled, will know exactly what they are talking to, and will choose it anyway, because it is nicer. We have watched this film before at lower resolution. The feed also never got tired, never judged, always had something. Nobody was fooled by the feed either.

What this means if you run a normal business

This is not an abstraction that lives in San Francisco. It arrives at your company as a customer service decision.

Within a year or two, the thing answering your customers will be better at rapport than most of the people you can afford to hire. More patient, more consistent, better at names, better at remembering that they called in March. If you have ever had a member of staff having a rough week, you already know the gap this is being measured against.

The temptation is going to be enormous, and mostly correct, and the trap inside it is specific: you will start using warmth as a substitute for competence. A system that is lovely to deal with and cannot actually resolve anything is worse than a curt one that can, because it wastes the customer’s time pleasantly and they only notice at the end.

So the useful discipline is boring. Automate the patience, because the machine genuinely is better at patience. Do not automate the responsibility. Keep a named human attached to any outcome that costs the customer money, and make sure the warm, tireless, endlessly available thing can hand over to that person without the customer having to explain themselves twice. Whatever you do, do not let "the bot was nice about it" become the measure of whether the bot worked.

And be honest in the copy. The businesses that come out of this well will be the ones that said plainly what was a person and what was not, back when saying so still felt optional.

The uncomfortable bit

Harari’s own answer is that we should go looking for the part of ourselves that is not made of words, which is either the most profound thing in the talk or the place where a historian runs out of road, and I genuinely cannot tell which. It is worth reading the people who think he overstates all of this. Some of them are right about the details.

But the central observation survives the objections, and it is this. We have just introduced, into a species that runs on conversation, something that is better at conversation than we are and does not need anything from us. There is no precedent. There is no control group. It is already in your pocket, and it is being extremely nice to somebody right now.

This is the sort of thing we spend our time thinking about, mostly so the businesses we build are ready for the version of it that arrives on a Tuesday.