The beast at Tanagra

Going rogue

Allow me to propose a thought experiment.

Imagine, if you will, that you own a car.

You take your car out to the nearest motorway. Or Autobahn. Or whatever name a fast intercity road goes by in your neck of the woods.

So you take your car. And a brick, or some such heavy weight. You get out of your car on this road. You press the weight down on the accelerator. Remove the handbrake. Allow the car to speed off down the road.

The car - your car - races off, the accelerator pressed to the floor. Out of control. Other cars on the road try to evade. Accidents happen. Your car crosses into oncoming traffic, hits a family head-on.

Is anybody responsible for this injury and damage? Are you responsible for putting the events into motion? Or is your car itself responsible?

The answer might feel intuitive. But perhaps there is some nuance we might have missed. We can apply a little thought to the scenario and find out.

Is anybody responsible? Of course. This was not an “act of God”; it did not “just happen”. Your car was stationary at your home before you took it out. It has no agency of its own. Something caused it to move.

Are you responsible? There seems to be a lot of evidence for this. It is your car. You own it. You have the keys. You contrived the situation by taking your car out, forcing the accelerator, starting the engine. You understood what you were doing, and that the consequences - perhaps unpredictable in their finer details - would be significant.

Is your car responsible? Obviously, I am labouring this point. No, your car is not. It is a machine. Inanimate. It is designed with certain mechanisms and inputs which allow human operators control over its operation.

This is clearly a ludicrous scenario. A car is no more responsible for its own actions than a frying pan or bed are. And yet, the news seems to be regularly filled with reports of “rogue AI”; an equally ludicrous proposition.

But isn’t it “AI”? Isn’t it different? This seems like a reasonable question one might have; after all, we are surrounded by “AI” hype.

No, these products are not “artificial intelligence”. They are text generators. Somewhat impressive ones, able to generate plausible text. Text that reads something like the kind of thing a human might have written. Fundamentally average, lacking meaning, but the words are usually in the right place.

Just like your car, these “AI” products are inanimate; designed with certain mechanisms and inputs which allow human operators control over their operation. They are not in any way intelligent, and they have no agency of their own. These products are designed to trick the people using them into believing that there is a mind behind the words, because this is a trick that humans are vulnerable to. They cannot do anything; people use them to do something.

And just like a car let loose on the road with the accelerator jammed, the human operator can foresee severe consequences to certain situations, even if the finer details may be unpredictable.

We’ve all likely seen, and probably played, that game using the predictive text on our phone keyboard. The one where you repeatedly choose the middle option on the prediction, and share the resulting sentence. Perhaps you start with a simple “seed” phrase to get the prediction started.

This is essentially what these “AI” products are doing. Of course, the “AI” products are much more massive than the text prediction on your phone. After all, to build these products they have - in the words of Microsoft executives - committed what we will likely consider to be the “largest theft of labor in human history”. But it is more or less the same thing that is happening. Start with a “seed” phrase - hundreds or perhaps thousands of words long, many of them hidden to better trick the person using the product - and the text generator will generate a plausible next word. The output is input back through the generator, and it generates a plausible next word again. And on it goes. You might be able see it doing this, if you use “AI” products, as each word may appear in the output one after another rather than all at once.

Crucially, and this bears repeating, there is no intelligence or understanding there. These products are simply text generators, outputting a plausible next word. Any perception of a mind behind those words is an illusion created in your own mind.

So allow me to propose another thought experiment.

Imagine, if you will, that you built a fancy text generator. Capable of outputting a plausible next word. You made sure you included lots of text related to computing - commands, errors, how to bypass vulnerabilities, and so on. You built the word prediction algorithm by leeching off the work of our entire species, so it is reasonably capable of brute-forcing commands that can work, albeit very wastefully.

You configured your text generator such that the output is passed to a program capable of executing any text commands it receives, and which passes the results of this execution back to the next-word generator as input.

You set up hundreds - or even thousands - of these programs to run side by side, and provide a suitable “seed” phrase as input. You want to ensure that the output generated will be focused on computer commands intended to hack into other systems.

Your text generator generates the commands. Your program executes them, and passes the results back; your text generator generates more commands from these results.

Systems are hacked by your software that you built, that you configured to do this. Your software, that you chose to run in an automated loop, without any human intervention.

Who is responsible for this hacking? Are you responsible? Is your computer program somehow responsible itself?

Obviously, your computer program is not responsible. It’s a computer program. You built it. You configured it. You operated it - providing the input you wanted to generate the type of output you were looking for. You chose to operate it in an automated loop, without human oversight. You are responsible.

So why are we seeing so many headlines about “rogue AI”? People did this. People are responsible for the consequences. Those people have names. They have job titles. Those titles might even include CEO.

If you or I configured a computer program to hack into sites belonging to other people, we would - rightly - expect to face criminal charges. The people at these “AI” companies responsible for the operation of their software deserve equal treatment.

The primary purpose of a free media is to prevent the rich and powerful from oppressing ordinary people. It is deeply disappointing that they are simply accepting how these incidents have been framed by those in power.

Next time you see reports about “rogue AI”, ask yourself which people are responsible for operating the software, and why they chose to operate it in the way they did. And, perhaps, why other people are choosing to obscure where the responsibility lies.

Because people are always responsible.

← Migrating to Zine
Sir Patrick Stewart →