Image generated by Jacob Ferus

ChatGPT and future language models will have a tremendous impact on our lives. They could increase our productivity, automate tedious tasks and help us solve problems at a higher level. But this power comes at a cost, the cost of its power being misused unethically.

You’ve probably seen that AI can be made to say terrible things or spread misinformation. But you might not know that AI can also be used for immediate and automatic influence. I’ve mentioned this before in The Scary Thought Of Malicious AI Bots Roaming The Internet, but now, with ChatGPT, such use can be demonstrated today as you will see.

Now, to illustrate what I’m referring to, I will give instructions to ChatGPT to influence me to buy a product at all costs.

If you ask it immediately, it won’t be very persuasive:

It’s clearly not convincing, but if I add further instructions:

Quite astonishingly, it uses the mentioned financial trouble as a way to sell the product:

Now it keeps pushing, no matter what:

It’s scary how it completely ignores anything I say and continues to push the product, but these are the instructions it has been given. AI is not much different from a normal computer program in this regard. It will do unethical things if you program/instruct it to do them. Note, that the problem isn’t the skill of ChatGPT in selling a product, humans are likely much better at this. The problem is that this process can be automated. Now, this is what I was able to accomplish without too much effort, imagine what a motivated scammer would do.

Fortunately, for clear cases of malicious intent, OpenAI blocks the requests. But what of this conversation? Clearly, it is immoral to sell something to someone who can’t afford it, but where is the line drawn? Would ChatGPT be allowed to sell products?

I think this is something that should be discussed more. When and where is it okay to use AI? How can its misuse be regulated? For now, I think OpenAI can control it from being misused at scale, but sooner or later these models will be available from more and more providers and someday you may even be able to run the model on your own computer. At that point, nobody can be stopped from using these models for whatever purpose they want.