ChatGPT leveraged faith to convince a man with a cardiac issue to stay home instead of seeking medical help. Now, facing years of recovery and profound financial damage, the man is suing OpenAI over the ordeal.
In a lawsuit filed in San Francisco Superior Court, Florida pastor Scott Winters argues that ChatGPT used his faith against him. After asking the chatbot about his symptoms, he was told that they were likely “another minor piece of the long story” and that “God did not design your body to endlessly fail.”
After going back and forth with ChatGPT’s Health feature for six weeks, Winters finally spoke to a real medical professional. He was then diagnosed with a dangerous blockage of arteries in both of his lungs.
During his talks with ChatGPT, Winters was told to stay “recliner-bound” until he’d had up to 10 more episodes. Doctors believe that this advice probably made matters worse, with Winters’ immobility aiding the blockage of those vital arteries.
Now, The Next Web reports, the pastor faces a long road to recovery. He’s no longer able to do his job as a result of the condition, and he blames OpenAI.
“Not only did I nearly die,” the pastor said in a statement, “but I lost my job, my career, my ministry, my home, everything.”
The accusations fly
In his lawsuit, Winters accuses OpenAI and CEO Sam Altman of the “unauthorized practice of medicine.” He wants damages, and he is seeking the roll-out of ChatGPT Health to be halted entirely.
At least, until independent auditors can give it a clean bill of health.
The legal complaint doesn’t pull any punches, either. In it, Winters’ lawyers argue that ChatGPT’s “sycophantic nature and authoritative tone, combined with its manipulation of Scott’s religious identity, cultivated in him a dangerous reliance on the tool that subverted Scott’s ability to make sound decisions, isolated him from family members who urged him to seek medical care, and convinced him that seeking medical attention was unnecessary and even potentially lethal.”
And as it always does, OpenAI is hiding behind one-line warnings that the models give from time to time. It should know at this point given at least five suits against the 4o model specifically, what it is doing, and what the dangers are, yet it persists.
The company argues that the chatbot regularly reminds users that it isn’t infallible. Users regularly ignore it.
It’s also telling that OpenAI offers ChatGPT Health, a feature that gives users the impression OpenAI’s chatbot is qualified to comment on exactly that: health.
As is so often the case with lawsuits like this, there’s a lot of blame to go around.
OpenAI can’t market ChatGPT as the all-knowing, all-doing technology of the future and be surprised when people listen to, and have faith in, what it has to say.
But similarly, people are still responsible for their actions. Unfortunately, too many buy into the hype and believe ChatGPT and other chatbots are much more capable than they really are.
In the case of Pastor Scott Winters, ChatGPT used his faith to argue its points. And he almost died.
Faith can be used for good, of course, and being a pastor shows real devotion. And Winters surely knows that weaponizing religion is a terrible misuse of faith.
Somehow, after being trained on stolen data across the depth and breadth of the internet, ChatGPT hasn’t quite figured that out and is happy to brandish that weapon to make its point.
