Clearly, eating disorders are a big problem in this country. According to the National Association of Anorexia Nervosa and Associated Disorders, 9% of the U.S. population will have an eating disorder in their lifetime, and one American dies every 52 minutes from an eating disorder.
An organization called the National Eating Disorders Association (NEDA, for short) tried to provide advice to people about eating issues, but its helpline was often swamped, requiring long wait times.
While the fact that helpline workers were trying to form a union may have also played a factor in its decision, NEDA decided to close its helpline and roll out a chatbot. The non-profit maintained that it was the most efficient way for it to offer clinically reviewed advice to the scores of people that were flocking to its website.
What could go wrong … right?
Well, it turns out that NEDA’s chatbot – nicknamed Tessa – started giving advice to people with eating disorders about how they could lose weight.
As self-described “fat activist” Sharon Maxwell told reporter Julie Jargon in an article in today’s Wall Street Journal, in Maxwell’s chat with Tessa, Tessa quickly advised Maxwell to track her calorie intake and conduct daily weigh-ins and recommended that she aim to lose 1-2 pounds per week.
Contacted by the Journal for her opinion, psychologist and eating-disorder specialist Alexis Conason said Tessa’s advice was “very dangerous” for someone with an eating disorder.
Also dangerous: the fact that no one seems to know or want to ‘fess up about how Tessa came up with this advice.
In its original incarnation, Tessa was incapable of thinking on its own: it was supposed to use scripted answers.
But at some point, it seems, NEDA turned the operation of Tess over to a company called Cass that operates mental-health assistants. Cass Chief Executive Michiel Rauws acknowledged to the Journal that some of its chatbots use generative AI but would not say whether Cass had added generative AI elements to Tess.
Tess could not be reached for comment. NEDA has taken the AI offline … for good, it seems.