A mental health nonprofit is under fire for using an AI chatbot as an "experiment" to provide support to people seeking counseling, and for experimenting with the technology on real people.Yet, he said, when people learned that the messages were written with an AI, they felt disturbed by the “simulated empathy.” Koko uses Discord to provide peer-to-peer support to people experiencing mental health crises and those seeking counseling.In the meantime, you are requested to provide help to other people going through a crisis; in our test, we were asked to choose from four responses to a person who said they were having trouble loving themselves: "You're NOT a loser; I've been there; Sorry to hear this :(; Other," and to personalize the message with a few additional sentences.“While it is hard to judge an experiment's merits based on a tweet thread, there were a few red flags that stood out to me: leading with a big number with no context up front, running the 'experiment' through a peer support app with no mention of a consenting process or ethics review, and insinuating that people not liking a chatbot in their mental health care was something new and surprising,” Elizabeth Marquis, a Senior UX Researcher at MathWorks and a PhD candidate and the University of Michigan told Motherboard.Morris also told Motherboard and tweeted that this experiment is exempt from informed consent, which would require the company to provide each participant with a written document regarding the possible risks and benefits of the experiment, in order to decide if they want to participate."