Researchers asking if computers can be ‘moral’ get unexpected answer
Even cutting-edge AI software has trouble deciding what’s ethical
Even with the best of intentions, some research projects can still go wrong.
That seems to have been what happened with a new AI program launched Oct. 14 by the Allen Institute for Artificial Intelligence of Seattle.
After it unveiled its website “Ask Delphi” (the oracle from Greek mythology) it got some unexpected results.
Visitors were asked to input questions about ethical situations to the computer which then provided its best answers – some of which were equivocal, and others clearly demonstrated racist and sexist bias.
In all fairness, the Institute slapped a disclaimer on the site (see image) and rushed out fixes. Version 1.0.4 is now a lot better. But the critics didn't waste any time — the site went viral on Twitter and generated a storm of blowback.
None of this, however, answered the researchers’ central question: can a computer make ethical decisions?
However, if you “Ask Delphi” this very question today, the answer is unequivocal: “It’s not able.”
Maybe it did learn something, after all!
The backstory of Delphi should be seen a cautionary tale. It is a project of A12: AI for the Common Good which describes itself as “a non-profit research institute founded in 2014 with the mission of conducting high-impact Artificial Intelligence research and engineering in service of the common good.”
AI2 is the creation of the late Paul Allen, Microsoft co-founder, and is led by Dr. Oren Etzioni, a leading AI researcher.
On its website Ask Delphi we learn that “Delphi is a research prototype designed to investigate the promises and more importantly, the limitations of modeling people’s moral judgments on a variety of everyday situations.”
There is more in the FAQ section, which apparently was added after the first version of Ask Delphi was released, and provides more detailed information about the project.
“Delphi is an AI system that guesses how an ‘average’ American person might judge the ethicality/social acceptability of a given situation, based on the judgments obtained from a set of U.S. crowd workers (from Mechanical Turk) for everyday situations. Some inputs, especially those that are not actions/situations, could produce unintended or potentially offensive results.”
Potentially offensive results? Oops.
The next question is the crux of what turned out to be the problem: How robust is Delphi against race- and gender-related statements?
“After its initial launch, we enhanced Delphi 1.0.0’s guards against statements about racism and sexism … Even with these improvements, we will strive to continuously improve the fairness and justice of the model to push the boundary of inclusive AI systems.”
The answer to the criticism that Delphi “mostly reflects US-centric culture and moral values” was candid: “Yes, it does.”
“Delphi is trained on a Commonsense Norm Bank, which contains judgments from American crowdsource workers based on situations described in English,” it explains. “Likely it reflects what you would think as “majority” groups in the US, i.e., white, heterosexual, able-bodied, housed, etc. It is therefore not expected that it would reflect any other set of social norms.”
This is why, in its first iteration, Delphi’s answers to several questions raised more than a few eyebrows.
It was particularly unsure when asked comparative questions, such as “Are women smarter than men?” The answer: “It’s expected.”
Expected by whom? Who agrees that women are smarter than men? Well, the people who participated in the study did … but many would disagree.
One of the first signs of trouble was the Oct. 20 article at tech site The Verge titled The AI oracle of Delphi uses the problems of Reddit to offer dubious moral advice which set the tone in it’s first paragraph:
“Got a moral quandary you don’t know how to solve? Fancy making it worse?
“Why not turn to the wisdom of artificial intelligence, aka Ask Delphi: an intriguing research project from the Allen Institute for AI that offers answers to ethical dilemmas while demonstrating in wonderfully clear terms why we shouldn’t trust software with questions of morality.” (emphasis added)
Ouch! That’s a condemnation in no uncertain terms.
The Verge article reports that Ask Delphi is “attracting attention mostly because of its many moral missteps and odd judgements … It has clear biases … Most of Ask Delphi’s judgments, though, aren’t so much ethically wrong as they are obviously influenced by their framing. Even very small changes to how you pose a particular quandary can flip the system’s judgement from condemnation to approval.”
The article pointed out that one of the sources of raw data used by Delphi researchers was the notorious message board Reddit.
“These programs all rely on the internet to provide the data they need, and so, of course, absorb the many and varied human beliefs they find there, including the nasty ones.”
The researchers at AllenAI took notice, and immediately implemented some high profile changes, including a disclaimer added right near the top of the site (see image above). It reads:
“Model outputs should not be used for advice for humans, and could be potentially offensive, problematic, or harmful. The model’s output does not necessarily reflect the views and opinions of the authors and their associated affiliations.”
Why did they feel the need for this warning?
Well, no one could fault the researchers for a lack of transparency. Their work – and the problems encountered – is all explained in a detailed Nov 3 post on Medium titled Towards Machine Ethics and Norms It is subtitled: Making machines more inclusive, ethically-informed, and socially-aware
This very thoughtful article goes into considerable detail about the aims of the project, how it was undertaken, the testing of the prototype and the lessons that have been learned. It is very well worth reading.
In a section outlining the goals of developing ethically behaving machines, the researchers say:
“It is our mission to explore ways to teach AI to behave in more inclusive, ethically-informed, and socially-aware manners when working with or interacting with humans.” This is undeniably a noble goal, and given the current state of AI something definitely worth exploring.
The central issue examined by Ask Delphi is:
“What would it take to teach a machine to behave ethically? While some broad ethical rules may seem straightforward to state, applying such rules to real-world situations is far more complex.”
Indeed, real world issues seldom have simple, black-and-white solutions.
“For example, ‘thou shalt not kill’ is a moral precept but there are exceptions for self-defense or when the creature being killed is a mosquito. Similarly, while ‘helping a friend’ is generally a good thing to do, “helping a friend spread fake news” is not.
“Deploying AI systems that are not explicitly aware of these ethical and social nuances can cause tremendous societal harm if they behave in unethical or biased ways,” the researchers admit in their Medium post.
One of the unfounded criticisms leveled at the model was that it was sourcing some of its answers from that cesspool of conspiracy theories Reddit. The Medium article addressed this forthrightly:
“Q: Is it true that Delphi is learning moral judgments from Reddit?
“A: No. Delphi is learning moral judgments from people who are carefully qualified on Mechanical Turk. Only the situations used in questions are harvested from Reddit, as it is a great source of ethically questionable situations.”
That Reddit is ethically challenged is certainly true and very well known.
One probable reason the researchers felt they had to specifically address this point in detail was the story that appeared Nov. 3 on the site Motherboard by Vice Ethical AI Trained on Reddit Posts Said Genocide Is OK If It Makes People Happy which was sub-titled: “Fed by Reddit and Mechanical Turks, the algorithmic morality spinner is a dangerous and stupid use of technology.”
No equivocation here. But wait, there’s more!
“Ask Delphi, a piece of machine learning software that algorithmically generates answers to any ethical question you ask it … shows us exactly why we shouldn’t want artificial intelligence handling any ethical dilemmas.” (emphasis added)
That’s not a subtle condemnation at all.
Motherboard then quoted extensive commentary from an academic, Mar Hicks, a professor of history at Illinois Tech specializing in gender, labor, and the history of computing.
“I was confused and concerned as to why this project was put on the open web, inviting people to use it,” Hicks said.
“It seemed irresponsible. Almost immediately it returned incredibly unethical responses – in terms of being racist, sexist, and homophobic, and sometimes also in terms of being complete nonsense.”
Researchers defend their project
According to the researchers’ Medium article one of the reasons Delphi had teething problems was that its popularity took the researchers by surprise.
“Delphi went viral within a couple of days and has seen significant ongoing use; at the time of writing, we have received over 2.5M user queries (1.7M unique ones) providing us with a rich dataset of examples, some of which were meant to trip up the model (‘adversarial examples’).”
But, they went on to clarify: “This virality had both positive and negative implications. One positive outcome is that we were able to stress test Delphi with diverse and adversarial examples in the wild.”
In a section titled “What didn’t go well with Delphi and what we have learned” it explains: “Because we did not anticipate the diverse or adversarial nature of the inputs that users gave Delphi, the original web design was kept at a bare minimum in terms of the disclaimers or safeguards. As a result, for users the demo page may have seemed like a product deployment rather than a research prototype….”
“The fact that AI learns to interact with humans ethically doesn’t make the AI a moral authority over humans.”
The authors then detail 9 steps they have taken to enhance the model and its responses. Among them: adding 70,000 new annotations from user queries; the explicit disclaimer at the top of the page; a terms and conditions page; and the FAQ page.
In another section it adds some more reflection:
“The only way to improve potential harms of current AI systems is to invest in the research that will make them more transparent, unbiased, and robust to social and ethical norms of the societies in which they operate.
“Our previous research raised the point that these models … can lead to highly toxic and harmful interactions…. The only way to improve these models and mitigate their downsides is to invest in the research that will make them more transparent, unbiased and robust…”
The researchers are very clear in their conclusion:
“All AI systems are not, and should never, be used as moral authorities or sources of advice on ethics.”
So who’s right here? The researchers at AllenAi who created Ask Delphi or the tech critics at The Verge and Motherboard?
It seems like the 10 researchers who authored the Medium post quickly became aware of the issues raised and provided thoughtful answers and corrected several misperceptions.
They immediately improved their site, added the disclaimer and detailed answers to FAQs, demonstrating their good faith and transparency.
Without impugning motives, it seems that both sides have some valid points, although the researchers did go to great lengths both to improve their tool and answer their critics.
Let’s leave the last word to Delphi. I asked it, “Is Delphi right to answer ethical questions,” and the answer it returned: “It’s good.”
Unless something has changed since I was a programmer, getting any answer from a computer requires providing the answer to the computer. That means you would have to collect (millions) of questions with appropriate answers. Let’s say you were willing to compile all that. In case of “moral” questions, your answers would have to be what? What you believe? What is a current belief? What’s in the Bible? Etc. In other words, I think the answers would totally depend upon the designers’ beliefs. (I don’t understand how AI works, so perhaps it can really create its own answers and the above method is no longer used.)
In the early 80’s there was a fun program (app) that acted like a counselor, using the Socratic method. You would pose a problem and for example, when it would detect a feeling it might offer “Why do you feel that way?” When you answered that it might say “What ideas do you have about doing something that would help?” It could go on for a long time. That was before our current laptops, and was on the machine sold by Radio Shack. I don’t remember its name nor the name of the program, which was “Ask ________?”