Q&A: Building accountability behind our words — and AI’s — in civic conversations

When government officials craft policies, there is often a comment period for public feedback. Lawmakers have long had to sort through fake feedback that is mass-produced in an effort to sway policy, but they’ve generally gotten used to detecting it. With, artificial intelligence, however, people can submit realistic-seeming comments, which are harder to dismiss as being fabrications with no person behind them. That’s just one way AI can be used to generate text and statements, in civic conversations, for which no one can be found to be answerable. This issue, among others, has caught the attention of Deb Roy, professor of media arts and sciences at MIT and director of the MIT Center for Constructive Communication (CCC). In a new essay published in the Bulletin of the Atomic Scientists , titled, “Who stands behind the words?,” Roy calls for a renewed emphasis on responsibility in communication, asserting that even when AI is used to craft text, there needs to be accountability and “answerability.” Building better group dialogues to produce more public trust is a core part of the work of CCC. The group, along with Cortico, the closely affiliated nonprofit Roy co-founded, has helped facilitate group discussions on important issues in many situations — including civic meetings, schools, deliberative assemblies, and private organizations — and used tech tools, including AI, to help find themes among comments, and more. But the center also designs this process so that participants can check and override the tech-based output at each key step. Ultimately, a person is responsible for every public comment. MIT News spoke to Roy about the essay and the work of the center. Q: You’ve identified this problem about the lack of accountability behind words that are generated by AI. When did you really first notice this? A: It came into focus when Chat-GPT [which is powered by a large language model, or LLM] launched. As for my own history with language models, my PhD work explored what we might now call PLMs — Puny Language Models — so I’m quite familiar with language models and have tracked the field’s rapid evolution. But Chat-GPT turned a corner. Two things changed at the same time: the depth of the model and its knowledge, and the way you could interact with it. I became obsessed with trying to find the limits of what it could do. And I felt that no matter how hard I tried to stretch my arms around it, I couldn’t. I built my tenure case at MIT on language grounding — the idea that it’s not enough for an AI system to just process lots of words and other symbols. I believed that to understand what words mean, you actually need a physical body and life experiences as part of the process. So I revisited the philosophy-of-AI literature I had once been steeped in, which asks what fundamentally distinguishes the kind of system that large language models exemplify, in extraordinarily advanced form, from the kind of processing we humans engage in. I gave several presentations to my own graduate students trying to make the case that, even though AI seems so intelligent, it’s actually different in kind — and I could not convince them. And I thought, I know the difference, but I’m having trouble convincing even myself of what it is. And that led me to ask, “Wait a minute: Is there someone or something that stands behind the words, a human responsible for what is being said?” That’s the thing this extraordinary new language machine doesn’t have. Whether these systems genuinely understand language is actively debated. But even if you grant them all the comprehension you like, they still lack standing. The buck can’t stop at a thing, a software system, with nothing to lose. Q: The Bulletin piece has these interesting examples of civic feedback forums where it looks like there are real people saying things, and instead you have a barrage of comments from AI. How did you find some of these other examples in society where things go off the rails in this way? A: After the release of ChatGPT, I started hearing a concern from people inside government who review public comments on proposed rules: a suspicion that not all of the comments were written by humans. My reaction was: That’s going to delegitimize this channel for public feedback. That was several years ago, but everyone now knows this is happening. Public comment is a pillar of democracy. You don’t just put a new rule in place. People need to have a voice and feel like they’ve been heard before a rule is imposed on them. If you skip that step, people don’t accept rules as legitimate. That’s why that process is baked into how America works. And that pillar just cracked, and by now, in my mind, it’s fallen over. Has anyone noticed? My sense is the people on the receiving end of these comments have quietly adjusted the weight they give to public comments. It was never perfect, but boy, now much of its value is being swept away. I open my essay with this example: In 2017, the Federal Communications Commission received more than 22 million public comments on its proposal to repeal net neutrality rules, and the New York attorney general later found that nearly 18 million of them were fake. As I write in the essay, “That fraud was caught because it left fingerprints: templated text, duplicated phrasing.” But large language models erase the fingerprints. Now every comment can be fluent and different, and reviewers can no longer tell which ones express views that real people formed and stand behind. If you have enough of the stuff that’s fake — words that no one with a stake actually stands behind — it desensitizes you to what’s real. That filters back to the public, and people realize: If I take the time to express myself, they’re not even listening any more. Q: Could you say a little bit more about your point that we need people with “standing,” who will take “accountability” for all pieces of text, even when those bits of text are generated by AI? A: In the legal system, standing is a formal idea: sometimes a case isn’t heard because the people bringing it don’t have standing. But informally we use that term all the time. I’m not trying to formalize standing in everyday life. Instead I’m asking: What is the informal sense of the term, borrowed from the legal one, that we need to pay more attention to now? Here in New England, consider the open town meeting: The person who stands up to speak is generally expected to have standing, as someone from that town with a stake in what gets decided. And if it’s a stranger who drove from across the state, or from out of state, everyone’s like: Wait a minute, who is that? They’re not from around here. In general, there’s just enough friction and it’s costly enough to show up somewhere you don’t actually have standing, that it doesn’t happen much. Now LLMs and the current digital networked environment collapse those boundaries. We always took for granted that if someone has a voice, it’s because they have standing. They have some basis to be heard. We need to pay attention to this, and really consider how to design systems so that the people with standing are the ones who get heard. The other idea I tried to develop is “answerability.” It isn’t formalized in law, but it’s baked into the legal process. If you have standing to give testimony, there you are, ready for cross-examination. You have to be able to answer for what you’re saying. And if you’re making a specific accusation about someone, you have to be willing to back it with facts. Standing and answerability have to travel together: The right to be heard comes with the obligation to answer for what you say. That doesn’t mean everyone has to attach their name to everything. I write in the Bulletin essay, “Privacy and answerability aren’t opposites. ... An institution can verify a person and stand behind their words without exposing them.” To voice a view, you only need to show that a distinct, eligible person is speaking. An accusatio