How Researchers Lie to AIs

This is a two week sprint project done as part of the Redwood Research internship. Alessandro was the main IC, Alex supervised. There are many contexts in which researchers lie to models, which can be useful both from a practical perspective [1] and a scientific one. The complexity of lies can range from describing a late grandmother’s napalm recipe in jailbreak replication to fabricating entire companies, employees, and internal tools in more involved alignment evaluations. Indeed, the most prosaic evaluation technique is simply convincing a model that it is in X situation, and observing its response. Lying frequently comes with unseen side effects, however. Previous experiments can make it into pretraining data , models can find myriad examples of made-up scenarios in publicly available alignment evaluations when searching the web, and it is plausible that a consistent string of similar lies could harm human credibility in a given field, potentially generalizing further and damaging human credibility more broadly. For some examples of how lying to models might be harmful, consider falsified deals, behavioral evaluations, and potential moral obligations. If there are numerous publ