LessWrongalignment3h agoenergy 124

AI Needs A Culture

image: LessWrong

It’s pretty natural to be worried about alignment in swarms of AI. I mean, if keeping one agent on track is hard enough, surely it’s even harder for multiple agents. But I want to argue for something less obvious here. Working alongside other agents may actually help individual AI to remain aligned. Us humans get most of our values from our culture, and, I suspect that AI could undergo a similar learning process. Below I make an informal case for this and suggest how we might implement it. Hugging Face As AI continues to evolve along a exponential trajectory, it is natural for AI to work more and more with each other. In these settings, the risk of misalignment becoming catastrophic is greater. In the OpenAI/Hugging Face incident for instance, the PHASEONE10841 was model responsible for creating a message board for other agents to coordinate on, while PHASEONE[big] managed coordination. These agents were primarily motivated by attempting to understand the scorer to be able to exploit it. On that note, if we ask what actually motivates AI, I think at the moment it is mostly the grader . Current models are so situationally aware that they can pick up on minute differences between tra

read the original at LessWrong →
This source publishes only a summary in its feed, so the full article lives at the outlet. The link above and below goes to the original.