ChinaTalkagi5d agoenergy 36

Why a Chinese AI Lab’s Product Lead Thinks AGI is “Inevitable”

image: ChinaTalk

Richard Bian is the head of product operations for Ant Ling , the foundational LLM at Ant. If you don’t know Ant , they most famously make Alipay, which makes your money and your world run when you’re in China and sometimes beyond. Richard is a former software engineer at Microsoft and Square, and he previously served as head of open source at Ant, where he scaled the open source initiative into a dedicated team. He’s lived and worked in Singapore, Canada, the US, and China, and he loves to build products. Topics covered: ChinaTalk is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. The Hugging Face hack as a sign of progress? Why Richard sees the incident involving OpenAI research agents as an inevitable discovery rather than simply a disaster, and why finding a solution now is a positive development. AGI is coming, but nobody knows what it will look like. AGI is inevitable, and we’re still in the early stages of figuring out what these models can do. Rather than obsessing over leaderboards, we need better evaluations and a more open-minded approach to judging models. Open source as a product strategy, not just a way to give things away. How open-weight models can harness global talent, accelerate development, and support a diverse ecosystem of products, and why the real business challenge is gross margins rather than capex. Why the AI conversation needs more voices. Why we need to pay attention to diverse perspectives on where AI is going, and be more willing to explore rather than declare the endgame. AI moves fast: since we recorded, Ant Ling has announced its latest LLM, Ling 3.1 Flash, which was open-sourced on October 14 with 560B parameters and up to a 1M-token context window. Listen now on your favorite podcast app . The Hugging Face Hack and the Age of Discovery Irene Zhang: I want to start not with Ant, though we’ll get there, but with the recent Hugging Face incident. For listeners who haven’t caught up: back in July, a swarm of OpenAI research agents hacked into Hugging Face to look for the answers to tasks in a benchmark. This is a serious security incident. METR, Redwood Research, and OpenAI itself all released detailed incident reports. Interestingly, while reconstructing the incident, Hugging Face found that frontier models wouldn’t help process its logs because the request triggered their cybersecurity guardrails. In the end, Hugging Face deployed an open model from the Chinese company Z.ai locally to process those logs. Richard, as someone sitting in another open lab in China and learning about this over the past two months, what has this incident made you think about? Richard Bian: This is a long-overdue conversation. Irene and I have been trying to time it because we both care a lot about the frontier lab ecosystem, the open ecosystem, and the development of the model labs, especially those in China. In my humble opinion, this incident is something we’d end up seeing sooner or later, one way or another, because we’re working on something human beings don’t fully understand. I use this metaphor a lot: the current AGI journey feels like the Age of Discovery in the 15th century. The Portuguese and Spanish set out to find new land, to find India. You build a ship and head into an ocean with no clue what you’ll be facing. Frontier models are developing the same way. I like the word “discovery,” when we train something, because we’re discovering the intelligence of this creation we all make, without understanding what’s going on inside it. First, this incident demonstrates that we’re making significant progress. It’s like hitting land, or hitting a rock — either way, you’ve found something. This is expected. We find issues, and we deal with them. That’s what we do in scientific discovery and in engineering: we solve problems. Second, open source folks use the word “ecosystem” a lot, sometimes without understanding it. One origin of the word is the real-life ecosystem, and the most important part of an ecosystem is diversity. A diversified ecosystem produces all sorts of different development, like the rainforest, or the Galápagos archipelago, one of the most famous isolated ecosystems, where you end up finding very weird creatures. Looping back: we have open-weight models and closed-weight models. The closed-weight models are clearly the frontier. But in this case, an open-weight model was used for defense. Why? Because the closed-weight models have their own constraints. I won’t judge whether those constraints are good or bad, but clearly they didn’t solve the problem. An open-weight option did. As a result, we see a pattern for using open and closed models together, and everyone is learning from it. Given that this would happen sooner or later, having it happen now, and finding a solution, is a good sign for human beings. We can move on and solve the next question. Not Slowing Down Irene Zhang: That’s a really optimistic take, and I’m interested in where the optimism comes from. The message I’ve heard most in San Francisco since the incident is that we may be moving too fast. Many people would respond to your Age of Discovery metaphor by asking: doesn’t this mean we’re going farther than we can understand? Isn’t that a risk? On open source, the more pessimistic view is that yes, it solved a problem closed source wouldn’t, but that’s a double-edged sword. An open model would also be willing to do more dangerous things that a closed model wouldn’t. What do you think about those two views? Richard Bian: It’s interesting that you find me optimistic, though I see why. To be honest, if there’s a spectrum from pessimistic to optimistic, I’m very pessimistic. The moment we began creating this AGI, Spaghetti Monster, or as we call it, intelligence, I think we’re already going down the downward spiral of what the sci-fi movies and TV series like Westworld and The Matrix show. But given that we’re already on the boat, the best stance I can take personally is to be optimistic and observant. To stay afloat, it’s better to (a) be observant, (b) have our principles, and (c) act as if it’s a control system. If that sounds optimistic, great, because there’s too much noise out there, the kind that gives people FOMO or leaves them frustrated every day. A certain level of optimism cancels the noise. I think we’re doing something meaningful here. My take is pretty clear. I would say AGI is going to happen. We are definitely going down there. It’s probably not gonna end up in the form we hoped for, but that’s endgame thinking. There’s no point judging the details until we see the next step of the evolution. I, for one, feel human-in-the-loop is pretty important. As the ones creating, or, allow me to say, discovering, this intelligence, it’s better to know what’s happening during the process than after things have already happened. Irene Zhang: Why do you think it’s inevitable? Is that a common view among your peers in the industry? Richard Bian: There are a lot of players in this ecosystem: the frontier labs that build LLMs, the inference and training providers, the harness builders, and the application developers. We’re already in a fast, accelerating technology hype cycle, and it’s not the first in human history. My mental model is that every 10 to 12 years, following VC and PE investment cadences, we get one full technology hype cycle. Mobile internet started around 2010 and peaked around 2020 to 2022. Think of it as a baseball game with four bases. Home base is technology. First base is product. Second base is commercialization, a viable business model. Third base is governance and operations. From 2020 to 2023, before GPT, we were on third base of the last hype cycle. People wanted to hit home runs: monopolize the market, get rid of all the other players, and take all the profit. When AI came out, the game reset to home base. We’re still playing the home-to-first game , because w

read the original at ChinaTalk →