China’s AI Safety Money Problem

Here are two claims I hear about China and AI safety: China is intrinsically less concerned about AI risk . Chinese people are techno-optimists, while the Party sees doom talk as either Western paranoia or a containment strategy designed to hold China back — such as many of the dismissive responses to Dario’s call to pace the frontier. China is now safety-pilled . This refrain gains traction once or twice a year when a high-up like Xi gives a speech about controlling AI risks (as he did at the World AI Conference ), or China puts forth a new international initiative, such as the September UN Global Dialogue on AI Governance . There’s evidence for each of these takes (see footnote). 1 But I think both sides miss the messier reality on the ground. I met plenty of ambitious and talented people when I lived in Beijing who would have loved to work on AI safety full-time. But a graduate in China who cares about AI risk is still expected to get a “real” job eventually. Outside an underfunded PhD track or a handful of positions in a country of 1.4 billion people, there isn’t a well-trodden path. In San Francisco, DC, or London, worrying about AI professionally can be a very-well-paid career. In China, it’s more of a hobby. As a result, I think the main bottleneck facing China’s AI safety ecosystem is not ideological but material ... Independent Chinese AI safety has basically no money. 2 And the limiting funding structures that do exist determine what kinds of safety work can survive, and where. Subscribe now Today: Why China’s philanthropic rules make it difficult to fund independent organizations, even if the money exists. Why this leaves it up to the Chinese government to lead on safety work Technical safety research and how it is continuing to grow China’s “safety-as-a-service” companies which try to make safety profitable A “ third wave of American philanthropy ” appears to be on the horizon, alongside initiatives like Coefficient Giving’s Project Tailwind , potentially enabling much more ambitious AI safety efforts in the West. Understanding how this influx of resources could reshape the gap between Western and Chinese AI safety ecosystems (and the relationship between them) therefore seems increasingly urgent. [Nathan Labenz explores related themes in his dispatch from China , which I’d recommend checking out.] Share A Brief Overview of Chinese Philanthropy China has plenty of billionaires. Forbes puts the number at 539 billionaires in China worth $2.2 trillion. (The US counts 989 billionaires worth $8.4 trillion.) It would take only one or two of them getting really into AI safety to finance a sizable funding ecosystem. So why has no Chinese equivalent of Coefficient Giving emerged? The answer is related to the strange structure of Chinese philanthropy. The modern Chinese charitable sector as it exists today expanded rapidly after the 2008 Sichuan earthquake, but it was plagued from the start by corruption scandals, inconsistent regulation, and government concerns about grassroots organizations operating outside of the state’s purview. To streamline the system, the 2016 Charity Law 慈善法 clarified the legal status of domestic philanthropy, tax incentives, and fundraising. Charities would have to register, disclose their finances, and accept government supervision. The goal wasn’t to reduce or get rid of philanthropy. Beijing still wanted private money to support education, health, poverty relief, and other social services. But it wanted civil society to help deliver state-approved public goods, rather than become an independent source of political organization or agenda-setting. Article 4 of the Charity Law made this principle explicit enough by stating that “charity work shall be conducted under the leadership of the Communist Party of China.” 3 Scholars Angela Bies and Scott Kennedy write : “There exists a current paradox of philanthropy and the state in China: a stronger social sector accompanied with growth in philanthropy is desired, but such strength and growth must also contend with the state’s own policy agenda and desire to limit aspects of advocacy and civil society typically associated with the third sector.” Subscribe now The Charity Law may have legitimized the sector, but it has not erased longstanding distrust. China has long ranked near the bottom of the Charities Aid Foundation’s World Giving Index , which measures behaviors including donating money and volunteering. In off-the-record conversations for this piece, I likewise heard skepticism about donating due to corruption . ^The 2011 Guo Meimei 郭美美 fiasco, one of many scandals that badly damaged public confidence in major charities. Perhaps some of this lingering distrust also reflects another impetus behind the 2016 reforms: limiting the influence of foreign organizations operating inside China — such as groups focused on human rights . This led to the Overseas NGO Law 境外非政府组织境内活动管理法 (2016), which, for AI safety, may actually be more consequential than the Charity Law. The Overseas NGO law mandates that foreign nonprofits must register an office or work through an approved Chinese partner, with the system overseen by public-security authorities. Shawn Shieh, a research associate at the British charity ODI, found that many Chinese non-profits went from receiving 80–90% of their funding internationally to 80–90% from domestic Chinese sources in the years following the Overseas NGO Law, effectively flipping the philanthropic funding ecosystem on its head. This should help clarify why supporting AI safety work in China is unlikely to be as straightforward as simply giving more money to Chinese non-profits to work on safety. In practice, foreign funding has to flow through organizational structures that are acceptable to the government. Even if Beijing fully accepted the case for addressing AI risks, it would still be unlikely to permit foreign funding on a large scale to flow into Chinese organizations. Share Chinese charitable organizations received roughly $21 billion in donations in 2023 , compared with $557 billion in the US . That’s a 27x difference. Companies supplied 77% of Chinese giving in 2023, while individuals provided only 22%. In the US, the pattern is reversed . Individuals supplied 67% of charitable giving in 2023, while corporations accounted for just 7%, with foundations providing another 19%. Specifically for AI, I estimate Western AI safety philanthropy is $250-600 million per year. 4 And I predict that number will push well beyond $1 billion by next year, after companies like Anthropic and OpenAI go public and usher in a potential “ third wave ” of American philanthropy. What Chinese money does exist flows toward established causes. Health received 37% of donations in 2023, education 25%, and scientific research around 10%. Among the country’s largest philanthropists, 70% donated to education . Harvard’s China Philanthropy Project similarly finds that education dominates elite giving. This fits the logic of the Charity Law, which encourages philanthropy in politically approved areas already recognized as socially useful. State-Sanctioned Safety Perhaps in part because of these restrictions on charitable giving, AI safety in China tends to be treated less as a philanthropic domain than as a responsibility of the state. China’s largest AI safety institutions are government or government-backed, like Shanghai AI Lab (SHLAB), the Beijing Academy of Artificial Intelligence (BAAI), the China Academy of Information and Communications Technology (CAICT) and TC260. Subscribe now None of these institutions primarily focus on safety. CAICT is a massive Ministry of Industry and Information Technology (MIIT) think tank. BAAI trains models for capabilities, such as Emu3.5. Safety is one function inside a much broader mission, competing for resources with everything else that mission entails. ^BAAI’s Emu3.5. Looks interesting but doesn’t seem to have anything to do with safety.