Take Ten-Thousand: Egocentric Data Collection for Humanoid Robots

The ChinaTalk team is going to CoRL (the Conference on Robot Learning) in Austin! Reply to this email if you’d like to hang out between November 9th and 12th. We’re also doing karaoke on the 11th — RSVP here if you’d like to join! A. Chen writes about people, technologies, and their relationships to power. Her submission, “Take Ten-Thousand: Egocentric Data Collection for Humanoid Robots,” won one of 7 Grand Prizes at the ChinaTalk Essay Contest. We’re thrilled to share this report from the frontlines of China’s robot data collection industry! Where is the center of the universe? For the residents of Beijing, the answer is Wudaokou. The neighborhood houses China’s top universities, research institutions, and labs, drawing talent and funding from every corner of the country. The streets are radiant at night, packed full of bars and clubs that have somehow bounced back after the COVID-19 pandemic. And the number 13 metro line, which rattles overhead every five minutes, is never empty. I came to Wudaokou to understand China’s AI industry — particularly, the hype surrounding humanoid robots, and the methods used to train them. Everyone has seen videos of these robots: eerily tall and lanky, some of them shedding nuts and bolts as they sprint down a track, others backflipping repeatedly on a dance stage. The global market for humanoid robotics, or embodied intelligence, is said to be worth $38 billion by 2035: the latest chapter in the U.S.-China technology race that has dominated international headlines for over a decade. This year, China’s 15 th Five-Year Plan declared embodied intelligence a matter of national priority, then laid a strategy for developing training grounds across the country. Over 150 humanoid robotics companies and 40 state-owned robot data collection centers were reported to be in operation in China by the end of 2025. According to the Global Times, Chinese embodied intelligence startups drew 50 billion yuan in investment deals by September 2025: an increase of 2.5 times from the previous year. More recently, embodied intelligence startups have begun to focus on the human hand, whose articulation and coordination has yet to be replicated by robots. Data of the human hand can’t be generated passively, in the same way that an Instagram user’s scroll or like might be harvested, and researchers of embodied intelligence are racing to solve this dilemma with a number of techniques. Most recently, videos have begun to circulate of people wearing headsets and filming themselves at work — a technique called egocentric data collection — to train robot hands. I sat down with the researchers and entrepreneurs behind China’s egocentric data collection space to understand the industry. Beneath the speculation, the social media virality, and the staggering amounts of investor money, what — and who — does it actually take to train a robot? The Third Wave of Data Collection Dr. Jackson Guo, a researcher at the Beijing-based Kunlun Nest Robot Open Laboratory (昆仑巢人机器人开放实验室), invited me to his workspace to discuss the history and technicalities of egocentric data collection. He identifies three distinct periods of robot data collection: teleoperated, isomorphic, and egocentric. Teleoperated data collection, the first generation of data collection techniques, appeared around 2017. That year, Stanford Professor Fei-Fei Li piloted a program that allowed users to connect their phones to remote controls that could control robot arms. Teleoperated data collection methods have since become more streamlined: a user stands several feet away from a robot and moves its body using remote controls, sometimes with a VR headset to simulate a different environment. According to Dr. Guo, the individuals responsible for teleoperated data collection often suffered from fatigue and joint pain after hours of performing slow, repetitive motions. Teleoperation was also expensive; hardware could cost hundreds of thousands of yuan per set. Even so, companies such as Helix, OpenXE, and PI (Physical Intelligence) collected tens of thousands of hours of data using this method. Most of the data collected was for gripper hands, which operated on a simple binary command: 0 for closed, 1 for open. Fine-grained, dexterous manipulations — such as cutting an apple, folding a shirt, or playing the piano — proved too complex for a user to simulate through a remote control. In 2024, Stanford researchers Cheng Chi and Zhenjia Xu and Professor Shuran Song published a thesis on Universal Manipulation Interface (UMI). This marked the second generation of data collection techniques, or isomorphic data collection, which took Silicon Valley by storm. Using Velcro straps, elastics, or metal hinges, human subjects connected their hands to the inside or underside of a robotic exoskeleton, feeding information to a robot more swiftly and precisely than a teleoperated remote control could. Gripper hands were still the easiest model to train, as exoskeletons were often too rigid or heavy to allow for more articulated movements. Dr. Guo credits Generalist AI with the most successful isomorphic data collection program in the U.S.: within six months, they used one-thousand workers to collect over 250,000 hours of data. By the start of 2026, egocentric data collection had begun to define a third generation of data collection techniques. On February 18, 2026, NVIDIA introduced the concept of egocentric data collection in its EgoScale Project, which shifted the primary method of data collection from isomorphic exoskeletons to camera footage. NVIDIA’s team attached cameras to the foreheads and wrists of over 1,000 people, which they used to film 20,854 hours of first-person footage: mostly everyday tasks, such as washing dishes and cutting vegetables. Researchers mapped the positions of the subjects’ wrists, fingers and joints through each frame of the footage. This preliminary data was then augmented and refined using a variety of isomorphic techniques. In their paper, the NVIDIA team claimed that training a robot on egocentric data increased the chances of it successfully folding a shirt by 88%. “Humans can be treated as a scalable embodiment for learning general embodied intelligence,” celebrated the authors of the paper, before encouraging future researchers to record as much video as possible for robot training. Imperfect and low-quality footage can always be cleaned: what matters is getting enough video to clean. Today, among most companies in the egocentric data business, the consensus is, the more, the merrier. As was the case for teleoperated and isomorphic data collection, egocentric techniques quickly latched on across the U.S., before spilling across the world to China. The U.S.-based EgoVerse uses Meta’s Project Aria glasses, or their Mecka app downloaded to an iPhone 15+, to record an individual in a variety of settings. Build AI has raised over $30 million with the goal of collecting over 1 billion hours of egocentric footage by 2028, a goal shared by a dozen other startups in Silicon Valley. And in an interview with Wall Breaker, World Intelligence founder Grace Zhang declared her ambitions of creating the world’s biggest physical AI data marketplace. “I want people to film and record data of themselves doing everyday chores in their homes,” she said. “That is my ideal future.” The Chinese companies jumping on the trend Today, Chinese companies have been collecting egocentric data for their own in-house use, or to sell to domestic and overseas consumers. Jingdong, or JD.com, was one of the first Chinese companies to jump on the egocentric data collection wave. On March 16, 2026, founder and chairman Liu Qiangdong announced the creation of a data collection center in his hometown of Suqian (宿迁), Jiangsu Province, with the goal of collecting over 5 million hours of footage by 2027 and 10 million hours of footage by 2028. Over 100,000 Suqian citizens have signed up to film themselves performing fi