Import AI 479: Spacewar; low-latency tasks versus LLMs; Genie 4

Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe. Subscribe now LLMs are very bad at low-latency tasks: ...Realtime-Bench says it’ll be a while till AI systems can play games or do online auctions for you... Researchers have built Realtime-Bench, “a suite of carefully curated dynamic tasks where delayed actions directly affect success”. The test helps evaluate how well modern, general-purpose AI systems can perform tasks where decisions need to be made in real-time. The benchmark consists of 50 interactive game and web tasks, split across 30 web tasks in four categories (travel, e-commerce, service, and security), and 20 game tasks (arcade, puzzle, strategy, and simulation). Who did the research: The benchmark was built by people at the University of Texas at Austin, Ohio State University, University of Illinois at Urbana-Champaign, NVIDIA, Northwestern University, and Steadyworks. What it measures: The test is designed to probe for three qualities: temporal state awareness (”RT-Perceive”), where models can use observations to track changes in the world; adaptive deliberation (”RT-Deliberate”), where models alter their reasoning length and depth according to available time; and time-aware action selection (”RT-Act”) where models need to appropriately time their actions for the situation. “For example, an agent purchasing a ticket must track the sale status and assess whether observed availability is still current (RT-Perceive), limit further deliberation when the purchase window is closing (RT-Deliberate), and choose between waiting for sales to open and purchasing while tickets remain available (RT-Act).” They have three different conditions: Wall clock, which is designed to test out performance under real-world latency. Paused clock, which lets you arbitrarily pause things so you can get a sense of baseline capability, and Token clock, which “advances the environment at a fixed rate per generation token. An action is applied at the simulated time reached when it is committed, so consuming more tokens delays execution regardless of actual inference latency”. How well do models do? The authors tested out models from OpenAI (GPT-6-Astra, GPT-5.6 Sol, GPT-Realtime-2, OpenAI Decisions), Anthropic (Claude Fable 5.1, Claude Opus 5), Google (Gemini 3.8 Flash, Gemini 3.1 Flash Live, Gemini 3.8 Live), Jev 1.13, as well as Chinese models from Deepseek (V4.1) and Moonshot AI (Kimi-K3). GPT-6 Astra (high) got the highest pass rate on the ‘wall clock’ evaluation setting, with 36%. By comparison, with a paused clock, models with high reasoning get closer to 70% or even 80% success. Why this matters: This benchmark is an effective proxy for where AI agents will break – the low scores here point to tasks where AI agents cannot yet be expected to perform well. Check out the leaderboard : Realtime-Bench (official website) . Read the paper : Realtime-Bench: Can Frontier Models Solve Real-time Interactive Problems? (arXiv) . *** The future of spacewar, courtesy of the Defense Intelligence Agency: ...Anti-satellite tech, space nukes, and more... Working in AI can be stressful so sometimes to relax myself I like to read about other topics to get a sense for what horrors lie elsewhere. I take the same approach with depression, where if I feel sad I will try and listen to music by people sadder than me in an attempt to cheer myself up (e.g., Radiohead, Neil Young, and in very dark moments Bill Callahan / Smog, etc). So with that in mind I recently read “Challenges to Security in Space”, by the Defense Intelligence Agency. What’s going on in space: As with conflict on Earth, space is defined by conflict between the US versus China, Russia, Iran, and North Korea, with China the most technologically sophisticated, and Russia the most seemingly gung ho. “China and Russia are developing a formidable spectrum of offensive capabilities to degrade or destroy space services of other countries, from kinetic destruction to sophisticated jamming and cyber operations,” the DIA writes. “China has more than tripled its space launches per year in the past decade, with 22 launches in 2016 and 93 launches in 2025... Beijing judges space is central to the United States and its allies’ operations. Beijing also considers space an offensive military domain, which has shaped integral components of PLA military planning and campaigns.” Spacewar: The general vibe you get from reading this 100+ page report is there is a huge amount of ugly, frightening stuff happening in space, or being done to things in space by things on the ground. Some example quotes: “By the late 2020s, China may field space-based directed energy systems and higher-powered ground-based directed energy systems capable of threatening satellites in higher orbits” “China is developing space-based technology, including on-orbit counterspace weapons, capable of holding space systems at risk” “Russia’s Peresvet, a ground-based laser system, is designed to dazzle enemy satellites’ optical systems to mask the movement of Russia’s strategic missile systems” “Russia has launched several satellites over the last few years which conducted test activities on orbit for technologies that could enable attacks on other countries’ satellites and result in temporary or permanent damage” China has a couple of spaceplanes, akin to the US’s hyper-secret X-37B vehicle, and they appear to be getting more sophisticated. In 2023, one of China’s aforementioned vehicles, “released and interacted with an object.” Satellite-on-satellite fights: It’s obvious throughout the report that we’re in a world where rival nations have satellites that try to go up and mess with other satellites from other people. “In 2019, Russia launched satellite Cosmos 2542, which released a second satellite, Cosmos 2543. Cosmos 2543 then began following a U.S. national security satellite and approached it multiple times at a distance close enough to create potentially dangerous operating conditions,” the DIA writes. “Russia is developing kinetic orbital systems to permanently disable, or “kill,” satellites. Moscow’s Burevestnik is a developmental, co-orbital air-launched ASAT weapon for killing spacecraft in LEO”. “In February 2022, Russia launched a satellite called Cosmos, which Moscow claimed at the time was testing the effects of space radiation on electronic systems. This satellite was launched into an unusual and isolated orbit not used by any other spacecraft”. Space nukes: Russia is experimenting with nukes in space. “A nuclear detonation in space would create an electromagnetic pulse, which would immediately affect satellites within range and could impact civilian infrastructure,” the DIA writes. “A nuclear detonation in space could be tailored to impact satellites in most orbits—LEO for commercial and reconnaissance satellites, GEO for critical communications, and MEO for navigation and timing satellites”. Why this matters – infrastructure for the next phase of the AI power struggle: As people spend more resources on building space-based data centers (and eventually, perhaps, lunar-based ones), space will become ever more strategically and commercially valuable and ever more contested. Research like this gives us a sense of the hardware which is already in space and will arrive in the future. Perhaps in some far-off future an AI might seek to take control of such space assets as well. Read more: DIA Releases Challenges to Security in Space (Defense Intelligence Agency, press release) . Read the report : Challenges to Security in Space (DIA, PDF) . *** Metis: A startup trying to fuse symbolic and neural reasoning: ...Towards Vulcan-like machine minds... Metis, a startup that wants to build novel AI architectures oriented around developing machines with better reasoning, has published a research paper laying out its research approach. The high-level summa