NaiveAI Promises 2,100 Tokens per Second. How Well Does It Code?

Editor’s note: With this post, we are launching a new series focused on trying new models and products with unbiased reviews. Reach out with suggestions to tips@latent.space. Naive-N0.5-Flash is a new open-weights model built for AI coding tasks and tuned to generate as many tokens per second as possible . It was developed by a Chinese company called NaiveAI , with the aim of speeding up engineering and research workflows so that teams can experiment, iterate and solve complex problems faster. The model is an evolution of Xiaomi’s MiMo-V2.5 , which NaiveAI has trained further for long-context reasoning and paired with its own NaiveRT inference engine. Naive-N0.5-Flash arrives with a bold pitch: a model built largely by AI and trained to do AI research itself, with recursive self-improvement as the long-term goal. To see what changed, it helps to know where it started. MiMo-V2.5 is a 310B-parameter sparse Mixture-of-Experts model with 15B active parameters, trained on 48T tokens. Its language backbone inherits the hybrid design of MiMo-V2-Flash, which mixes sliding-window attention (SWA) with global self-attention. NaiveAI's recipe starts from those weights. Through continued, multi-stage training, the team adapted the checkpoint to DeepSeek Sparse Attention layers. According to the NaiveAI technical blog , this took roughly 3.25T additional tokens on top of MiMo's original 48T. About 3T of those additional tokens were trained at a fixed 1M context length and with a constant learning rate. Benchmarking NaiveAI vs. DeepSeek and Z.ai We benchmarked the ultrafast mode of Naive-N0.5-Flash against DeepSeek V4.1 Flash 1017 and GLM 5.3 Flash , and then gave all three the same app-building prompts. The trade-off, we discovered, is that Naive-N0.5-Flash takes longer to reason through harder problems, yet it still delivers answers faster overall. However, that comes at a slightly higher cost per question — although that may be an acceptable price for applications where speed comes first. DeepSeek V4.1 Flash 1017 and GLM 5.3 Flash are two of the most popular models in OpenRouter . Both belong to the Flash model class and demonstrate excellent capabilities in AI-assisted coding . As of this writing, performance statistics for NaiveAI models are not yet available on Artificial Analysis . Therefore, we report the Intelligence Index scores of the models used in our comparison: Model Intelligence Index DeepSeek V4.1 Flash 1017 (Max): 45 GLM-5.3-Flash: 43 Naive-N0.5-Flash: not yet available Published benchmark results offer some insight into the new model's capabilities. They place its coding ability between the two comparison models, and ahead of both on NL2Repo-Bench . The table below shows benchmark scores reported in the official model cards and technical blog posts of the respective models. Sources: GLM-5.3-Flash – Technical Blog , DeepSeek V4.1 Flash 1017 – Model Card , Naive AI – Research . Model NL2Repo-Bench DeepSWE v1.1 Agent's Last Exam (ALE) GLM-5.3-Flash 56.3 63.4 26.3 DeepSeek V4.1 Flash 1017 68.2 77.0 35.4 Naive-N0.5-Flash 71.9 67.8 32.4 Our own evaluations below look at the differences between these models in more detail. Speed and verbosity experiments Speed is the headline feature. 1 NaiveAI promises faster research iteration than comparable models, which matters to developers and AI researchers who need to test ideas quickly. 2 We evaluated each model on 10 randomly sampled, text-only questions at each of three difficulty levels: 10 easy questions from MATH-500 , 10 medium questions from GPQA-Diamond and 10 hard questions from HLE-Diamond . That makes 30 questions per model. All experiments used the same maximum completion-token limit of 256K. For each request, we recorded response speed (tokens per second, TPS), the total number of completion tokens, and end-to-end response time. The code for the experiments is at https://github.com/gbeltramo/llm-verbosity . The tables below report the mean and median across all 30 successful requests for each configuration. Throughput figures come from different sources: OpenRouter for DeepSeek V4.1 Flash 1017 and GLM 5.3 Flash, and server-side measurements for Naive-N0.5-Flash. They should be compared with that in mind. Key observations Naive-N0.5-Flash reports exceptionally high throughput , approaching or passing 1,000 tokens per second in both thinking configurations. Because its figures are server-side and the other models' figures come from OpenRouter, we do not claim an exact speed multiple over DeepSeek V4.1 Flash 1017 or GLM 5.3 Flash. Thinking has a large effect on verbosity. On the hard HLE questions, Naive-N0.5-Flash with thinking on averaged about 108,500 completion tokens, roughly nine times the GLM 5.3 Flash (high) average and 11 times the DeepSeek V4.1 Flash 1017 (high) average. The medians tell a different story from the means. Across all 30 questions, the median completion for Naive-N0.5-Flash with thinking on was about 3,200 tokens, far below its average of more than 51,000. A small number of requests produced exceptionally long reasoning traces and pulled the average up. The same pattern appears, less sharply, in the other models. The tables also include end-to-end response time, which is the fairer basis for judging speed. The median question took 3.1 seconds with Naive-N0.5-Flash (thinking), against 5.3 seconds for GLM 5.3 Flash (high) and 10.1 seconds for DeepSeek V4.1 Flash 1017 (high). The means are closer: 45.3 seconds for Naive-N0.5-Flash, 45.0 for DeepSeek V4.1 Flash 1017, and 86.9 for GLM 5.3 Flash. On the hard questions, Naive-N0.5-Flash (thinking) was slower on average than DeepSeek V4.1 Flash 1017 (high) , at 89.8 seconds against 79.7. Two caveats apply. The sample is small, at 10 questions per difficulty level, and we did not grade answer accuracy. Short, fast replies are only useful if they are correct. App experiments We also asked the models to build the same web apps from the same initial prompt. We tested six combinations of model and thinking effort, with the thinking effort in parentheses: Naive-N0.5-Flash (no thinking) Naive-N0.5-Flash (thinking) DeepSeek V4.1 Flash 1017 (low) DeepSeek V4.1 Flash 1017 (high) GLM 5.3 Flash (low) GLM 5.3 Flash (high) The gallery of apps is available online. In summary, the three prompts asked the six model variants to do the following: Create an interactive 3D view of the Earth and the Solar System. Grow a 3D tree from a seeded random generator, with branches that taper and fork, thousands of leaves, and bark and leaf textures. Do a browser port of the book Ray Tracing in One Weekend . Below we’ve embedded results for the thinking variants of each model . Naive-N0.5-Flash (thinking) DeepSeek V4.1 Flash 1017 (high) GLM 5.3 Flash (high) Solar system The first app is an interactive 3D view of the Earth and the Solar System. Viewers can orbit the camera around the scene and watch the planets circle the sun, with controls to speed up or slow down their motion. Every surface is drawn from generated textures, with no image files. Naive-N0.5-Flash (thinking) produced the best result of the three. Its sun shows sunspots, and the dark side of its Earth glows with city lights at night. DeepSeek V4.1 Flash 1017 (high) came close behind. Its version lacks an asteroid belt, and its Earth has less textural detail. GLM 5.3 Flash (high) stumbled on lighting, which makes the planets hard to tell apart. Its speed controls are also too sensitive, so a small nudge sends the planets racing or crawling. Tree modeling The second app grows a 3D tree from a seeded random generator, with branches that taper and fork, thousands of leaves, and bark and leaf textures. Wind makes the branches sway and the leaves flutter, and the shadows on the grassy ground move with them. The tree stands under a sky gradient and light fog. A control panel adjusts the seed, branching depth, leaf count, wind strength and direction, and sun position. There’s also a switch to autumn colors