Machine Studying with Jacob X. Li - Weaviate Podcast #148!

Jacob X. Li , a Ph.D. student at MIT joins the Weaviate Podcast to explain Machine Studying, a new framing for how AI agents can prepare themselves for a domain defined only by a corpus of documents, whether that's a code base, a pile of drug discovery papers, or a system's logs. Just like a student studying for an exam they haven't seen, the agent studies the corpus without knowing the downstream tasks, and still has full access to the corpus at test time. Rather than leaning entirely on inference-time compute the way Codex or Claude Code explore a repository from scratch, can an agent develop arbitrarily deep understanding ahead of time?The conversation dives into how the paper defines expertise: an expert is an agent that efficiently turns inference-time search into accurate work, measured as the weighted area under the performance versus compute curve, with the search budget of a ReAct-style agent loop as the knob instead of reasoning effort. Stacking that one level higher gives studying intelligence, expertise as a function of study-time budget, with retraining from scratch as the baseline. From there, Jacob and Connor discuss learning to search, including a literature review experiment where an outdated model with BM25 finds roughly the same papers as a newer model but is far worse at deciding which ones matter, and why studying is more than retrieval.Jacob then separates Machine Studying from continual learning, which is framed around sequences of benchmarks and often collapses into long-context processing, and lays out the three bets every studying method makes: self-supervised objectives or synthetic data, updating weights through LoRA, KV-cache cartridges or a trained retriever, and amortized context management like notes and cheat sheets. Cheat sheets helped on DSPy but less on the larger, newer OpenClaw code base, and weights versus context, along with where prompt optimization like GEPA fits, is left deliberately open.The episode closes with how human study habits like self-quizzing and annotating inspire these algorithms, agents that study you before teaching you, and why evaluation as a science matters when recursive self-improvement happens in domains with no benchmark at all!