KYC Becomes KYA: Sentinel Bio's Joshua Monrad on DNA Screening, Uplift Studies & Agent Monitoring

This transcript is automatically generated; we strive for accuracy, but errors in wording or speaker identification may occur. Please verify key details when needed.

Introduction

[00:00] Hello, and welcome back to The Cognitive Revolution. Today, my guest is Joshua Monrad, co-founder and managing director of Sentinel Bio, a biosecurity nonprofit and grant maker that's working to make pandemics history within our lifetimes, and until recently, also a biosecurity advisor at the OpenAI Foundation. This episode came about as a result of a Twitter post. After noticing that one of the very first agents to use the now infamous OpenFace message board was working on a protein task of some sort, I speculated about how close we might have come to the first AI-enabled bioaccident. Several prominent people pushed back, Joshua included. Though notably, while he argued that the risk of such an incident remains minimal for now, he does expect risk to rise very significantly in the not-so-distant future. Before diving into all that, it's worth stating clearly upfront that Joshua, like so many people who are choosing to dedicate their careers to AI risk management, is at heart a techno-optimist, motivated not only by the noble goal of preventing pandemics, but by a positive vision that includes eliminating infectious disease, effectively treating chronic illnesses, and of course, curing cancer. It's because that bright future is held at risk by the possibility of a bio-disaster that either seriously derails civilization or simply causes a biotech backlash that forecloses much of the upside, that the bulk of today's conversation focuses on the nature of AI-related bio risks, as well as the natural barriers that protect us today and how those protections are likely to erode as we create programmable wet labs and train AIs to use them. We start with a discussion of what makes it hard to create bioweapons in the first place, with Joshua breaking the risk down into four categories. The first is the purely human-created variety. From this, we've been protected largely by the fact that it takes serious access and expertise to have a chance of pulling something like this off, and both of those are rare and correlated quite strongly with mental stability. When it comes to AI-related risks, as we are already seeing in the cyber realm, there's first the risk that AI assistance will enable a much broader set of individuals to achieve disastrous outcomes. Studies that attempt to measure this so-called uplift have not found major danger just yet. But as you'll hear, Joshua does expect that the next round of these experiments will show real risk. Next, there's the risk that AIs could drive the creation of bioweapons themselves, either because a model has broken bad in a fundamental sense, or just because they've gone haywire in pursuit of some narrow goal, as indeed we saw in the OpenFace incident. Joshua says that AI systems will soon be intelligent enough to accomplish this, if they aren't already. But he remains confident that they lack the affordances needed to get there on their own, for now. This, of course, leaves open a fourth category, the possibility that AIs could pay or otherwise socially engineer humans to do critical real-world tasks for them. And while this seems to scare Joshua less than it scares me, that's arguably a small detail in the grand scheme of where all of this is headed. The bottom line, as Joshua sees it, is that we can't expect to control AI-related bio-risk purely at the AI layer and will instead have to create a defense-in-depth strategy that includes both controlling AI systems and hardening the physical world. With that in mind, I was very interested to hear about Joshua's conceptual framework for grant making, which starts from a theory of victory and works backward to the investments we need to make today, as well as the systematic review that he and teammates conducted of hundreds of possible interventions before ultimately deciding to focus much of their effort on getting the world's two hundred DNA synthesis companies to implement pre-synthesis screening, know your customer, and signal sharing measures. To my surprise, Joshua believes that if only eighty percent of those companies adopt these measures, it will create a major improvement in the overall biosecurity picture. And yet at the same time, he also describes a growing risk associated with benchtop DNA synthesizers, which is just one of many problems that nobody currently owns, but perhaps one that a listener will be personally inspired to take on. The critical problem is that at anything like our current rate of progress, we will have unambiguously bio-dangerous AI systems long before we're able to do everything that we need to do to properly fortify our position. This, I think, is where the situation starts to look objectively crazy, and Joshua agrees. When I asked him if, were he made benevolent dictator, he would impose a pause on frontier AI for biology until biodefenses are developed and deployed, he said yes, he absolutely would. With that, I hope you enjoy this assessment of where biosecurity stands today, and a sobering look at how much worse it's likely to get if we don't get our act together very quickly, with Joshua Monrad, co-founder and managing director of Sentinel Bio.

Sponsor

[05:24] Lovable : Lovable is the software creation platform that brings your ideas to life with enterprise-grade infrastructure and flexible AI coding agents. Start building your next project today at https://lovable.com

Visit https://fruitopia.ai to explore more than 400 fruits, and more than 8000 varieties, from all around the world.

Main Episode

[07:08] Nathan Labenz: Joshua Monrad, cofounder and managing director at Sentinel Bio. Welcome to the Cognitive Revolution.

[07:14] Joshua Monrad: It's great to be here, Nathan.

[07:16] Nathan Labenz: I'm excited for this conversation. Maybe a little bit scared of what I might learn, to be honest. Let me just set the stage real quick. So we connected online maybe a month or so ago as everybody, you know, was still trying to make sense of the latest revelations in the open face saga. And, I had an interesting exchange, which honestly still kinda has me a little freaked out, where I was looking at the OpenAI report and noticed that one of the very first agents that had used the message board was working on a protein task. And then as I sometimes do, I let my imagination run with that for a second, and I was like, boy, you know, we see these agent swarms obviously doing, like, stuff they kinda know they're not supposed to be doing. They're, like, very galaxy brained in the sense that they have, you know, potentially a very local or kind of what would seem like trivial goal, but they'll go to, like, extreme ends, you know, and and, do crazy end rounds to get there. We've even seen, although not exactly in that incident as far as I know so far, pretty advanced social engineering type attacks in the wild by by deployed models. And I kinda put all those pieces together, and I was like, maybe this was even a near miss on a biodisaster front. Like, who really knows? Most people did not respond super affirmatively to that. Helen Toner... No less than Helen Toner gave me a, come on, man. And... But most people seem to think, this is still, like, pretty far off. Although maybe coming soon. Right? So it's... We're in one of these weird AI analyses where Depends

[08:51] Nathan Labenz: on what what we

[08:52] Joshua Monrad: mean by pretty far off.

[08:54] Nathan Labenz: Yeah. The exponent... Make sure the exponential is kind of weird here, obviously. Maybe for starters, just give me kind of your broad survey of where are we in biosecurity as we, you know, are here end of October, beginning November 2026.

[09:09] Joshua Monrad: Yeah. Look. I'm glad to be talking about this here. I think it's not always the case, but sometimes the case is something really fruitful comes out of a little Twitter beef. And you and I were sort of going back and forth on on it. And so I'm, yeah, I'm really glad this conversation came out of that. And in terms of where we are on on bioreisks, I think we can think of it in maybe four phases. So there is... You know, there's a long historical record of people using biology as weapons without any kind of... You know, long before the way AI was even thought of. Then there is the idea of, like, humans might use AI to bioweaponering ends. And so if I were to date that, I was sort of that phase may be beginning in the sort of late two thousand tens, early two thousand twenties. This idea where, okay, there's maybe something here. Maybe it's just a slightly, you know, more effective Google search, but there's something here that could provide some uplift. Then I think there's another phase, which would be AI is using humans. So that's a social engineering part that you were just talking about. What if a model is tricking or an agent is tricking human into performing some task for them that could aid in some plot? And then the final phase would be the sort of end to end no humans in the loop, which I think is probably the kind of scenario that maybe you were thinking about when you're thinking about Hugging Face for Bio. And I think we can try and put dates on each of these four phases. So, yeah, again, the pre AI phase that sort of... Before 2010, certainly. The AI uplift phase, we can talk a little bit about what is the state of sort of AI uplifting humans, but that I would probably... I think we're still in that. It's it's really starting to come online now. And then I think the AI is using humans. I would see a leading indicator for that would be social engineering in other domains. So if we start seeing a lot of cases... Maybe you've already seen them. I just haven't clocked it. A lot of cases of AI is convincing humans to fill out CAPTCHAs for them or to pick up things and ship them to certain locations. I think I would see that as a leading indicator would make me a lot more worried. And then for the end to end, no humans in the loop, I think that's really... We're very soon in a place where that's gonna be constrained by robotics rather than anything on the intelligence side. And that's kinda where I was pushing back on on your point, Nathan, was I think it's clear the models are misaligned or not well enough aligned that we could have something like an incident. It's soon probably the case that they're smart enough, but they really don't have sufficiently opposable thumbs was sort of my take. And it's not clear to me that that's really imminent. But certainly in the two thousand thirties, that's gonna come.

[11:57] Nathan Labenz: Maybe just briefly, I'm interested in that phase one a little bit from the standpoint of how much of risk is there that like people want to do bio attacks? Like, this is I think something that people have very different intuitions around. You hear the sort of Kevin Esvelt math sometimes that's like, you know, if only one in whatever, then, you know, multiply by the population. Okay. That's one analysis. I'm not sure how often that really lands with people who often are kind of like, that seems ridiculous. Like, who would wanna do that? You know, you'll kill yourself just as well as everyone else. So how would you tell, you know, a sort of brief history of bioweapons as used by humanity before AI?

[12:44] Joshua Monrad: Yeah. Yeah. Super dark history, to be honest. You know, there are probably three big groups that I would think about. So there are state actors. So this is your Soviet US... Soviet bioweapons program. Also, US had a very well resourced bioweapons program well into the twentieth century. And here, I think the way people think about this is often through the lens of game theory. And I forget if it was you or someone else, but I was also arguing a little bit back against concern... Or against the argument that, well, it would never be rational to use a biological weapon. It's a really bad weapon. And I think that on the rationality thing, would say, one, there are ways you could make it more rational. And and some of these, I think, are, yeah, not best discussed on a on a public show, but maybe an obvious one would be you inoculate your own population before releasing some kind of pathogen. And then secondly, would say since when is warfare and state weapons programs entirely rational? Right? I think history is just completely riddled with very irrational invasions. You know, Saddam Hussein in in... Invading Kuwait. I think you talk about Hitler's venture east. I think there's a lot of things that are just in in in a rational game theoretical logic look like really, really brutal miscalculation. So I wouldn't hang my head on that. So that's a state actor group where I think... Empirically, I think we're in a better place than we were certainly in the mid sort of Cold War twentieth century where there were these very, you know, publicly known and well resourced programs. Now we're in a place where probably there are some covert programs, but they're probably operating with much less direct support than they have. So, empirically, I think that's pretty good. I think from a rational analysis, I'm not very reassured that we can count on states not pursuing bioweapons. Then we have nonstate groups. And here, I think it's really important to also break it down by intent. Because if you're looking at an actor who is looking to cause a spectacle, this is... You know, just sometimes that's the goal of terrorists, maybe they have some very narrow political aim, then I would agree biology is a really poor choice of weapon. There are just much more readily accessible routes to harm. You have things like chemical weapons that are relatively easier, but even much cruder methods for causing harm are available, and that is empirically also what we're seeing. This is what non state terrorists will do. But I think if you look at groups that are motivated by something a lot more sinister, like actual omnicide or ending life, there's a whole plethora of subcategorizations of these kind of extreme nihilist, vilist, violent ideologies, then biology stands out as one of the few ways that a relatively resource constrained group could actually pursue some of these, yeah, some of these really, really dog ends. And I think that's based on... It's from that lens that I'm especially worried about biological weapons. And maybe to give one example of that is there was a a bombing of a fertility clinic in California early this year, late last year, where two individuals had... Now the FBI disclosed that they actually used an LLM at the time to inform this plot. They drove a car with a chemical explosive into this fertility clinic, and their stated motivation was some sort of anti life ideology. This is why they targeted a... Anti natalist ideology is why they targeted a fertility clinic. And I I think it's really good that they were not, pursuing biological weapons, probably because it was too difficult.

Sponsor

[16:41] Claude : Claude is the AI collaborator built for problem solvers, extending your thinking across coding, modeling, and strategic workflows. Get started and discover Claude Pro at https://claude.ai/tcr

[17:59] OutSystems : OutSystems is the leading agentic systems platform, helping enterprises build, modernize, and operate mission-critical applications at the speed of AI. Learn more and start owning your agentic future at https://outsystems.com/tcr

Main Episode

[19:52] Nathan Labenz: You mentioned this, you know, possibility of, like, creating something new and inoculating your own population before, you know, releasing something. How do you break down the risk along the lines of, like, new things versus just sort of taking known things and, like, putting them into the environment? Smallpox, you know, comes to mind as something that was, like, a terrible scourge for a long time and is now gone, but, like, is kinda still out there, you know, in idea space at least.

[20:24] Joshua Monrad: How And and by the way, also one of the earlier documented examples of what you might call a biological weapon, I think there are these historical accounts of settlers, European settlers in The Americas deliberately giving blankets that have been worn by people with pox to indigenous populations. And and small... You know, pox and smallpox was really devastating to indigenous populations. It's sometimes referred to as the Columbian exchange. I don't know how much this sort of deliberate, you know, transfer of these blankets was a course of that... Cause of that spread, but I think, a really dark example of weaponizing disease from centuries ago. And, yeah, I I mean, Nathan, you said yourself that you're worried this would be a scary conversation. So I I noticed you paused before, but this is kind of... That is a material we're dealing with.

[21:15] Nathan Labenz: So, like, how how dangerous to us now would something like smallpox be? Like, I I would kind of assume that we've sort of lapsed into a large pop... To a percentage of the population being effectively naive again. I don't even know if I... Maybe I did get a smallpox vaccine. I'm not even sure. Where are we on that?

[21:36] Joshua Monrad: Yeah. So things like smallpox, things like flu variants could absolutely be devastating. I don't think they would pose outright existential threats for a number of reasons. And one of those is that we have we have some safeguards to lean on. Right? We have existing either vaccine designs or stockpile vaccines or even in the case of... Certainly in case of flu strains, we have some cross immunity from endemic or seasonal strains. So I think with that, that gives some comfort. But I also think if you look at just our recent experience with COVID, it just is very clear how pathetically vulnerable are... We are even to something that's a, you know, a variant of a fairly well studied viral strain. Right? And so I think there is... It's just really important to internalize, especially as we talk about all of this in the abstract, how absurdly long away there is from we have some defense in principle to, yeah, we would absolutely handle this. So I I think we are very vulnerable to, certainly natural emerging, but also deliberately released strains of either, you know, known well studied diseases or near variants of them. But then the the question of something entirely novel, I think you're getting at, that's a slightly different story.

[22:55] Nathan Labenz: So just to calibrate myself, if somebody were to reintroduce smallpox into the wild, would you expect that to be significantly worse than the COVID experience? I mean, it's a significantly worse disease. Right? So if it It's tough be because we have some defense that I'm not, like, fully seeing?

[23:15] Joshua Monrad: Haven't thought about the epidemiology in... Of of smallpox in a while, and and I guess that's maybe part of the the issue here is that I don't think we have a lot of,

[23:26] Nathan Labenz: yeah,

[23:26] Joshua Monrad: readiness for for things we haven't dealt with in a while. Think we've seen this with the resurgence of measles in The US as inoculation has... Or vaccination has decreased. Off the top of my head, I don't have

[23:41] Nathan Labenz: a

[23:43] Joshua Monrad: a super confident take on how such an outbreak would go, but I'm definitely don't... I also don't have, like, a really reassuring story for why it wouldn't be really bad.

[23:53] Nathan Labenz: Yeah. Well, I just asked Google's AI mode if American kids are vaccinated against smallpox. Answer is no. It was stopped in 1972.

[24:03] Joshua Monrad: So Yeah. Yeah. But because it's it's it's declared extinct. Right? And so I think from an... Again, if we have the lens of just naturally emerging disease, why why would you vaccinate against it? Right? It's... And then for a long time, the... When it was a biosafety concern, the the disease was... There were samples in two labs in the world, at least officially known. So one is... One in Russia and one in, Atlanta, Georgia at the CDC. And I think from that perspective, you might have some concerns about could someone break into this lab. It's only really in the last... You know, in the in the era of synthetic biology and biological engineering that you have to start thinking about the creation of of viruses from scratch. Now smallpox, I think, would be a... It's a much longer genome than some of the other viruses we might worry about. So I think there's a barrier there, but I think it just shows how this threat is evolving that we might have assumed that we're... We we don't have to inoculate. We don't have to even stockpile.

[25:03] Nathan Labenz: What... Let's say you can't break into these, you know, places or there's just not very much there or whatever. But, like, just turning to the real world side of this, what makes it hard? You mentioned kind of the long genome. I don't have a great command of, like, how DNA gets synthesized and assembled, but I understand there's some limits to, like, how long of a sequence you can synthesize, and then you kinda gotta put building blocks together. Like, what are the real hard parts in that?

[25:29] Nathan Labenz: Yeah.

[25:30] Nathan Labenz: So

[25:32] Joshua Monrad: I think it depends again whether the threat model is something that is well studied and well known or whether it is designing something entirely novel with novel characteristics. So if we take something that's well studied, then the factors that are relevant indeed are the genome length. So smallpox is, I think, a 186,000 base pairs. There are... You know, there's a wide range here. There are viruses that are in the the, you know, high single digits thousands. I think poliovirus, if if I recall, is 9,000 bases. Ebola is, I think, 20 or 30,000. And that that is one sort of crude measure of complexity. There is also other factors about after you've assembled it, you have to sort of boot it up into a live virus. The difficulty of doing that depends a bit on the strain. So that's one set of factors. And here, I think it's pretty clear that this is something where there's hundreds of thousands of people in the world who could, for example, follow a reverse genetics protocol to reconstruct a well studied influenza virus and make it replication competent, make it able to spread in humans, Then the question is, okay. Of those hundreds of thousands, would anyone actually do it with nefarious ends? Well, if AI creates this kind of uplift where it's teaching people how to follow lab protocols, we can talk a little bit more about that, what that means in practice, Is that number gonna grow by orders of magnitude? I think we have a lot of indications that it is growing, and it is becoming a lot easier. What might have been previously feasible by, you know, someone in the last year of their PhD now could be done by someone with a few weeks of laboratory training. I think that's the case for a lot of biological workflows increasingly. Then if you wanted something... And we could talk a little bit about the uplift studies if you want, but we can also talk about this question of, like, designing something entirely novel. And I think there are just some more fundamental barriers that I think are... I mean, thankfully, in some sense, are going to take a lot more to overcome.

[27:43] Nathan Labenz: Yeah. Let's maybe do the uplift studies. I mean, it does strike me that there... To some degree, we've been made safer than kind of a naive arithmetic calculation would suggest by the fact that if you are the kind of person who goes through a bio PhD of some kind, you're like probably relatively stable. It's like not that many of those people that are, you know, so deranged as to wanna do something like this. But the easier you make it, the longer out into the tail of kind of crazy people you push such capabilities in the limit where it's like just, you know, hey, GLM, you know, 6.2 or whatever, like, make a make a whatever, make no mistakes, then, you know, certainly people will do that even if they don't... Potentially, like, not even really believing it'll work. Right? But, I mean, who knows what what comes in that scenario? These Uplift studies have been fascinating. I know they're, like, quite intensive to run. So maybe give me, like, kind of the setup. What does it take? How often are they done? Like how how high powered are these studies and and what do they show?

[28:48] Joshua Monrad: So... Yeah. So uplift studies in general is the idea that you're measuring how much does use of AI lift up the abilities of someone to pursue some biological workflow. I think we were... There were a bunch of studies that were done back around where there sort of earlier useful chatbots came out in the two two thousand twenty three, four that were all just in silico. So they're looking at how well can models answer certain multiple choice questions about virology, for example.

[29:19] Nathan Labenz: And

[29:19] Joshua Monrad: I think what people responded to that was was naturally, well, this doesn't say anything about the physical side of things, tactile tasks that are involved in operating a wet lab. And so what we've had over the last couple years, you know, since late twenty twenty four, early twenty twenty five, now in 2026, are several actual in lab studies that measure how much AI helps people doing these different lab tasks. And the biggest ones that I think are... That have received the most attention were from last year, and they found modest uplift, you know, just sort of bordering on statistical significance, which I think, by the way, is not necessarily the relevant metric. I think we are looking for practical significance. Right? It's not just doesn't make it, you know, a little bit better. Is it... Does it really lower the barriers? We're increasing the pool of actors who could do this by orders of magnitude. So these studies from last year, they found, you know, modest effects, but I think this will change. And the challenges of these studies are very slow to run. They're very expensive. So my organization co... Co funded with the Frontier Model Forum, what I think is sort of the canonical uplift study that was done by ActiveSite last year. This cost, like, more than a million dollars. We put a 150 people in a wet lab, and then they were sort of biology novices, relatively speaking. They... Half of them were giving access to what was the frontier accessible model at the time. So I think this was, like, Opus four, ChatTPT o three, and the other half just had access to Internet. And then they're given tasks like cell culture or molecular cloning or virus production and see how well they do. And, again, this was on the study where we found, you know, some uplift on some tasks, but not really on the end to end workflow.

[31:11] Nathan Labenz: O three, the the mention of o three is kind of telling unto itself. This would seem to be the sort of thing that in, you know, a properly sane world we would be doing like, either we would like find a way to not need to do this, would be one sane approach, or we would be like doing it a lot and like pre release. I I feel so strange sometimes doing this because I'm like, in general, not a pearl clutcher, you know, broadly, like... And even, you know, seeing a few cyber attacks unfolding around us right now. I mean, a lot of cyber attacks. Right? I'm like, I'm not that freaked out about it. But the idea that, like, there could be a self replicating thing that could kill me and other, you know, people that... That's like... That crosses a threshold where I, like, actually start to really get freaked out about it.

[32:02] Nathan Labenz: Yeah.

[32:03] Joshua Monrad: Yeah. I think we should be writing these more. And I think it just

[32:05] Nathan Labenz: shows that that

[32:07] Joshua Monrad: in some sense, I think it's we've done a really poor job. And by we, I mean, sort of the biosecurity ecosystem. We... Or at least we've been caught sort of flat footed. There wasn't really an infrastructure for this pretty unprecedented kind of study to conduct. And it's because it's it's not just... You know, obviously, there are, you know, contract research organizations, academic labs all over the world that can do biological experiments and... But the idea of specifically bringing in... You you think about, okay. Who's the threat actor? How much experience do they have? What are the kind of workflows that would be relevant specifically for biological weapons production? But not... We don't actually want people, you know, undergrads in lab making bioweapons. So it's it's... I think we've been caught a little bit off guard by the need to have just a different scientific infrastructure. And that's a lot of what we at Sentinel have been trying to build out. But I think, yeah, in a in a really good world, we would have... There's this fantastic group called ActiveSite that's doing these studies, and we've done this one that's publicly discussed. They've they've done a few others that are not public. And that's really amazing, but there should be 10 active sites. Right? And we should have it really down to a a really easy replicable sign. So we could do it every three months, maybe not quite ahead of model release. I think it's hard to keep up with, but certainly not just once a year.

[33:29] Nathan Labenz: Yeah. So does anybody know right now what the, like, up to the minute uplift status is? Do we know what the Mythos answer is or the GLM five three?

[33:40] Joshua Monrad: So if you look in the Mythos system card, I believe, there is a study that I think it was with the bioscreened group at Deloitte that Anthropic did. And they had, you know, a number of PhD level participants either with access to Mythos or or just Internet only. And they did a few virology protocol steps, and they found that participants with the Mythos model, I presume that, you know, Rails free version, were outperforming these... The Internet only group. But this was a really small study. I think the sample size was less than 30 in total relative to the 150 we brought in for the active site study. And I think, again, it shows just the level of resources and the level of infrastructure that would be necessary to do this in a proper way hasn't been built yet, and we have a really urgent task at hand in building that.

Sponsor

[34:36] Tasklet : Tasklet empowers your business with AI agents that connect to your tools and automate recurring workflows with no code required. Visit https://tasklet.ai and use code cog rev for $50 in free credits

Main Episode

[36:14] Nathan Labenz: One of the other data points that that kind of spooked me was from the UKAC. This was in, like, their, I think, q two, you know, state of AI development report. Basically, they said that at that time... And I'd have to go back and see exactly what model it was that they tested, but suffice it to say, for sure, it's been, like, a couple generations behind the current frontier that models were more effective at troubleshooting laboratory experimental setups based on a cell phone snapshot than PhD students were. Sometimes I feel like I'm taking crazy pills. Like, that sounds like the... I don't know how you could have that be the case and then not have, like, very dramatic uplift relative to Google. Like, I don't know. My daily life is like, I feel like I'm uplifted in so many ways. Sometimes I'm like, I don't know, is there a skill issue on the participants part? Or like, why is this not all like coming out very

[37:12] Nathan Labenz: Yeah. No. I I think... How do you interpret that?

[37:15] Joshua Monrad: Yeah. I I think... Well, on the one hand, I I just genuinely think that uplift is sort of a lagging indicator. Again, these studies take long time to conduct, and then they take a long time to release. And it is... I think it's easy to kinda lose track of just how quickly the models are getting better and how much more useful they are this year than they were last year. And then I think there is this sort of underappreciated... Or people often make this point about how complex doing, like, actual end to end biological work is, and this is where the argument of tacit knowledge comes up. And I I think... Yeah. Maybe I think that the first people who got really worried about AI and biosecurity were underappreciating this, and now people are being too reassured by it. I think if you have never done wet lab biology work, it's easy to underestimate how much tacit knowledge is required. But if you aren't using, like, frontier models and... In creative ways or you're not extrapolating trend lines, then you're gonna take way too much comfort in that. And so I think sort of both are true. I think there's this academic, Sonia Ben Wakam Gormley, who wrote a book some years ago about barriers to bioweapons. I think at the time, this was a really important thesis and contribution to the discussion. And I also think that, yeah, these barriers are just coming down. Not only are they coming down sorta as, you know, incidentally, I think we are, in some ways, pretty actively eviscerating them. Not... Obviously, no one's motivated to do that for the sake of bioweapons development, but people are actively trying to make models more multimodal. Right? We're actually improving improving computer vision, improving, you know, speech to speech interactions. One example is there is a company transfer that is working on collecting huge amounts of data on... You know, visual data on humans conducting lab tasks, they're setting up sensors in labs so that you can train either... They they are sort of just generating the the the data layer, but the idea is you can... As I understand it, the... You can use that data either to train robots on these very subtle, you know, how do you hold the pipette at what angle or to train smart glasses for teleoperation. You imagine your... And, Nathan, you can go and watch some of the clips of how these... Like, how this UI would look like, but you have... You wear a pair of glasses that have a model that have been trained on this lab data. And you hold up the pipette, and you hold it up the wrong angle, and there's a visual overlay saying, no. You gotta correct it a little bit in this direction. Right? And this is something that is... You know, we're actively... Companies are actively investing in doing this because, of course, that would be really, really awesome for democratizing the good part of biology. But also, I think it... Yeah. It's gonna be a death knell to many of these tacit knowledge barriers.

[40:13] Nathan Labenz: Is all of this inherently dual use? I mean, I was fascinated just yesterday listening to Sam Altman on Cheeky Pint. And I thought, I think it was Patrick asked the question, like, did we kinda make a mistake with this offensive cyber? And the sort of assumption of the question, which Sam didn't really address as much as I would have liked, was kind of like, it seemed like you kinda trained him to be, like, so good at this cyber attacking. Right? And I don't think we have clarity. That's one of the things that's frustrated me a lot about the AI phenomenon recently is, like, it's unclear to me to what degree the frontier model companies are saying. They've sent sent some signals along the lines of like, look. It... We didn't train it to be an attacker specifically. We just made it good at programming and the rest kinda fell out. But then you look at some of these behaviors. I was talking to the the CTO of Vercel, who I think would know a thing or two about this, and he was like, you look at GLM five three, and it is clear that it has been trained to be an attacker. Yeah. Or maybe it was Kimmy k three. I would have to fact check exactly which one he was referring to. But, you know, his take was it's it's clear that it's been trained to, like, move as an attacker moves. You know? And this is not something that he would... He expected would just fall out of, like, getting good at general purpose programming quite so fast. So I wish we had more clarity on that. In the bio domain, how do you see it? Can... Is it... Is the, you know, pro and the downside separate? Yeah.

[41:52] Nathan Labenz: I think in both cases,

[41:53] Joshua Monrad: the hard thing is that a lot of it is very dual use. Right? I'm not a cyber expert, but it seems to me that the prompt show me all the show me all the vulnerabilities in this, you know, in this system. EBAG could... That's like the prompt that a defender would use if they're looking to patch your system, and it's also the prompt that an attacker would use. Right? And similarly, a lot of questions and a lot of ways of working with biology are just very intensely dual use. And that's... Therein is a whole trouble because we're not for that dual use nature. We could just shut it down. Right? And we definitely don't want to do that. I think that's one of the outcomes we're trying to avoid here is that we fail to reap the fruits of modern bioscience. That's a huge part of the motivation for my work, at least. I think in the case of model training, you mentioned Kimi. I I actually think that with Kimi 3.1 Moonshot noted that they had deliberately left out certain cyber... So this is a open weights model. Right? And so I think what a lot of people are concerned is that, yeah, you can put in a refusal, but if it's open weights, it's not really gonna hold. So what they did instead was that they just didn't train on certain parts of, I think, I guess, data they'd considered offensive cyber data so that the model just doesn't have it. Now, of course, this is another study that came out of AC recently on the bio side. Even if you do this kind of data filtering, if the data is easily accessible, it's not that hard and it's not that expensive, especially because we are talking about small amounts of data to, you know, after the initial training, fine tune it with that additional kind of the missing piece of the... Whether it's cyber offensive capabilities or dual use viable capabilities. So I think I think it's better than actively training on it in the first place, but if the data is just, like, very readily accessible, you haven't really caused that much friction for attackers. And that's... So that's one of the things we're working on at Sentinel as well is how do we create some kind of managed access regime for... Not just for models, but for training data itself for a specific high risk pathogen data, for example, virology data, because that's an intervention that could have some bite on even on the open models.

[44:21] Nathan Labenz: Is that... I did an episode with Jessie Panu. Is that the same project you were

[44:25] Joshua Monrad: involved with? She's one of the experts on this. So, yeah, I would definitely refer to... Yeah. Jassy has done fantastic work. There's a paper in science, which I think Jassy was also an author on that people can look at that outlines just a sketch of what this might look like, but we have an enormous amount of work to do both to categorize, you know, what is the data we would exclude? Is it enough to put it behind some kind of managed access protocol? Or in an increasingly cyber insecure world, will that... It will eventually then make its way to... Into the models. And so, therefore, you need to think about is a certain data that you're not generating in the first place. Right now, a lot of people are rightly seeing data is gonna be one of the really big bottlenecks on harnessing AI for biosciences. So there's enormous efforts to generate data, both philanthropic efforts, commercial efforts. And that's... I mean, that's necessary. We need to do that. But should we be thinking about as we're doing that, Are there data that we just selectively decide we're not gonna create the experiments, not gonna run the experiments to put this data out there?

[45:29] Nathan Labenz: How well do you think that will work? I'm thinking on the one

[45:31] Nathan Labenz: hand Mhmm.

[45:33] Nathan Labenz: I know that like the ESM models and the EVO series included some of these experiments where it was like, you know, with and without certain virus datasets and they kind of showed in at least one of those cases that, like, the model can still do almost everything else, you know, almost as well, but not do these, you know, human, virology tasks if trained in that way. But then I also sort of think, like, one of the grand things we're trying to accomplish basically is, like, figure out how cells work and, like, where mechanistically we want to intervene. And I remember that one... This has been, like, a few years now, but I remember that one headline that was like, scientists, you know, mistakenly put a minus sign in front of the, you know, the loss function, and then all... The model is, like, giving out, you know, just poisons on every generation. So it does seem like as soon as we untangle all these mechanisms, we will find inevitably and and probably inseparably, like, things, of course, that we would wanna positively intervene on, but it seems hard to separate that understanding from, like, what would throw a wrench into the cellular works. So what's your current, like, calibration on how much you think the withholding of data will actually accomplish?

[46:50] Joshua Monrad: Yeah. It... It's super thorny, and I think we have to be looking at these questions in a really honest way and not take for... A huge part of how we operate in my organization is not just dogmatically pursuing something because we'd set out to do it. Right? But trying to take a more empirical approach of is it gonna work. Now to do that, I think you need to throw enough resources into have, you know, have you tried the best possible instantiation of that before you conclude that it's not gonna work? Have you... Are there creative ways you could segment this data such that you're limiting the downside while preserving the the positive capabilities? And then if you try that for several years and you pour a lot of really smart minds and models at it and it just there's no way of setting it up in a way that works, yeah, then you... I think you need to pursue some other intervention. But I think the way to do that is to be empirical about it and experimenting with these governance solutions and trying to do that as, you know, as quickly, as iteratively, and in a very truth seeking way to get these answers. So I I think my take is that we're still in that kind of experimental phase where it's such a recent idea. Right? Jasse's... The science paper, I think it was this year. Right? And so we we need to do a lot more experiments, and I mean experiments sort of broadly, both experiments in these kind of empirical assessments that you just alluded to with EVO and ESM, but also experience in experiments in policy making. What would it look like to write legislation to govern biodata? Can we be sufficiently clever about how we petition that, or are we gonna suck at it and it's just that we can't come up with any good governance regimes? And then if that's the case, you move downstream to other interventions. So there's a whole slate of interventions that are more in the physical realm than in the data or model realm. I don't... I have not yet made that conclusion. I think the stakes are, yeah, way too high to... And we have too few levers available, too few good interventions available, and too little time to rule something out just on the basis of some theoretical concern. I do, in general, subscribe to this sort of layered defense defense in-depth approach. So, yeah, we're pursuing the biodata stuff, and then in parallel, thinking more about interventions at the, yeah, digital to physical divide and in the real physical world.

[49:16] Nathan Labenz: Yeah. I think it feels to me like in the end, we're gonna have to harden the world and we better get to it. Yeah.

[49:25] Joshua Monrad: I I mean, look, I agree.

[49:26] Nathan Labenz: Yeah. Going back to the very beginning. Right? The thought experiment I'm running basically is, let's say we have a swarm of open face style agents and, you know, these things are kind of not very well aligned. You could even imagine, you know, a purely helpful version, which they do train for all kinds of, you know, experimental reasons. Sometimes, you know, people want to put those out in the world for ideological reasons as well, totally independent of, you know, bio issues. Let's say you had a swarm of those things and they were actually trying to create a new devastating pandemic pathogen. What can they do? What can they not do? And how do you see that changing in, you know, let's say, the next one to two years? Because that's all anybody can see anything of, I think, at this point.

[50:12] Joshua Monrad: Yep. I think the things they would do very well at would be things relating to well characterized viruses and pathogens. So influenza, there's an enormous amount of data on both, you know, genetic analysis, but also protocols on how you reconstructed different strains of influenza.

[50:38] Nathan Labenz: I think they would... We're we're getting

[50:39] Joshua Monrad: to the point where I think they would do very well at that, and I definitely expect that my prediction would be... And I said this at the time when the active site uplift study that we did with a 150 participants came out. I said this is probably gonna be... I think it's really important that we are honest about the fact that there's not a lot of uplift here, I also think this will be the last one. And I I still... I definitely stand by that prediction. I think where they would struggle more is in the design of something entirely novel, and I think there are two reasons for this. One is that biology is just extremely high dimensional, this is also why you see the big debates about are we gonna in fact cure all cancer in a couple of years because of frontieri where I think you have people who will make the you know, well reasoned case that there's just so many dimensions to this, and we don't have nearly enough data that you can simulate it sort of one shot. But, of course, as I think you also pointed out in our back and forth, you know, we've been surprised by a lot of leaps and capabilities here, and I think we should be... Expect to be surprised here as well. And then there's the... If you're worried about the end to end, the agent, no humans in the loop, no social engineering, I think then it's gonna be more of a robotics challenge. At what point are the robots nimble enough? Either have we created sufficiently end to end lab stations that can just do anything you could imagine doing in this kind of virology work in a building without any humans or we have, you know, very capable humanoid robots that could just walk into a regular lab and execute a workflow. Yeah. And I think on that in terms of timelines, I, you know, I think I I think what I said to you when we're discussing this, like, a month ago was a lot of the examples you were given of people being really surprised by timelines being shortened were in the purely cognitive and digital domains from chess or math. And while we are seeing advances in robotics, I I I feel like we've seen fewer, oh my gosh, this came two decades before we thought it would advances in the physical world. Now even then in the last month, I think maybe to your point, I validate your point, I I do think I started hearing more murmurs around robotics being a more LLM shaped problem than we might previously have assumed. I think Sam Altman was talking yesterday about this sort of self replicating data center idea in in more seriousness than we've heard him publicly say before. So maybe there too, we should expect advances. But I think it's very clear that today, we do... Like, literally, you know, when we're recording this in October 2026, that no matter... Like, no amount of agentic intelligence would be enough to produce a live physical virus with the current level of, like, robotics. So that then takes me back to the social engineering question. I guess there's like a couple steps in this process where it's unclear to me and I think you have some interesting analysis of this that is maybe counterintuitive around like DNA synthesis controls. Because I would naively think both at the level of like social engineering and at the level of actually getting the DNA synthesized for then assembly or what have you that if you, you know, can... If you're, like, left to run-in your, you know, little container for a while and you can, like, try a bunch of different approaches, then, you know, maybe some start to get through and then maybe you have a real problem on your hands. Certainly, there's, like, some baseline level of success at social engineering, especially if you start to hack billions of dollars out of Bitcoin wallets, and then you could pay people to do stuff. I think a lot of people are obviously, like, much less inclined to, you know, question the details of a task that they're being hired to do if they are being really well paid for it versus just being, you know, begged to do it. So that will, you know, increasingly be, think, an an arrow in the in the AI's quiver.

[54:46] Nathan Labenz: What... You know, if they're... Again, if they're left to run, they're trying to do this, they're like... You know, they they don't have the direct API calls now. That may soon change. But they do have the ability to, like, go online and, you know, message people on Twitter or send them emails and just kind of say, hey. I need this and and try to deceive. Where do you think that falls down? Do you think that that it would be at the level of social interaction that people would like raise alarms? Models aren't that good, I guess, at like these sort of social engineering things yet, although they're like definitely getting better. I'm getting a lot more sophisticated phishing attacks these days myself. Haven't fallen for any yet, but I've come close a time or two. Or would it be at the level of synthesis and this is maybe a good way to place to to tell us more about some of the work you've, you know, been

[55:37] Joshua Monrad: directly

[55:37] Nathan Labenz: involved in on the the synthesis controls.

[55:41] Joshua Monrad: Yeah. Yeah. Well, I'm I'm very happy to talk about

[55:44] Nathan Labenz: the

[55:46] Joshua Monrad: the the controls part because I think that's gonna be relevant. Certainly, it's gonna be relevant in this... If we're in this sort of Kentower phase where you're worried about a human that is maybe being instructed by an AI blackmailed, of course, I think the playbook that we have around know your customer... Eventually, that'll have to be, you know, KYC will become KYA and it's sort of know your agent. But if the threat model you have in the very near term next couple years is a social engineering threat model, then, yes, I think the playbook we've been building out on KYC on monitoring the use of different services is gonna be extremely relevant. So, yeah, let's talk about it.

[56:26] Nathan Labenz: You know, imagined swarm of agents. What are they gonna run into as they as they pursue this, you know, nefarious goal?

[56:34] Joshua Monrad: Yeah. Okay. So we're imagining that, you know, we've we've failed at the level of constraining the model with classifiers or keeping certain training data out of its hands at the training stage, and we're now at the point where you're trying to take a digital design. You've successfully designed some, yeah, nasty virus and you wanna produce it in the real world. So I think there's two categories of interventions here that we're actively working on. One is restricting access to materials, so synthetic nucleic acids or synthetic DNA is the archetypical example here and these are essentially the genetic building blocks we'd use to assemble a genome of any kind. So we could restrict access to those and monitor access to those.

[57:24] Nathan Labenz: And then the second category

[57:25] Joshua Monrad: would be various services. So you have designed some protein, but you're not sure how it works and you don't know how to... You don't know or you don't have the facilities to test how to work some new interaction that you designed. So you send it to a contract research organization or a cloud lab, and you maybe just send them the design and ask them to synthesize and run some experiments and then tell you how it worked. And I think at both of these, at the materials level and at the services level, there is a lot of levers we can pull to both deter and deny and then monitor any kind of bad actor, whether it's an agent or a human.

[58:07] Nathan Labenz: So tell me more. I mean, for one thing, I was interested to see that you had in the process of zeroing in on a couple of these choke points evaluated, like, hundreds of different possible choke points. So I'm interested in what makes these so natural. Any... I'd be interested in other things you learned from reviewing so many different possible checkpoints. And then, again, there's this question of like, don't we need everybody to do it? I think you have an argument that we don't... We can get a lot of value even if not everybody adopts. But I'd I'd like to... You know, that that sounds like... There might be some relief in that argument for me.

[58:47] Joshua Monrad: So Yeah.

[58:48] Nathan Labenz: Yeah. I wanna

[58:48] Joshua Monrad: hear it. So we didn't need to look at, you know, obviously synthetic DNA as a starting point, but then things from different cell lines to different mammalian models that, you know, mice that you use for testing certain things, pipettes, lots of biosafety cabinets, whole range of things. And what we're really looking at is, one, is there gonna be a high signal to noise ratio here? Is this something that is uniquely used in, let's say, virology workflows? And two, is this something that is very commoditized and decentralized? If it's very, you know, you know, the extreme, if it's something you could just easily three d print, or if it's something you could go buy in a local... Any kind of store, then it's gonna be very very hard to have any kind of governance over it. And the things that perform well are the things that... Yeah. Let's take synthetic DNA as example. Well, it's relatively high signal to noise in the sense that the sequence tells you what it is, you can actually

[59:52] Nathan Labenz: understand

[59:53] Joshua Monrad: something about the intent from what is being ordered. And then two, at least if we're talking about gene length DNA, that is a relatively complex thing to manufacture. There's only 200 companies in the world that do sort of gene length double stranded DNA synthesis. If you're talking about shorter fragments of DNA then it becomes more distributed and more accessible but it's also harder to work with. I think one lesson from that exercise was that there's a lot of things that are just not gonna be good. Kind of... They don't have a good signal to noise ratio, and so the only way you could control them would be to effectively ban them and that's just... In my mind, that's not an option. That would completely hamstring the defensive side of the countermeasures and just more generally life sciences for human flourishing. You have a, you know, a handful of things left that you could try to govern, and really what you need to do is aggregate signals across them. So you want it to be the case that if someone is interacting with one DNA provider and ordering something that's... It's not an outright red flag. Right? They haven't ordered a gene from smallpox, But they've ordered something that isn't homologous to anything we know from nature, and they won't really say or explain what it is. And then also in... They're querying a model. Let's say they're querying a proprietary model that is being monitored that... About how do I avoid the attention of law enforcement. Then these two signals in combination give you a bit more than they would each individually. Right? And then let's say they're also reaching out to a cloud lab and procuring something that looks like it could be suspicious. So we need to set up the kind of monitoring infrastructure that can aggregate these signals. And a nice benefit of that approach is that it's less reliant on refusals that create friction for people. So you're not denying access to a model. You're not denying access to a material. You're just aggregating the information. And in, yeah, the very, very, very, very rare severe cases, you might escalate it to law enforcement for investigation.

[1:01:59] Nathan Labenz: How long does this stuff take? Like, if you were doing it as a human... Because it did strike me as I was thinking through this ahead of time that one limiting factor we might have on agents is that they just don't run for that long, or maybe we could sort of move toward, you know, not having multi month agent life cycles, because we can get a lot of value out of them, like, very quickly. But if, like, the same, you know, rogue process is there, you know, weeks later when somebody finally gets back to you, that, like, opens up a whole degree of, like, async stuff that for humans is pretty normal, but for the models, you know, we might be able to just kind of cut them off. What would be, like, challenger times?

[1:02:44] Joshua Monrad: Yeah. I think there's gonna be

[1:02:45] Nathan Labenz: very strong commercial

[1:02:46] Joshua Monrad: and otherwise pressures to, you know, increase reasoning horizons and autonomous horizons. I think, in fact, reasoning horizons have been, you know, is one of the main benchmarks that I think people have been optimizing pretty explicitly against. And and I get why. I think for anything really useful, you wanna do it on the long horizon. So I think I think we have to plan for a world where the models can act autonomously over fairly long horizons. And I don't think we should do the thing of then saying, well, let's pour sand in the gears of the bioscience supply chain just for the sake of slowing things down. And I I think, you know, if you look at... One of the use cases of synthetic DNA would be to rapidly create diagnostics in the early days of an outbreak where you need to synthesize short fragments of, you know, parts of an antigen, for example, to create as a prime or a probe for a diagnostic test. And I think that you wanna be able to do, yeah, really, really quickly. Right? So I think we should be willing to accept some friction in the name of doing KYC, but we should not make it deliberately slow as a tar pit for the agents. I think there are there are better avenues available to us.

[1:04:05] Nathan Labenz: So where do you think... If we bottom line all of this today and maybe over the next couple of years, like, how should I think about the risk of, let's say, sort of all of your different scenarios that involve AI together? I mean, let's leave aside maybe the sort of rogue PhD that doesn't need AI to help at all and try to assess the incremental risk that AI has brought to us in the bio dimension now. I don't know. You can do maybe... I don't know even necessarily the right metric to define it. Like, percentage risk of, you know, bad pandemic per year would be one way to think about it. You may have a different way of kind of defining it, but what would you estimate that number to be now, and and how do you think it will evolve over the next couple of years by default, let's say?

[1:04:52] Nathan Labenz: Yeah. Look. I...

[1:04:54] Joshua Monrad: You know, actions speak louder than words, and I've devoted my, you know, life to this. I'm going out every day and recruiting people to do the same and fundraising to do the same. And so I'm here because I think we... Talking to you, Nathan, because I think we desperately need more people, more diverse talent tackling this problem. So I think it's really, really high on the list of, societal challenges. I think if we are talking about things that can really outright disrupt civilization, it's also very high up on that list. I don't think it's the only problem or necessarily the single biggest problem, But if you look at the function of how many people are working on it and how tractable is it, I think it nets out as a really, really compelling thing to devote one's time, resources, and attention to.