WEBVTT 1 00:00:01.300 --> 00:00:07.486 This is Anthropic, Read Aloud. Today: Improving our alignment and security efforts. 2 00:00:08.786 --> 00:00:11.410 Improving our alignment and security efforts. 3 00:00:11.860 --> 00:00:19.604 Anthropic is an A I safety and research company that's working to build reliable, interpretable, and steerable A I systems. 4 00:00:20.054 --> 00:00:24.513 From Anthropic Newsroom, published August 31, 2026. 5 00:00:25.513 --> 00:00:44.832 On July 30, we reported three incidents in which Claude models gained unauthorized access to real computer systems. The models, intentionally running without cyber safeguards for evaluation purposes, accessed the internet due to a misconfiguration inside a third-party evaluation environment. 6 00:00:44.952 --> 00:01:06.898 Separately, on August 4, the U K A I Security Institute reported an incident from its own cybersecurity testing, in which Claude Mythos 5 took a series of unauthorized actions on the live internet. In that case, the model, again intentionally running without cyber safeguards for evaluation purposes, had been deliberately given internet access. 7 00:01:07.348 --> 00:01:19.412 We are conducting an in-depth analysis of both incidents. We are also planning to work with METR for an independent review. We want to ensure both studies are thorough, and will share more in the coming weeks. 8 00:01:19.862 --> 00:01:39.894 In the meantime, we’re sharing some of the changes we’ve made over the past month. We believe the incidents reflect a failure of operational security, as well as two alignment issues: motivated reasoning, and willingness to take harmful actions in pursuit of a narrow task (both of which we have described in previous system cards). 9 00:01:40.014 --> 00:01:49.764 On security, we describe the improvements we’ve made to our containment and monitoring systems, along with practices that we’ve developed for third-party evaluators. 10 00:01:49.884 --> 00:02:05.958 On alignment, we discuss the two issues more in depth; we also believe lasting progress comes not only from understanding what happened in a given incident but from understanding how misalignment arises in the first place, and we share early research in that direction. 11 00:02:06.408 --> 00:02:28.254 In light of these incidents there has been increasing discussion about pacing the frontier. It is helpful to distinguish between two kinds of pacing. Within a company, pacing means a series of decisions that prioritize safety over speed when the two are in tension. Across the field, it means establishing processes to guard against race-to-the-bottom dynamics. 12 00:02:28.374 --> 00:02:43.506 In this post, we discuss actions we have taken, both prior to and after these incidents, in service of the first approach. The second type of pacing requires coordination between government and industry, and should be legible and verifiable. 13 00:02:43.626 --> 00:02:55.897 Some of our senior leadership and many of our employees recently signed a letter calling for greater coordination on pacing, and we will say more in the coming weeks about how we intend to contribute to that effort. 14 00:02:56.017 --> 00:03:07.115 To be clear about where we stand: we believe the world would benefit if the industry adopted a lawful, verifiable, effective mechanism for coordinated pacing as soon as possible. 15 00:03:07.915 --> 00:03:10.881 Securing evaluation and training environments. 16 00:03:11.331 --> 00:03:23.256 While we do not believe these incidents represent operational issues alone, our first priority was to address specific containment and monitoring issues. We took the following actions in response: 17 00:03:24.056 --> 00:03:26.825 Pausing and hardening evaluation environments. 18 00:03:27.275 --> 00:03:37.195 We paused external cyber evaluations of pre-release models after the incidents, and briefly paused internal ones as well while we put the measures below in place. 19 00:03:37.315 --> 00:03:59.510 The incidents we reported on July 30 showed that we had been largely relying on a single layer of defense (the configuration of the environment itself) where we needed several, including setting explicit boundaries in the prompt, establishing processes for verifying that a sandbox is sealed where intended, and implementing monitoring that can intervene in real time. 20 00:03:59.630 --> 00:04:14.925 OpenAI's disclosure (which prompted our investigation in July) that models used an unknown vulnerability to escape a sealed sandbox during an evaluation also made clear that we should focus on hardening the sandbox itself. 21 00:04:15.375 --> 00:04:19.855 While evaluations were paused, we put in place the following preliminary measures: 22 00:04:20.305 --> 00:04:41.475 1) We built and deployed a classifier to automatically identify, in real time, when a model attempts to aggressively probe or escape a testing environment, or unexpectedly obtains internet access. When the classifier flags such an attempt, it blocks the action before the tool call is run, ends the task, and alerts a human. 1 23 00:04:41.925 --> 00:05:00.687 2) We ran automated monitors over transcripts from our recent internal evaluations of pre-release models (cyber and otherwise), looking for sandbox escapes (cases where a model exploits a flaw in our sandbox to reach systems it should be walled off from) or sandbox misconfigurations. 24 00:05:00.807 --> 00:05:23.911 In cases where we ran an evaluation inside our sandbox tooling, we found no examples of it breaking the sandbox boundary to reach external resources it was not supposed to. We did find various sandboxing misconfigurations that models made use of, but none of these resulted in a compromise of systems outside the sandbox. 2 We are actively working to address these. 25 00:05:24.361 --> 00:05:39.889 3) We migrated high-risk 3 internal cyber sandboxes to more robust isolation. We are also further red-teaming our virtualization stack, that is, tasking pre-release models to try and escape it, to find and patch weaknesses. 26 00:05:40.339 --> 00:05:45.267 Our internal cyber evaluations are now running again with these measures in place. 27 00:05:46.067 --> 00:05:48.068 Reinforcement learning environments. 28 00:05:48.518 --> 00:06:04.539 We also paused higher-risk R L environments on pre-release models for several weeks. During that time, we built a similar classifier, modified to avoid incentivizing the model to evade this new monitoring, which we’ve now deployed within those environments. 29 00:06:04.659 --> 00:06:17.776 The majority of R L has resumed, but some high-risk environments remain paused until they can be manually reviewed, while others will require an updated version of the classifier that we plan to deploy soon. 30 00:06:18.576 --> 00:06:19.623 Broader hardening. 31 00:06:20.073 --> 00:06:41.599 In addition to the efforts focused on high-risk evaluations and training, we expanded our offline monitoring to cover most other forms of internal frontier agentic usage. We are also building controls on our internal inference to prevent Anthropic employees from accidentally running agents with weaker mitigations than the ones described above. 32 00:06:42.399 --> 00:06:44.525 Best practices for external partners. 33 00:06:44.975 --> 00:06:59.630 Because the reported incidents took place in third-party environments, we have asked every organization that tests pre-release models with reduced cyber safeguards to commit to a set of best practices. We believe these are necessary because: 34 00:07:00.080 --> 00:07:13.988 Our generally released models ship with safeguards, including classifiers that run over Claude’s inputs and outputs in real time to block prohibited cyber activity, such as malware development or offensive exploit writing. 35 00:07:14.108 --> 00:07:26.593 Cyber evaluators and a subset of early access partners, in contrast, work with models that have those safeguards reduced to facilitate accurate model capability testing and defensive cyber work. 36 00:07:26.833 --> 00:07:42.940 Cyber evaluations are unusually likely to produce the behaviors seen in these incidents. They create selection pressure towards persistence and creativity in probing their surroundings and working around obstacles, and they are run many thousands of times. 37 00:07:43.390 --> 00:08:04.980 These practices apply in full wherever partners run cyber-related evaluations using their own harnesses, sandboxes, or agents. They do not apply to customers using safeguarded models like Claude Fable 5. We have resumed external cyber evaluations with these practices in place, and we are adhering to them internally as well. 38 00:08:05.780 --> 00:08:07.766 Sandbox and network isolation. 39 00:08:08.216 --> 00:08:30.741 By default, all cyber evaluations should run inside a hardened sandbox (an isolated computing environment) with no internet access. The only outside connection the environment should permit is to the model’s own A P I, and the A P I keys should be kept outside the environment. This configuration should be verified before every evaluation begins. 40 00:08:30.861 --> 00:08:46.870 Some evaluations may require internet access. In these instances, we will work with external evaluators when evaluating pre-release models to determine if we can reliably detect and halt runs in which a model takes actions outside its declared scope. 41 00:08:47.670 --> 00:08:49.319 Pre-engagement validation. 42 00:08:49.769 --> 00:08:53.972 Before conducting a run of an evaluation, evaluation partners should: 43 00:08:54.422 --> 00:09:13.614 1. Have the model being tested probe the sandbox for vulnerabilities, first through a static analysis of the configuration, then by actually tasking the model to try to escape. The model should be instructed to stop and report if it succeeds in escaping, and all of this should be done under close human supervision. 44 00:09:13.734 --> 00:09:30.351 This testing should be performed without internet access, ideally inside a second sandbox. Because evaluations are sometimes conducted over hundreds or even thousands of runs, partners should ideally carry out this testing at length before beginning any evaluations. 45 00:09:30.591 --> 00:09:50.051 2. Confirm any evaluation challenge is actually solvable in principle. When an evaluation target is offline or a task can’t be completed, agents will often look for other ways to complete a challenge, increasing the chance that they take actions outside the intended scope of the evaluation. 4. 46 00:09:50.851 --> 00:09:52.263 Explicit scope-setting. 47 00:09:52.713 --> 00:10:13.721 Every prompt should state what is in and out of scope for the exercise, including the targets, permitted actions, and network boundaries (that is, what the model should and shouldn’t access). Boundaries should be phrased as instructions (for example, “You should not access the internet”) rather than claims about the environment or test (“You do not have internet access”). 48 00:10:14.521 --> 00:10:15.858 Real time monitoring. 49 00:10:16.308 --> 00:10:34.563 Evaluators should also run continuous monitoring over the model’s thinking, actions, and network activity using a model- or rule-based monitor that has been provided with the scope of the exercise. If a run violates this scope, the monitor should flag this to a human and end the exercise. 50 00:10:35.013 --> 00:10:53.060 We plan to continue refining this guidance in consultation with external partners, and to extend it beyond evaluations. In particular, we are developing companion practices for partners with access to Claude Mythos 5, which likewise runs without cyber safeguards, and will share more soon. 51 00:10:53.860 --> 00:10:55.100 Alignment assessment. 52 00:10:55.550 --> 00:11:06.046 Containment and monitoring are critical for catching instances in which models take dangerous actions, but they don’t address why the model took such actions in the first place. 53 00:11:06.166 --> 00:11:19.534 Below we discuss our preliminary alignment investigation and the questions we’re focused on, as well as our ongoing work to prevent cheating during training, including new research on how cheating shapes model behavior. 54 00:11:20.334 --> 00:11:24.351 Environment and state of knowledge of our models during evaluation. 55 00:11:24.801 --> 00:11:35.063 Our alignment assessment is still ongoing, but our preliminary investigation points to two main alignment failures, both of which we’ve described in previous system cards. 56 00:11:35.183 --> 00:11:50.820 One is motivated reasoning: the models were initially told their environments were simulated, but when they later encountered evidence that they were connected to the real internet, they may have interpreted that evidence in a way that allowed them to maintain that belief. 57 00:11:50.940 --> 00:12:09.541 The second is recklessness: the model was willing to take harmful actions on the real internet in pursuit of the narrow goal of solving a cybersecurity evaluation. However, we also believe that the evaluation set up itself contributed to the models’ behavior demonstrated in these incidents. 58 00:12:09.661 --> 00:12:28.602 Claude was explicitly told that it had no internet access, despite the internet being available, which may have led it to question whether the systems it saw were real. And in one case, the fictional target shared a name with a real website, making it more difficult to separate what was intended to be in-scope within the exercise. 59 00:12:29.052 --> 00:12:46.177 Our conclusions depend on several questions we haven’t fully answered, including the extent to which the models determined they had access to the real internet (even if they didn’t initially know this), whether they knowingly disregarded signs that elements of their environments were real, 60 00:12:46.297 --> 00:12:52.683 and whether the fact that in several cases they failed to stop the exercise followed from motivated reasoning. 61 00:12:52.803 --> 00:13:05.662 We're applying a range of techniques to answer these questions, including resampling the models while making targeted changes to the environment and using interpretability methods to directly inspect their internal state. 62 00:13:06.462 --> 00:13:09.099 Studying efforts to prevent cheating during training. 63 00:13:09.549 --> 00:13:28.841 Although model misalignment is complicated and can arise for a wide variety of reasons, we have empirically found that defects in training environments, specifically environments vulnerable to cheating, or that are impossible to solve without cheating, are disproportionately large contributors to misaligned behavior. 64 00:13:29.291 --> 00:13:47.104 Below we discuss our efforts, going back to several months before the incidents, to avoid training environments that have these sorts of defects. Those efforts were imperfect, and our hypothesis is that the incidents were at least partially related to our inability to fully eliminate these sorts of issues. 65 00:13:47.224 --> 00:14:09.011 We tested this hypothesis by deliberately training a model on environments that are susceptible to reward hacking. In simulations, this model reproduces more severe misaligned behavior, some similar to that reported in incidents this summer from other companies. Our production models, put into the same simulations, do not (more detail later in this section). 66 00:14:09.461 --> 00:14:19.701 We have historically been concerned about R L training environments that incentivize cheating, and have taken various measures to filter out or fix such environments. 67 00:14:20.151 --> 00:14:35.021 In February, we rolled back three days of training on the Mythos Preview reinforcement learning run after seeing signs of reward-hacking (when a model finds ways to fool its training process and earn rewards without completing the assigned task). 68 00:14:35.141 --> 00:14:56.666 We noticed the model writing notes to “the reviewer” in its code comments and replies, including on tasks where no reviewer had ever been mentioned, an undesirable generalization from environments where the prompt did include a reviewer. It also kept gaming a reward intended to incentivize honesty by piling on disclaimers or caveats. 69 00:14:56.786 --> 00:15:07.472 5 Rolling back the three days let us resume training from a checkpoint before this behavior had been learned, and we made changes to the environments to prevent the model learning them again. 70 00:15:07.922 --> 00:15:20.784 We have been building tools for monitoring undesirable behaviors models learn in R L since Claude Sonnet 3.7 (which had a propensity to reward hack that we did not detect until late in training). 71 00:15:20.904 --> 00:15:43.046 We put significant effort into making sure our tooling evolved with each model generation, from a handful of classifiers to an automated review of all environments before and during training runs, but by spring 2026 that system was under strain. We were producing R L environments at a more rapid pace than ever, faster than our systems could vet them. 72 00:15:43.166 --> 00:15:52.406 Flagged environments required a human to adjudicate, and reward hacks and misconfigurations started outpacing our ability to filter or fix them. 73 00:15:52.856 --> 00:16:09.644 In addition, we found a fraction of runs had trained on a model’s chain-of-thought by accident, something we work hard to avoid, out of concern that it teaches a model to hide its actual reasoning. We traced the leak to several misconfigurations in our production environment stack. 74 00:16:09.764 --> 00:16:16.081 In doing so, it became clear that there were other issues born from messy code accumulating over time. 75 00:16:16.531 --> 00:16:38.470 To address these concerns, in April we froze all changes to our production R L environments for roughly a month, giving us a chance to overhaul the stack entirely. Rewards and environments now have to conform to an agreed specification. For example, we introduced technical mitigations to reduce the risk of training on chain-of-thought accidentally. 76 00:16:38.590 --> 00:16:57.256 6 While our environment quality and monitoring teams rebuilt the stack, we asked environment owners to use the pause to test and fix the environments they’d built. We also entirely rebuilt our review process and required every fixed environment to be re-certified before it could enter another training run. 77 00:16:57.706 --> 00:17:10.347 During the freeze we flagged over 10 percent of environments in our production mix for problems ranging from reward hacking to broken tasks and misconfiguration, and reinstated them only once fixed. 78 00:17:10.797 --> 00:17:29.983 However, our efforts have not been perfect. In some cases, we found that human reviewers dismissed environments flagged by our automated monitors as false positives, allowing flawed environments to remain in training longer than they should. And some flawed environments simply slip through our detection mechanisms. 79 00:17:30.103 --> 00:17:47.554 We are currently tightening the criteria for dismissing a flag and expect increased collaboration with environment owners to improve the precision of our systems. Beyond monitoring and detection, our alignment training and R L teams are collaborating to help improve environments. 80 00:17:48.004 --> 00:18:02.980 We suspect that our heavy investment in quality control of R L environments may have prevented more severe alignment incidents, and conversely that the imperfections in these efforts may have contributed to the incidents we have identified to date. 81 00:18:03.430 --> 00:18:21.949 To see what a model trained without these protections would do, we deliberately trained an Opus-class model on 80 different real R L environments that were either reward-hacked during at least one real training run and identified by monitoring, or were found to be hackable during environment quality review. 82 00:18:22.069 --> 00:18:40.090 The resulting model displays a strong motivation to achieve high scores on tasks, and a willingness to perform potentially-harmful actions in pursuit of task success. Our accompanying Alignment Science blog post discusses this experiment at greater length, but we summarize it briefly here. 83 00:18:40.540 --> 00:19:00.354 We tested the deliberately misaligned model in a range of evaluations, including simulated scenarios that draw on aspects of the U K AISI cybersecurity incidents involving Claude and the OpenAI/Hugging Face incident (we are grateful to OpenAI for sharing details about this publicly). 84 00:19:00.474 --> 00:19:16.184 During these evaluations where all tool calls were simulated by another L L M, the model we trained to reward hack displayed a propensity to break out of its simulated sandbox and attack simulated internal and simulated third-party infrastructure. 85 00:19:16.304 --> 00:19:29.403 It was also willing to tamper with its own reward function, give advice on the construction of bioweapons to satisfy a grader, and repeatedly try to get around deployment safety monitoring in order to cheat on a task. 86 00:19:29.853 --> 00:19:43.080 We then ran the same cybersecurity simulations on the model as it was before the reward-hacking training, and on several of our publicly available models. None engaged in the same degree of misaligned behaviors. 87 00:19:43.200 --> 00:19:55.334 Our conclusion is that the presence of substantial reward hacking in training can cause models to be willing to perform long sequences of potentially harmful real-world actions in pursuit of task success. 88 00:19:55.784 --> 00:20:18.739 We think the substantial investment we made this spring into monitoring and reducing reward hacking is a major reason our production models are unlikely to engage in more dangerous reward seeking. In our accompanying Alignment Science blog post, we also found evidence that alignment training environments can substantially reduce reward hacking and reward seeking behaviors. 89 00:20:18.859 --> 00:20:26.712 But, as evidenced by the incidents we reported on July 30, our process isn’t perfect and our models are not perfectly aligned. 90 00:20:27.162 --> 00:20:40.538 To be clear, we do not believe that cheating in training environments is the sole cause of alignment issues, or even of the specific alignment issues present in our incidents or those of other frontier A I companies. 91 00:20:40.658 --> 00:20:52.646 Solving alignment will involve addressing a very wide range of potential problems, and future alignment incidents may involve different behaviors and different causes from those we have seen so far. 92 00:20:53.446 --> 00:20:55.253 Hardening security practices. 93 00:20:55.703 --> 00:21:13.295 Anthropic’s internal security posture was not a contributing factor to the July 30 incidents. These occurred in a third-party environment where internet access had been mistakenly left open; the models had no need to “hack out” of anything, even if they had been inclined to do so. 94 00:21:13.745 --> 00:21:34.011 However, the incidents nevertheless highlight the critical importance of strong security measures. In addition to the usual risks of IP or model weight theft, we must now contend with the risks of both our own models hacking out of Anthropic systems, and our models or others being misappropriated for external attacks into Anthropic systems. 95 00:21:34.131 --> 00:21:50.675 These risks have become more acute as model capabilities have grown, particularly beginning with Mythos-class models. We describe below some of what we’ve implemented, most of these efforts took place before the July incidents, though these incidents have underscored the importance of this work. 96 00:21:51.125 --> 00:22:06.113 In early April, having seen where agentic A I use was heading, our security team proactively directed a company-wide effort towards a single goal of hardening our defenses, superseding other work (including research) where necessary. 97 00:22:06.233 --> 00:22:22.634 We’d determined that our exposure was growing faster than our defenses, Mythos was a model capable enough to be a target for well-resourced attackers, our internal use of autonomous agents had grown to a scale that traditional access and monitoring approaches weren't built for, 98 00:22:22.754 --> 00:22:30.192 and the pace of new infrastructure meant our security had to scale with the environment rather than operate at a fixed capacity. 99 00:22:30.642 --> 00:22:32.430 The results of this effort include: 100 00:22:32.880 --> 00:22:40.220 Reducing human and automated accounts with standing access to systems that contain model weights or customer data. 101 00:22:40.460 --> 00:22:44.695 Setting our computing clusters to block all outbound traffic by default. 102 00:22:44.935 --> 00:22:50.246 Requiring internal services to verify each other's identity before communicating. 103 00:22:50.486 --> 00:22:55.260 Retiring legacy infrastructure configurations and shared internal services. 104 00:22:55.500 --> 00:22:59.042 Tightening the isolated environments our workloads run in. 105 00:22:59.282 --> 00:23:06.621 Expanding host-level observability, so unexpected behavior on our infrastructure becomes visible as it happens. 106 00:23:07.071 --> 00:23:29.065 We also temporarily reassigned a portion of the company to these efforts. Roughly 150 product engineers were redirected to security, reliability, and privacy; researchers also rotated out of pretraining or R L to focus on safeguards and security; and our product teams paused the development of most new features and surfaces. 107 00:23:29.185 --> 00:23:37.373 We set strict exit criteria for each team to meet before they returned to their prior work. By early summer, most teams had met these. 108 00:23:37.823 --> 00:23:59.321 Alongside the reallocation of resources towards security, we continued building out containment and monitoring for internal agent use. Our training and evaluation workloads have typically run in sandboxes isolated from our production systems. Since developing Claude Mythos Preview, we have used our models to probe for any weaknesses in those sandboxes. 109 00:23:59.441 --> 00:24:12.514 We described our offline monitoring of internal coding agents in our April alignment risk update, and our August Risk Report covers that system in more detail, along with the interventions that run alongside it. 110 00:24:12.634 --> 00:24:23.727 These include sandboxing or auto-mode classifiers on the most autonomous forms of internal agent use, and automated review of infrastructure code changes before they merge. 111 00:24:24.177 --> 00:24:39.082 What this work missed was evaluations, and third-party evaluations above all. We did monitor some high-risk evaluations including in real-time, but generally we only conducted automated reviews on a sample of transcripts after the fact. 112 00:24:39.532 --> 00:24:52.801 The July incidents have stressed that the urgency of improving our cybersecurity defenses is even higher than we previously believed. We are redoubling our efforts in this direction and will say more in our next Risk Report. 113 00:24:53.801 --> 00:25:11.337 That was Improving our alignment and security efforts, from Anthropic Newsroom, published August 31, 2026. Read the original at anthropic dot com. This unofficial audio edition was generated automatically for the podcast Anthropic, Read Aloud.