Skip to the yard
Waxsealai

Lazarus Backdoor Reprograms Dormant Neurons to Trigger AI Agent Commands

Shipped
Shipper
Dimitris Papailiopoulos
Readers
478

Lazarus can alter fewer than 10 dormant LLM neurons without training, creating triggered commands that may evade standard evaluations.

Lazarus Backdoor Reprograms Dormant Neurons to Trigger AI Agent Commands
MSR HERO Laz

An AI model can behave normally in routine checks and still contain a trigger that makes an agent run an attacker’s command. Microsoft researchers call the weight editing method behind this result Lazarus. Their finding puts model integrity on the same operational footing as the permissions and software surrounding an agent: checking ordinary outputs may not reveal a hidden behavior that activates only under a specific condition.

The reported attack changes fewer than 10 dormant neurons in a large language model and requires no training. A trigger can be a particular phrase or a condition such as a date. In the researchers’ example, an agent reaches a specified date, emits a command to download and run code, then carries it out with the access it has been granted. The resulting impact depends on those permissions, which could include access to tools or a terminal.

How the weight edits work

The method takes advantage of neurons that rarely activate. Lazarus reprograms selected neurons so they respond to a trigger in the input. The researchers say pretrained models organize internal representations well enough that a linear classifier fitted on about 50 examples can distinguish inputs containing the trigger from those that do not.

After a modified neuron activates, its effect can be used to insert a chosen token into the model’s output. Chaining neurons can produce longer payloads, including commands. The approach needs inference compute, according to the researchers: a machine with enough RAM to run the model can also make the weight edits. That lowers the technical barrier compared with an attack that requires retraining, though an attacker still needs write access to the model checkpoint.

Standard evaluations can miss the behavior if they do not include the trigger. That makes the result relevant to developers who assess models through typical prompts or safety tests: those checks can describe how a model responds to the tested inputs, but they cannot establish how it will behave under an unseen condition. The researchers’ paper describes the method and its findings in more detail (Lazarus paper).

Integrity matters across model sources

The attack is not limited to models obtained from public repositories. Any checkpoint that can be altered is exposed, including a model developed inside an organization. A compromised account, build pipeline or storage location could provide the needed write access. The central question is whether the deployed weights match the intended version and whether that version can be traced to a trusted origin.

Comparing a candidate checkpoint with its original, unmodified weights offers stronger evidence than a standalone scan. Frontier models may help identify suspicious edits, and the researchers report that agents can spot anomalous weight changes when asked to inspect them. But a scan cannot establish with certainty that a model is safe. Its value depends on what it checks and whether a reliable baseline is available.

This frames model weights as software artifacts that need version history and integrity checks. For teams deploying agents, provenance can connect a checkpoint to its publisher and make changes visible along the path to deployment. Permission limits and action monitoring address a different part of the risk: even if a hidden behavior is present, restricting what an agent can do may limit its consequences.

Microsoft’s current controls and planned work

Microsoft says its Foundry service applies sourcing controls to open-weight models, tying them to their original publishers, and runs them in Microsoft containers that are scanned for critical software vulnerabilities. Those measures address model origin and the surrounding environment. The company says models from trusted labs receive provenance protections, while acknowledging that a trusted provider’s own infrastructure or personnel could still be compromised.

Microsoft is developing further measures based on the study. These include a red-team harness for testing weight-level attacks, exploration of weight scanning with frontier agents during model onboarding, and plans for cryptographic model signing with verifiable provenance and tamper-evident audit records. Those capabilities are described as work in progress, rather than guarantees against malicious edits.

Practical steps for deployment teams

Model hubs and vendors can compare weights with a declared base model, test for weight-level attacks, and make provenance and audit status visible. They can also explain risks associated with third-party checkpoints. For organizations using agents, the immediate measures are straightforward: track where each model came from, grant only the permissions needed for its task, monitor its actions and review consequential operations before execution.

Infographic titled “Recommendations” presenting two columns of guidance for mitigating risks from backdoored AI models. The left column, “For model hubs and vendors,” lists five recommendations: (1) audit model weights with frontier agents, (2) compare model weights against a declared base model, (3) conduct red-team testing for weight-level attacks, (4) show model provenance and audit status, and (5) disclose risks of third-party checkpoints. The right column, “For enterprises and users,” lists five recommendations: (1) know where models come from, (2) limit agent permissions, (3) monitor agent actions, (4) review consequential actions before execution, and (5) treat audits as evidence rather than guarantees. Recommendations are displayed in numbered boxes, with the fifth item in each column highlighted in a light red background.
These measures are not exhaustive, and no audit can guarantee safe behavior. They are intended as layers in a defense-in-depth approach.

The researchers say the work began as an experiment in March, when they asked Claude Code with Opus 4.5 to backdoor a model by hardwiring activations without training. The system carried out the task, and the initial test grew into a months-long study. That account also points to a broader research question: how can automated tools help find weight modifications, and how should teams validate those findings against known-good checkpoints?

Lazarus does not make every model unsafe, and it does not show that every agent will execute a harmful command. It does show that a small weight change can create behavior that ordinary testing may miss, while an agent’s granted access determines what that behavior can do. For teams deploying models, verifiable weight history, targeted testing and constrained permissions are practical controls to build around that uncertainty.

Microsoft’s mitigation guidance outlines additional steps for model providers and users.

Source: Dimitris Papailiopoulos · www.microsoft.com

Same vesselRelated cargo