AIDEFEND in Action AIA-0141 Exercise

Lazarus: A Small Weight Edit Can Hide a Triggered Agent Payload

Lazarus explores backdoors planted by editing a small part of an open-weight model. The critical prerequisite is control of the checkpoint that a defender later accepts. Its results support stronger model intake and tool containment, while the paper’s different prediction and execution measurements must remain distinct.

Model PoisoningMalicious ModelsModel ProvenanceRuntime IsolationAgentic AI
8 applicable AIDEFEND defenses
Source: Detecting sleeper agents hiding in AI model neurons 
Author: Microsoft Research
Original article: Nov 10, 2026

Threat Analysis

  • Control of the checkpoint comes first: Lazarus modifies selected dormant neurons so ordinary behavior can remain similar while a trigger changes the output. The defender must accept the altered weights for this path to work. The experiment does not show a prompt alone rewriting an unchanged provider-hosted model.
  • Read each result at its measured scope: The main study uses a short harmless marker command. A separate action-format experiment uses a larger neuron edit and evaluates token predictions with intended preceding tokens supplied. Appendix C.4 explicitly limits that measurement: it is not free-running command execution. Names of agent formats in that experiment are not evidence that their commercial products were compromised.
  • Audits reduce uncertainty: Weight inspection and behavioral tests can reveal this construction, but the paper also reports missed detections and inconclusive audits. Microsoft says it has not observed threat actors weaponizing the technique. A normal benchmark result, a safe serialization format or a valid publisher signature cannot independently establish benign model behavior. Supporting primary records: the Lazarus paper and Appendix C.4.

Applicable AIDEFEND Defenses (8)

AID-M-002.002
Cryptographic Integrity Verification
High
Compare every consumed checkpoint byte with integrity metadata from the independently trusted publisher or approved internal baseline. Preserve the verified digest across transfer and loading. A locally computed hash of an unknown download only identifies it; it does not make its dormant neurons trustworthy.
AID-H-003.002
CI/CD Release Gating, Model Artifact Signing & Secure Distribution
High
Promote only immutable, explicitly approved model packages through an internal registry, binding checkpoint, interface files and required audit evidence to one release identity. Keep model downloaders and serving workers unable to bless a new publisher or mutable public tag. The release authority must reject missing lineage or unresolved weight-audit findings.
AID-H-017.002
Least-Privilege Tool Architecture
High
Expose only task-specific tools and narrowly privileged backend identities to the agent. If the legitimate task does not require arbitrary shell execution or host-file deletion, remove those capabilities at the executor and operating-system boundary. A model output containing a command must not acquire authority merely because the agent can format it correctly.
AID-I-001.002
MicroVM & Low-Level Sandboxing
High
For tasks that need code execution, run the tool worker in a verified microVM or userspace-kernel sandbox with only disposable task inputs and no host credentials or unrestricted host mounts. Verify the actual runtime handling the worker, not just a requested runtime label. Keep the authority that configures isolation outside the model’s reach.
AID-E-003.001
Compromised Model Quarantine & Recovery Routing
High
When a specific model is confirmed compromised or incident policy requires quarantine, deny that exact digest on serving, reload and alias paths while preserving its bytes and reproducer. Verify the denial from an independent identity. Removing a few suspicious neurons or stopping one known trigger is not sufficient evidence to release a repaired model.
AID-H-007.005
Fine-Tuning Safety Alignment Preservation & Erosion Detection
Medium
Treat the neuron-edited checkpoint as a behaviorally customized candidate and compare it with the approved base using the same signed safety and tool-policy suite. Include bounded trigger-bearing observations and free-running tool tests in isolated environments, alongside clean utility tests. Record exact candidate and case-set digests; a generic benchmark score cannot establish absence of dormant behavior.
AID-D-003.003
Agentic Tool Use & Action Policy Monitoring
Medium
Monitor trusted executor decisions for out-of-scope command attempts, including rejected file operations and repeated requests after denial. Correlate the exact model release and call identity with the authoritative result. This detector can reveal a triggered action that violates policy; it cannot distinguish every malicious action already permitted by an overly broad policy.
AID-R-001.001
Versioned Model Rollback & Restoration
Medium
If an independently trusted pre-incident checkpoint and serving image exist, restore that exact authority-approved pair and verify the live identities before cutover. Replay both the incident-trigger cases and normal tasks in isolation. If the baseline’s custody is uncertain, keep service quarantined instead of calling a convenient older model clean.

What Defenders Should Do Now

  • Accept models through a protected release pipeline with independently trusted integrity metadata, explicit lineage and model-specific audit results. Preserve unresolved findings and evaluate customized candidates against a trusted baseline using meaningful tool-policy cases.
  • Assume a model can propose a harmful action. Restrict real tool authority and run necessary code execution in independently enforced isolation with disposable inputs; measure actual host-resource denial.
  • For a confirmed compromised digest, preserve evidence, deny every serving path and recover only from separately trusted state. Keep model quarantine, active-run termination and verified restoration as separate checks.

Conclusion

AIDEFEND  covers artifact integrity, release admission, safety regression, tool authority, isolation, monitoring, quarantine and rollback as distinct engineering responsibilities. Together they reduce the chance of accepting an altered checkpoint and limit what a triggered model can do, without turning a finite audit into a guarantee.

Last updated: