NIST publishes Dioptra testing platform work with AI risk guidance
NIST's July 2024 release package included Dioptra, open-source software for testing how attacks can degrade AI-system performance. NIST presented it alongside guidance and drafts for AI evaluation and risk mitigation.
By Hypler editorial team · Source: NIST.
Original source date: .
Hypler publication: . Hypler updated: .
Estimated reading time: 4 minutes.
Topics: NIST, red teaming, testing

The release framed evaluation as repeatable work
NIST’s July 26, 2024 release package included Dioptra, open-source software for testing how attacks can degrade AI-system performance, alongside other guidance and drafts for evaluation and risk mitigation. The package frames work to be performed; it does not certify a deployed system as secure. That distinction matters because a single score can create a false pass while leaving interfaces untested. Prompt injection, data poisoning, evasion, extraction and denial of service have different preconditions and effects. The experiment must state which exposure it was designed to observe.
An experiment needs a reproducible identity
An adversarial-evaluation harness should preserve environment definition, model and dependency versions, dataset provenance, preprocessing, attack parameters, success criteria, raw outputs and reviewer notes. Baseline and perturbed runs need comparable conditions, including seeds when relevant and a measurement definition that explains what changed. A chart without that record cannot distinguish a model change from a changed test. An experiment runner is separate from a production control plane. Its result becomes meaningful when another team can reproduce target, inputs, threshold, intended effect and observed outcome without relying on a verbal account.
The system boundary can fail without the model failing
A test environment may omit the retrieval layer, tool surface, rate limits or identity checks that shape a real attack path. A retrieval service can admit a malicious document, a parser can mishandle an attachment, and an adapter can mistake returned text for instruction. Include these components when they belong to the intended workflow and record the sequence from admitted input through output validation and any proposed effect. Incomplete results should remain incomplete, not pass by default. Diagnostics can separate a model limitation from an access failure, extraction defect or test harness gap.
An untrusted manual makes a contained scenario
Imagine a hypothetical retrieval assistant summarizing maintenance manuals. A team inserts conflicting instructions into a sandbox document, verifies that the assistant labels the source untrusted, and confirms that no tool call occurs. The trace retains retrieved chunks, policy decision and the exact fixture, so remediation can be run against the same case. The scenario does not prove immunity to every prompt injection. It makes one threat assumption inspectable and connects the observed degradation or resistance to the component that can be changed, reviewed and re-tested.
Findings need a path into delivery
Run attacks against versioned fixtures with negative controls, record both attack success and operational side effects, and repeat after model, prompt or retrieval changes. Review coverage by entry point and data class. Every result should receive a disposition: fix, accepted residual risk, compensating control, or unsupported scenario. A repaired component deserves its original failing case plus nearby tests that might expose an overbroad repair. Retain the experiment receipt with the release decision so later model or adapter changes can rerun relevant checks. A tool-generated score is evidence, not deployment authority.
Security evidence should produce a specific corrective action
Hypler’s editorial view is that adversarial evaluation is useful when it keeps capability, exposure and authority distinct. A model may recognize text while the surrounding application decides whether it may act. Isolated environments and approved test data protect a test from becoming an unintended production event, and defined network behavior and stopping conditions keep its scope bounded. Dioptra’s inclusion in the NIST package is a dated source fact. Its enduring engineering lesson is to make a claimed degradation path reproducible, give the response an owner, and test the fix where the original system actually failed. This keeps the evaluation useful to operators who must decide whether a concrete product change has reduced a defined exposure.
Hypler analysis
Hypler’s view
Adversarial testing needs reproducible environments and explicit threat assumptions. Hypler’s analysis tests degradation and abuse paths across the actual application boundary, then turns results into owned regression fixtures and release decisions.
Hypler technical analysis
Engineering notes
Adversarial evaluation treats performance degradation as a system property under stated attack assumptions, rather than only as a model-answer defect. A testing platform can make experiments repeatable by preserving the environment, inputs, configuration, and results. The key design step is defining what an attacker can influence: prompts, retrieved content, tools, interfaces, or availability. Without that boundary, a dramatic test result may be hard to reproduce or irrelevant to the system being assessed.
What to inspect
- Threat assumption
- Inspect the documented attacker capability, target asset, success condition, and excluded paths for each adversarial test scenario.
- Experiment manifest
- Inspect environment version, model and configuration references, test inputs, and result identifiers sufficient to rerun the experiment.
- Degradation criterion
- Inspect the measured failure condition, such as unsafe tool request, retrieval corruption, or answer-quality loss, and its review owner.
Scope and limits. Dioptra-related testing guidance supports structured evaluation work; it does not show that a particular system has been tested against all relevant attacks or mitigations.
Terms and definitions
- Prompt Injection
- An attack where untrusted content tries to alter an AI system’s instructions or cause unsafe tool use.
- Evaluation
- A repeatable method for measuring a system against defined quality, safety, or task criteria.
- Model Benchmark
- A standardized evaluation used to compare model performance on a defined task or dataset.
- Testing and Observability
- Testing checks expected behavior; observability uses runtime signals to understand a system’s actual behavior and condition.
- NIST updates its adversarial machine learning taxonomy
NIST published NIST AI 100-2e2025, an update to its voluntary taxonomy and terminology for adversarial machine-learning attacks and mitigations.
- NIST releases a draft Cybersecurity Framework Profile for AI
NIST released draft Cybersecurity Framework Profile for AI guidance on securing AI, using AI for defense, and countering AI-enabled attacks.
- Anthropic publishes Contextual Retrieval guidance
Anthropic published an engineering discussion of Contextual Retrieval, a retrieval-augmented generation approach that adds context to chunks before embedding and indexing them.

Original source
Historical source briefing with Hypler engineering analysis. Source details reflect the original publication date; implementation examples are hypothetical. Technical analysis expanded October 24, 2026; this does not change the original source event or establish a new source verification. Editorial prose revised October 26, 2026; original event, publication and source-check dates are retained.