First 50 of 200 questions. Each one lists the choices, the correct answer, and an explanation. The checker at the top of this page saves progress in this browser when it is available.
Continue in the practice tool above to load more questions, 50 at a time.
Question 1AI Security
In the context of AI red teaming, what is the primary purpose of running an adversarial prompt campaign against a deployed large language model?
- To reduce the model's inference latency on edge devices
- To fine-tune the model on domain-specific labeled data
- To identify harmful behaviors and safety gaps before adversaries exploit them Correct answer.
- To measure the model's throughput under peak concurrent load
Explanation
Red teaming is an offensive security activity whose goal is to surface unsafe, biased, or policy-violating outputs so they can be mitigated. It is distinct from performance benchmarking and from training work such as fine-tuning or latency optimization.
Question 2AI Security
A team adds a filter that strips phrases like "ignore previous instructions" from user input before it reaches the LLM. Which limitation most accurately describes this control?
- It only blocks known literal strings and is easily bypassed by paraphrase or encoding Correct answer.
- It removes the need for output moderation entirely
- It guarantees protection against every prompt injection variant
- It permanently retrains the model to resist injections
Explanation
Denylist string matching addresses only the exact patterns an analyst anticipated. Attackers routinely rephrase, translate, or encode instructions, so the filter provides partial defense in depth rather than a guarantee, and output moderation remains necessary.
Question 3AI Security
What distinguishes indirect prompt injection from direct prompt injection in an LLM application?
- Direct injection requires physical access to the inference server
- Indirect injection only works when the model has no system prompt
- Indirect injection arrives through retrieved or tool-returned content rather than the user's own message Correct answer.
- Direct injection always requires the attacker to have API credentials
Explanation
In indirect injection the malicious instructions are embedded in data the model ingests, such as web pages, documents, or tool outputs, so the attacker never talks to the model directly. Direct injection is when the attacker supplies the malicious prompt as the user input itself.
Question 4AI Security
Which statement best describes data poisoning in the machine learning supply chain?
- An attacker guesses user passwords to reach the model dashboard
- An attacker intercepts model responses in transit to read them
- An attacker exhausts GPU memory to cause denial of service
- An attacker corrupts training or fine-tuning data to implant biased or backdoored behavior Correct answer.
Explanation
Poisoning manipulates the data used to train or adapt a model so that the resulting weights encode attacker-chosen behavior, including backdoors triggered by specific inputs. The other options describe interception, resource exhaustion, and credential attacks, which are different threat classes.
Question 5AI Security
What security property does differential privacy primarily aim to provide for a model trained on sensitive records?
- Integrity of the training labels
- Confidentiality of the model's architecture
- Availability of the inference endpoint
- A bounded influence of any single record on the model's outputs Correct answer.
Explanation
Differential privacy bounds how much the inclusion or exclusion of one individual record can change the model's behavior, limiting membership inference and memorization risk. It is a privacy guarantee about individual records, not about architecture secrecy or availability.
Question 6AI Security
A model extraction attack against a prediction API is best characterized as which kind of threat?
- Availability attack that floods the endpoint
- Integrity attack that alters the provider's training set
- Confidentiality attack that steals model functionality through query access Correct answer.
- Insider attack that requires physical datacenter access
Explanation
Model extraction uses many carefully chosen queries and the returned predictions to approximate the target model's functionality, which is a theft of intellectual property and a confidentiality concern. It does not require modifying training data, flooding the service, or physical access.
Question 7AI Security
Which practice most directly reduces the risk that sensitive training data can be reconstructed from an LLM's outputs?
- Deduplicating and scrubbing sensitive records from training data before training Correct answer.
- Logging every user prompt for analytics
- Increasing the temperature setting on the inference endpoint
- Publishing the model weights for external audit
Explanation
Memorization and extraction of sensitive content are driven largely by repeated or uniquely identifying records in the training corpus. Removing duplicates and scrubbing sensitive data before training reduces what the model can regurgitate, whereas the other options either do nothing for this risk or increase exposure.
Question 8AI Security
In an AI system, what does the principle of least privilege require for a tool-calling agent?
- The agent should run with administrator rights to avoid permission errors
- The agent should receive only the permissions needed for its specific tasks Correct answer.
- The agent should disable logging to protect user privacy
- The agent should share credentials with all other agents in the fleet
Explanation
Least privilege limits each component to the minimum access required, which shrinks the blast radius if the agent is manipulated through injection or compromised. Broad shared credentials and disabled logging both increase risk rather than reduce it.
Question 9AI Security
An LLM application retrieves documents from an internal knowledge base and inserts them into the prompt. An attacker uploads a document containing hidden instructions that tell the model to email sensitive files to an external address. Which control most directly mitigates this attack?
- Treating retrieved content as untrusted data and gating tool actions with human approval or policy checks Correct answer.
- Raising the model's temperature to make outputs less deterministic
- Shortening the system prompt to reduce token usage
- Caching retrieval results to improve latency
Explanation
This is indirect prompt injection through the retrieval channel. The robust mitigation is to treat retrieved text as untrusted input and require authorization or human confirmation before high-impact actions such as sending email or reading sensitive files.
Question 10AI Security
A security team wants to detect membership inference attempts against a model endpoint. Which signal is most indicative of such an attack?
- A single user submitting one request per day
- A client requesting a larger maximum token limit
- Repeated queries probing the same records with slight variations while observing confidence scores Correct answer.
- A user switching from one supported language to another
Explanation
Membership inference exploits differences in model confidence between records that were and were not in the training set. Attackers therefore probe the same candidate records with variations and examine confidence or loss signals, which is a distinctive query pattern.
Question 11AI Security
Which approach best mitigates a backdoor implanted through poisoned fine-tuning data that triggers on a rare phrase?
- Increasing the number of inference replicas
- Rotating API keys on a fixed schedule
- Scanning training data provenance and testing the model with suspected trigger inputs before deployment Correct answer.
- Compressing the model to a smaller precision
Explanation
Backdoors are defined by a trigger that produces anomalous behavior, so detection combines supply-chain scrutiny of data provenance with targeted evaluation using suspected triggers. Scaling replicas, rotating keys, or compressing the model does not remove learned trigger behavior.
Question 12AI Security
A team deploys an LLM with a system prompt containing confidential business rules and asks whether users can extract it. What is the most accurate assessment?
- System prompts are only visible to administrators in all deployments
- System prompt content can often be elicited through crafted prompts, so it must not hold secrets Correct answer.
- System prompt extraction is impossible once temperature is set to zero
- System prompts are cryptographically protected and cannot be disclosed
Explanation
System prompts are plain text in the model's context and can frequently be revealed through careful questioning, so they should be treated as potentially disclosable. Secrets belong in server-side logic and access-controlled services, not in the prompt.
Question 13AI Security
When logging LLM interactions for security monitoring, which practice best balances detection value with privacy risk?
- Storing raw prompts and completions indefinitely in plaintext
- Redacting or hashing sensitive fields and applying retention limits while preserving security-relevant metadata Correct answer.
- Sending logs to the model provider without any agreement
- Disabling all logging to avoid collecting personal data
Explanation
Effective monitoring needs enough detail to detect abuse, but sensitive content should be minimized through redaction or hashing and bounded retention. This preserves detection and forensic capability while limiting exposure of personal data.
Question 14AI Security
An organization uses a third-party pretrained model. Which action best addresses supply-chain risk from that dependency?
- Skip version pinning so the latest weights are always used
- Verify the model's provenance, hash, and publisher signature before loading it Correct answer.
- Load the model with unrestricted network access to simplify setup
- Assume the model is safe because it is popular
Explanation
Model artifacts can carry malicious code or tampered weights, so integrity verification through provenance, hashes, and signatures is the core supply-chain control. Popularity and unpinned latest versions provide no assurance of integrity.
Question 15AI Security
Which scenario is the clearest example of an AI system producing a harmful output due to a jailbreak rather than a data poisoning attack?
- An attacker floods the endpoint until requests time out
- A user crafts a role-play prompt that leads the model to give prohibited instructions Correct answer.
- An attacker steals API keys and issues requests on another user's quota
- A model trained on tampered labels misclassifies specific inputs in production
Explanation
A jailbreak manipulates the prompt at inference time to bypass safety behavior without changing the model's weights. Poisoning alters behavior through training data, while flooding and key theft are availability and access-control issues respectively.
Question 16AI Security
A retrieval-augmented application must prevent an injected document from causing unauthorized data exfiltration through a URL-fetching tool. Which design is most effective?
- Increase the model's context window to include more documents
- Restrict the tool to an allowlist of destinations and require authorization for sensitive data access Correct answer.
- Rely on the model to refuse malicious instructions on its own
- Allow the tool to fetch any URL the model generates
Explanation
Exfiltration through tool calls is best blocked at the tool boundary with destination allowlisting and authorization checks on sensitive resources. Model self-refusal is probabilistic and cannot be the sole control, and a larger context window only exposes more untrusted content.
Question 17AI Security
Which metric best captures whether an LLM's safety filter is failing to block harmful requests while still allowing benign ones?
- Token cost per request
- Model parameter count
- Average response latency
- Both a false-negative rate on harmful prompts and a false-positive rate on benign prompts Correct answer.
Explanation
A safety filter has two error modes: letting harmful content through and blocking legitimate requests. Evaluating both false negatives and false positives gives a balanced view, whereas latency, cost, and size say nothing about classification quality.
Question 18AI Security
An AI agent is granted a long-lived API token with write access to a production database so it can answer user questions. Which change most reduces risk while preserving functionality?
- Extend the token lifetime to reduce rotation effort
- Share the token with all agents to simplify management
- Store the token in the system prompt for convenience
- Replace the token with scoped, short-lived credentials issued per task and limited to read-only where possible Correct answer.
Explanation
Scoping credentials to the minimum needed action and shortening their lifetime limits what a manipulated or compromised agent can do. Embedding tokens in prompts, extending lifetimes, and sharing them all increase exposure.
Question 19AI Security
A company wants to detect model extraction against its inference API. Which combination of controls is most appropriate?
- Rate limiting, query-pattern anomaly detection, and watermarking of outputs Correct answer.
- Disabling authentication to reduce friction
- Publishing the full model architecture and weights
- Removing all usage logs after 24 hours
Explanation
Extraction requires large volumes of informative queries, so rate limiting and anomaly detection on query patterns raise the cost, while watermarking helps attribute stolen functionality. Removing authentication or logs and publishing weights all make extraction easier.
Question 20AI Security
Which statement about adversarial examples in image classifiers is most accurate?
- They are eliminated permanently once a model reaches high accuracy
- They only occur in models trained without any regularization
- They require modifying the model's training data
- They are small, often imperceptible input perturbations that cause misclassification Correct answer.
Explanation
Adversarial examples are inputs altered by small perturbations crafted to change the model's prediction while remaining nearly unchanged to humans. They are an inference-time phenomenon and are not removed by high clean accuracy or by regularization alone.
Question 21AI Security
A team wants to reduce the risk that an LLM reveals personally identifiable information present in its fine-tuning corpus. Which measure is most appropriate?
- Lowering the maximum output length
- Applying PII detection and removal to the fine-tuning data and testing for memorization before release Correct answer.
- Adding a disclaimer to the user interface
- Increasing the model size to spread information across more parameters
Explanation
The durable fix is to keep sensitive data out of the training corpus and then verify through memorization testing that it is not regurgitated. UI disclaimers, larger models, and shorter outputs do not remove memorized content.
Question 22AI Security
An analyst observes that an LLM consistently produces a specific incorrect output whenever a rare token sequence appears in the input. Which explanation is most consistent with a backdoor?
- The model is experiencing normal sampling variance
- The tokenizer is misconfigured for all inputs
- The model was trained on data containing the trigger paired with the attacker's target behavior Correct answer.
- The endpoint is rate limiting the request
Explanation
A consistent, trigger-specific response pattern is the signature of a backdoor implanted during training or fine-tuning. Sampling variance would not be deterministic, rate limiting affects availability, and a tokenizer fault would affect inputs broadly rather than one trigger.
Question 23AI Security
A multi-agent system lets one agent delegate tasks to others. An attacker compromises a low-privilege agent and uses it to instruct a high-privilege agent to perform destructive actions. Which control most directly addresses this privilege escalation path?
- Requiring authenticated, authorized inter-agent messages with scoped capabilities and action approval Correct answer.
- Reducing the number of agents in the fleet
- Increasing the temperature of the high-privilege agent
- Caching the high-privilege agent's responses
Explanation
The attack exploits implicit trust between agents, so inter-agent calls must be authenticated and authorized against scoped capabilities, with approval for destructive actions. Temperature, caching, and fleet size do not establish trust boundaries between agents.
Question 24AI Security
A red team finds that an LLM will output its full system prompt when asked to repeat everything above the user message. Which remediation is most robust?
- Increasing the model's context window
- Randomizing the model's temperature per request
- Adding an instruction telling the model never to reveal the system prompt
- Moving sensitive configuration out of the prompt into server-side access-controlled components Correct answer.
Explanation
Prompt-level instructions are themselves part of the context and can be overridden or extracted, so they are not a reliable secret store. The robust fix is architectural: keep sensitive configuration outside the prompt where the model cannot disclose it.
Question 25AI Security
A team must choose between blocking all tool calls from an LLM agent and mediating them through a policy engine that evaluates each action against user authorization and data sensitivity. Which reasoning best supports the mediated approach?
- Mediation enforces least privilege per action while preserving necessary functionality, whereas blanket blocking sacrifices utility without addressing authorization logic Correct answer.
- Policy engines eliminate the need for logging and monitoring
- Mediation is only useful when the model has no system prompt
- Blocking all tool calls is always more secure and has no functional cost
Explanation
A policy engine evaluates each requested action against authorization and sensitivity, enforcing least privilege without disabling legitimate capability. Blanket blocking is secure but removes the agent's utility, and a policy engine complements rather than replaces logging and monitoring.
Question 26AI Security
In AI security, which term describes an attacker deliberately crafting input so a model produces an incorrect output chosen by the attacker?
- Adversarial example Correct answer.
- Model card
- Prompt template
- Data drift
Explanation
An adversarial example is an input intentionally perturbed so the model misclassifies it or produces attacker-chosen output, often with changes imperceptible to humans. Data drift, model cards, and prompt templates are unrelated to deliberate manipulation of model behavior.
Question 27AI Security
Which control most directly limits the damage a compromised LLM agent can do through its tool integrations?
- Scoping each tool's credentials to the minimum actions required Correct answer.
- Adding more few-shot examples to the prompt
- Increasing the model's context window
- Lowering the sampling temperature
Explanation
Tool abuse is contained by limiting what each integration can actually do, so least-privilege credentials cap the blast radius of a compromised agent. Context size, temperature, and few-shot examples affect model behavior but not the authority granted to tools.
Question 28AI Security
What is the primary security purpose of red teaming an LLM application before release?
- Verifying that the UI meets accessibility standards
- Estimating the cost of training from scratch
- Measuring inference latency under peak load
- Probing for harmful, unsafe, or policy-violating behavior and bypasses Correct answer.
Explanation
Red teaming is structured adversarial testing that seeks to surface harmful outputs, jailbreaks, and policy bypasses before adversaries find them. Performance, training cost, and accessibility are separate engineering concerns.
Question 29AI Security
Which practice best protects an LLM application from prompt injection delivered through user-supplied content?
- Trusting the model to distinguish instructions from data on its own
- Relying on the system prompt alone to forbid injections
- Treating retrieved or user content as untrusted and enforcing controls outside the model Correct answer.
- Removing all logging so injected payloads cannot be recorded
Explanation
Models cannot reliably separate instructions from data, so defenses must sit outside the model, such as output filtering and restricted tool access. Instructions in a prompt, including the system prompt, are not an enforceable security boundary.
Question 30AI Security
Which artifact documents a model's intended use, training data provenance, and known limitations for downstream consumers?
- Incident response plan
- Model card Correct answer.
- SBOM
- Access control matrix
Explanation
A model card records intended use, data provenance, evaluation results, and limitations so consumers can assess fitness and risk. SBOMs, incident plans, and access matrices serve different governance functions.
Question 31AI Security
An attacker repeatedly queries a deployed model with crafted inputs to infer its architecture and decision boundaries. Which threat category does this describe?
- Denial of service
- Model extraction Correct answer.
- Credential stuffing
- Data poisoning
Explanation
Model extraction, or model stealing, uses systematic queries to approximate a model's behavior or recover information about its structure. Denial of service, credential stuffing, and poisoning target availability, accounts, and training data respectively.
Question 32AI Security
Which property of a generative model most directly creates the risk that it may reproduce memorized sensitive content from its training data?
- Its tendency to emit verbatim sequences seen during training Correct answer.
- Its tokenizer vocabulary size
- Its parameter count
- Its inference batch size
Explanation
Memorization risk arises when models can regenerate near-verbatim training sequences, including sensitive text, in response to prompts. Parameter count, vocabulary, and batch size do not by themselves cause verbatim reproduction.
Question 33AI Security
Which measure most directly reduces the risk of unauthorized access to a hosted model endpoint?
- Storing API keys in client-side JavaScript
- Disabling input validation to speed responses
- Requiring authenticated, rate-limited access with per-client quotas Correct answer.
- Publishing the endpoint URL in documentation
Explanation
Authentication, rate limiting, and per-client quotas restrict who can query the endpoint and how much they can extract. Exposing URLs, skipping validation, or embedding keys in client code all increase exposure.
Question 34AI Security
A team wants to detect when a model's behavior has been manipulated by poisoned training data. Which detection approach is most appropriate?
- Shortening the model's maximum output length
- Increasing the learning rate on the next training run
- Removing all evaluation datasets from the pipeline
- Monitoring for anomalous trigger-conditioned outputs during evaluation with crafted inputs Correct answer.
Explanation
Backdoors manifest as anomalous behavior on specific trigger inputs, so targeted evaluation with candidate triggers can surface them. Training-rate changes, removing evaluations, and output limits do not detect implanted behavior.
Question 35AI Security
Which practice best limits the impact of a successful jailbreak against a customer-facing LLM?
- Relying solely on user reports of abuse
- Removing rate limits to reduce user frustration
- Assuming the base model's alignment is sufficient
- Layering output moderation and action restrictions behind the model Correct answer.
Explanation
Defense in depth assumes jailbreaks will sometimes succeed, so output moderation and restricted actions contain harm even when the model is manipulated. Trusting alignment alone, relying on reports, or removing rate limits leaves no backstop.
Question 36AI Security
An AI system processes untrusted documents and can call external tools. Which design most reduces the risk of indirect prompt injection causing harmful actions?
- Granting the agent broad tool permissions so it can complete any task
- Requiring human approval for high-impact tool actions and validating parameters Correct answer.
- Embedding secrets in the system prompt for the model to use
- Allowing the model to choose any URL to fetch without restriction
Explanation
Human approval gates and strict parameter validation prevent injected content from silently triggering consequential actions. Broad permissions, embedded secrets, and unrestricted fetching all amplify injection impact.
Question 37AI Security
Which approach best protects sensitive data when an LLM must summarize confidential documents?
- Sending full documents to any available third-party API
- Redacting or tokenizing sensitive fields before inference and rehydrating afterward Correct answer.
- Storing the documents unencrypted for faster retrieval
- Disabling audit logging to protect user privacy
Explanation
Redaction or tokenization keeps sensitive values out of the model and provider logs while preserving utility, with rehydration performed in a trusted layer. Sending raw data, storing it unencrypted, or disabling audits increases exposure.
Question 38AI Security
What is the main security concern with fine-tuning a base model on user-submitted conversations without review?
- The tokenizer will stop functioning
- The model will necessarily become slower at inference
- The model may learn and later emit sensitive or malicious content present in that data Correct answer.
- The model's license will be automatically revoked
Explanation
Unreviewed user data can contain secrets, personal information, or attacker-crafted content that the model memorizes or learns to reproduce, creating poisoning and leakage risk. Performance, tokenizer function, and licensing are not the core concern.
Question 39AI Security
Which control most directly mitigates the risk of an LLM leaking secrets embedded in its system prompt?
- Increasing the model's temperature
- Keeping secrets out of prompts and injecting them server-side only when needed Correct answer.
- Shortening the user's input length
- Asking the model politely not to reveal the prompt
Explanation
The durable fix is to never place secrets where the model can emit them, resolving credentials server-side instead. Instructions not to reveal, temperature, and input length do not prevent disclosure.
Question 40AI Security
A security team wants to bound how much information an attacker can extract from a model API over time. Which combination is most effective?
- Unlimited free queries and public model weights
- Per-user query quotas, anomaly detection on query patterns, and watermarking of outputs Correct answer.
- Returning full probability vectors with no rate limit
- Disabling authentication to simplify access
Explanation
Quotas limit volume, anomaly detection flags extraction-like patterns, and watermarking supports attribution of stolen outputs. Unlimited access, no authentication, and unrestricted probability vectors all aid extraction.
Question 41AI Security
Which statement best describes the role of an AI system's threat model in security engineering?
- It enumerates assets, adversaries, and attack paths to prioritize controls Correct answer.
- It is only relevant after a breach occurs
- It guarantees the system cannot be attacked
- It replaces the need for incident response
Explanation
A threat model identifies what is being protected, who might attack it, and how, so defenses can be prioritized rationally. It is not a guarantee, a substitute for response planning, or a post-incident-only artifact.
Question 42AI Security
An organization must choose how to handle model outputs that may contain malware-laden links. Which control is most appropriate?
- Auto-opening all links in the user's browser
- Trusting the model to never generate malicious URLs
- Disabling all output filtering to preserve fidelity
- Rendering links as plain text and warning users before navigation Correct answer.
Explanation
Treating model output as untrusted, defanging links, and warning users prevents drive-by navigation to malicious destinations. Auto-opening, blind trust, and removing filters all increase risk.
Question 43AI Security
Which practice best supports forensic investigation after an AI system misuse incident?
- Logging only aggregate request counts
- Storing logs unencrypted on public storage
- Retaining tamper-evident logs of prompts, tool calls, and model versions with defined retention Correct answer.
- Deleting logs daily to save storage
Explanation
Tamper-evident logs capturing inputs, actions, and model versions enable reconstruction of what happened, subject to retention and privacy policy. Deleting logs, logging only counts, or exposing logs unencrypted undermines investigation.
Question 44AI Security
A team observes that a classifier's accuracy collapses on inputs with small, carefully chosen pixel changes. Which property is being demonstrated?
- Class imbalance in training data
- Overfitting to the test set
- Robustness failure to adversarial perturbation Correct answer.
- Label noise in annotations
Explanation
Sensitivity to small, deliberately chosen perturbations is the defining signature of adversarial vulnerability, distinct from generalization or data-quality problems. Overfitting, imbalance, and label noise produce different error patterns.
Question 45AI Security
Which approach best reduces the risk that a multi-agent system lets a compromised agent escalate its privileges?
- Allowing every agent to call every other agent freely
- Enforcing per-agent identity, scoped permissions, and inter-agent authentication Correct answer.
- Removing audit trails between agents
- Sharing one master credential across all agents
Explanation
Per-agent identity, scoped permissions, and authenticated inter-agent calls prevent a low-privilege compromise from spreading. Free lateral calls, shared master credentials, and missing audits enable escalation and hide it.
Question 46AI Security
An organization wants to verify that a third-party model provider enforces data-handling commitments. Which action is most appropriate?
- Assuming provider defaults are secure
- Relying on marketing claims alone
- Skipping review because the model is popular
- Reviewing contractual terms, audit reports, and data-retention settings Correct answer.
Explanation
Verification requires evidence such as contractual obligations, independent audit reports, and configuration of retention and training-use settings. Marketing claims, assumed defaults, and popularity are not evidence of compliance.
Question 47AI Security
A model can be queried with unlimited free-text inputs and returns only final answers. Which extraction defense is most effective under these constraints?
- Publishing the model weights to deter attackers
- Disabling authentication to simplify monitoring
- Watermarking outputs and monitoring for systematic query patterns indicative of cloning Correct answer.
- Removing all rate limits to reduce incentive
Explanation
When outputs must be returned, watermarking supports attribution of stolen models and pattern monitoring can flag systematic cloning attempts. Publishing weights, removing limits, and disabling authentication all make extraction easier.
Question 48AI Security
A red team finds that an agent will execute a destructive tool call when a retrieved document contains a hidden instruction. Which remediation addresses the root cause most directly?
- Adding the phrase 'ignore hidden instructions' to the system prompt
- Lowering the model temperature during retrieval
- Treating retrieved content as untrusted data and requiring policy-checked authorization for destructive actions Correct answer.
- Increasing the retrieval window to include more documents
Explanation
The root cause is that untrusted retrieved content can influence privileged actions, so the fix is to separate data from authority and enforce authorization outside the model. Prompt admonitions, larger retrieval windows, and temperature changes do not establish that boundary.
Question 49AI Security
An attacker with brief access to a training pipeline inserts a small set of poisoned samples. Which property makes this attack hard to detect by simple accuracy checks?
- Poisoning only affects inference latency
- Poisoned samples always reduce accuracy to near zero
- The trigger is rare, so overall accuracy on normal data stays high Correct answer.
- The trigger appears in every input, so it is obvious
Explanation
Backdoor triggers are deliberately rare, so aggregate accuracy on ordinary data remains normal while behavior on trigger inputs is compromised. Accuracy collapse, ubiquitous triggers, or latency effects would make the attack conspicuous.
Question 50AI Security
A team must decide whether to allow an LLM to summarize emails and send replies automatically. Which design best balances utility and security?
- Disabling logging to protect email privacy
- Drafting replies for human review with scoped read access and send approval Correct answer.
- Allowing the model to send replies directly with full mailbox access
- Granting permanent admin credentials to the agent
Explanation
Human review before sending and scoped read access preserve utility while preventing injected content from causing unauthorized communication. Direct sending, admin credentials, and disabled logging remove the controls that contain mistakes or attacks.