I85PREP
⌘ K
EXIN AI Security Professional study materials
EXIN AI Security Professional study resources

EXIN AI Security Professional practice questions

One question at a time. Build your EXIN AI Security Professional understanding.

First 50 of 200 questions. Each one lists the choices, the correct answer, and an explanation. The checker at the top of this page saves progress in this browser when it is available.

Continue in the practice tool above to load more questions, 50 at a time.

Question 1AI security (technical security and governance)

An organization wants to reduce the risk of a prompt injection attack reaching its customer-facing LLM application. Which control is most directly aimed at the injection input path?

  1. Encrypting the model weights at rest in the hosting environment
  2. Rotating the API keys used by the application every 90 days
  3. Enforcing multi-factor authentication for the model's administrative console
  4. Isolating the model's tool permissions and validating tool arguments before execution Correct answer.

Explanation

Prompt injection becomes dangerous mainly when the model can act through tools, so constraining tool permissions and validating arguments limits what an injected instruction can achieve. Encryption at rest, key rotation, and admin MFA protect infrastructure but do not stop malicious instructions embedded in user or retrieved content from influencing model behavior.

Question 2AI security (technical security and governance)

In the context of AI security governance, what does model provenance documentation primarily establish?

  1. The maximum number of concurrent inference requests a model can serve
  2. The origin, training data lineage, and modification history of a model Correct answer.
  3. The physical location of the GPU cluster used for inference
  4. The licensing cost of the model per token at production scale

Explanation

Provenance records where a model came from, what data and process produced it, and how it has been changed since, which supports supply chain trust and accountability. Capacity, pricing, and hosting location are operational facts rather than provenance.

Question 3AI security (technical security and governance)

A security team discovers that an attacker extracted a near-identical copy of a proprietary model by sending large volumes of crafted queries through the public API. Which threat category does this describe?

  1. Data poisoning
  2. Adversarial evasion
  3. Membership inference
  4. Model extraction Correct answer.

Explanation

Model extraction, also called model stealing, occurs when an adversary queries a deployed model extensively to train a substitute that approximates its behavior. Poisoning corrupts training data, membership inference tests whether specific records were in the training set, and evasion crafts inputs to cause misclassification.

Question 4AI security (technical security and governance)

Which practice best mitigates training data poisoning risk in an AI supply chain?

  1. Validating and signing training datasets before they enter the pipeline Correct answer.
  2. Publishing the model architecture in open documentation
  3. Reducing the context window size of the deployed model
  4. Increasing the learning rate during fine-tuning

Explanation

Poisoning is introduced through data, so integrity controls such as dataset validation, provenance checks, and cryptographic signing at ingestion reduce the chance that tampered data trains the model. Hyperparameters, architecture disclosure, and context length do not verify data integrity.

Question 5AI security (technical security and governance)

An AI governance policy requires that every production model have a named accountable owner and an approved risk assessment before deployment. Which governance mechanism does this most directly represent?

  1. A model inventory with gated approval workflow Correct answer.
  2. An incident response playbook
  3. A red team exercise schedule
  4. A data retention policy

Explanation

A model inventory captures each model along with its owner and risk assessment, and gating deployment on approval enforces accountability before release. Red teaming, retention, and incident response are separate controls that do not by themselves assign ownership or require pre-deployment approval.

Question 6AI security (technical security and governance)

A team deploys a retrieval-augmented generation system where documents from external sources are inserted into the prompt. What is the primary security concern this design introduces?

  1. Slower embedding generation throughput
  2. Loss of model quantization precision
  3. Indirect prompt injection through poisoned retrieved content Correct answer.
  4. Increased token cost per request

Explanation

When retrieved content is placed in the prompt, an attacker who can influence that content can embed instructions the model may follow, which is indirect prompt injection. Quantization, token cost, and embedding throughput are performance or cost considerations, not the primary security exposure.

Question 7AI security (technical security and governance)

Which control most directly limits the impact of an AI agent that has been manipulated into attempting unauthorized actions?

  1. Disabling logging to reduce storage costs
  2. Granting the agent broad standing credentials to reduce integration effort
  3. Increasing the model temperature for more creative responses
  4. Applying least-privilege scoped tokens and human approval for high-impact actions Correct answer.

Explanation

Least-privilege scoped credentials and human approval for consequential actions bound what a manipulated agent can do, which is a core agentic security control. Broad credentials expand blast radius, disabling logging removes detection evidence, and temperature affects output variability rather than authorization.

Question 8AI security (technical security and governance)

What distinguishes a membership inference attack from a model inversion attack?

  1. Membership inference targets API availability, while model inversion targets billing systems
  2. Membership inference modifies training data, while model inversion modifies model weights
  3. Membership inference determines whether a record was in the training set, while model inversion reconstructs representative features of training data Correct answer.
  4. Membership inference requires physical access, while model inversion requires source code

Explanation

Membership inference answers a yes-or-no question about whether a specific record contributed to training, whereas model inversion attempts to reconstruct sensitive attributes or representative inputs from model outputs. Both are privacy attacks on a deployed model and neither requires physical access or weight modification.

Question 9AI security (technical security and governance)

An organization uses a third-party foundation model via API. Which governance artifact best documents the security responsibilities split between the provider and the organization?

  1. A copy of the provider's public pricing page
  2. An internal style guide for prompt writing
  3. A marketing brochure describing model capabilities
  4. A shared responsibility matrix covering data handling, logging, and incident notification Correct answer.

Explanation

A shared responsibility matrix explicitly maps which party handles data protection, logging, monitoring, and breach notification for a third-party model service. Capability brochures, pricing pages, and prompt style guides do not allocate security obligations between parties.

Question 10AI security (technical security and governance)

Which technique is designed to detect whether a model's output was influenced by an adversarial input perturbation?

  1. Compressing the model with pruning
  2. Input preprocessing with anomaly detection and output consistency checks Correct answer.
  3. Caching frequent responses at the edge
  4. Increasing the batch size of inference requests

Explanation

Anomaly detection on inputs plus consistency checks on outputs can flag perturbations that push behavior away from expected distributions. Batch size, pruning, and edge caching change performance characteristics and do not detect adversarial influence.

Question 11AI security (technical security and governance)

What is the primary purpose of a model card in an AI security program?

  1. To replace the need for access control on the model endpoint
  2. To record the model's intended use, limitations, and evaluation results for informed risk decisions Correct answer.
  3. To store the encrypted weights of the model
  4. To serve as the runtime API gateway configuration

Explanation

A model card documents intended use, known limitations, and evaluation results so consumers and reviewers can judge suitability and residual risk. It is a documentation artifact, not a weights store, gateway configuration, or substitute for access control.

Question 12AI security (technical security and governance)

A red team finds that adding a small patch to an image causes a classifier to mislabel it consistently. What property does this demonstrate?

  1. The model is overfit to the test set
  2. The model lacks a model card
  3. The model has a data retention defect
  4. The model is vulnerable to adversarial examples Correct answer.

Explanation

A small, deliberately crafted perturbation that reliably changes predictions is the definition of an adversarial example, showing a robustness weakness. Overfitting, retention defects, and missing documentation are different issues that this behavior does not by itself establish.

Question 13AI security (technical security and governance)

An AI system processes personal data, and regulators ask how the organization prevents training data from being memorized and leaked. Which combination is most appropriate?

  1. Disabling input validation plus raising the temperature
  2. Publishing raw training data plus open-sourcing weights
  3. Differential privacy during training plus output filtering for sensitive patterns Correct answer.
  4. Increasing model size plus removing audit logs

Explanation

Differential privacy bounds the influence of any single record on the trained model, and output filtering catches residual leakage of sensitive patterns. Larger models, removed logs, raw data publication, and disabled validation all increase exposure rather than reduce memorization risk.

Question 14AI security (technical security and governance)

Which logging practice best supports forensic investigation of an AI application incident while respecting privacy?

  1. Logging redacted prompt metadata, model version, and decision outcomes with defined retention Correct answer.
  2. Disabling logging for requests that contain personal data
  3. Logging full raw prompts with user identifiers retained indefinitely
  4. Logging only the HTTP status code of each request

Explanation

Redacted metadata, model version, and outcomes give investigators the context needed to reconstruct events while limiting personal data exposure through defined retention. Full raw prompt logging with indefinite retention increases privacy risk, and minimal or disabled logging removes forensic value.

Question 15AI security (technical security and governance)

In an AI incident response plan, why is a model rollback capability important?

  1. It guarantees the model will never produce harmful output
  2. It encrypts the model's training data automatically
  3. It allows rapid reversion to a known-good model version when a deployed model is compromised or misbehaving Correct answer.
  4. It removes the need for monitoring after deployment

Explanation

Rollback lets responders restore a previously validated model version quickly, containing harm from a compromised or faulty deployment. It cannot guarantee harmless outputs, does not replace post-deployment monitoring, and has no encryption function.

Question 16AI security (technical security and governance)

A company wants to verify that a model artifact downloaded from a public repository has not been tampered with. Which control is most appropriate?

  1. Running the model once to see if outputs look reasonable
  2. Verifying a cryptographic signature and hash against a trusted publisher Correct answer.
  3. Reviewing the repository's star rating
  4. Checking the model's parameter count

Explanation

Signature and hash verification against a trusted publisher confirms artifact integrity and origin, which is the standard supply chain control. Parameter count, popularity metrics, and spot-checking outputs cannot detect tampering.

Question 17AI security (technical security and governance)

Which statement best describes the security rationale for using a guardrail model alongside a primary LLM in production?

  1. The guardrail model stores the primary model's training data
  2. The guardrail model increases the primary model's parameter count
  3. The guardrail model replaces the need for authentication on the endpoint
  4. The guardrail model independently classifies inputs and outputs to block policy-violating content Correct answer.

Explanation

A guardrail model acts as an independent classifier that screens prompts and responses for policy violations, adding a layer the primary model cannot reliably enforce alone. It does not provide authentication, alter parameter counts, or store training data.

Question 18AI security (technical security and governance)

What is the key security difference between open-weight and closed API-only model deployment?

  1. Open-weight models allow the organization to inspect and harden the artifact locally but shift more patching responsibility to it Correct answer.
  2. Closed API models cannot be used for regulated workloads
  3. Closed API models never require authentication
  4. Open-weight models cannot be fine-tuned

Explanation

With open weights the organization can inspect, modify, and harden the artifact, but it also assumes responsibility for patching and securing the runtime. Closed API models still require authentication and can serve regulated workloads under appropriate contracts.

Question 19AI security (technical security and governance)

Which metric is most relevant when evaluating whether an AI content filter is appropriately tuned for security screening?

  1. The disk space consumed by the filter's container image
  2. The filter's false positive and false negative rates on a representative labeled set Correct answer.
  3. The filter's average response time in milliseconds
  4. The number of parameters in the filter model

Explanation

Security screening quality depends on how often the filter wrongly blocks benign content and, more importantly, how often it misses malicious content, which false positive and false negative rates quantify. Latency, parameter count, and image size are operational metrics unrelated to detection quality.

Question 20AI security (technical security and governance)

An attacker manipulates a model's behavior by embedding hidden instructions in a document that a summarization tool later processes. Which term best describes this attack?

  1. Data exfiltration via side channel
  2. Gradient masking
  3. Model inversion
  4. Indirect prompt injection Correct answer.

Explanation

Indirect prompt injection occurs when malicious instructions arrive through content the model consumes, such as a document to be summarized, rather than directly from the user. Model inversion is a privacy attack, gradient masking is an obfuscation pitfall, and side-channel exfiltration is a different mechanism.

Question 21AI security (technical security and governance)

Which governance control best addresses the risk that an AI vendor silently changes the model behind a stable API endpoint?

  1. Increasing the organization's own inference batch size
  2. Relying on the vendor's uptime SLA alone
  3. Contractual change notification requirements combined with periodic behavioral regression testing Correct answer.
  4. Removing version identifiers from internal documentation

Explanation

Silent model updates are detected by requiring contractual notice of changes and by periodically running regression tests that reveal behavioral drift. Uptime SLAs cover availability, batch size is a performance setting, and removing version identifiers harms traceability.

Question 22AI security (technical security and governance)

What is the purpose of adversarial training in an AI security program?

  1. To encrypt model weights before distribution
  2. To improve model robustness by including adversarial examples in the training process Correct answer.
  3. To reduce the number of API endpoints exposed
  4. To teach users how to write better prompts

Explanation

Adversarial training augments the training set with crafted adversarial examples so the model learns to resist similar perturbations. It is not user education, weight encryption, or an endpoint reduction measure.

Question 23AI security (technical security and governance)

A security architect must decide where to place content filtering in a RAG pipeline. Which placement provides the strongest defense against poisoned retrieved documents?

  1. Filtering only the embedding model's outputs
  2. Filtering retrieved documents before they are inserted into the prompt, plus filtering the final response Correct answer.
  3. Filtering only the final response after generation
  4. Filtering only the user's initial question

Explanation

Filtering retrieved content before insertion blocks malicious instructions from reaching the model, and filtering the final response catches any harmful output that still emerges. Filtering only the question, only the response, or only embeddings leaves a gap on the retrieval path.

Question 24AI security (technical security and governance)

Which practice best supports auditability of human oversight over high-risk AI decisions?

  1. Relying on the model to self-certify that its decisions were reviewed
  2. Recording reviewer identity, timestamp, decision, and rationale for each override Correct answer.
  3. Allowing reviewers to act without logging to speed up processing
  4. Storing review records only in individual reviewers' local machines

Explanation

Auditability of human oversight requires durable records linking each override to a reviewer, time, decision, and stated rationale. Unlogged actions, model self-certification, and local-only storage all undermine independent verification.

Question 25AI security (technical security and governance)

Why does differential privacy offer only a bounded, probabilistic guarantee rather than absolute protection against training data leakage?

  1. It removes all rare records from the dataset before training
  2. It relies on the model never being queried more than once
  3. It adds calibrated noise, so privacy loss is limited by a parameter but not eliminated for every possible record Correct answer.
  4. It encrypts each record with a unique key that attackers can brute force

Explanation

Differential privacy injects calibrated noise so that any single record's influence is bounded by a privacy budget parameter, which limits but does not eliminate inference risk. It is not a per-record encryption scheme, does not simply delete rare records, and does not assume a single query.

Question 26AI security (technical security and governance)

An organization wants to reduce the risk that a public-facing LLM assistant generates harmful content even if the base model is later swapped or updated. Which architectural pattern most directly provides a model-independent safety layer?

  1. Fine-tuning the base model on a curated dataset of safe responses
  2. Adding a separate input/output moderation service that screens prompts and completions before and after the model call Correct answer.
  3. Increasing the context window so more policy text fits in the prompt
  4. Lowering the model's temperature parameter to reduce randomness

Explanation

A separate moderation service enforces policy outside the model itself, so it continues to apply even if the underlying model is replaced or retrained. Fine-tuning and prompt-based policy text are tied to a specific model version and can be lost or bypassed. Temperature only affects sampling variability, not policy enforcement.

Question 27AI security (technical security and governance)

Which concept describes an attacker supplying crafted inputs specifically designed to make a deployed model produce incorrect or attacker-chosen outputs?

  1. Data drift
  2. Concept shift
  3. Adversarial input Correct answer.
  4. Label noise

Explanation

Adversarial inputs are deliberately crafted to exploit model weaknesses and cause misclassification or manipulated output. Data drift and concept shift are natural changes in input or target distributions over time, not intentional attacks. Label noise refers to errors in training labels.

Question 28AI security (technical security and governance)

A team wants to protect an AI inference API from credential-stuffing and automated abuse. Which control is most appropriate at the API gateway layer?

  1. Rate limiting combined with anomaly detection on request patterns Correct answer.
  2. Encrypting model weights at rest
  3. Publishing the model architecture in documentation
  4. Rotating the model's training dataset

Explanation

Rate limiting and behavioral anomaly detection at the gateway directly counter automated abuse such as credential stuffing and scraping. Encrypting model weights protects the artifact at rest but does not stop abusive request patterns. Dataset rotation and architecture disclosure are unrelated to API abuse prevention.

Question 29AI security (technical security and governance)

What does the principle of least privilege mean when applied to an AI agent that can call external tools and APIs?

  1. The agent should run with administrator rights to avoid permission errors
  2. The agent should log every action it takes for audit purposes
  3. The agent should use the largest available model for reliability
  4. The agent should be granted only the permissions strictly needed for its defined tasks Correct answer.

Explanation

Least privilege means scoping an agent's credentials and tool access to the minimum required for its tasks, limiting blast radius if it is manipulated. Administrator rights and broad access increase the impact of compromise. Logging and model size are separate concerns from permission scope.

Question 30AI security (technical security and governance)

Which artifact records the intended use, known limitations, and evaluation results of a trained model so downstream teams can assess its fitness?

  1. A penetration test report
  2. A service-level agreement
  3. A model card Correct answer.
  4. A data retention schedule

Explanation

A model card documents intended use, limitations, and evaluation results, helping consumers judge suitability and risk. An SLA covers service commitments, a penetration test report covers discovered vulnerabilities, and a retention schedule governs data lifecycle.

Question 31AI security (technical security and governance)

An AI application stores user prompts containing sensitive data. Which practice best limits exposure if the underlying storage is later compromised?

  1. Storing prompts in plaintext for easier debugging
  2. Encrypting sensitive fields with keys managed separately from the storage system Correct answer.
  3. Retaining all prompts indefinitely for future model training
  4. Sharing the prompt database with all engineering teams

Explanation

Field-level encryption with separately managed keys means a storage compromise alone does not reveal plaintext prompts. Plaintext storage, indefinite retention, and broad sharing all increase exposure and blast radius. Separating key management from storage is the core control.

Question 32AI security (technical security and governance)

Which term describes the risk that a model trained on historical hiring decisions reproduces and amplifies past discriminatory patterns?

  1. Data augmentation
  2. Prompt injection
  3. Algorithmic bias Correct answer.
  4. Model quantization

Explanation

Algorithmic bias occurs when a model learns and perpetuates unfair patterns present in training data, such as historical discrimination. Quantization reduces numeric precision, prompt injection is an input manipulation attack, and data augmentation expands training data synthetically.

Question 33AI security (technical security and governance)

Why must an AI security program treat the training data supply chain as part of its threat model?

  1. Training data has no effect on model behavior after deployment
  2. Model weights fully determine behavior regardless of training data
  3. Training data is always public and therefore cannot be tampered with
  4. Poisoned or unvetted training data can embed backdoors or biases that persist in the deployed model Correct answer.

Explanation

Training data directly shapes model behavior, so tampered or unvetted data can introduce backdoors, biases, or degraded safety that survive into production. Treating data as inherently safe or irrelevant ignores a major attack surface. Weights are derived from data, so data integrity matters.

Question 34AI security (technical security and governance)

A security team wants to detect when a deployed model's behavior diverges from its validated baseline after an update. Which practice is most direct?

  1. Removing all logging to reduce storage costs
  2. Continuous monitoring of output distributions and quality metrics against a baseline Correct answer.
  3. Increasing the model's parameter count
  4. Disabling version control for model artifacts

Explanation

Comparing live output distributions and quality metrics against a validated baseline reveals behavioral drift or regressions after updates. Adding parameters does not detect divergence, and removing logging or version control eliminates the evidence needed to detect and investigate change.

Question 35AI security (technical security and governance)

An attacker gains read access to a model's weights and uses them to craft inputs that reliably evade detection. Which property of the deployment most increased this risk?

  1. The model used a transformer architecture
  2. The model was trained on a large dataset
  3. The model was served over HTTPS
  4. The model weights were exposed to the attacker Correct answer.

Explanation

Access to weights enables white-box adversarial techniques that can craft highly effective evasive inputs, so weight exposure is the risk amplifier. HTTPS protects traffic in transit but not weights already exposed. Architecture and dataset size are not the direct cause of the evasion advantage.

Question 36AI security (technical security and governance)

Which control best reduces the risk that an AI coding assistant leaks secrets from a developer's environment into its suggestions or logs?

  1. Storing assistant conversation logs unencrypted for transparency
  2. Disabling code review for AI-generated changes
  3. Secret scanning and redaction applied to context before it is sent to the model Correct answer.
  4. Allowing the assistant to read all environment variables by default

Explanation

Scanning and redacting secrets from the context before transmission prevents sensitive values from reaching the model or its logs. Broad environment access increases leakage, unencrypted logs expose captured secrets, and skipping review removes a key detection point.

Question 37AI security (technical security and governance)

An organization must decide how much human review to require before an AI system's high-impact decisions take effect. Which approach best reflects a risk-proportionate governance design?

  1. Require no human review because the model was validated before deployment
  2. Require human sign-off for decisions above a defined impact threshold, with sampling review below it Correct answer.
  3. Delegate review entirely to the model's own confidence score
  4. Require human review of every decision regardless of impact

Explanation

Risk-proportionate oversight concentrates scarce human attention on the highest-impact decisions while using sampling for lower-risk ones. No review ignores residual risk, reviewing everything is often infeasible, and relying solely on model confidence lets the model grade its own work.

Question 38AI security (technical security and governance)

Which security property does a signed model artifact with provenance metadata primarily provide to a deployment pipeline?

  1. Assurance that the model will meet latency targets
  2. Guarantee that the model cannot be adversarially attacked
  3. Guarantee that the model is free of bias
  4. Assurance that the artifact came from a trusted source and was not altered Correct answer.

Explanation

Signing and provenance metadata let the pipeline verify origin and integrity of the artifact before deployment. They say nothing about bias, adversarial robustness, or performance, which require separate evaluation.

Question 39AI security (technical security and governance)

A team is deploying a RAG assistant and wants to limit the damage if a retrieved document contains malicious instructions. Which design choice best supports this?

  1. Allow retrieved documents to modify the assistant's system instructions
  2. Give the model broad tool access so it can act on any instruction it finds
  3. Treat retrieved content as untrusted data and constrain the model's tool permissions accordingly Correct answer.
  4. Merge retrieved content directly into the system prompt without separation

Explanation

Treating retrieved content as untrusted data and limiting tool permissions reduces the impact of embedded instructions. Broad tool access, merging content into the system prompt, and allowing documents to alter system instructions all increase the chance that injected content is followed.

Question 40AI security (technical security and governance)

Which practice best supports reproducibility when investigating an AI security incident involving a specific model version?

  1. Deleting old model versions to save storage
  2. Relying on the current production model to reconstruct past behavior
  3. Recording model version, configuration, and data snapshots used at the time of the incident Correct answer.
  4. Relying on staff memory of configuration changes

Explanation

Capturing version, configuration, and data snapshots enables investigators to reconstruct the exact conditions of the incident. Using the current model, deleting old versions, and relying on memory all destroy or distort the evidence needed for reliable reconstruction.

Question 41AI security (technical security and governance)

What is the main security reason to enforce strict schema validation on tool outputs that an AI agent consumes?

  1. Schema validation eliminates the need for authentication
  2. Schema validation reduces the model's parameter count
  3. Malformed or unexpected output can cause the agent to take unintended actions Correct answer.
  4. Schema validation improves the model's training accuracy

Explanation

Strict schema validation ensures an agent only acts on well-formed, expected data, reducing the chance that malformed or manipulated outputs trigger unintended actions. It does not affect training accuracy or parameter count, and it is not a substitute for authentication.

Question 42AI security (technical security and governance)

An organization shares a dataset with a research partner for model development. Which control best reduces the risk of re-identification of individuals in the dataset?

  1. Applying de-identification and aggregation techniques before sharing Correct answer.
  2. Sharing the raw dataset under a confidentiality clause alone
  3. Removing only the name column and sharing the rest
  4. Hashing identifiers with a public, unsalted function

Explanation

De-identification combined with aggregation reduces the linkage risk that enables re-identification. Removing only names leaves quasi-identifiers, a confidentiality clause is legal rather than technical protection, and unsalted hashing is reversible via lookup tables.

Question 43AI security (technical security and governance)

Which statement best captures why AI systems require security controls beyond those used for traditional software?

  1. They never process untrusted input
  2. Their behavior is learned from data and can be manipulated through inputs and training data rather than only through code Correct answer.
  3. They cannot be deployed in networked environments
  4. They do not require authentication because they are probabilistic

Explanation

Learned behavior creates attack surfaces such as data poisoning and adversarial inputs that do not exist in deterministic code, so AI systems need additional controls. AI systems do process untrusted input, are commonly networked, and still require authentication.

Question 44AI security (technical security and governance)

A governance team wants to ensure that a vendor's AI service meets agreed security requirements over time. Which mechanism best supports ongoing verification rather than one-time assurance?

  1. A single security questionnaire completed before contract signing
  2. Verbal assurances from the vendor's sales team
  3. A press release describing the vendor's security commitment
  4. Contractual audit rights combined with periodic evidence reviews of the vendor's controls Correct answer.

Explanation

Audit rights plus periodic evidence reviews create a repeatable verification loop that catches control degradation over time. A one-time questionnaire, verbal assurances, and marketing statements provide no ongoing assurance.

Question 45AI security (technical security and governance)

Which risk is most directly associated with allowing an AI model to autonomously execute shell commands in a production environment?

  1. Increased model training time
  2. Unintended or malicious command execution with production-level impact Correct answer.
  3. Reduced model interpretability
  4. Higher inference latency

Explanation

Autonomous shell execution lets a model, if manipulated or mistaken, run commands with real production consequences, so it is a high-impact capability requiring strict sandboxing and least privilege. Training time, interpretability, and latency are not the primary security risk of this capability.

Question 46AI security (technical security and governance)

An attacker repeatedly queries a deployed model through its public API to reconstruct a functionally similar model. Which control most directly raises the cost of this attack?

  1. Publishing the model's full architecture
  2. Disabling authentication on the API
  3. Removing all usage logs
  4. Rate limiting and query monitoring with anomaly detection Correct answer.

Explanation

Model extraction relies on large volumes of queries, so rate limiting and anomaly monitoring raise the cost and detect the pattern. Publishing architecture, disabling authentication, and removing logs all make extraction easier or harder to detect.

Question 47AI security (technical security and governance)

A security analyst observes that a model's accuracy drops sharply only for inputs containing a specific rare token pattern. Which explanation is most consistent with a backdoor attack?

  1. The model is overfitting to the general training distribution
  2. The model has been quantized too aggressively
  3. The training dataset was too small overall
  4. A trigger pattern was embedded during training that causes targeted misbehavior Correct answer.

Explanation

Backdoors produce behavior that is normal on typical inputs but fails on a specific trigger, matching a drop confined to a rare token pattern. Overfitting, quantization, and small datasets tend to cause broader degradation rather than trigger-specific failure.

Question 48AI security (technical security and governance)

Which approach best mitigates the risk that an AI system's training pipeline pulls dependencies from an unverified public package registry?

  1. Trusting packages with high download counts without verification
  2. Allowing any version to be installed to stay current
  3. Pinning versions and verifying package signatures or hashes against an approved internal mirror Correct answer.
  4. Disabling dependency scanning to speed up builds

Explanation

Pinning versions and verifying signatures or hashes against an approved mirror prevents tampered or unexpected packages from entering the pipeline. Unpinned installs, disabled scanning, and popularity-based trust all leave supply-chain compromise undetected.

Question 49AI security (technical security and governance)

A red team demonstrates that an LLM can be induced to reveal its system prompt through multi-turn conversational pressure. Which mitigation most directly reduces this exposure without retraining the model?

  1. Extending the maximum response length
  2. Adding more detailed instructions to the system prompt
  3. Treating the system prompt as non-secret and moving sensitive logic to server-side controls Correct answer.
  4. Increasing the model's temperature

Explanation

Since system prompts can often be extracted through conversational pressure, sensitive logic and secrets should live in server-side controls the model cannot disclose. Adding detail to the prompt increases what can leak, and temperature or response length do not address exposure.

Question 50AI security (technical security and governance)

An organization wants to evaluate whether its AI system remains robust when an attacker can adapt to its defenses over time. Which evaluation design is most appropriate?

  1. A survey of end users about perceived safety
  2. Iterative adversarial testing in which attackers adjust tactics against the current deployed defenses Correct answer.
  3. A one-time benchmark run before launch
  4. A static checklist review of documentation

Explanation

Adaptive adversaries change tactics in response to defenses, so iterative adversarial testing better reflects real-world robustness than static evaluation. One-time benchmarks, documentation checklists, and user surveys do not model an adapting attacker.