I85PREP
⌘ K
CAIS study materials
CAIS study resources

CAIS practice questions

One question at a time. Build your CAIS understanding.

First 50 of 200 questions. Each one lists the choices, the correct answer, and an explanation. The checker at the top of this page saves progress in this browser when it is available.

Continue in the practice tool above to load more questions, 50 at a time.

Question 1AI governance, risk management and policy

What does the MANAGE function of the NIST AI RMF primarily involve?

  1. MANAGE Correct answer.
  2. GOVERN
  3. MEASURE
  4. MAP

Explanation

The NIST AI RMF organizes activities into GOVERN, MAP, MEASURE, and MANAGE. MANAGE is where prioritized risks are treated, resources are allocated, and monitoring is used to adjust responses. MAP identifies and contextualizes risks, while MEASURE analyzes and tracks them.

Question 2AI governance, risk management and policy

What is the main purpose of maintaining an AI model inventory within an AI governance program?

  1. To certify every model against all global AI laws
  2. To record AI systems and their risk attributes Correct answer.
  3. To replace the organization's data classification scheme
  4. To increase training throughput for machine learning teams

Explanation

An AI model inventory provides a central record of systems, owners, data dependencies, intended use, and risk ratings so governance controls can be applied consistently. It supports traceability and reporting but does not by itself certify legal compliance or replace data classification.

Question 3AI governance, risk management and policy

Which principle commonly requires that affected individuals receive meaningful information about an AI system's operation and outcomes?

  1. Federated learning
  2. Data minimization
  3. Homomorphic encryption
  4. Transparency Correct answer.

Explanation

Transparency requires that people affected by an AI system can obtain meaningful information about its purpose, logic, and outcomes. Data minimization limits collection, while federated learning and homomorphic encryption are technical privacy approaches rather than governance principles.

Question 4AI governance, risk management and policy

Under the EU AI Act, which practice is prohibited?

  1. Using an AI system to generate synthetic training data
  2. Conducting a conformity assessment for a high-risk system
  3. Documenting a general-purpose AI model's training data summary
  4. Using public-authority social scoring to disadvantage people Correct answer.

Explanation

The EU AI Act prohibits certain practices outright, including social scoring by public authorities that leads to detrimental or unfavorable treatment. Conformity assessment and documentation are obligations for certain systems, not prohibited practices.

Question 5AI governance, risk management and policy

In an AI risk register, which entry best represents a residual risk?

  1. The risk of model drift before monitoring is deployed
  2. The risk that a regulator introduces a new AI law
  3. The risk that a vendor refuses to share training data
  4. Risk remaining after existing mitigations are applied Correct answer.

Explanation

Residual risk is the risk that remains after planned or existing controls have been applied. Inherent risk is the risk before controls, and risk treatment can reduce, transfer, avoid, or accept risk. A risk register should distinguish inherent and residual risk so treatment effectiveness can be evaluated.

Question 6AI governance, risk management and policy

Which control best supports human oversight for a high-impact AI decision support system?

  1. Automatic retraining on every user decision
  2. Expansion of model parameter count
  3. Human review and override of high-impact outputs Correct answer.
  4. Removal of audit logs to protect model trade secrets

Explanation

Human oversight controls are intended to catch errors, provide accountability, and allow intervention in consequential decisions. Reviewers need authority, information, and logging to exercise meaningful override rather than superficial approval.

Question 7AI governance, risk management and policy

What is a key governance reason to document an AI system's intended use and reasonably foreseeable misuse?

  1. To set context for risk assessment and controls Correct answer.
  2. To guarantee that the model will not be biased
  3. To avoid classifying the system as high-risk
  4. To eliminate the need for post-market monitoring

Explanation

Documenting intended use and reasonably foreseeable misuse establishes the boundaries for risk assessment, testing, and control selection. It helps teams identify affected stakeholders and failure modes, but it cannot guarantee fairness or remove the need for monitoring.

Question 8AI governance, risk management and policy

In third-party AI risk management, which artifact most directly helps an adopter evaluate a vendor's AI security and governance posture?

  1. The vendor's stock price history and recent earnings calls
  2. A completed vendor AI risk questionnaire with evidence Correct answer.
  3. The vendor's marketing brochure
  4. The vendor's office floor plan

Explanation

Third-party AI risk assessment depends on evidence about the vendor's governance, security, data handling, and model lifecycle controls. A completed risk questionnaire with supporting evidence gives the adopter comparable information, while marketing materials and stock data do not demonstrate control effectiveness.

Question 9AI governance, risk management and policy

Which statement best distinguishes AI risk appetite from AI risk tolerance?

  1. Appetite applies only to vendors; tolerance applies to staff
  2. Appetite is the maximum vendor spend; tolerance is the audit budget
  3. Appetite is a model metric; tolerance is a legal penalty
  4. Appetite is risk sought; tolerance is acceptable variation Correct answer.

Explanation

Risk appetite is the amount and type of risk an organization is willing to accept in pursuit of its objectives. Risk tolerance translates that appetite into acceptable variation or thresholds for specific activities, so the two concepts are related but not interchangeable.

Question 10AI governance, risk management and policy

A company uses AI to screen job applicants. Which governance response best reflects proportionality?

  1. Full delegation of accountability to the model vendor
  2. Logging disabled to reduce privacy complaints
  3. The same minimal controls as a spam filter because both are software products
  4. Heightened oversight and bias testing for employment decisions Correct answer.

Explanation

Proportionality means controls should match the likelihood and severity of harm, so employment screening warrants stronger oversight because it affects access to livelihoods and may engage legal protections. Vendor contracts can allocate some duties, but accountability for the deployer's use of the system remains with the organization.

Question 11AI governance, risk management and policy

Which artifact establishes the organization-wide authority, decision rights, and accountability structure for managing AI risk?

  1. A data sheet describing dataset collection methods
  2. A penetration test report for the AI inference API
  3. A model card published for each production model
  4. An AI governance charter approved by executive leadership Correct answer.

Explanation

An AI governance charter is the instrument that assigns decision rights, oversight responsibilities, and accountability for AI risk across the enterprise. Model cards, data sheets, and penetration test reports are supporting technical artifacts; they document specific systems or datasets rather than establish organization-wide governance authority.

Question 12AI governance, risk management and policy

Which international standard specifies requirements for establishing, implementing, maintaining, and continually improving an artificial intelligence management system (AIMS)?

  1. ISO/IEC 38500
  2. ISO/IEC 23894
  3. ISO/IEC 42001 Correct answer.
  4. ISO/IEC 27001

Explanation

ISO/IEC 42001 is the management system standard for artificial intelligence, specifying requirements for an AIMS that can be audited and continually improved. ISO/IEC 27001 covers information security management, while ISO/IEC 23894 provides AI risk management guidance and ISO/IEC 38500 addresses corporate IT governance.

Question 13AI governance, risk management and policy

Within the NIST AI Risk Management Framework, which function is primarily concerned with establishing the context of an AI system and identifying relevant risks?

  1. GOVERN
  2. MANAGE
  3. MAP Correct answer.
  4. MEASURE

Explanation

The MAP function establishes the context of the AI system, identifies stakeholders, and surfaces relevant risks before measurement or treatment. GOVERN sets culture and accountability, MEASURE analyzes and assesses risks, and MANAGE prioritizes and treats them.

Question 14AI governance, risk management and policy

A public authority in the EU plans to deploy an AI system to prioritize welfare benefit applications for investigation. Before deployment, which assessment is most directly required to examine the system's impact on fundamental rights?

  1. A fundamental rights impact assessment Correct answer.
  2. An environmental impact assessment
  3. A model card describing model performance
  4. A data protection impact assessment only

Explanation

The EU AI Act requires a fundamental rights impact assessment for certain high-risk AI systems deployed by public bodies or private providers of public services, including in access to benefits. A DPIA focuses on personal data protection, a model card documents model characteristics, and an environmental impact assessment addresses ecological effects.

Question 15AI governance, risk management and policy

A vendor of an internal summarization tool cannot provide training data provenance. The tool will process only non-sensitive internal documents. Which governance response is most defensible?

  1. Approve with documented compensating controls, limited scope, and review Correct answer.
  2. Rely on the vendor's claim that the model is trustworthy
  3. Reject the tool because provenance is always mandatory for every AI system
  4. Approve the tool without conditions because the data is non-sensitive

Explanation

Risk-based governance allows conditional approval when residual risk is documented and within appetite, using compensating controls such as scope limits and scheduled review. Provenance gaps are a real concern but do not automatically require rejection in every low-sensitivity context, and vendor assurances alone are insufficient evidence.

Question 16AI governance, risk management and policy

A production AI model begins issuing biased recommendations that affect customers. Which action best aligns with AI incident response governance?

  1. Wait for the next quarterly risk review before taking action
  2. Immediately retrain the model and redeploy it without documenting the event
  3. Activate the incident response plan and preserve logs Correct answer.
  4. Delete the model's logs to prevent reputational harm

Explanation

AI incident response governance requires activating the plan, containing the issue, preserving logs for investigation, and assessing affected customers before remediation decisions. Retraining or deleting evidence without documentation undermines root-cause analysis and accountability.

Question 17AI governance, risk management and policy

Which measure best evaluates whether an AI governance training program is changing employee behavior rather than merely recording attendance?

  1. The percentage of employees who completed the annual module
  2. The number of policy documents published on the intranet
  3. Scenario-based assessment results showing staff escalate AI concerns correctly Correct answer.
  4. The total training hours delivered per department

Explanation

Training effectiveness is best judged by evidence that employees can apply governance expectations in realistic situations, such as correctly escalating an AI concern. Completion rates, policy counts, and training hours measure activity or outputs, not changed behavior.

Question 18AI governance, risk management and policy

Under the EU AI Act, which set of obligations applies to providers of general-purpose AI models that are classified as having systemic risk?

  1. Reporting every downstream use of the model to the national supervisory authority
  2. Only publishing a summary of training content and registering a contact point
  3. Obtaining prior conformity assessment from a notified body before each release
  4. Performing model evaluation, adversarial testing, incident reporting, and cybersecurity protection Correct answer.

Explanation

Providers of general-purpose AI models with systemic risk must perform model evaluation and adversarial testing, report serious incidents, and ensure cybersecurity protection under the EU AI Act. Documentation and a contact point apply more broadly to GPAI providers, while conformity assessment is a high-risk AI system requirement rather than a per-release GPAI model requirement.

Question 19AI governance, risk management and policy

An organization relies on a vendor-hosted AI model for fraud detection. Which contract provision best addresses accountability when the vendor silently changes the model in ways that affect performance?

  1. A clause limiting the vendor's liability to the prior year's subscription fees
  2. A clause granting the vendor ownership of all customer transaction data
  3. A clause requiring notice of material model changes and updated performance documentation Correct answer.
  4. A clause requiring the customer to accept all model updates automatically

Explanation

A change-notification clause with updated performance documentation enables the customer to monitor drift, reassess risk, and hold the vendor accountable for material modifications. Automatic acceptance, liability caps, and data ownership clauses do not provide visibility into silent model changes.

Question 20AI governance, risk management and policy

A product team wants to launch a high-risk AI feature, but the risk team and legal team disagree on whether the residual risk is acceptable. Which governance action is most appropriate?

  1. Let the product team decide because it owns the launch timeline
  2. Ask the vendor to certify that the residual risk is acceptable
  3. Launch the feature and address concerns after customer complaints
  4. Escalate the unresolved risk decision to the accountable AI governance body Correct answer.

Explanation

Unresolved disagreements about residual risk should be escalated to the accountable governance body, which can make and document a risk-based determination. Letting the product team decide creates a conflict of interest, and post-launch reaction or vendor certification does not substitute for organizational accountability.

Question 21AI governance, risk management and policy

In a three-lines-of-defense model for AI risk, which function provides independent assurance to the board that the AI control environment is operating effectively?

  1. Internal audit, which reports independently on the effectiveness of the AI control environment Correct answer.
  2. The product team that owns and operates the AI system day to day
  3. The risk function that sets the AI risk framework and monitors compliance
  4. The board's AI committee, which approves the annual risk appetite statement

Explanation

Internal audit is the third line of defense and supplies independent assurance on whether AI controls and risk processes work as intended. Product teams own risk in the first line, while the risk function performs second-line oversight and challenge. A board committee provides governance direction but is not an independent assurance function.

Question 22AI governance, risk management and policy

An AI team finds that a recommendation model could produce unlawful disparate impact and decides to stop offering the feature in a jurisdiction rather than add further controls. Which risk treatment option does this describe?

  1. Risk acceptance, by documenting the disparate impact as tolerable
  2. Risk avoidance, by discontinuing the feature in that jurisdiction Correct answer.
  3. Risk mitigation, by adding a bias monitoring dashboard
  4. Risk transfer, by purchasing cyber insurance for the model

Explanation

Risk avoidance eliminates the activity that creates the risk, which is what discontinuing the feature does. Mitigation would reduce the risk while continuing the activity, transfer would shift financial consequences to another party, and acceptance would retain the risk with documented approval. The decisive fact is that the risky activity stops.

Question 23AI governance, risk management and policy

Which artifact most directly gives downstream users a standardized summary of an AI model's intended use, performance metrics, and known limitations?

  1. A data processing agreement
  2. A SOC 2 Type II report
  3. A model card Correct answer.
  4. An incident response log

Explanation

A model card is designed to document a model's intended use, performance characteristics, and limitations so that consumers and reviewers can assess fitness for purpose. A data processing agreement governs personal data handling, a SOC 2 report covers service controls, and an incident log records past events. Only the model card directly summarizes the model itself.

Question 24AI governance, risk management and policy

Before deploying an AI system that ranks emergency callers for dispatch, which assessment most directly evaluates effects on individuals and groups and documents mitigations?

  1. A vendor security questionnaire
  2. An algorithmic impact assessment covering affected groups and mitigations Correct answer.
  3. A model performance benchmark on a holdout dataset
  4. A business impact analysis focused on service downtime

Explanation

An algorithmic impact assessment examines how an AI system may affect people and groups, including fairness, rights, and safety concerns, and records mitigations. A benchmark measures technical performance, a business impact analysis focuses on continuity, and a security questionnaire addresses vendor controls. The impact assessment is the governance tool aimed at downstream human and societal effects.

Question 25AI governance, risk management and policy

Within the NIST AI Risk Management Framework, which activity belongs primarily to the MEASURE function?

  1. Defining the AI risk appetite and assigning accountability
  2. Prioritizing identified risks for treatment after deployment
  3. Mapping the system's purpose and deployment context
  4. Testing the AI system against metrics for accuracy, robustness, and bias Correct answer.

Explanation

The MEASURE function uses quantitative, qualitative, or mixed-method techniques to analyze, assess, benchmark, and monitor AI risks and impacts. Defining risk appetite and accountability is part of GOVERN, mapping purpose and context is part of MAP, and prioritizing risks for treatment is part of MANAGE. Testing against accuracy, robustness, and bias metrics therefore fits MEASURE directly.

Question 26AI governance, risk management and policy

Under the EU AI Act, a provider of a chatbot that interacts with humans must generally ensure that users are informed they are interacting with an AI system. Which obligation does this illustrate?

  1. The post-market monitoring duty for general-purpose AI models
  2. The conformity assessment requirement for high-risk AI systems
  3. The transparency obligation for AI systems that interact with humans Correct answer.
  4. The prohibition on social scoring by public authorities

Explanation

The EU AI Act includes transparency duties for certain AI systems, including informing natural persons when they are interacting with an AI system unless this is obvious from the circumstances. Social scoring is a prohibited practice, conformity assessment applies to high-risk systems, and post-market monitoring applies to general-purpose AI models under specified conditions. The chatbot disclosure is a transparency requirement.

Question 27AI governance, risk management and policy

A governance team is classifying third-party AI vendors. Which factor should most directly raise a vendor's risk tier?

  1. The size of the vendor's sales team and the length of its free trial
  2. The vendor's model can access regulated personal data and make decisions affecting access to credit Correct answer.
  3. The vendor's headquarters country and the polish of its marketing materials
  4. The vendor's use of a click-through contract for online acceptance

Explanation

Vendor risk tiering should reflect the potential impact of the service, including data sensitivity, decision authority, and harm to individuals. Access to regulated personal data combined with credit-access decisions creates high privacy, fairness, and legal exposure. Headquarters marketing, sales team size, and contract acceptance mechanics are weak indicators compared with inherent risk and use context.

Question 28AI governance, risk management and policy

An AI fraud-detection system begins declining legitimate transactions for one customer segment. Which criterion is most appropriate for assigning the incident a high severity level?

  1. The potential for material financial harm and loss of access to essential services across an affected population Correct answer.
  2. The cloud provider that hosts the model
  3. The number of internal staff who reported the issue
  4. The duration of the incident response meeting

Explanation

AI incident severity should be based on actual or potential impact, such as financial harm, denial of services, safety effects, and the size of the affected population. Internal reporting volume and meeting duration are process indicators, and the hosting provider is not a harm criterion. The segment-wide denial of legitimate transactions is the fact that drives high severity.

Question 29AI governance, risk management and policy

A company wants to train an AI model in a cloud region abroad using customer records from the EU. Which governance control most directly addresses the legal basis and safeguards for the transfer?

  1. A standard contractual clause or adequacy decision covering the transfer Correct answer.
  2. An internal model card describing training metrics
  3. A red-team report on prompt injection
  4. A service-level agreement for model uptime

Explanation

Cross-border transfers of personal data require a lawful transfer mechanism, such as an adequacy decision or appropriate safeguards like standard contractual clauses. A model card, red-team report, and uptime SLA address documentation, security testing, and availability rather than transfer legality. The transfer mechanism is therefore the control that directly governs the legal basis and safeguards.

Question 30AI governance, risk management and policy

A product team requests an exception to an AI policy because residual risk exceeds the stated tolerance but is expected to last three months. Who should generally decide whether to accept this risk, and on what basis?

  1. The product manager, because they own the feature roadmap
  2. The vendor, because it hosts the model
  3. End users, because they can choose not to use the feature
  4. The AI governance body with delegated authority, applying documented risk acceptance criteria and time-bound conditions Correct answer.

Explanation

Risk acceptance above tolerance should be decided by a body with delegated authority, using documented criteria and, where appropriate, time-bound conditions and monitoring. Product managers may propose exceptions but normally lack authority to accept risk above the approved appetite. Vendors and end users do not hold the organization's accountability for residual AI risk.

Question 31AI governance, risk management and policy

Which artifact typically sets mandatory, organization-wide rules for acceptable employee use of AI tools and states the consequences of violating them?

  1. A model card published for each deployed AI model
  2. A system architecture diagram approved by the engineering team
  3. AI acceptable use policy Correct answer.
  4. An AI incident log maintained by the security operations team

Explanation

An acceptable use policy translates the organization's risk appetite into mandatory rules for users, including prohibited uses, approved data handling, and sanctions. Model cards document model characteristics and limitations rather than employee behavior. Incident logs and architecture diagrams support operations and design review but do not set user conduct rules.

Question 32AI governance, risk management and policy

In an AI governance framework, what is the primary purpose of designating a named model owner for each AI system?

  1. To perform independent assurance over the AI risk function for the board
  2. To ensure a specific person is accountable for the model's lifecycle risks and decisions Correct answer.
  3. To transfer all legal liability for the model from the organization to that individual
  4. To approve the enterprise-wide AI budget and vendor procurement strategy

Explanation

A named model owner creates clear accountability across the model lifecycle, including risk acceptance, performance monitoring, and escalation decisions. Legal liability generally remains with the organization and cannot simply be reassigned to an employee. Budget approval and independent assurance are separate governance functions.

Question 33AI governance, risk management and policy

Which indicator is most useful as an early warning that a deployed machine-learning model may be degrading before outcome metrics show harm?

  1. Significant drift in the distribution of production input features Correct answer.
  2. The count of completed AI awareness training modules
  3. The total cloud compute cost of model training
  4. The number of confirmed AI incidents reported during the last quarter

Explanation

Input feature drift can signal that production data no longer resembles the data used to develop or validate the model, which may precede measurable performance loss. Incident counts are lagging indicators because harm must already have occurred. Training activity and compute cost do not directly measure production model behavior.

Question 34AI governance, risk management and policy

An AI policy states that high-risk systems require human review before adverse decisions. Which artifact should specify how reviewers document that review and retain evidence?

  1. The board charter that establishes the AI governance committee's authority
  2. An operational procedure for the review step Correct answer.
  3. The vendor contract covering the AI platform's service levels
  4. A model card describing the system's intended use and performance

Explanation

Policies set the mandatory requirement, while operational procedures or work instructions define how staff execute the step, including required evidence and retention. A board charter establishes authority, a model card documents model characteristics, and a contract governs vendor obligations. None of those artifacts normally tells a reviewer how to record and retain a human review decision.

Question 35AI governance, risk management and policy

A company is selecting a third-party generative AI service for customer support. Which due-diligence request most directly tests whether the vendor can support the company's records-retention and legal-hold obligations?

  1. A copy of the vendor's latest generative AI benchmark results
  2. Retention schedule, storage locations, and deletion/export controls Correct answer.
  3. The vendor's marketing description of its responsible AI principles
  4. A summary of the vendor's uptime service-level commitment

Explanation

Records-retention and legal-hold obligations depend on knowing where data is stored, how long it is kept, and whether it can be preserved or exported on request. Benchmark results address model quality, responsible AI statements are not operational evidence, and uptime commitments address availability rather than retention. The retention and deletion controls therefore provide the most direct evidence.

Question 36AI governance, risk management and policy

A company plans to use biometric facial recognition to verify employee attendance. Which characteristic most directly makes a data protection impact assessment appropriate before deployment?

  1. The vendor hosting the system is headquartered outside the company's country
  2. The system is expected to reduce attendance administration time by five percent
  3. The system relies on an open-source face-matching library
  4. Biometric processing that systematically identifies employees Correct answer.

Explanation

A data protection impact assessment is particularly appropriate when processing involves biometric data used to identify individuals or systematic monitoring of people. Using open-source components does not remove that risk, and vendor location may raise transfer issues but is not the primary trigger. Administrative efficiency gains do not offset the privacy risk of biometric identification.

Question 37AI governance, risk management and policy

Which control most directly reduces the risk that a multi-tenant generative AI assistant will expose one customer's confidential prompts to another customer?

  1. Running periodic fairness testing on model outputs
  2. Adding a disclaimer that model outputs may contain inaccuracies
  3. Increasing the model temperature to produce more varied responses
  4. Tenant isolation and clearing prompt data between sessions Correct answer.

Explanation

Cross-customer disclosure is a confidentiality and access-control risk, so tenant isolation and prompt-data clearing directly limit one customer's ability to reach another customer's content. Disclaimers address reliance on output accuracy, temperature changes affect variability, and fairness testing addresses disparate outcomes. None of those controls prevents cross-tenant data exposure.

Question 38AI governance, risk management and policy

A team wants to reuse customer transaction data, originally collected for fraud monitoring, to train a new marketing-recommendation model. Which governance control most directly addresses purpose limitation?

  1. Publish a model card describing the recommender after deployment
  2. Increase the training dataset by adding synthetic transaction records
  3. Move the model to a cloud region with lower inference latency
  4. Confirm purpose compatibility and document the new use Correct answer.

Explanation

Purpose limitation requires that a new use be compatible with the purpose for which the data was originally collected and that the new use be documented. Adding synthetic records, changing hosting location, or publishing a model card after deployment does not establish compatibility. The compatibility assessment and documentation are the direct controls.

Question 39AI governance, risk management and policy

An enterprise consumes a foundation model through an API, and the provider can silently update the underlying model. Which governance mechanism best preserves the enterprise's ability to detect and respond to changed model behavior?

  1. A contractual clause requiring the provider to publish a general responsible AI statement
  2. A user-facing disclaimer stating that model outputs may change without notice
  3. An annual review of the provider's financial statements and insurance coverage
  4. Pinned versions, change notification, regression tests, rollback criteria Correct answer.

Explanation

Pinning model versions and requiring advance change notification give the enterprise a stable baseline and time to test updates. Regression tests and rollback criteria then provide a controlled way to detect behavioral changes and revert if risk exceeds tolerance. General statements, financial review, and disclaimers do not create operational control over model updates.

Question 40AI governance, risk management and policy

A board asks for evidence that the AI governance program is effective rather than merely active. Which measure set best answers that question?

  1. Number of AI policies issued, training sessions delivered, and governance meetings held
  2. Count of registered AI use cases, signed vendor contracts, and trained employees
  3. Employee satisfaction with AI tools and the number of prompts submitted
  4. Residual-risk trends, control test results, incident recurrence, remediation timeliness Correct answer.

Explanation

Effectiveness is shown by whether controls reduce risk and whether problems are detected, remediated, and prevented from recurring. Residual-risk trends, control testing, incident recurrence, and remediation timeliness are outcome-oriented measures. Activity counts and user satisfaction may show adoption or effort but do not demonstrate that governance is working.

Question 41AI threat landscape and adversarial machine learning

An attacker adds a small, human-imperceptible perturbation to an image so that a deployed classifier assigns a wrong label at inference time. Which category best describes this attack?

  1. Training-data poisoning with a trigger
  2. Membership inference on a training record
  3. Model inversion of training records
  4. Evasion using an adversarial example Correct answer.

Explanation

Evasion attacks craft inputs at inference time so a trained model misclassifies them, and adversarial examples are the classic form of this. The perturbation is optimized to change the model's output while remaining difficult for humans to detect. Poisoning instead alters training data, and inference attacks target privacy rather than prediction integrity.

Question 42AI threat landscape and adversarial machine learning

During training, an attacker inserts a rare pixel pattern into a small subset of images and labels them with a target class. At inference, images containing the pattern are classified as that target class. What is this attack called?

  1. Backdoor poisoning via a trigger pattern Correct answer.
  2. Model extraction through API queries
  3. Adversarial patch applied at inference
  4. Gradient inversion of the training set

Explanation

Backdoor or trojan poisoning implants a trigger during training so the model behaves normally on clean inputs but produces the attacker's target output when the trigger appears. Because the trigger is learned from poisoned training samples, it is activated later at inference. Adversarial patches are crafted at inference time and do not require modifying training data.

Question 43AI threat landscape and adversarial machine learning

An attacker uses a model's confidence scores to determine whether a specific patient record was part of its training set. Which privacy attack is this?

  1. Adversarial evasion
  2. Model inversion
  3. Membership inference Correct answer.
  4. Model extraction

Explanation

Membership inference exploits differences in model behavior between training and non-training records to decide whether a given record was seen during training. It is a privacy attack because that membership fact may itself be sensitive. Model inversion reconstructs features of records, while model extraction clones model behavior.

Question 44AI threat landscape and adversarial machine learning

Which description matches the Fast Gradient Sign Method (FGSM)?

  1. Random search over noise magnitudes until the predicted label flips
  2. One epsilon-scaled step along the sign of the input gradient Correct answer.
  3. Fine-tuning on correctly classified samples after each query round
  4. An iterative multi-step optimization projected back into an epsilon ball

Explanation

FGSM is a single-step attack that perturbs the input in the direction of the sign of the gradient of the loss with respect to the input, scaled by epsilon. This keeps the change within a small bound while seeking to increase the loss. Iterative projected variants extend the idea over multiple steps.

Question 45AI threat landscape and adversarial machine learning

An attacker repeatedly queries a public machine-learning API with synthetic inputs and trains a substitute model on the returned labels. Which threat does this describe?

  1. Data poisoning
  2. Prompt injection
  3. Membership inference
  4. Model extraction Correct answer.

Explanation

Model extraction, also called model stealing, uses many queries and the target's outputs to approximate its decision function with a substitute model. The attacker gains a reusable copy that may support further attacks without continued API access. Membership inference and poisoning have different objectives and do not primarily aim to clone the model.

Question 46AI threat landscape and adversarial machine learning

An attacker with no access to a target model's weights trains a local substitute, generates adversarial examples on it, and finds that many also fool the target. Which property explains this?

  1. Transferability Correct answer.
  2. Post-training quantization
  3. Data augmentation
  4. Differential privacy

Explanation

Transferability is the tendency of adversarial examples crafted for one model to also mislead another model trained on a similar task. It lets an attacker operate without white-box access to the target. Differential privacy, quantization, and data augmentation do not explain cross-model attack success.

Question 47AI threat landscape and adversarial machine learning

A fraud model passes pre-deployment tests. Months later, transactions containing a specific merchant-code sequence are consistently approved, although similar transactions without the sequence are flagged. Investigation shows attackers had injected labeled samples with that sequence during periodic retraining. Which description best fits?

  1. Pure evasion: attackers perturb each transaction at inference without affecting training.
  2. Poisoning that installs a trigger, later activated by attacker-crafted inputs. Correct answer.
  3. Model inversion: attackers reconstruct typical legitimate transactions from outputs.
  4. Membership inference: attackers infer whether a transaction was in the training set.

Explanation

The injected training samples teach the model to associate the merchant-code sequence with a benign or approved outcome. That is a backdoor-style poisoning attack, and the trigger is later exploited at inference. Pure evasion would not require influencing the retraining data.

Question 48AI threat landscape and adversarial machine learning

A team wants to reduce a vision model's susceptibility to small input perturbations used at inference. Which control addresses this most directly?

  1. Differential privacy during training
  2. Output watermarking
  3. Query rate limiting
  4. Adversarial training Correct answer.

Explanation

Adversarial training includes adversarial examples in the training process so the model learns to classify them correctly, directly targeting evasion robustness. It is not a complete defense, but it addresses the perturbation threat more directly than privacy, watermarking, or rate limiting. Those other controls serve different security objectives.

Question 49AI threat landscape and adversarial machine learning

A vendor claims a classifier is robust because FGSM succeeds only 1% of the time; the defense uses input quantization and a non-differentiable preprocessing step. An independent tester trains a surrogate and transfers perturbations, achieving 80% success. What is the most likely explanation?

  1. Gradient masking Correct answer.
  2. Training data leakage
  3. Catastrophic forgetting
  4. Label noise

Explanation

Non-differentiable preprocessing can break white-box gradient attacks and produce misleadingly low attack success rates, a failure mode known as gradient masking or obfuscated gradients. It does not establish true robustness because adaptive or transfer attacks can still succeed. The large gap between reported and independent results is characteristic of this problem.

Question 50AI threat landscape and adversarial machine learning

A retrieval-augmented LLM assistant summarizes external web pages. An attacker publishes a page containing hidden text instructing the model to ignore prior instructions and reveal the system prompt; when the page is retrieved, the assistant complies. What is this?

  1. Membership inference
  2. Model inversion
  3. Indirect prompt injection Correct answer.
  4. Poisoning of the model weights

Explanation

Indirect prompt injection places malicious instructions in content that the model later retrieves or processes, rather than in the user's direct input. The retrieved page becomes an untrusted instruction channel that can override intended behavior. It does not require changing the model's trained weights.