I85PREP
⌘ K
AI Transformation Leader study materials
AI Transformation Leader study resources

AI Transformation Leader practice questions

One question at a time. Build your AI Transformation Leader understanding.

First 50 of 200 questions. Each one lists the choices, the correct answer, and an explanation. The checker at the top of this page saves progress in this browser when it is available.

Continue in the practice tool above to load more questions, 50 at a time.

Question 1AI Strategy and Business Alignment

Which statement best describes the primary purpose of an AI strategy document for an enterprise?

  1. It serves as the procurement contract for purchasing AI software licenses
  2. It prioritizes AI investments against business goals and defines how value will be measured Correct answer.
  3. It lists the technical specifications of every machine learning model the company will build
  4. It records the daily operational tasks assigned to the data science team

Explanation

An AI strategy connects AI initiatives to business objectives, sets priorities, and defines success metrics so investments can be evaluated. It operates at the portfolio and governance level rather than the level of model internals or daily task tracking.

Question 2AI Strategy and Business Alignment

A company wants to reduce customer churn. Which framing best aligns an AI initiative with business value?

  1. Collect as much customer data as possible before scoping
  2. Deploy the most advanced deep learning architecture available
  3. Define the churn reduction target and the decision it will change Correct answer.
  4. Build a model with the highest possible accuracy on a benchmark

Explanation

Business alignment starts from the outcome and the decision the AI will inform, then works back to the model. Technology choice and data collection follow from that framing rather than leading it.

Question 3AI Strategy and Business Alignment

What does a business case for an AI initiative typically need in order to justify investment?

  1. A commitment to hire only PhD-level researchers
  2. A list of open-source libraries the team will use
  3. A guarantee that the model will never make errors
  4. A comparison of expected benefits against costs and risks Correct answer.

Explanation

A business case weighs projected benefits, costs, and risks so decision-makers can compare the initiative against alternatives. It does not require zero errors or a specific staffing pedigree.

Question 4AI Strategy and Business Alignment

Which metric is a lagging indicator of AI program success?

  1. Count of experiments run in the quarter
  2. Number of models deployed to production
  3. Realized revenue uplift attributable to AI Correct answer.
  4. Percentage of employees who completed AI training

Explanation

Lagging indicators measure realized outcomes such as revenue or cost savings that appear after the initiative has taken effect. Deployment counts, training completion, and experiment volume are activity or leading measures.

Question 5AI Strategy and Business Alignment

What is the main role of an executive sponsor in an enterprise AI program?

  1. Writing the production code for the core models
  2. Performing daily model performance monitoring
  3. Removing organizational barriers and securing resources for the initiative Correct answer.
  4. Labeling training data during model development

Explanation

An executive sponsor provides authority, resources, and air cover, and resolves cross-functional blockers. Hands-on technical and operational work is delegated to the delivery team.

Question 6AI Strategy and Business Alignment

Which sequence reflects a sound order for aligning AI initiatives with business strategy?

  1. Identify business objectives, then prioritize use cases, then define measures of success Correct answer.
  2. Select a model, then find a problem it can solve, then estimate value
  3. Hire a data science team, then decide which department to serve, then set goals
  4. Collect data, then choose a vendor, then define the objective

Explanation

Alignment flows from business objectives to prioritized use cases and then to explicit success measures. Starting from technology or staffing inverts the alignment logic.

Question 7AI Strategy and Business Alignment

A retailer's AI team proposes a demand forecasting model. Which question most directly tests strategic alignment?

  1. Whether the data scientists hold relevant academic degrees
  2. Which business decision and financial outcome the forecast will improve Correct answer.
  3. Whether the model uses gradient boosting or a neural network
  4. How many lines of code the implementation requires

Explanation

Strategic alignment is tested by linking the model to a specific decision and the financial or operational outcome it improves. Algorithm choice, credentials, and code size are secondary considerations.

Question 8AI Strategy and Business Alignment

What distinguishes a leading indicator from a lagging indicator in an AI portfolio review?

  1. Lagging indicators are collected only from model training logs
  2. Leading indicators predict future outcomes and can be acted on early Correct answer.
  3. Lagging indicators are always more actionable for course correction
  4. Leading indicators can only be measured after the fiscal year closes

Explanation

Leading indicators, such as adoption rates or pilot conversion, move before outcomes and allow early intervention. Lagging indicators confirm results after the fact and are less actionable for mid-course correction.

Question 9AI Strategy and Business Alignment

An organization wants to build an AI capability roadmap. Which element belongs at the portfolio level rather than the individual project level?

  1. The sequencing of use cases across business units over several quarters Correct answer.
  2. The choice of activation function in a neural network
  3. The schema of a single training dataset
  4. The learning rate used during model training

Explanation

A roadmap sequences initiatives across the organization over time and allocates resources among them. Hyperparameters and dataset schemas are project-level technical details.

Question 10AI Strategy and Business Alignment

Which factor most often causes AI pilots to fail to scale into production value?

  1. Excessive executive interest in the initiative
  2. Too few machine learning frameworks available
  3. Overly detailed success metrics
  4. Weak integration with existing business processes and workflows Correct answer.

Explanation

Pilots frequently stall because they are not embedded into the processes and systems where decisions are made. Framework availability, executive attention, and clear metrics generally help rather than hinder scaling.

Question 11AI Strategy and Business Alignment

A company sets a goal to 'become AI-driven' without further detail. What is the main strategic weakness of this goal?

  1. It is too specific about which technologies to use
  2. It specifies too many success metrics
  3. It commits the firm to a single vendor
  4. It lacks measurable outcomes and a defined scope Correct answer.

Explanation

A goal must be measurable and scoped to guide prioritization and investment. An aspirational slogan without metrics or boundaries cannot direct decisions or be evaluated.

Question 12AI Strategy and Business Alignment

Which stakeholder group is most important to engage early when defining AI use-case priorities?

  1. External auditors who review financial statements
  2. Open-source maintainers of the modeling library
  3. Competitors operating in adjacent markets
  4. The business owners of the processes the AI will affect Correct answer.

Explanation

Process owners understand where AI can change decisions and where adoption barriers lie, making them essential early partners. Auditors, competitors, and library maintainers are not the primary stakeholders for use-case prioritization.

Question 13AI Strategy and Business Alignment

What is the best way to translate an AI capability into an explicit business value hypothesis?

  1. List the data sources the model will consume
  2. Describe the mechanism by which the capability changes a decision and its financial effect Correct answer.
  3. State the model's F1 score as the expected business benefit
  4. Name the cloud provider that will host the model

Explanation

A value hypothesis explains the causal path from the AI capability to a changed decision and then to a financial or operational effect. Technical scores and infrastructure choices are inputs, not value hypotheses.

Question 14AI Strategy and Business Alignment

An AI initiative has strong expected benefits but requires changes to a regulated process. What should the strategy team do first?

  1. Abandon the initiative because regulation forbids AI
  2. Deploy the model and address compliance after launch
  3. Assess regulatory constraints and design compliance into the rollout plan Correct answer.
  4. Outsource the entire initiative to avoid accountability

Explanation

Regulated processes require constraints to be identified up front and built into the design and rollout. Proceeding first and fixing compliance later creates legal and operational risk.

Question 15AI Strategy and Business Alignment

Which practice best ensures that AI investment decisions remain aligned with strategy over time?

  1. Freezing the strategy for five years to ensure stability
  2. Reviewing the portfolio against business objectives at regular governance checkpoints Correct answer.
  3. Approving every proposed use case to encourage experimentation
  4. Delegating all investment decisions to the data science team

Explanation

Regular governance reviews let leadership reallocate resources as business priorities and results change. Freezing strategy, delegating to one function, or approving everything removes the alignment mechanism.

Question 16AI Strategy and Business Alignment

A firm compares two AI use cases: one with high certainty and modest value, another with high uncertainty and large potential value. Which approach best reflects portfolio thinking?

  1. Fund a mix that balances expected value against risk and strategic fit Correct answer.
  2. Fund only the high-certainty case to minimize risk
  3. Fund only the high-uncertainty case to maximize upside
  4. Defer both until uncertainty is fully eliminated

Explanation

Portfolio thinking balances expected value, risk, and strategic fit across a set of bets rather than optimizing a single dimension. Waiting for full certainty is rarely feasible and forfeits advantage.

Question 17AI Strategy and Business Alignment

Which statement best captures the relationship between an AI operating model and business strategy?

  1. Strategy should be rewritten to match whatever operating model exists
  2. The operating model is independent of strategy and set by IT alone
  3. Operating models only matter after AI reaches full maturity
  4. The operating model should be derived from the strategy it must execute Correct answer.

Explanation

The operating model, covering roles, processes, and governance, exists to execute the strategy and should follow from it. Treating it as independent or as something to retrofit strategy to inverts the relationship.

Question 18AI Strategy and Business Alignment

A leader must decide whether to build an AI capability in-house or buy a vendor solution. Which consideration is most strategic rather than tactical?

  1. The programming language used by the vendor
  2. The color scheme of the vendor's user interface
  3. Whether the capability is a source of durable competitive differentiation Correct answer.
  4. The number of employees on the vendor's support desk

Explanation

Build-versus-buy is a strategic decision when it hinges on whether the capability creates durable differentiation. Capabilities that are not differentiating are often better bought, while differentiating ones may justify in-house investment.

Question 19AI Strategy and Business Alignment

Which set of measures best balances an AI scorecard across the value chain?

  1. Only the number of models registered in the model catalog
  2. Only the count of data scientists hired
  3. Only model accuracy metrics reported by the data science team
  4. Adoption, decision quality, and realized financial impact alongside model health Correct answer.

Explanation

A balanced scorecard tracks whether the capability is used, whether decisions improve, and whether financial impact is realized, alongside technical model health. Single-dimension measures miss parts of the value chain.

Question 20AI Strategy and Business Alignment

What does 'AI value realization' most precisely mean in a strategy context?

  1. Increasing the number of AI tools the company licenses
  2. Completing model training runs on schedule
  3. Converting AI capability into measurable business outcomes Correct answer.
  4. Publishing research papers about AI techniques

Explanation

Value realization refers to translating capability into measurable business outcomes such as revenue, cost, or risk reduction. Training completion, publications, and tool counts are activities rather than realized value.

Question 21AI Strategy and Business Alignment

Which is an example of an AI use case that is primarily defensive rather than offensive in strategic terms?

  1. Creating a new AI-powered revenue stream
  2. Entering a new market with an AI service
  3. Launching a new personalized product recommendation engine
  4. Automating fraud detection to reduce losses Correct answer.

Explanation

Defensive use cases protect existing value, such as reducing fraud losses or managing risk. Offensive use cases aim to grow revenue or create new offerings.

Question 22AI Strategy and Business Alignment

Why is executive-level AI literacy important for strategic alignment?

  1. It allows leaders to write production machine learning code
  2. It enables leaders to evaluate AI proposals and set realistic expectations Correct answer.
  3. It guarantees that every AI project will succeed
  4. It replaces the need for data science specialists

Explanation

Leaders with AI literacy can judge proposals, ask the right questions, and set realistic expectations about timelines and risk. It complements rather than replaces specialist roles and does not guarantee project success.

Question 23AI Strategy and Business Alignment

A business unit proposes an AI project that duplicates an existing initiative in another unit. What is the most strategically sound response?

  1. Let each unit proceed without sharing data or results
  2. Approve both to encourage internal competition
  3. Reject both projects outright
  4. Consolidate or coordinate the efforts to avoid duplicated spend and conflicting models Correct answer.

Explanation

Portfolio governance exists partly to detect duplication and coordinate related efforts so spend is efficient and outputs are consistent. Uncoordinated duplication wastes resources and can produce conflicting systems.

Question 24AI Strategy and Business Alignment

Which statement about measuring AI ROI is most accurate?

  1. ROI is irrelevant once a model is deployed
  2. ROI requires attributing business outcomes to the AI intervention, often with a baseline comparison Correct answer.
  3. ROI can be computed reliably from model accuracy alone
  4. ROI is only measurable for generative AI systems

Explanation

ROI depends on attributing outcomes to the AI intervention, typically through a baseline or control comparison. Model accuracy is an input metric and does not by itself establish ROI.

Question 25AI Strategy and Business Alignment

What is the primary purpose of an AI governance council in a strategy-aligned organization?

  1. To market AI products to external customers
  2. To write all machine learning code for the company
  3. To approve and oversee AI initiatives against policy and strategic priorities Correct answer.
  4. To replace the finance department's budgeting process

Explanation

A governance council oversees AI initiatives for policy compliance and strategic fit, making funding and risk decisions. It does not perform engineering work, replace budgeting, or handle external marketing.

Question 26AI Strategy and Business Alignment

Which artifact states the specific business outcome an AI initiative is expected to produce, the metric that will track it, and the accountable owner?

  1. A value hypothesis with a named metric and owner Correct answer.
  2. A model architecture diagram
  3. A vendor security questionnaire
  4. A data lineage register

Explanation

Strategic alignment requires that each initiative be tied to an explicit, measurable business outcome with clear accountability. An architecture diagram, lineage register, or security questionnaire supports delivery but does not define the expected business result.

Question 27AI Strategy and Business Alignment

A logistics firm's executive committee wants AI spending tied to corporate priorities. Which approach best accomplishes this?

  1. Rank proposals by their contribution to stated strategic objectives Correct answer.
  2. Fund every proposal that shows a positive model accuracy gain
  3. Approve projects in the order they are submitted
  4. Allocate budget by department headcount rather than by initiative

Explanation

Strategic alignment means investment choices are evaluated against the organization's declared priorities, not against technical merit alone. Ranking by contribution to objectives keeps funding decisions connected to strategy.

Question 28AI Strategy and Business Alignment

What is the core purpose of an AI strategy document for an enterprise?

  1. To state how AI investments will advance the organization's goals Correct answer.
  2. To specify the technical stack the engineering team must purchase
  3. To record the minutes of model review meetings
  4. To catalog every algorithm the data science team can implement

Explanation

An AI strategy connects AI activity to enterprise objectives and guides where to invest. It is a directional and prioritization instrument rather than a technical inventory or a meeting record.

Question 29AI Strategy and Business Alignment

Which signal most clearly indicates that an AI initiative has genuine executive sponsorship rather than passive approval?

  1. The sponsor attends the kickoff presentation
  2. The initiative appears in the annual report narrative
  3. The initiative has a dedicated project logo and name
  4. The sponsor reallocates budget and staff when milestones slip Correct answer.

Explanation

Real sponsorship is demonstrated by the willingness to move scarce resources when the initiative needs them. Symbolic gestures such as naming, narrative mentions, or kickoff attendance do not show that commitment.

Question 30AI Strategy and Business Alignment

A bank's AI program reports high model deployment counts but flat customer retention. Which interpretation is most defensible?

  1. Retention is an inappropriate metric for an AI program
  2. The program lacks a causal link between deployed models and the retention outcome Correct answer.
  3. Deployment count should replace retention as the headline measure
  4. The models must be retrained on more data

Explanation

Output volume such as deployment counts measures activity, not business effect. When the targeted outcome does not move, the strategy team should question whether the deployed work actually influences that outcome.

Question 31AI Strategy and Business Alignment

When a strategy team defines AI use-case priorities, which criterion most directly reflects strategic fit?

  1. The novelty of the modeling technique
  2. The size of the available training dataset
  3. The number of engineers available to build it
  4. The degree of alignment with a stated corporate objective Correct answer.

Explanation

Strategic fit is judged by how directly a use case advances a declared organizational objective. Data size, technique novelty, and staffing are feasibility factors rather than measures of strategic relevance.

Question 32AI Strategy and Business Alignment

An AI team proposes a project whose business case rests entirely on cost reduction in a unit scheduled for divestiture. What should the strategy function do?

  1. Escalate it for review against the divestiture plan before funding Correct answer.
  2. Fund it but remove the cost-reduction target
  3. Approve it because cost reduction is always valuable
  4. Reject it because cost reduction is never strategic

Explanation

Alignment requires checking a proposal against the organization's portfolio direction, including planned divestitures. Escalation allows the conflict between local savings and corporate direction to be resolved deliberately.

Question 33AI Strategy and Business Alignment

Which practice best converts an AI capability into a testable business hypothesis?

  1. Listing the capability's technical specifications
  2. Documenting the capability's software license terms
  3. Stating the expected change, the metric, and the timeframe Correct answer.
  4. Benchmarking the capability against competitor tooling

Explanation

A testable hypothesis names an expected effect, how it will be measured, and over what period, which makes it falsifiable. Specifications, benchmarks, and license terms describe the capability rather than predict its business effect.

Question 34AI Strategy and Business Alignment

A leadership team wants AI to improve both customer experience and internal efficiency. Which portfolio design best serves that dual objective?

  1. One portfolio ranked solely by short-term cost savings
  2. Separate portfolios each with objectives, owners, and metrics Correct answer.
  3. A portfolio that excludes projects touching customer data
  4. A single large program scored on one blended metric

Explanation

Different strategic objectives require distinct success measures and accountability, which a blended metric obscures. Separate portfolios let each objective be tracked and governed on its own terms.

Question 35AI Strategy and Business Alignment

What is the most reliable way for a strategy team to keep an AI portfolio aligned as business priorities shift?

  1. Extend every initiative's timeline until priorities stabilize
  2. Freeze the portfolio at annual planning and avoid changes
  3. Review the portfolio against refreshed priorities at fixed intervals Correct answer.
  4. Delegate all portfolio decisions to the data engineering team

Explanation

Regular, scheduled re-review lets the portfolio be re-scored against current priorities rather than stale assumptions. Freezing, delegating, or deferring removes the mechanism that keeps investment aligned.

Question 36AI Strategy and Business Alignment

Which pair of measures best shows whether an AI program is delivering strategic value rather than merely operating?

  1. Number of models registered and number of dashboards created
  2. Server uptime and average model training duration
  3. Adoption by target users and movement in the targeted business metric Correct answer.
  4. Count of experiments run and count of notebooks committed

Explanation

Strategic value is evidenced by whether intended users adopt the capability and whether the business outcome it targets moves. Operational and activity counts describe effort and system health, not delivered value.

Question 37AI Strategy and Business Alignment

A firm's AI unit reports to the CIO while its growth strategy depends on AI-enabled products sold by business lines. What structural misalignment does this create?

  1. Business lines cannot sell products that use data
  2. Product accountability sits apart from the AI capability that enables it Correct answer.
  3. The AI unit lacks the authority to set enterprise strategy
  4. The CIO cannot approve any technology spending

Explanation

When AI capability is housed in a technology function while product outcomes are owned by business lines, accountability for AI-enabled revenue is split. Aligning ownership with the strategic objective reduces that gap.

Question 38AI Strategy and Business Alignment

Two business units propose nearly identical AI capabilities with different vendors. Which action best serves enterprise strategy?

  1. Let both proceed independently to preserve unit autonomy
  2. Cancel both and defer AI investment for a year
  3. Consolidate into one initiative with shared ownership and a single business case Correct answer.
  4. Award the work to the unit that submitted first

Explanation

Duplicated capability fragments investment and dilutes accountability, so consolidation under shared ownership captures scale and produces one coherent business case. Independent pursuit, cancellation, or submission-order awards all forfeit that benefit.

Question 39AI Strategy and Business Alignment

A regulated insurer must choose between an AI pricing model with large projected margin gains and an AI claims-triage model with moderate gains and no regulatory exposure. Which decision framework is most strategically sound?

  1. Compare both on risk-adjusted strategic value, including regulatory and capability effects Correct answer.
  2. Choose claims triage because it carries no regulatory exposure
  3. Defer both until competitors demonstrate results
  4. Choose the pricing model because projected margin is highest

Explanation

Sound prioritization weighs expected benefit against risk and strategic side effects such as regulatory standing and capability building. Selecting on raw margin or on risk avoidance alone ignores half of the trade-off.

Question 40AI Strategy and Business Alignment

Which statement best describes how an AI capability roadmap should relate to the enterprise strategy?

  1. The roadmap should be derived from strategic objectives and sequenced by dependency Correct answer.
  2. The roadmap should list every available AI tool in the market
  3. The roadmap should remain independent of business planning cycles
  4. The roadmap should be set by vendors' published release schedules

Explanation

A capability roadmap translates strategic objectives into a sequenced plan, respecting technical and organizational dependencies. Tool catalogs, vendor schedules, and detachment from planning cycles disconnect the roadmap from strategy.

Question 41AI Governance, Risk, and Responsible AI

Which statement best describes the purpose of a model card?

  1. It defines the service-level agreement for model uptime.
  2. It stores the encrypted weights used to deploy the model into production.
  3. It logs every inference request for security auditing purposes.
  4. It documents a model's intended use, training data, and known limitations for stakeholders. Correct answer.

Explanation

A model card is a transparency artifact that summarizes a model's intended use, data provenance, evaluation results, and limitations so downstream users can judge fitness. It is documentation, not an operational log or deployment artifact.

Question 42AI Governance, Risk, and Responsible AI

A team wants a lightweight record of who approved a model for release, when, and on what evidence. Which governance artifact fits best?

  1. A prompt template library
  2. A data ingestion pipeline diagram
  3. A hyperparameter tuning report
  4. A model risk register entry with approval sign-off and linked evidence Correct answer.

Explanation

A risk register entry captures the decision, approver, date, and supporting evidence, providing an auditable approval trail. The other artifacts describe technical implementation rather than governance decisions.

Question 43AI Governance, Risk, and Responsible AI

What is the primary role of a human-in-the-loop control in a high-stakes AI decision process?

  1. To allow a qualified person to review and override consequential AI outputs Correct answer.
  2. To increase model throughput by parallelizing inference
  3. To replace the need for monitoring after deployment
  4. To reduce the size of the training dataset

Explanation

Human-in-the-loop controls place a qualified reviewer in the decision path so consequential outputs can be checked and overridden. They supplement, rather than replace, monitoring and other safeguards.

Question 44AI Governance, Risk, and Responsible AI

Which practice best supports fairness assessment before a model is deployed?

  1. Measuring outcome disparities across defined groups using disaggregated evaluation data Correct answer.
  2. Removing all demographic fields from the training data
  3. Reporting only the aggregate accuracy on the full test set
  4. Increasing the model size until accuracy stops improving

Explanation

Fairness assessment requires disaggregated metrics so disparities between groups become visible and can be judged against chosen criteria. Aggregate-only reporting hides subgroup differences.

Question 45AI Governance, Risk, and Responsible AI

What does data minimization require in an AI system handling personal information?

  1. Collecting only the data reasonably necessary for the stated purpose Correct answer.
  2. Storing data in the smallest possible file format
  3. Retaining all data indefinitely in case it becomes useful
  4. Restricting data access to a single engineer

Explanation

Data minimization limits collection and use to what is necessary for a defined purpose. It is about scope of collection and retention, not storage format or headcount.

Question 46AI Governance, Risk, and Responsible AI

A governance policy states that AI systems must have a named accountable owner. What problem does this requirement primarily address?

  1. Insufficient model training data
  2. Diffusion of responsibility across teams Correct answer.
  3. Overly large prompt contexts
  4. Slow inference latency

Explanation

Naming an accountable owner prevents responsibility from being spread across multiple teams with no clear decision-maker. The other options are technical performance concerns unrelated to accountability.

Question 47AI Governance, Risk, and Responsible AI

Which of the following is an example of a technical guardrail applied to a generative AI assistant?

  1. Adding the assistant to the corporate software catalog
  2. Publishing a blog post about the company's AI values
  3. Requiring employees to attend an annual ethics briefing
  4. Filtering outputs that match prohibited content patterns before they reach users Correct answer.

Explanation

Technical guardrails are automated controls embedded in the system, such as output filters that block prohibited content. Training, communications, and catalog listings are organizational measures, not runtime controls.

Question 48AI Governance, Risk, and Responsible AI

What is the main purpose of conducting a red-team exercise on an AI system?

  1. To benchmark raw inference speed against competitors
  2. To probe for harmful, unsafe, or unintended behaviors before adversaries exploit them Correct answer.
  3. To generate marketing material showcasing model capabilities
  4. To reduce cloud hosting costs

Explanation

Red teaming deliberately attempts to elicit failures so they can be remediated before real-world harm. It is an adversarial safety exercise, not a performance or cost benchmark.

Question 49AI Governance, Risk, and Responsible AI

A bank deploys a credit-scoring model. Regulators require that applicants receive a meaningful explanation for adverse decisions. Which capability is most directly required?

  1. A larger training corpus of historical loan data
  2. A quarterly report on aggregate portfolio performance
  3. Post-hoc explanation of the factors that drove the individual decision Correct answer.
  4. A faster scoring endpoint to reduce wait times

Explanation

Individual adverse-action explanations require the ability to attribute a specific decision to its influential factors, often through post-hoc explanation methods. Aggregate portfolio reporting does not satisfy an individual's right to understand their own decision.

Question 50AI Governance, Risk, and Responsible AI

A model performs well on its test set but degrades sharply after deployment on new customer segments. Which governance response is most appropriate?

  1. Remove the new segments from the customer base
  2. Establish production monitoring with drift detection and a defined retraining trigger Correct answer.
  3. Disable logging to reduce noise in the metrics
  4. Immediately retrain on the test set to raise its score

Explanation

Distribution shift between test and production data is detected through ongoing monitoring for drift, with pre-agreed thresholds that trigger review or retraining. Retraining on the test set corrupts evaluation and does not address the shift.