Config Drift Between Stage and Prod After a Manual UI Change
A team uses the standard Acquia workflow: code moves Dev → Stage → Prod via git tags and Acquia Cloud pipelines, and configuration is exported/imported with Drupal's config management. During an incident, an engineer enabled a new module and changed a view's pager settings directly in the Prod UI to restore service quickly. Prod is now healthy, but Stage and Dev do not have the module enabled or the view change. Two weeks later, a routine deployment exports config from Dev and imports to Prod. The import fails with a dependency error because Prod has a module enabled that Dev's export does not reference, and the view config in the export would overwrite the incident fix. The team wants to keep the incident fix and get back to a clean pipeline.
1. Explain why the import failed and why simply forcing the import is the wrong move.
Read the worked answer
The failure is expected given the workflow. Config export/import is a full-state sync of the configuration entities in the export directory. Prod's active config now contains a module enablement and a view change that were never exported back into code. When Dev's export is imported to Prod, Drupal validates dependencies against Prod's active state: the export does not declare the module that Prod has enabled, and the view config in the export is a different revision than the incident fix. The dependency check fails, which is Drupal protecting you from an inconsistent state.
Forcing the import (e.g., ignoring validation or importing with --force-style overrides) would silently revert the incident fix and could leave the module enabled without matching config, producing a broken or partially configured site. The correct framing is that Prod's UI edits are uncommitted configuration that must be reconciled into the codebase, not overwritten.
2. Lay out the steps to reconcile Prod, Stage, and Dev and restore a single source of truth.
Read the worked answer
- Freeze changes and capture Prod's current config: export the active config from Prod (or at minimum the module's config and the view) so the incident fix is not lost.
- Diff Prod's export against the Dev export to enumerate every drift item - the module enablement, the view pager change, and anything else changed during the incident.
- Decide intent per item: keep the incident fix (module + view) as the desired state, and encode it in code. Add the module to the codebase's dependency set (composer/composer.json and the config export), and commit the corrected view config.
- Propagate through the pipeline in order: merge into Dev, run a config import on Dev, verify; deploy to Stage, import, verify; then deploy to Prod.
- Before the Prod import, confirm Prod's active config matches the export you are about to import (or that the only differences are the ones you intentionally reconciled), so the dependency check passes.
- Add a guardrail: a CI check that runs a config-import dry run against a copy of Prod's config to catch drift before deployment, and a rule that incident UI changes must be exported and committed within the same incident window.
3. What monitoring or process control would prevent this class of incident from recurring?
Read the worked answer
Introduce a configuration drift detector: a scheduled job that exports active config from each environment and diffs it against the code branch, alerting on any non-empty diff. Pair it with a deployment gate that refuses to promote to Prod if the dry-run import reports dependency errors or would change config that isn't in the release. Process-wise, document an incident-change protocol: any UI change made under pressure must be immediately exported, committed, and noted in the incident ticket, with a follow-up to run it through the normal pipeline. This converts the 'quick fix' from an out-of-band change into a tracked one.