GitLab CI/CD Specialist: Pipeline efficiency: caching, artifacts, and parallelism

The Pipeline efficiency domain tests whether you can make pipelines faster without making them unreliable: cache versus artifacts, artifact expiry and reports, parallel jobs, matrix expansion, and DAG scheduling with needs.

Cache and artifacts

Cache is a best-effort speed feature stored on the runner or a distributed cache backend; it is keyed by cache:key, optionally from lockfiles via key:files, and can use fallback_keys and policy: pull for read-only consumers. Cache is not a delivery mechanism and is not guaranteed across runners. Artifacts are uploaded to GitLab, expire via expire_in, and are passed to later jobs. needs downloads artifacts only from the named dependencies; dependencies cannot be combined with needs in the same job. Test reports such as JUnit and coverage should use artifacts:reports with when: always so failures still publish results.

Parallelism

parallel: N creates N copies of a job and exposes CI_NODE_INDEX and CI_NODE_TOTAL for manual sharding. parallel:matrix expands named variables into one job per combination, which is better for cross-version or cross-cloud testing. interruptible: true lets auto-cancel redundant pipelines stop superseded jobs. resource_group deliberately serializes jobs that must not overlap, such as production deploys.

Worked example

test:
  parallel:
    matrix:
      - PYTHON: ["3.11", "3.12"]
  image: python:$PYTHON
  script: pytest --junitxml=report.xml
  artifacts:
    when: always
    paths: [report.xml]
    reports:
      junit: report.xml

This creates two jobs, one per Python version. Each runs the full test command; the matrix does not shard tests automatically. when: always preserves the JUnit report even when pytest fails, and the report is uploaded as an artifact rather than cached.

Traps

Using cache for artifacts that downstream jobs must consume is a common failure. A cache key that ignores the lockfile serves stale dependencies. Assuming parallel jobs split work automatically is wrong; you must shard. needs does not download artifacts from jobs it does not name. Expired artifacts break downstream retries even if the cache still exists.