Private Build Runner for On-Premises Integration Tests

Decision: private build runners inside sn-build; do not try to route the managed runner. The managed build runner is Oracle-hosted and cannot be attached to your VCN, so the only shortcut would be exposing Nexus and the database through an internet gateway - already rejected by security.

Topology:

• Compute instances in sn-build with no public IP, built from a hardened image that already has the DevOps build-runner agent installed. Register them against the project's build-runner resource and select that runner in each build stage.
• Route table: 10.40.0.0/16 -> DRG for Nexus and the database, all-fra-services-in-oracle-services-network -> Service Gateway for the DevOps endpoint, OCIR, Object Storage, Vault and Logging. No 0.0.0.0/0 rule. If Maven Central must be reached directly instead of through Nexus, add NAT - otherwise omit it.
• NSG egress only: TCP 8081 to 10.40.12.9, TCP 1521 to 10.40.20.15, TCP 443 to the service gateway. No ingress rules at all; the runner polls outbound for work, so nothing needs to connect in.
• Private DNS forwarding so fra.ocir.io and the DevOps endpoint resolve from the private subnet.

IAM before the first build: a dynamic group matching the runner instances (compartment plus freeform tag role=build-runner), with a policy granting use devops-family in the project compartment and read repos in the artifact compartment. Deliberately keep the runner's rights to build-and-test only: publishing is performed by a Deliver Artifact stage, which uploads the declared output artifact to OCIR under the pipeline's own principal, so no registry auth token ever has to exist on the runner.

The symptom is queueing capacity, not network. A private build runner executes one job at a time. With 25 concurrent builds at the Monday peak and only two or three runners, most builds spend their life waiting, and the build stage's clock can expire while the job is still queued or just starting. Prove it before changing anything by comparing queue wait timestamps with actual job execution time in the build-run log; a healthy 11-minute job behind a 20-minute queue looks exactly like a timeout.

Fixes in order of value:

1. Shorten the job. Cache the Maven local repository outside the runner (object storage cache or Nexus proxy) so dependencies are not re-downloaded every run, and split unit and integration into parallel stages so wall clock drops rather than just stage count.
2. Size to the peak, not the average. 900 builds a month is about 30 a day, but the binding constraint is 25 concurrent. Use an instance pool of 2–12 runners with a scheduled scale-out on Sunday night. Autoscaling on CPU is a poor proxy for queue depth, so either schedule the scale-out or drive it from a custom metric.
3. Set timeouts deliberately. Raise the stage timeout only after queueing is addressed; otherwise one hung build occupies a runner for the entire window and amplifies the queue.
4. Recycle runners between jobs and rebuild the image monthly so a poisoned workspace cannot leak state from one build to the next.

Vault secret resolved by the DevOps service, not by the runner. Store the credential in OCI Vault in the vault compartment, declare a build pipeline parameter of type Vault secret, and reference it in build_spec.yaml as an environment variable (for example ${NEXUS_CREDS}). The value is injected when the stage starts and is masked in build logs.

IAM: the DevOps pipeline's principal needs use vaults plus read secret-bundles on the matching secret. Deliberately do not grant the runner dynamic group any Vault policy - the runner should be unable to read secrets on its own, which limits what an attacker with code execution on the runner can exfiltrate.

Rotation: reference the secret by OCID and rotate by creating a new secret version. The next build picks up the new value with no pipeline edit, which is exactly the failure mode a console-typed environment variable has.

Anti-patterns to reject explicitly: base64-encoded credentials in build_spec.yaml, an environment variable typed into the build stage in the console, or credentials committed to the OCI DevOps code repository. All three are visible to every project user and tend to surface in build logs. Also verify the secret is not echoed by the Maven settings.xml generation step - a set -x in that step is a common leak.