CS AO cheat sheet
CS AO · Keep the key ideas within reach.
System Administration Fundamentals
Principle of least privilege: every account, service, and process runs with the minimum rights needed for its function; root/admin use is exception-based and logged.
Idempotence: a configuration action is safe to repeat and converges to the same state; non-idempotent steps (appending to a file, creating a user) break re-runs.
Boot sequence (Linux): firmware POST → bootloader (GRUB) → kernel + initramfs → init system (systemd) → targets/units; failures localize by stage.
Runlevels vs systemd targets: 0/6 = poweroff/reboot, 1 = rescue/single-user, 3 = multi-user text, 5 = graphical; map to poweroff.target, rescue.target, multi-user.target, graphical.target.
Process states: R running, S interruptible sleep, D uninterruptible (often I/O wait), Z zombie (exited, parent has not reaped), T stopped.
Load average counts runnable plus uninterruptible tasks over 1/5/15 minutes; compare to CPU core count, not to a fixed threshold.
Filesystem hierarchy: /etc config, /var variable data and logs, /usr read-only user binaries and libraries, /tmp world-writable scratch, /proc and /sys kernel interfaces.
Exit codes: 0 success, 1-125 command-defined failure, 126 not executable, 127 command not found, 128+N killed by signal N (137 = SIGKILL).
Change management: record change, test in non-production, define rollback, schedule window, verify after apply; unlogged changes are the top cause of undocumented drift.
Installation, Configuration, and Deployment
Unattended Linux installs use Kickstart (RHEL family) or preseed/autoinstall (Debian/Ubuntu); Windows uses answer files (unattend.xml) with Windows Setup.
Partitioning decisions: separate /var and /home to contain growth, /boot outside LVM for bootloader compatibility, swap sized to hibernation need rather than an old 2x-RAM rule.
Package managers: apt/dpkg (Debian), dnf/rpm (RHEL family), zypper/rpm (SUSE); verify signatures and pin versions for reproducible builds.
Configuration management tools (Ansible, Puppet, Chef) declare desired state; Ansible is agentless over SSH, Puppet/Chef use an installed agent and periodic convergence.
Golden image vs configuration management: images give fast, identical baseline deploys; config management handles post-boot drift and per-host variation.
Blue-green deployment keeps two identical environments and switches traffic; canary releases route a small traffic percentage first and expand on health signals.
Rollback requires versioned artifacts plus a tested revert path; database migrations usually need a backward-compatible schema step before code rollback.
Secrets belong in a vault or injected environment, never baked into images or committed to version control; rotate on exposure and audit access.
User and Access Management
Linux account files: /etc/passwd (UID, GID, shell), /etc/shadow (hashed password, aging), /etc/group (membership); UID 0 is root regardless of name.
Password aging fields in /etc/shadow: last change, min days, max days, warn days, inactive days, expiry date, reserved.
PAM controls Linux authentication stack (pam_unix, pam_tally2/faillock for lockout, pam_pwquality for complexity); sshd and login both call into it.
sudo grants command-level elevation via /etc/sudoers (edit with visudo); NOPASSWD and wildcard rules are common privilege-escalation paths.
Windows AD: users and computers are objects; Group Policy applies settings by OU; Kerberos is the default auth protocol, with NTLM as legacy fallback.
Access models: RBAC assigns permissions to roles then users to roles; ABAC evaluates attributes; MAC (SELinux, AppArmor) enforces policy above discretionary permissions.
MFA combines factors (know, have, are); TOTP codes are time-based and phishable, FIDO2/WebAuthn hardware keys are phishing-resistant.
Joiner-mover-leaver process: provision on hire, re-scope on role change, disable and revoke sessions/keys on exit; orphaned accounts are a top audit finding.
Storage and File System Administration
Linux disk stack: block device → partition table (GPT/MBR) → LVM physical volume → volume group → logical volume → filesystem → mount point.
LVM commands: pvcreate, vgcreate, lvcreate, lvextend then resize filesystem (xfs_growfs for XFS, resize2fs for ext4); XFS cannot shrink.
Filesystem choices: ext4 general purpose and shrinkable offline; XFS high-throughput, online grow only; ZFS/Btrfs add checksums, snapshots, and send/receive replication.
/etc/fstab fields: device, mount point, type, options, dump, fsck pass; use UUID= or LABEL= to survive device renaming; a bad entry can block boot.
RAID levels: 0 stripe (no redundancy), 1 mirror, 5 stripe plus one parity (survives one disk), 6 two parity, 10 mirror plus stripe; RAID is not backup.
NFS exports in /etc/exports with options like rw, sync, root_squash, no_subtree_check; client mounts type nfs4 and verifies with showmount -e.
SMB/CIFS shares mount with credentials in a protected file (credentials=), and mount options include vers=, uid=, gid=, and ro.
Capacity monitoring must track inodes (df -i) as well as blocks (df -h); a filesystem can be full of small files while showing free space.
Quotas limit per-user or per-group blocks and inodes; enable in mount options, initialize with quotacheck, then activate with quotaon.
Networking and Connectivity
TCP/IP layers: link, internet (IP, ICMP), transport (TCP, UDP), application; TCP is connection-oriented with handshake and retransmission, UDP is not.
Private ranges: 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16; 169.254.0.0/16 is link-local (APIPA) and signals a failed DHCP lease.
CIDR math: /24 = 256 addresses, /25 = 128, /26 = 64, /30 = 4 (2 usable); usable hosts = 2^(32-prefix) - 2.
Ports to know: 22 SSH, 25 SMTP, 53 DNS, 80 HTTP, 123 NTP, 389/636 LDAP/LDAPS, 443 HTTPS, 445 SMB, 3306 MySQL, 3389 RDP, 5432 PostgreSQL.
DNS resolution order and record types: A/AAAA (address), CNAME (alias), MX (mail), TXT (SPF/DKIM/DMARC), PTR (reverse); TTL controls cache lifetime.
DHCP DORA: Discover, Offer, Request, Acknowledge; reservations bind a MAC to a fixed address, scopes define range, lease time, gateway, and DNS.
Diagnostic order: link (ip link), address (ip addr), route (ip route), name resolution (dig/getent), then port reachability (ss -lntp, nc -zv).
VLANs segment at layer 2; routing between them requires a layer 3 device or router-on-a-stick subinterface with 802.1Q tagging.
Security, Hardening, and Compliance
CIA triad: confidentiality, integrity, availability; controls are preventive, detective, or corrective, and technical, administrative, or physical.
SSH hardening: disable PermitRootLogin and password auth, use key pairs, restrict AllowUsers/AllowGroups, change or firewall port 22, enable fail2ban-style lockout.
Host firewall defaults: deny inbound, allow established/related, permit only required services; nftables/firewalld on Linux, Windows Defender Firewall with profiles.
Patch cadence by severity: critical internet-facing flaws on an emergency timeline, routine patches on a fixed cycle; track CVSS score and exploit availability.
SELinux modes: enforcing, permissive, disabled; troubleshoot denials with ausearch and audit2why, fix labels with restorecon rather than disabling.
Encryption: TLS for data in transit, full-disk or volume encryption for data at rest; key management and rotation matter as much as the cipher.
Compliance frameworks: PCI DSS (card data), HIPAA (health), GDPR (EU personal data), ISO 27001 (ISMS), SOC 2 (trust services); each maps to controls and evidence.
Audit logging: record authentication, privilege escalation, and configuration changes; ship logs off-host with NTP-synced timestamps to preserve evidentiary value.
Least-function services: remove unused packages, disable unneeded daemons, and bind administrative interfaces to management networks.
Monitoring, Backup, and Recovery
Backup types: full (everything), incremental (changes since last backup of any type), differential (changes since last full); restore chain length differs accordingly.
3-2-1 rule: three copies, two media types, one offsite; add immutability or offline copies to resist ransomware.
RPO is the maximum acceptable data loss (drives backup frequency); RTO is the maximum acceptable downtime (drives recovery method and staffing).
Snapshots are not backups: they share the source volume and are lost with it; they are fast rollback points, not offsite copies.
Monitoring signals: availability checks, resource saturation (CPU, memory, disk, network), error rates, and latency; alert on symptoms users feel.
SNMP polls OIDs for metrics and sends traps for events; syslog centralizes logs by severity (0 emerg to 7 debug) with facility codes.
Verification: a backup is unproven until a test restore succeeds; schedule restores, record restore time, and compare against RTO.
Test-restore checklist: retrieve media, restore to isolated hardware, validate data integrity and application startup, document elapsed time and gaps.
Troubleshooting and Operational Procedures
Structured method: identify symptom, gather data, form hypothesis, test the cheapest likely cause, apply fix, verify, document; change one variable at a time.
Layer isolation: start at the layer where the failure is reported (physical, network, OS, service, application) and confirm each layer below before moving up.
Log locations: /var/log/syslog or messages, /var/log/auth.log or secure, journalctl -u -b for systemd; Windows Event Viewer System/Application/Security logs.
Resource triage commands: top/htop, free -m, df -h, df -i, iostat, vmstat, ss -lntp, dmesg for kernel and hardware errors.
Service failure triage: systemctl status, then journalctl -xe; check config syntax (nginx -t, sshd -t, named-checkconf) before restart.
Connectivity triage: ping loopback, ping gateway, ping external IP, resolve name, then test port; each success narrows the remaining scope.
Escalation and communication: define severity levels, notify stakeholders on impact and ETA, keep a timeline in the incident record, and hold a blameless review.
Documentation standard: record hostname, change, command output, time, and operator; undocumented fixes recur and defeat root-cause analysis.
Operational Procedures and Documentation
Runbooks list exact commands and expected output for routine tasks; a runbook that fails in a drill is a defect to fix, not a reader error.
Maintenance windows define start, end, rollback deadline, and communication channel; changes that overrun the deadline are reverted.
Ticketing fields that matter for audit: requester, change description, approval, implementation time, verification evidence, rollback result.
Asset inventory records hostname, owner, OS version, IP, role, and patch state; unknown assets are unmanaged risk.
Capacity planning uses trend data (growth per month) against thresholds, not a single snapshot reading.
Configuration baselines and drift reports compare live hosts to a defined standard; drift is remediated by automation to stay consistent.
Virtualization, Cloud, and Automation Basics
Hypervisor types: type 1 bare-metal (ESXi, Hyper-V, KVM) and type 2 hosted (VirtualBox, Workstation); type 1 is standard for production.
Virtual machine vs container: VMs carry a full guest OS and hypervisor boundary; containers share the host kernel and isolate via namespaces and cgroups.
Cloud models: IaaS (raw compute/storage), PaaS (managed runtime), SaaS (finished application); shared responsibility splits provider and customer duties.
Infrastructure as code stores environment definitions in version control; plan/apply or dry-run previews changes before they touch production.
Automation scripts need idempotence, logging, error handling, and a rollback path; untested scripts run against production are an outage waiting to happen.
Secrets in automation come from a vault or short-lived tokens; embedded credentials in scripts or CI variables leak through logs and repositories.