Google IT Automation with Python: Managing files, processes, and system resources with Python

Managing files, processes, and system resources with Python

This domain tests practical system administration in Python: locating and manipulating files, reading structured data, launching external commands, and checking disk, memory, and process state. Questions often ask which module or flag is appropriate, or what a short script will do under a given filesystem or process condition.

Core rules and concepts

• Use pathlib.Path for path joins, existence checks, globbing, and reading or writing. Path('logs').glob('*.log') matches one level; rglob('*.log') recurses.
• Manage file handles with with open(path, 'r', encoding='utf-8') as f:. Text mode returns str; binary mode returns bytes. JSON and CSV need the matching json and csv modules rather than manual splitting.
• shutil.copy2() preserves metadata; shutil.move() moves or renames. os.remove() deletes a file, not a directory; shutil.rmtree() deletes a tree and must be used cautiously.
• Run commands with subprocess.run([...], capture_output=True, text=True, check=True, timeout=30). Pass an argument list and leave shell=False unless shell features are genuinely required.
• Check return codes. A process that exits nonzero has failed even if it printed useful output. check=True raises CalledProcessError; otherwise inspect result.returncode.
• Resource checks include shutil.disk_usage(path), os.statvfs(path), and process inspection with psutil or os facilities. These return snapshots, not guarantees.

Worked example

Find log files larger than 10 MB, gzip them, and remove the originals only after a successful compression.

from pathlib import Path
import gzip, shutil

for path in Path('/var/log/app').rglob('*.log'):
    if path.stat().st_size <= 10 * 1024 * 1024:
        continue
    target = path.with_suffix(path.suffix + '.gz')
    with path.open('rb') as src, gzip.open(target, 'wb') as dst:
        shutil.copyfileobj(src, dst)
    path.unlink()

Reasoning: rglob finds nested logs; the size test uses bytes; streaming avoids loading a large file into memory; unlink() runs only after the compressed file is closed. A safer production version would compare sizes or use a temporary file and atomic rename.

Traps that cost marks

os.system() gives poor error handling and invites shell injection. shell=True with an f-string built from user input is dangerous. Relative paths depend on the current working directory, which may differ under cron or a service manager. Forgetting encoding causes platform-dependent text handling. Treating a directory as a file, or deleting before verifying a copy, causes data loss.