Ruankao Multimedia Application Designer (Intermediate) cheat sheet

Von Neumann bottleneck: CPU and memory share one bus, so multimedia pipelines add DMA, dedicated codecs, and GPU/ISP blocks to move frame data without CPU copy per pixel.

Multimedia system layers: capture device → driver/API (DirectShow, V4L2, AVFoundation) → codec/processing graph → renderer/synchronizer → application. Synchronization sits above decoding.

Throughput rule: raw video rate = width × height × bytes-per-pixel × frame rate. 1920×1080, 3 B/px, 25 fps ≈ 155 MB/s; this exceeds many storage and bus budgets, motivating compression.

Latency budget rule for interactive A/V: end-to-end glass-to-glass delay under ~150 ms keeps talk and control natural; buffering trades latency for jitter tolerance.

Amdahl's law: speedup = 1 / ((1−p) + p/s), where p is the parallelizable fraction. Encoding gains from multicore are capped by serial stages such as entropy coding.

Real-time constraint: a stream is schedulable only if per-period processing time plus jitter margin ≤ period (e.g., 40 ms per frame at 25 fps). Missed deadlines cause dropped frames, not slowdowns.

Endianness and sample layout matter: PCM WAV is little-endian interleaved; planar YUV (I420) stores Y, then U, then V planes; packed formats interleave components per pixel.

Multimedia OS support: priority inversion and unbounded blocking in drivers cause audio glitches; real-time or elevated-priority threads plus lock-free ring buffers are standard remedies.

Sampling theorem: sample rate must exceed twice the highest frequency (Nyquist). 44.1 kHz audio captures up to 22.05 kHz; anti-alias filtering is mandatory before decimation.

Quantization error SNR for uniform b-bit PCM: SNR(dB) ≈ 6.02b + 1.76. Each added bit buys about 6 dB of dynamic range.

Color models: RGB is additive display space; CMYK is subtractive print; HSV/HSL separate hue from intensity for editing; YCbCr separates luma from chroma to exploit eye sensitivity.

Chroma subsampling notation J:a:b: 4:4:4 keeps full chroma; 4:2:2 halves horizontal chroma; 4:2:0 quarters chroma. 4:2:0 video needs 1.5 bytes per pixel at 8-bit luma.

Lossless vs lossy: RLE, Huffman, LZW, and arithmetic coding are entropy coders (lossless); quantization and DCT coefficient truncation are the lossy steps in transform codecs.

Bits per pixel (bpp) formula: bpp = total bits / (width × height). A 640×480 image stored in 300 KB (2,400,000 bits) is ≈ 7.8 bpp.

Run-length coding suits data with long repeats (palette images, fax); LZW builds a dictionary on the fly and is used in GIF and TIFF; Huffman assigns shortest codes to most frequent symbols.

Color depth and palette: indexed color stores k-bit indices into a palette of at most 2^k colors; 8-bit indexed images cannot represent smooth gradients without dithering.

JPEG baseline pipeline: 8×8 block split → level shift −128 → forward DCT → quantization by scaled table → zig-zag scan → DC differential + AC run-length → Huffman. Decoding reverses these steps.

JPEG quality factor scales the quantization table; lower quality means larger quantization steps, more zeroed coefficients, smaller files, and visible blocking/ringing artifacts.

JPEG is intra-frame only and suited to still images; Motion JPEG applies it per frame and lacks temporal prediction, so it is inefficient for video.

MPEG video structure: I-frames are intra-coded, P-frames predict from past references, B-frames bidirectionally predict from past and future references and require frame reordering.

GOP trade-off: longer groups of pictures improve compression but increase seek latency and error propagation; a closed GOP starts each group with an I-frame.

H.264/AVC adds variable block-size motion compensation, multiple reference frames, in-loop deblocking, and CABAC/CAVLC entropy coding; H.265/HEVC adds larger coding tree units and better prediction.

MPEG audio: Layer III (MP3) uses a polyphase/MDCT filterbank, psychoacoustic masking, and Huffman coding of quantized subband samples; the masking threshold determines allowed quantization noise.

AAC improves on MP3 with a pure MDCT filterbank, more flexible window switching, and tools such as SBR (HE-AAC) and parametric stereo; it is the usual audio partner of MP4 containers.

Storage capacity math: capacity = sectors × bytes per sector; sustained transfer rate, not burst rate, determines whether real-time capture succeeds.

RAID levels: RAID 0 stripes for speed with no redundancy; RAID 1 mirrors; RAID 5 stripes with distributed parity tolerating one disk failure; RAID 6 tolerates two.

Optical media: CD-ROM ≈ 650–700 MB, DVD-5 ≈ 4.7 GB single layer, Blu-ray ≈ 25 GB single layer; read speed multipliers are relative to base rates (CD 150 KB/s, DVD ~1.32 MB/s).

Flash storage has no seek time but suffers write amplification and wear; wear leveling and TRIM extend life. Sustained write speed can collapse after SLC cache is exhausted.

Capture interfaces: USB (isochronous transfers suit real-time audio/video), Thunderbolt/PCIe for high bandwidth, SDI/HDMI for uncompressed professional video.

Display metrics: pixel density = diagonal pixels / diagonal inches; refresh rate in Hz sets the maximum displayed frame rate; response time and persistence affect motion blur.

Audio hardware: ADC converts analog to digital on capture; DAC converts digital to analog on playback; sample clock jitter causes audible distortion, so word clock sync matters in studios.

Colorimeters and calibrators set monitor gamma and white point; without calibration, colors approved on one display can shift on another, breaking color-managed workflows.

Authoring paradigms: page/card-based (sequential screens), timeline-based (time-ordered tracks), icon/flow-based (event-driven flowchart), and scripted (code-driven). Choose by interactivity and synchronization needs.

Non-linear editing model: source clips → timeline tracks → transitions and effects → render/export. Editing decisions are non-destructive when the source media is only referenced, not rewritten.

Proxy workflow: transcode high-resolution footage to lightweight proxies for editing, then relink to originals for final render; preserves quality while lowering CPU and storage load.

Timecode: HH:MM:SS:FF, where FF counts frames and wraps at the frame rate. Drop-frame timecode compensates for 29.97 fps versus 30 fps drift in NTSC-derived formats.

Export settings must match delivery: resolution, frame rate, codec, bitrate, color space, and audio sample rate. Mismatched frame rates cause judder or duplicated/dropped frames.

Container versus codec: MP4, MKV, and MOV are containers holding video, audio, and subtitle streams; H.264, HEVC, and AV1 are codecs inside them.

Asset management: consistent naming, metadata/tags, and versioned project files prevent broken links; offline media errors occur when referenced paths change.

Rendering cost scales with resolution, effects, and codec complexity; GPU acceleration helps effects and encoding but not all codecs or filters are GPU-enabled.

Raster versus vector: raster images store a pixel grid and degrade when scaled; vector graphics store geometric primitives and render crisply at any size.

Alpha compositing (Porter-Duff over): C_out = C_src·α_src + C_dst·(1−α_src). Premultiplied alpha multiplies color by alpha before blending to avoid dark fringes.

Color space conversions: sRGB is gamma-encoded; linear-light blending is required for correct compositing and lighting. Converting to linear and back changes apparent brightness if skipped.

Interpolation: nearest-neighbor preserves hard edges, bilinear is fast and smooth, bicubic is sharper with slight ringing; Lanczos gives high-quality resampling at higher cost.

Animation principles relevant to digital work: squash and stretch, anticipation, ease-in/ease-out, and timing. Keyframe interpolation (linear, Bezier) sets motion feel.

Frame rate and motion: 24 fps gives cinematic motion blur feel; 60 fps gives smoother motion; interpolation of missing frames (retiming) can produce artifacts on complex motion.

Image filtering: convolution kernels implement blur (averaging), sharpen (high-pass emphasis), and edge detection (Sobel/Laplacian). Kernel normalization keeps brightness constant.

Histograms show tonal distribution; gamma correction adjusts midtones without clipping highlights or shadows, unlike simple brightness/contrast sliders.

Loudness standard: EBU R128/ITU-R BS.1770 target −23 LUFS (broadcast) or −14 LUFS (streaming); peak normalization alone does not control perceived loudness.

Sample rate conversion requires anti-aliasing when downsampling and interpolation when upsampling; poor resampling causes aliasing or ringing artifacts.

Equalization and dynamics: EQ shapes frequency balance; compression reduces dynamic range above a threshold; limiting caps peaks; gating removes low-level noise.

Video color correction stages: primary correction balances exposure and white point; secondary correction targets specific hues or regions; grading applies a creative look.

Chroma keying works best with even lighting and a distinct background color; spill suppression and edge refinement reduce fringing around hair and translucent edges.

Frame rate conversion: pulldown (3:2) maps 24 fps film to 29.97 fps NTSC; inverse telecine restores the original cadence. Blending or optical-flow retiming is used for other conversions.

Audio-video sync tolerances: audio leading video by more than ~45 ms or lagging by more than ~125 ms is noticeable; ITU-R BT.1359 gives these perceptual bounds.

Delivery loudness and true peak: true-peak metering catches inter-sample overs that sample-peak meters miss; keep true peak at or below −1 dBTP for lossy encoding headroom.

Nielsen's usability heuristics include visibility of system status, match to real world, user control and freedom, consistency, error prevention, recognition over recall, and help.

Response time thresholds: ~0.1 s feels instantaneous, ~1 s keeps flow of thought, ~10 s is the limit before users switch tasks; progress feedback is needed beyond ~1 s.

Fitts's law: movement time = a + b·log2(2D/W). Larger and closer targets are faster to acquire; edge and corner targets are effectively infinite in one dimension.

Multimedia modality principle: narration plus relevant graphics beats narration plus redundant on-screen text; split attention across distant sources harms learning.

Accessibility: WCAG requires captions for prerecorded audio, transcripts, keyboard operability, and sufficient contrast. Captions also serve noisy or muted environments.

Interaction design patterns: direct manipulation, progressive disclosure, undo instead of confirmation dialogs, and consistent navigation reduce errors and cognitive load.

Perceived performance: skeleton screens, optimistic UI, and preloading hide latency; spinner-only waits feel longer than determinate progress indicators.

Multimedia synchronization in UI: audio is the master clock for lip sync; user events must not block the render thread or cause visible stalls.

Multimedia retrieval uses content-based features (color histograms, texture, shape, audio MFCCs) plus metadata; feature vectors are indexed with structures like R-trees or inverted files.

Similarity search: distance metrics such as Euclidean, cosine, and Earth Mover's Distance rank results; high-dimensional indexes suffer the curse of dimensionality.

Streaming protocols: HLS uses segmented MPEG-TS/fMP4 over HTTP with a playlist; MPEG-DASH uses MPD manifests. Both adapt bitrate to measured bandwidth (ABR).

Adaptive bitrate ladder: encode multiple renditions (e.g., 360p to 1080p) with aligned keyframes so the player can switch without artifacts.

Progressive download versus streaming: progressive files play before full download but cannot adapt quality; segmented streaming enables seeking and ABR at the cost of latency.

Web media: HTML5 / elements, Media Source Extensions for adaptive playback, and WebRTC for low-latency real-time communication.

CDN delivery reduces latency and origin load; cache-control headers, byte-range requests, and edge caching determine seek responsiveness and bandwidth cost.

Hypermedia structure: nodes linked by hyperlinks; broken links and orphan nodes are structural defects. Navigation depth and breadth affect findability.

Project constraints triangle: scope, time, and cost are interdependent; quality sits at the center. Changing one constraint forces trade-offs in the others.

Work breakdown structure decomposes deliverables into estimable tasks; critical path method identifies the longest dependent chain and the project's minimum duration.

Milestone reviews and stage gates (concept, script, prototype, production, QA, delivery) catch defects before expensive production stages.

Standards bodies: ISO/IEC JTC 1/SC 29 publishes JPEG and MPEG standards; ITU-T publishes H.26x video and audio recommendations; W3C publishes web standards.

Container and codec standards are distinct: ISO BMFF (MP4) defines file structure, while codecs such as AVC or HEVC define bitstream syntax.

Copyright protects original expression fixed in a tangible medium; it arises automatically. Ideas, facts, and methods are not protected.

Licensing models: royalty-free, rights-managed, and Creative Commons (which uses standardized conditions such as attribution and share-alike).

Fair use is a four-factor balancing test (purpose, nature, amount, market effect) and is a defense, not an automatic permission; it varies by jurisdiction.

Patents may cover codec techniques, so standards often have patent pools or licensing programs; using a standard does not automatically grant a license.

Von Neumann bottleneck: CPU and memory share one bus, so multimedia pipelines add DMA, dedicated codecs, and GPU/ISP blocks to move frame data without CPU copy per pixel.

Multimedia system layers: capture device → driver/API (DirectShow, V4L2, AVFoundation) → codec/processing graph → renderer/synchronizer → application. Synchronization sits above decoding.

Throughput rule: raw video rate = width × height × bytes-per-pixel × frame rate. 1920×1080, 3 B/px, 25 fps ≈ 155 MB/s; this exceeds many storage and bus budgets, motivating compression.

Latency budget rule for interactive A/V: end-to-end glass-to-glass delay under ~150 ms keeps talk and control natural; buffering trades latency for jitter tolerance.

Amdahl's law: speedup = 1 / ((1−p) + p/s), where p is the parallelizable fraction. Encoding gains from multicore are capped by serial stages such as entropy coding.

Real-time constraint: a stream is schedulable only if per-period processing time plus jitter margin ≤ period (e.g., 40 ms per frame at 25 fps). Missed deadlines cause dropped frames, not slowdowns.

Endianness and sample layout matter: PCM WAV is little-endian interleaved; planar YUV (I420) stores Y, then U, then V planes; packed formats interleave components per pixel.

Multimedia OS support: priority inversion and unbounded blocking in drivers cause audio glitches; real-time or elevated-priority threads plus lock-free ring buffers are standard remedies.

Sampling theorem: sample rate must exceed twice the highest frequency (Nyquist). 44.1 kHz audio captures up to 22.05 kHz; anti-alias filtering is mandatory before decimation.

Quantization error SNR for uniform b-bit PCM: SNR(dB) ≈ 6.02b + 1.76. Each added bit buys about 6 dB of dynamic range.

Color models: RGB is additive display space; CMYK is subtractive print; HSV/HSL separate hue from intensity for editing; YCbCr separates luma from chroma to exploit eye sensitivity.

Chroma subsampling notation J:a:b: 4:4:4 keeps full chroma; 4:2:2 halves horizontal chroma; 4:2:0 quarters chroma. 4:2:0 video needs 1.5 bytes per pixel at 8-bit luma.

Lossless vs lossy: RLE, Huffman, LZW, and arithmetic coding are entropy coders (lossless); quantization and DCT coefficient truncation are the lossy steps in transform codecs.

Bits per pixel (bpp) formula: bpp = total bits / (width × height). A 640×480 image stored in 300 KB (2,400,000 bits) is ≈ 7.8 bpp.

Run-length coding suits data with long repeats (palette images, fax); LZW builds a dictionary on the fly and is used in GIF and TIFF; Huffman assigns shortest codes to most frequent symbols.

Color depth and palette: indexed color stores k-bit indices into a palette of at most 2^k colors; 8-bit indexed images cannot represent smooth gradients without dithering.

JPEG baseline pipeline: 8×8 block split → level shift −128 → forward DCT → quantization by scaled table → zig-zag scan → DC differential + AC run-length → Huffman. Decoding reverses these steps.

JPEG quality factor scales the quantization table; lower quality means larger quantization steps, more zeroed coefficients, smaller files, and visible blocking/ringing artifacts.

JPEG is intra-frame only and suited to still images; Motion JPEG applies it per frame and lacks temporal prediction, so it is inefficient for video.

MPEG video structure: I-frames are intra-coded, P-frames predict from past references, B-frames bidirectionally predict from past and future references and require frame reordering.

GOP trade-off: longer groups of pictures improve compression but increase seek latency and error propagation; a closed GOP starts each group with an I-frame.

H.264/AVC adds variable block-size motion compensation, multiple reference frames, in-loop deblocking, and CABAC/CAVLC entropy coding; H.265/HEVC adds larger coding tree units and better prediction.

MPEG audio: Layer III (MP3) uses a polyphase/MDCT filterbank, psychoacoustic masking, and Huffman coding of quantized subband samples; the masking threshold determines allowed quantization noise.

AAC improves on MP3 with a pure MDCT filterbank, more flexible window switching, and tools such as SBR (HE-AAC) and parametric stereo; it is the usual audio partner of MP4 containers.

Storage capacity math: capacity = sectors × bytes per sector; sustained transfer rate, not burst rate, determines whether real-time capture succeeds.

RAID levels: RAID 0 stripes for speed with no redundancy; RAID 1 mirrors; RAID 5 stripes with distributed parity tolerating one disk failure; RAID 6 tolerates two.

Optical media: CD-ROM ≈ 650–700 MB, DVD-5 ≈ 4.7 GB single layer, Blu-ray ≈ 25 GB single layer; read speed multipliers are relative to base rates (CD 150 KB/s, DVD ~1.32 MB/s).

Flash storage has no seek time but suffers write amplification and wear; wear leveling and TRIM extend life. Sustained write speed can collapse after SLC cache is exhausted.

Capture interfaces: USB (isochronous transfers suit real-time audio/video), Thunderbolt/PCIe for high bandwidth, SDI/HDMI for uncompressed professional video.

Display metrics: pixel density = diagonal pixels / diagonal inches; refresh rate in Hz sets the maximum displayed frame rate; response time and persistence affect motion blur.

Audio hardware: ADC converts analog to digital on capture; DAC converts digital to analog on playback; sample clock jitter causes audible distortion, so word clock sync matters in studios.

Colorimeters and calibrators set monitor gamma and white point; without calibration, colors approved on one display can shift on another, breaking color-managed workflows.

Authoring paradigms: page/card-based (sequential screens), timeline-based (time-ordered tracks), icon/flow-based (event-driven flowchart), and scripted (code-driven). Choose by interactivity and synchronization needs.

Non-linear editing model: source clips → timeline tracks → transitions and effects → render/export. Editing decisions are non-destructive when the source media is only referenced, not rewritten.

Proxy workflow: transcode high-resolution footage to lightweight proxies for editing, then relink to originals for final render; preserves quality while lowering CPU and storage load.

Timecode: HH:MM:SS:FF, where FF counts frames and wraps at the frame rate. Drop-frame timecode compensates for 29.97 fps versus 30 fps drift in NTSC-derived formats.

Export settings must match delivery: resolution, frame rate, codec, bitrate, color space, and audio sample rate. Mismatched frame rates cause judder or duplicated/dropped frames.

Container versus codec: MP4, MKV, and MOV are containers holding video, audio, and subtitle streams; H.264, HEVC, and AV1 are codecs inside them.

Asset management: consistent naming, metadata/tags, and versioned project files prevent broken links; offline media errors occur when referenced paths change.

Rendering cost scales with resolution, effects, and codec complexity; GPU acceleration helps effects and encoding but not all codecs or filters are GPU-enabled.

Raster versus vector: raster images store a pixel grid and degrade when scaled; vector graphics store geometric primitives and render crisply at any size.

Alpha compositing (Porter-Duff over): C_out = C_src·α_src + C_dst·(1−α_src). Premultiplied alpha multiplies color by alpha before blending to avoid dark fringes.

Color space conversions: sRGB is gamma-encoded; linear-light blending is required for correct compositing and lighting. Converting to linear and back changes apparent brightness if skipped.

Interpolation: nearest-neighbor preserves hard edges, bilinear is fast and smooth, bicubic is sharper with slight ringing; Lanczos gives high-quality resampling at higher cost.

Animation principles relevant to digital work: squash and stretch, anticipation, ease-in/ease-out, and timing. Keyframe interpolation (linear, Bezier) sets motion feel.

Frame rate and motion: 24 fps gives cinematic motion blur feel; 60 fps gives smoother motion; interpolation of missing frames (retiming) can produce artifacts on complex motion.

Image filtering: convolution kernels implement blur (averaging), sharpen (high-pass emphasis), and edge detection (Sobel/Laplacian). Kernel normalization keeps brightness constant.

Histograms show tonal distribution; gamma correction adjusts midtones without clipping highlights or shadows, unlike simple brightness/contrast sliders.

Loudness standard: EBU R128/ITU-R BS.1770 target −23 LUFS (broadcast) or −14 LUFS (streaming); peak normalization alone does not control perceived loudness.

Sample rate conversion requires anti-aliasing when downsampling and interpolation when upsampling; poor resampling causes aliasing or ringing artifacts.

Equalization and dynamics: EQ shapes frequency balance; compression reduces dynamic range above a threshold; limiting caps peaks; gating removes low-level noise.

Video color correction stages: primary correction balances exposure and white point; secondary correction targets specific hues or regions; grading applies a creative look.

Chroma keying works best with even lighting and a distinct background color; spill suppression and edge refinement reduce fringing around hair and translucent edges.

Frame rate conversion: pulldown (3:2) maps 24 fps film to 29.97 fps NTSC; inverse telecine restores the original cadence. Blending or optical-flow retiming is used for other conversions.

Audio-video sync tolerances: audio leading video by more than ~45 ms or lagging by more than ~125 ms is noticeable; ITU-R BT.1359 gives these perceptual bounds.

Delivery loudness and true peak: true-peak metering catches inter-sample overs that sample-peak meters miss; keep true peak at or below −1 dBTP for lossy encoding headroom.

Nielsen's usability heuristics include visibility of system status, match to real world, user control and freedom, consistency, error prevention, recognition over recall, and help.

Response time thresholds: ~0.1 s feels instantaneous, ~1 s keeps flow of thought, ~10 s is the limit before users switch tasks; progress feedback is needed beyond ~1 s.

Fitts's law: movement time = a + b·log2(2D/W). Larger and closer targets are faster to acquire; edge and corner targets are effectively infinite in one dimension.

Multimedia modality principle: narration plus relevant graphics beats narration plus redundant on-screen text; split attention across distant sources harms learning.

Accessibility: WCAG requires captions for prerecorded audio, transcripts, keyboard operability, and sufficient contrast. Captions also serve noisy or muted environments.

Interaction design patterns: direct manipulation, progressive disclosure, undo instead of confirmation dialogs, and consistent navigation reduce errors and cognitive load.

Perceived performance: skeleton screens, optimistic UI, and preloading hide latency; spinner-only waits feel longer than determinate progress indicators.

Multimedia synchronization in UI: audio is the master clock for lip sync; user events must not block the render thread or cause visible stalls.

Multimedia retrieval uses content-based features (color histograms, texture, shape, audio MFCCs) plus metadata; feature vectors are indexed with structures like R-trees or inverted files.

Similarity search: distance metrics such as Euclidean, cosine, and Earth Mover's Distance rank results; high-dimensional indexes suffer the curse of dimensionality.

Streaming protocols: HLS uses segmented MPEG-TS/fMP4 over HTTP with a playlist; MPEG-DASH uses MPD manifests. Both adapt bitrate to measured bandwidth (ABR).

Adaptive bitrate ladder: encode multiple renditions (e.g., 360p to 1080p) with aligned keyframes so the player can switch without artifacts.

Progressive download versus streaming: progressive files play before full download but cannot adapt quality; segmented streaming enables seeking and ABR at the cost of latency.

Web media: HTML5 / elements, Media Source Extensions for adaptive playback, and WebRTC for low-latency real-time communication.

CDN delivery reduces latency and origin load; cache-control headers, byte-range requests, and edge caching determine seek responsiveness and bandwidth cost.

Hypermedia structure: nodes linked by hyperlinks; broken links and orphan nodes are structural defects. Navigation depth and breadth affect findability.

Project constraints triangle: scope, time, and cost are interdependent; quality sits at the center. Changing one constraint forces trade-offs in the others.

Work breakdown structure decomposes deliverables into estimable tasks; critical path method identifies the longest dependent chain and the project's minimum duration.

Milestone reviews and stage gates (concept, script, prototype, production, QA, delivery) catch defects before expensive production stages.

Standards bodies: ISO/IEC JTC 1/SC 29 publishes JPEG and MPEG standards; ITU-T publishes H.26x video and audio recommendations; W3C publishes web standards.

Container and codec standards are distinct: ISO BMFF (MP4) defines file structure, while codecs such as AVC or HEVC define bitstream syntax.

Copyright protects original expression fixed in a tangible medium; it arises automatically. Ideas, facts, and methods are not protected.

Licensing models: royalty-free, rights-managed, and Creative Commons (which uses standardized conditions such as attribution and share-alike).

Fair use is a four-factor balancing test (purpose, nature, amount, market effect) and is a defense, not an automatic permission; it varies by jurisdiction.

Patents may cover codec techniques, so standards often have patent pools or licensing programs; using a standard does not automatically grant a license.