Graphics Rendering and GPU Programming Questions

Overview (quick)
Camera transforms map vertices from local object space to pixels. Typical pipeline: Model -> World -> View -> Clip -> NDC -> Screen.

Typical matrices

• Model matrix (M): object local -> world (translation, rotation, scale)
• View matrix (V): world -> camera (inverse of camera transform)
• Projection matrix (P): view -> clip (perspective or orthographic)
• Full transform: MVP = P * V * M

Place formulas on their own lines:

position_clip = P * V * M * position_object

After clip-space you perform the perspective divide:

position_ndc = position_clip / position_clip.w

Then viewport transform maps NDC [-1,1] to screen pixels.

Key extra matrix

• Normal matrix = transpose(inverse(M)) — use this to transform normals when M has non-uniform scale/rotation.

Practical debugging example: incorrect lighting
Symptom: lighting looks wrong on a scaled model (highlights skewed). Cause: using M to transform normals directly instead of normal matrix or ignoring non-uniform scale. Fix: compute normal = normalize(normal_matrix * normal_object) in vertex/fragment shader. Outcome: correct specular and diffuse.

Practical debugging example: billboarding wrong
Symptom: sprite supposed to face camera but appears tilted. Cause: using world-space billboard against object-space camera or applying rotation before view. Fix: build billboard in view-space (transform sprite center to view space, zero out yaw/pitch, or construct quad in camera-facing basis) then project with P. Understanding View vs Model ordering ensures correct facing.

Knowing each space and appropriate matrix (and where to do perspective divide) is essential for fixing transform and lighting artifacts.

Brief intro
I'll describe four strategies with concrete tooling/algorithms and practical trade-offs — framed as a game-dev optimizing large scenes for memory & GPU bandwidth.

1) Index compression

• What: Use 16-bit indices when vertex count < 65536; otherwise use 32-bit. For large meshes, delta‑encode indices, or compress the index stream (entropy or mesh-specific).
• Tools/algorithms: use Google meshoptimizer's index encoder (indexvbyte / rANS) or general compressors (zstd) at build time. Meshoptimizer also supports triangle reordering + index encoding for better compression.
• Trade-offs: 16-bit reduces memory but requires splitting meshes; compressed indices reduce size on disk/stream but require CPU/GPU decode cost.

2) Interleaved vertex attributes with compact types

• What: Interleave attributes (pos, normal, uv, skin) to improve locality and use compact formats: fp16 for positions/normals, 8/16-bit UNORM for uvs/weights, octahedral pack for normals/tangents.
• Tools/algorithms: GLM/DirectXMath conversions, meshoptimizer pack/convert utilities, Xbox/PlayStation shader pack formats, or Draco for aggressive quantization. Use asset pipeline step to quantize per-attribute based on scene bounds / required precision.
• Trade-offs: Quantization saves bandwidth but may introduce artifacts—tune bit-depth per attribute.

3) GPU-friendly vertex cache optimization

• What: Reorder triangles to improve post-transform cache hits, reducing vertex shader invocations and memory reads.
• Tools/algorithms: Forsyth or Tipsify algorithms; Google meshoptimizer provides a fast implementation and GPU-friendly remapping + vertex remap generation to compact vertex buffers.
• Practical: Generate a vertex remap and a reordered index buffer at build time, then reorder vertex buffer accordingly. Also consider meshlets (meshoptimizer) for draw-call batching.

4) Reuse of vertex buffers for multiple LODs

• What: Produce LODs that share a base vertex buffer and only change index buffers (index-only LODs) or use vertex reordering so higher LOD is a superset.
• Tools/algorithms: Use edge-collapse / quadric decimation (Garland–Heckbert), or progressive meshes; Simplygon, Unity Mesh Simplify, or meshoptimizer's simplifier to generate index-only LODs and vertex remaps.
• Trade-offs: Index-only LODs minimize memory and switching cost; progressive meshes enable streaming and smooth transitions but require more complex runtime handling.

Putting it together

• Pipeline: quantize & pack attributes -> generate vertex remap -> optimize index order (Forsyth/Tipsify) -> compress indices for storage (meshoptimizer/zstd/Draco) -> produce index-only LODs.
• Measure: validate visual error (screen-space), GPU vertex shader invocations, bandwidth counters, and memory use on target platforms; iterate with different quantization and LOD thresholds.

Plan overview — goal: reduce fragment shader cost by cutting texture fetches, ALU work, dependent ops, and per-pixel branching; shift work to vertex stage and use lower precision. Measure with GPU profilers and controlled A/B tests.

Concrete optimizations

• Reduce dependent/redundant texture lookups
  • Merge maps into atlases / arrays; sample once and unpack (normal+roughness packed in RGBA).
  • Use textureGather where supported to fetch multiple texels in one instruction.
  • Cache a sampled value into a temporary rather than re-sampling the same UV multiple times.
  • Convert dependent lookups to independent by deriving UVs earlier in vertex shader when possible.
• Use lower precision
  • Mark intermediates mediump/half (GLSL/metal qualifiers) for color, roughness where visual loss is negligible.
  • Test on target devices — some GPUs execute mediump in faster 16-bit ALUs.
• Move work to vertex shader
  • Compute per-vertex approximations (light attenuation, sphere occlusion) and interpolate.
  • Precompute UV offsets, LOD selection hints in vertex stage to avoid per-pixel math.
• Precompute / pack constants
  • Bake complex math into textures or small uniform buffers; pack multiple constants into vec4 uniforms.
• Reduce branching & loops
  • Replace dynamic loops with unrolled or fixed-size loops; use lerp-based selection instead of if/else where both paths are cheap.
  • Use branch flattening for low divergence; avoid dynamic texture-dependent branching.
• Approximations for expensive math
  • Replace pow/exp/log with fast approximations or LUTs (small 1D texture).
  • Use reciprocal/sqrt approximations (fast inverse sqrt) and refine with one Newton iteration.
  • Replace high-precision analytic BRDFs with precomputed 2D LUTs for environment IBL.

Measurement & validation

• Profilers / counters
  • Use platform tools: Qualcomm Adreno Profiler / Perfetto, ARM Mali Graphics Debugger, Xcode GPU Frame Capture, RenderDoc shader tracing.
  • Collect: fragment shader GPU time, pixel/fragment count, texture read count, ALU occupancy, shader cycles, bandwidth (texel/sample bytes), coherency/tiler utilization.
• Controlled tests
  • Create microbenchmarks: fullscreen shader quad, representative scene with high overdraw, and production scene.
  • A/B test: baseline vs optimized; measure median and 95th percentile frame GPU time, thermal throttling behavior, and battery draw.
  • Visual validation: pixel-diff, SSIM/PSNR on reference frames to quantify visual regression.
• Iterate
  • Apply one change at a time, re-profile to attribute wins.
  • Use shader variants and hot-patching to guard precision/approximation by device capability.

Key trade-offs

• Visual fidelity vs perf: set acceptable error budget and use image metrics.
• Device variability: fallbacks for GPUs where mediump gives no speedup.

This plan yields measurable GPU-time reductions while preserving visual quality when validated with profiler metrics and image comparisons.

Overview
Design a 3-tier LOD for particle systems (High, Medium, Low) with clear switching rules, anti-pop measures, authoring controls, and runtime enforcement tuned for multi-platform games.

LOD tiers

• High (near): full simulation, GPU particles, full overdraw, sub-emitter detail.
• Medium (mid): reduced particle count (50–70%), simplified collisions, lower spawn rate, simpler shaders (no refraction).
• Low (far/cheap): impostor sprites/billboards, baked animated sprite sheets or GPU texture atlases, minimal simulation (update every N frames).

Switching rules

• Primary: camera distance bands (example: High <15m, Medium 15–60m, Low >60m).
• Secondary: screen coverage (percent of screen pixels the emitter occupies; switch to Medium if <2%).
• Tertiary: performance signals—global FPS or GPU load; downscale LOD if sustained FPS < target or thermal throttling detected.
• Combine via priority: if any condition requests cheaper LOD, pick lower; if conflicting, choose higher only if within hysteresis.

Transitions & artifact mitigation

• Hysteresis thresholds (1.2× distance or 0.5% screen coverage) to avoid flicker.
• Cross-fade blending between LODs over 0.25–0.5s using alpha and particle birth/death smoothing.
• Particle matching: map groups (e.g., 10 particles -> 1 impostor) so motion continuity is preserved.
• Temporal upscaling: staggered particle updates rather than abrupt stop to avoid popping.
• GPU-friendly fades (alpha + shader dissolve) to avoid CPU stalls.

Authoring-time controls

• Artist UI: set LOD ranges, screen-coverage thresholds, per-LOD overrides (count scale, shader variant), and preview sliders.
• Automated baked impostor generator (captures animated sprite sheet per viewpoint).
• Validation tools: visual diff, estimated drawcalls and memory per LOD, target device profiles.

Runtime enforcement

• Central LOD manager queries camera, screen coverage (bounding sphere projected size), and performance metrics.
• Per-system budgeter enforces particle count caps and clamps shader variants.
• Telemetry & dynamic tuning: record missed budgets and auto-adjust thresholds per device class.

This approach balances visual fidelity with performance and gives artists precise controls while ensuring stable runtime behavior across devices.

Clarify goals & constraints

• Real-time particle effects that must collide with complex static/dynamic world geometry at 60+ FPS on consoles/PC. Minimize CPU-GPU stalls and PCIe transfers; support thousands–millions of particles.

High-level partitioning

• GPU: particle integration, local particle–particle forces, broad-phase culling in particle-local space, final vertex positions for rendering.
• CPU: heavy-weight collision queries against complex world geometry (BVH traversal, dynamic object updates, exact contact generation) when geometry is too complex or uses engine-side data (NavMesh, skeletal meshes). CPU also maintains world BVH and answers queries for many particles in bulk.

Data structures

• World: CPU-side BVH / KD-tree per frame; accelerate dynamic objects with per-object local BVHs.
• Particles: GPU-side arrays (position, velocity, mass, life) in SSBOs/UAVs. Also store particle IDs or small keys.
• Collision batches: compact query buffers containing particle AABBs or ray segments and particle IDs prepared on GPU and copied to CPU as a single staging buffer.

Work flow & synchronization

1. GPU integrates N steps; computes per-particle collision queries (swept AABB or ray for displacement) and writes compact query records to a GPU staging SSBO.
2. Use async copy (glMapBufferRange / ID3D12Resource -> CopyQueue) to transfer staging to CPU without stalling GPU. Use double-buffered staging: while CPU handles batch k, GPU writes batch k+1.
3. CPU receives batch, performs BVH traversals and narrow-phase tests; produces result records (hit/no-hit, contact point, normal, penetration depth) in a compact array.
4. CPU writes results into a smaller staging buffer and issues async upload back to GPU. Use fences/futures to synchronize only when GPU needs results; avoid glFinish/Flush.
5. GPU consumes results in a compute pass that applies collision responses (position correction, impulse) and writes final vertex data for rendering. Use atomics or prefix-scan for compacting hit lists entirely on GPU where possible.

Minimizing transfers

• Transfer only compact queries (particle ID + ray/AABB: ~24 bytes) and compact results (ID + small contact). Batch many particles per transfer (e.g., 64K records).
• Use GPU-side broad-phase to reduce queries: spatial hashing / uniform grid or distance culling so only potentially colliding particles are sent to CPU.
• When geometry is mostly static, precompute signed distance fields (SDF) or heightmaps on GPU to handle local collisions fully on GPU, avoiding CPU entirely for many cases.

Performance & edge cases

• Use asynchronous queues and double/triple buffering to hide latency.
• Fallback small-particle collisions fully on GPU with SDFs; only send complex interactions to CPU.
• Handle dynamic objects by including per-object dirty flags to rebuild/partial-update BVH.
• Test worst-case when many particles collide — ensure CPU threads scale (job system) and backpressure limits query batch size.

This approach balances GPU throughput for bulk integration and rendering with CPU accuracy for complex geometry, using compact batched transfers and async synchronization to avoid stalls.