INSIDE EDGE
INFERENCE
A VISUAL FIELD GUIDE QUALCOMM / SNAPDRAGON

ONE REQUEST. CLOUD TO EDGE.

Inside an inference request.

Follow a request from prompt to streamed text.
Then move it onto a Qualcomm device.

CPUGPUNPU / HTPMEMORYClick a block to inspect

THE IMPORTANT BIT

Conceptual cloud + Llama-style edge paths · placement varies · Phone speaker notes ↗ · Live Ollama demo ↗← → chapters   Space step   R replay   F fullscreen
Layer details & speaker notes +
Animation direction inspired by Rudraksh Karpe. Original opening presentation reused with attribution; edge diagrams and implementation are original. Cloud extension: 10 October 2026 · edge review: 7 September 2026.

RESEARCH / IMPLEMENTATION

Sources & scope

Chapters 1–7 explain a conceptual cloud request and serving system. Chapters 8–21 follow a decoder-only LLM on a Qualcomm device using QNN / Hexagon HTP. It is not a captured hardware trace. Exact operator placement, fusion, precision, memory allocation and performance depend on the model, compiler, SDK and SoC.

Read the 14-chapter technical review · 7 September 2026 ↗

How to read the diagrams

Boxes show logical operations; one box is not necessarily one kernel. The animated matrix units illustrate tiling and data reuse, not a particular Snapdragon floorplan. CPU, GPU and NPU are participants with different roles, not three mandatory sequential stages.

Acknowledgements

Rudraksh Karpe’s interactive inference presentation and shared code informed the visual direction. Earlier talk inspiration: Suman Debnath and Linda Haviv. Technical documentation is credited beside the relevant chapters.