Understanding Flame Graphs
Flame Graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately. They are a powerful tool for performance optimization.
When Flame Graphs Help Most
Flame graphs are most useful when CPU usage is high, a request path feels unexpectedly slow, or you know a process is busy but cannot yet see which functions dominate runtime. They help narrow optimization work to the hottest call stacks first.
What is a Flame Graph?
A flame graph acts like an x-ray for your CPU usage.
- **X-Axis**: The population (width) shows the frequency of the function in the stack trace. The wider the bar, the more CPU time it's consuming (directly or via children).
- **Y-Axis**: The stack depth. The top-most boxes are the functions currently on the CPU. The bottom-most is the root.
Note: The x-axis order is **not** chronological; it is sorted alphabetically to merge identical stacks.
How to Read
1. **Look for wide plates:** The wider the bar at the top, the more time is spent in that function.
2. **Look for tall towers:** Deep stacks mean function A calls B calls C... this helps find where the logic is deep.
3. **Plateaus:** A large flat top suggests a single function is doing a lot of work (CPU bound).
4. **Spikes:** Many thin spikes suggest lots of small, fast function calls.
Common Optimization Patterns
| Key / Code | Description |
|---|---|
| Wide but Shallow | Function is doing a lot of loop iterations or heavy computation itself. |
| Deep Towers | Recursion or excessive abstraction layers. |
| Fragments | If proper stack merging isn't happening, check if you have valid stack traces (e.g., missing frame pointers). |
Generating Flame Graphs
You often generate these from `perf` (Linux) or other profilers.
# 1. Record profile with perf (99Hz for 60 seconds)
perf record -F 99 -a -g -- sleep 60
# 2. Convert to Flame Graph (using Brendan Gregg's tools)
perf script | ./stackcollapse-perf.pl | ./flamegraph.pl > flamegraph.svgCommon Flame Graph Mistakes
A common mistake is treating the left-to-right position as time order. In a flame graph, width matters, not chronology. Another frequent error is optimizing a wide parent frame before checking whether the real cost sits deeper in a child function.
Related
Latency targets and expectations
Use latency ranges to decide whether a hotspot is actually affecting user-facing performance.
Algorithmic complexity
Connect visible hotspots in profiles with the algorithmic cost behind them.
Docker cheatsheet
Helpful when you need to collect or inspect profiles inside containers.
top command reference
Start with system-level process observation before diving into detailed CPU profiles.
Knowledge is power.