docs(perf): split the findings into per-subject markdown files
PHASE2-FINDINGS.md had grown past 1,200 lines mixing server memory, client rendering, methodology and a long list of rejected experiments, which made it unusable as a reference. Split under docs/perf/: README.md headline results and the change-by-change ratio table SERVER-MEMORY.md what the container's RAM is made of, layer by layer CLIENT-PERFORMANCE.md dashboard cost in Chrome: DOM, fibers, images, paint REJECTED.md ideas implemented, measured and abandoned, with numbers METHODOLOGY.md the tools, and the measurement traps that gave wrong answers PHASE2-FINDINGS.md is now an index so existing links keep working. METHODOLOGY.md is the part worth keeping: every wrong number in this investigation came from an instrument rather than the application, and each trap is written down - snapshots cannot explain peaks, V8 truncates snapshot strings to ~1 KiB, snapshot objects are not live handles, measuring without a forced GC measures V8's growth, taking a CDP snapshot added 134 MiB to the process being measured, per-component React attribution silently returns nothing on a production build, a fixed port can measure a leftover container, and a server-side change that moves a client-side metric means the workload changed.
This commit is contained in:
6 files changed
+1120
-1215
No files matched your search
@@ -0,0 +1,112 @@
|
||||
# Methodology, and the traps that produced wrong answers
|
||||
|
||||
Every wrong number in this investigation came from an instrument, not from the application. They are
|
||||
recorded here because each one looked plausible at the time, and several survived long enough to be
|
||||
published before being caught.
|
||||
|
||||
## The harness
|
||||
|
||||
`scripts/benchmarks/stress-restore.mts` boots a fresh container, restores a supplied backup
|
||||
**through the onboarding restore UI** (including typing "I understand"), signs in, opens boards
|
||||
across several tabs, clicks into widgets, reloads, then soaks idle — sampling memory, CPU, browser
|
||||
heap, DOM nodes and listeners at each stage.
|
||||
|
||||
Each run proves it is actually behind a login rather than reading a public page: it asserts a
|
||||
cookie-less request to the private board is **denied**, requires the Auth.js session cookie after
|
||||
sign-in, and reports whether it modified the restored data at all.
|
||||
|
||||
The backup is passed by path via `STRESS_BACKUP_ZIP` and never enters the repository — it carries
|
||||
real integration secrets. `.gitignore` also refuses `homarr-backup*.zip` and loose `*.sqlite`.
|
||||
|
||||
## Tools
|
||||
|
||||
| tool | answers |
|
||||
| --- | --- |
|
||||
| `attribute-memory.mts` | cgroup → processes → address-space mappings → V8 spaces → loaded code, each layer reconciled against the one above |
|
||||
| `memory-probe.cjs` | injected `--require` preload: V8 spaces, module cache, buffer/fetch allocation sites, require graph, smaps, handles. Diagnostics only, never shipped |
|
||||
| `analyze-heap-dominators.mjs` | real retained sizes via a dominator tree, and expansion of the largest retainer |
|
||||
| `analyze-allocations.mjs` | allocation volume by site, diffable between two stages |
|
||||
| `analyze-smaps-stages.mjs` | RSS split into JS heap, JIT, native and file-backed, per stage |
|
||||
| `analyze-require-graph.mjs` | what pulled each module in |
|
||||
| `compare-stress-runs.mjs` | alternated runs, medians **and per-run spread** |
|
||||
| `measure-widget-cost.mts` | per-widget DOM nodes, fibers, heaviest components |
|
||||
| `measure-board-paint.mts` | image requests, bytes, style/layout/task time, layout objects |
|
||||
| `measure-widget-churn.mts` | idle churn attributed to a widget via DOM mutations |
|
||||
|
||||
## Metric choice
|
||||
|
||||
**cgroup `anon`** is the server-memory figure. It is non-reclaimable and is what a container limit
|
||||
acts on.
|
||||
|
||||
**`memory.current` is not used for comparisons.** It includes page cache and swung **192–261 MiB
|
||||
for the same image**, and 60+ MiB between runs of a byte-identical build.
|
||||
|
||||
**`heapUsed` is not server memory.** It is one component. An early version of this work reported a
|
||||
`heapUsed` improvement as though it were the whole result.
|
||||
|
||||
## The traps
|
||||
|
||||
**A heap snapshot cannot explain a peak.** V8 runs a full GC before serialising one, so it reports
|
||||
survivors only. The 94 MiB of transient ArrayBuffers that drove this container past 500 MB never
|
||||
appeared in a single snapshot. Peaks need an allocation profiler and wrapped allocators.
|
||||
|
||||
**V8 truncates snapshot string contents to ~1 KiB.** Grouping snapshot strings by content therefore
|
||||
compares only each string's first kilobyte. This produced a published claim of "7.86 MiB of literal
|
||||
duplicate chunk sources"; hashing the real files found **zero** byte-identical duplicates among all
|
||||
3,055 server JS files.
|
||||
|
||||
**Snapshot objects are not live native handles.** A snapshot showed 15 `Gzip` objects still present
|
||||
at idle, which read as leaked zlib streams pinned by native handles — a real leak class that Next
|
||||
documents. `process.getActiveResourcesInfo()` reported **Zlib=0**, before and after 300 deliberately
|
||||
aborted gzip requests.
|
||||
|
||||
**Measuring memory without forcing a GC measures V8's growth, not the module's cost.** A
|
||||
require-cost sweep reported `drizzle-orm` at 124.3 MiB; with `global.gc()` before each reading the
|
||||
same measurement was a fraction of that. Two database drivers measured 21.4 MiB, not the ~30 MiB the
|
||||
naive sum implied.
|
||||
|
||||
**The act of measuring can dominate the measurement.** Taking a heap snapshot over CDP delivers it
|
||||
as JS string chunks: for a ~135 MB snapshot that added **~134 MiB of ArrayBuffers to the process
|
||||
being measured**, and the next stage read 693.7 MiB. `v8.writeHeapSnapshot` streams from C++
|
||||
instead, and the probe now reports its own `observerCostMiB`.
|
||||
|
||||
**Per-component React attribution requires a profiling build.** `fiber.actualDuration` is only
|
||||
populated by one, so on a production build every fiber fails the "did it re-render" test and the
|
||||
attribution tables come back **empty** — which reads as "nothing re-rendered" rather than "this
|
||||
build cannot tell you". This produced a published claim that the App Router re-rendered once a
|
||||
second. The tool now says which case it is.
|
||||
|
||||
**When a server-side change moves a client-side metric, the workload changed.** `--jitless` looked
|
||||
like −24% peak RSS, −14% CPU and 30% *faster* board loads. Three of those are impossible for a
|
||||
server flag. The client had **80% fewer event listeners**: the page had simply under-rendered.
|
||||
|
||||
**A fixed port can silently measure the wrong container.** A readiness check passed against a
|
||||
leftover container on the same port while the container under test had not started. Ephemeral ports
|
||||
and asserting the container's own identity are the fix.
|
||||
|
||||
**A completed build is not a built image.** A `docker builder prune` running concurrently killed a
|
||||
BuildKit session; the build failed and the next step ran against a stale tag. Verify the tag exists.
|
||||
|
||||
**Boards differ, so measure the right one.** Peak memory is at `nav-01`, not the multi-tab stage — a
|
||||
GC lands during multi-tab and pulls RSS down. Measuring the obvious stage understates peak by
|
||||
37 MiB.
|
||||
|
||||
## Reproducing
|
||||
|
||||
```bash
|
||||
STRESS_IMAGE=<image> \
|
||||
STRESS_BACKUP_ZIP=~/Downloads/<backup>.zip \
|
||||
STRESS_BOARDS="default,defaultv2" STRESS_TABS_PER_BOARD=2 \
|
||||
STRESS_NAV_ITERATIONS=3 STRESS_SOAK_MS=120000 \
|
||||
node --experimental-strip-types scripts/benchmarks/stress-restore.mts
|
||||
|
||||
node scripts/benchmarks/compare-stress-runs.mjs .bench/stress <before-prefix> <after-prefix>
|
||||
```
|
||||
|
||||
For attribution, add the probe:
|
||||
|
||||
```bash
|
||||
STRESS_MOUNT="$PWD/scripts/benchmarks:/probe:ro" \
|
||||
STRESS_ENV="HOMARR_PROBE_PORT=9333,HOMARR_PROBE_TRACK_BUFFERS=32768" \
|
||||
STRESS_NODE_OPTIONS="--require /probe/memory-probe.cjs"
|
||||
```
|
||||
Reference in new issue
Block a user