forwardImageAsync awaited response.arrayBuffer(), so the whole image sat in
server memory before anything was sent to the client. A media board asks for
dozens of posters and backdrops at once, so that is dozens of full images held
simultaneously - and a client that gives up midway still cost a complete
server-side download.
Forwarding response.body instead lets backpressure reach upstream. Measured
with 30 concurrent 2 MiB images and deliberately slow clients, one fresh
process per arm because the allocator does not return freed spans:
buffered +116.0 MiB, +117.8 MiB peak RSS
streamed +74.2 MiB, +78.8 MiB peak RSS
-36% peak RSS on that load, and it scales with concurrency times image size.
Content-Length is now forwarded too, so clients still get a progress bar.
Next loads the module graph for all 74 app routes when the server starts. Measured
here: 1,114 files and 24 MiB of source resident before the first request, including
one shared 16 MiB subtree behind the API routes that reaches every integration
client. A dashboard serves a handful of routes per session.
`experimental.preloadEntriesOnStart: false` is Next's documented switch for this.
An earlier round of this work recorded route preloading as "framework behaviour, not
something this codebase sets" - that was wrong, and the findings doc is corrected.
Deterministic per-stage measurements, identical across both runs on each side:
stage source loaded files heapUsed
boot idle 24.0 -> 10.6 MiB 1114 -> 706 70.7 -> 44.8 (-37%)
post-login 27.9 -> 19.3 MiB 1215 -> 884 123.8 -> 109.8
board loaded 29.0 -> 21.7 MiB 1323 -> 1006 (run-to-run noise)
multi-tab 29.2 -> 22.9 MiB 1325 -> 1015 139.7 -> 127.8
post-soak 29.2 -> 22.9 MiB 1325 -> 1015 142.5 -> 127.4
The important part is the last two rows: 310 files and 6.3 MiB of source stay
unloaded even after a full multi-tab session, so this is not merely deferred cost -
routes the session never visits are never paid for at all. Live heap is lower at
every stage, by 26 MiB at idle and 12-15 MiB under load and at rest.
Peak node RSS does not move (350.3 -> 346.1, spreads overlap). V8 sizes its heap
against available headroom, so a smaller live set does not shrink peak RSS; what
improves is idle footprint and how much room is left before the heap cap. Median
board load is +3% with overlapping spreads, so the first-hit cost this trades for
did not show at this workload.
Full E2E suite passes 8/8 against the built image.
Tested the hypothesis that cached base64 images were bloating TanStack Query and
Redis. They are not, measured on a loaded 4-tab session:
base64 data-URLs, server heap 0.01 MiB (2 strings)
base64 blobs, browser heap 0.01 MiB (7 strings)
Redis total 2.17 MB
image bytes in Redis none - image-proxy stores URL + headers only
What the server heap is actually made of (141 MiB used): 41.4 MiB is bundled
JavaScript source retained as strings, ~29% of the heap. The memory is the code, not
cached data — which is why every cache-tuning idea in the fix catalog measured at
zero, and why serverExternalPackages was the only lever that moved idle memory.
7.86 MiB of that is literal duplicate copies: the same module inlined into several
Turbopack chunks and retained once per chunk. @ioredis/commands' command table was in
10 chunks and resident 6 times (2.07 MiB wasted), so it is now external — ioredis is
already external and owns it as a direct dependency, so it stays traced.
Recorded as bundle hygiene, NOT a memory win: idle 89.7 -> 89.4 MiB is inside noise,
and board-loaded/peak deltas were smaller than the run-to-run spread (one "before" run
read 220.6 MiB against 169-173 for identical code).
got and @octokit/request-error duplicate just as badly but were deliberately left
bundled: they are only transitive dependencies, and a check inside the image showed
got is not traced into node_modules, so externalising it would have turned a bundled
module into a require that resolves to nothing — a possible MODULE_NOT_FOUND for zero
measured gain.
analyze-heap-strings.mjs buckets a snapshot's strings by what they are, which is what
made "37 MiB of strings" actionable instead of just alarming.
Verified: 28/28 widgets render, auth control passes, zero module-not-found in the
container log, typecheck clean. pnpm test 1794/1795 with two different Testcontainers
flakes across runs (pi-hole, sabnzbd); both pass 18/18 in isolation.
DevTools still reported "Profiling not supported" because the build was never a
profiling one, despite three settings appearing to enable it. Next applies the React
swap through a webpack alias — getReactProfilingInProduction, mapping
'react-dom/client$' to 'react-dom/profiling' — that Turbopack never sees. A Turbopack
resolveAlias does not catch it either: the vendored entry requires
./cjs/react-dom-client.production.js relatively, not through a bare specifier.
The Dockerfile now performs that same swap on the vendored entry when
HOMARR_PROFILING=true, guarded so it fails the build loudly if React's file layout
changes rather than silently producing a normal build again.
Added `pnpm docker:profiling`, which builds and then verifies, because
`docker build . -t homarr:performance` alone yields a non-profiling image and that is
almost certainly what was being run.
verify-react-profiling.mts checks the bundle rather than the runtime: 2 client chunks
carry React's profiling renderer with the flag, 0 without. It replaces a Playwright
probe that could not tell the two apart — React sends a reduced inject payload to a
synthetic devtools hook, byte-identical for both builds, so that probe reported a
false negative for profiling and production alike. The bundle marker discriminates;
confirm the Profiler tab itself with the real extension.
Mantine's Menu mounts Popover + PopoverDropdown + several context providers even
while closed, so a menu per widget was ~13 fibers each for something almost never
opened. WidgetContextMenuProvider now mounts a single menu for the board, anchored
to the cursor; items keep only a contextmenu/long-press listener.
The dropdown takes the section's gridstack ref as a prop rather than reading
SectionContext — the shared menu lives outside any section, and reading it from
context there crashed the board with "useSectionContext must be used within a
SectionContext".
Verified against both builds with scripts/benchmarks/verify-context-menu.mts
(opens on right-click, shows actions, closes on Escape, re-targets to a second
widget):
idle fibers 18,670 -> 16,278 (-12.8%)
fibers after use +4,191 -> +4
JS event listeners 10,128 -> 7,774 (-23%)
DOM nodes 9,189 -> 8,287 (-10%)
browser JS heap 54.1 -> 52.7 MiB
median board load 520 -> 479 ms
container CPU 20.4 -> 20.0 s
Two runs per side, four tabs across two boards. Server memory is unchanged, as
expected for a client-only change. The listener reduction is the point: the 650 MB
session had ~88k listeners and this removes a per-widget source of them.
The gate is validated by passing against unmodified code first. An earlier version
used Playwright's click({ button: "right" }), which emits no contextmenu event
here and so failed shipped code identically — it wrongly condemned a previous
attempt at this optimisation. A gate never run against the un-optimised build
proves only that something changed, not that it still works.
pnpm test 1794/1795 (pre-existing pi-hole Testcontainers flake, 9/9 in isolation);
typecheck clean across 37 packages; 0 lint errors.
The container-only harness was measuring the wrong thing. Real usage — several
tabs, several boards, clicking into widgets — reaches ~880 MB server and a
650 MB *browser* heap, and the browser was never measured at all.
Profiling build: `docker build --build-arg HOMARR_PROFILING=true . -t
homarr:performance`. React's production bundle strips the hooks DevTools needs,
hence "Profiling not supported". Note reactProductionProfiling in next.config is
read only by the webpack path; under Turbopack it comes from `next build
--profile`, and turbo strips any env var not declared in the task's `env`, so all
three had to line up before the flag actually took effect — verified by markers
present only in React's profiling bundle, and static chunks going 20 -> 84 MB
with source maps.
inspect-container.mts attaches to an already-running container (SIGUSR1 opens
Node's inspector — no rebuild, no restart), reports cgroup/per-process/V8 memory
and can pull a heap snapshot out over CDP. The inspected context has no require
and no dynamic import, so it reads globals and uses CDP domains instead.
analyze-heapsnapshot / analyze-react-fibers / analyze-heap-retainers summarise a
snapshot too large for DevTools to open.
Harness now opens STRESS_BOARDS across STRESS_TABS_PER_BOARD tabs, clicks into
widgets, and records browser heap, DOM nodes and listener count via CDP.
Performance.getMetrics silently returns nothing unless the domain is enabled,
which is why the first run reported a 0 MiB client heap.
@scalar/api-reference-react is a documentation SPA reachable only from one
admin tab, and it renders nothing usable on the server, yet it was compiled
into the server graph for an SSR pass whose output is discarded. It is a
client component already, so next/dynamic with ssr: false is allowed in
place and needs no restructuring. @modelcontextprotocol/sdk is only reachable
through /api/mcp/[transport], so it moves to serverExternalPackages.
Honest accounting: this is an image-size and architecture change, not a
measured memory win. Idle landed at 89.4 MiB against 90.3/91.1/100.5 for the
previous build — the better end of that spread, but inside it, so on one
sample it cannot be claimed as a reduction. What is deterministic is image
size, 401 -> 398 MB. Kept because a docs SPA should not be server-rendered
regardless of what the memory number does.
pnpm typecheck and lint clean; pnpm test 1794/1795 with the known
Testcontainers pi-hole flake, which passes 9/9 in isolation.
Two further wins, both measured on a restored 28-widget board behind a real
demo session (scripts/benchmarks/stress-restore.mts).
--max-semi-space-size=4 in run.sh. --max-old-space-size only bounds the old
generation; V8 also sizes the young generation generously (16 MB per
semi-space) and that slack is pure resident memory. 4 MB cut idle anonymous
memory 120.0 -> 97.2 MiB and peak 710.6 -> 560.2 MiB. It costs CPU, as any
GC-space reduction does — 15.8 -> 17.1 CPU-seconds over the run, ~8% — with
no board-load latency regression (median 594 -> 570 ms). Overridable via
HOMARR_MAX_SEMI_SPACE_SIZE.
serverExternalPackages for the remaining server-only dependencies. Bundling
a package inlines its transitive graph into a chunk that is compiled at boot
even when the code path never runs; leaving it external makes it a lazy
require. Adding winston, drizzle-orm, ical.js, jszip, ldapts, node-unifi,
@kubernetes/client-node and linkedom took idle to 90.3 MiB and board-loaded
209.6 -> 164.6 MiB, with slightly lower CPU (16.3s) and latency (549 ms).
The trade-off is image size: 388 -> 401 MB, because those packages now ship
in node_modules instead of being tree-shaken into chunks.
Cumulative vs the merge-base: idle 141.7 -> 90.3 MiB (-36%), board loaded
257.8 -> 164.6 MiB (-36%), under load 392.7 -> 302.5 MiB (-23%).
The harness now proves the session rather than inferring it: it asserts an
anonymous request to the private board is denied, requires the Auth.js
session cookie after sign-in, and reports whether it had to modify the
restored data at all (now "none"). It also records cgroup CPU seconds and
board-load latency, so a memory win bought with CPU cannot hide.
pnpm test 144/144 files, 1795 tests; typecheck and lint clean.
Ablation (PHASE2-FINDINGS.md) attributed the memory win to the
serverExternalPackages line and the integrations barrel diet. These three
measured at exactly zero and were most of the file churn, so they go:
next-cache-handler: orphaned. cacheHandlers and cacheComponents are both
already gone from next.config.ts and nothing imported the package — it
was reachable only through a stale dependency entry.
cache-tags / cache-invalidation: there is no "use cache", cacheTag,
cacheLife or unstable_cache anywhere in app source, so revalidateTag was
firing on tags nothing registers, with no handler to receive them, inside
a catch that swallowed the resulting out-of-scope error. Removing the
calls also removed the SELECTs that existed only to supply a board name
or member list to them, so board.ts, group.ts, user.ts, serverSettings.ts,
section-router.ts and change-search-preferences.ts return to the
merge-base exactly. The live in-process integration data cache is
untouched.
Redis 6 -> 2 client consolidation: no measurable effect at idle (139.3 vs
139.2 MiB) or under load. Reverted, but the pubSub:last:* TTL is kept —
that one is an unbounded-growth fix, not a consolidation.
23 files, -883 lines. Diff vs merge-base: 100 files/3336 insertions ->
82 files/2484. pnpm test 144/144 files, 1795 tests, 0 failures;
typecheck and lint clean; app and Docker image build.
Revert layout.tsx, _layout-creator.tsx, manage/layout.tsx,
_theme.tsx, _creator.tsx, mantine.tsx, and login/page.tsx
to origin/dev — the Suspense provider splits caused
"EditMode is required", stream close, and React #419 errors.
Also remove cacheComponents/cacheHandlers from next.config.ts
(incompatible with auth/i18n/dynamic rendering) and fix
Boolean() truthy bug in Redis connection.
ALLOW-FROM is not supported by any modern browser. The CSP
frame-ancestors directive is the correct replacement and was
already present in both examples.
- Move getDependenciesAsync filesystem scan to scripts/dump-dependencies.mts
(runs at build time, outputs static-data/dependencies.json)
- About page now imports dependencies.json like other static data
(eliminates runtime glob of all package.json files)
- Delete dead getDependenciesAsync + parseDependencyCatalogsAsync from
package-reader.ts (only getPackageVersion remains)
- Add scripts/benchmarks/build-compare.sh for dev vs branch comparison
- BoardLayoutFallback: use HomarrLogoWithTitle (no board context required)
- integration-data-cache: guard cache.set with placeholder identity to
prevent stale write-back after invalidation + test
- next-cache-handler: memoize the promise (not resolved handler) to
prevent concurrent Redis connections
- redis-handler: tee ReadableStream before consuming in set(), hand fresh
branch to each get() caller
- redis-handler: base64-encode body instead of number[] (3-4x smaller)
Merge origin/dev to pick up service isolation (concurrently dev script,
turbopack aliases, instrumentation simplification), PostHog analytics,
color scheme, and other dev improvements while preserving all Phase 1
performance work (Suspense boundaries, cache components, Redis handler,
integration barrel diet, cache invalidation).
When looksLikeSvg detects SVG content under a non-SVG MIME type,
set Content-Type to image/svg+xml so browsers render correctly
with X-Content-Type-Options: nosniff.
- Detect SVG by content (magic bytes) in addition to MIME type to prevent
sanitization bypass via mislabeled uploads
- Add guard test verifying DOMPurify actually strips dirty payloads
- Add looksLikeSvg detection test
- Document REDIS_CACHE_DB in .env.example
Use greedy <svg>...</svg> match preferring closing tag over inner />
to prevent truncation at first self-closing child element.
Self-closing root <svg .../> handled as separate alternative.
Added test with <path/> + <circle/> children asserting well-formed output.
- Redis get() rejects legacy expire=0 entries and cleans them up
- transferOwnership invalidates previous owner in addition to new
- SVG regex uses non-greedy match and handles self-closing <svg />
- SVG fallback returns empty string instead of HTML wrapper
- Memory channel comment: FIFO eviction, not timestamp-based
- Fix integration data cache: check pending promise before TTL to
prevent returning undefined on concurrent cold-cache access
- Add concurrent-miss test (two parallel callers, one fetch)
- SVG sanitizer: extract <svg> from DOMPurify output to strip
residual HTML wrapper
- Narrow safeRevalidateTag catch: log unexpected errors, silence only
known "outside request scope" messages
- Run oxfmt across all packages (206 files formatted)
- Pass board.name to all invalidateBoardCache calls (permission mutations)
- Align integration create invalidation with update/delete
- Add initial-fetch-failure test for integration-data-cache
- Expand SVG XSS test corpus (javascript:, xlink:href, @import, onbegin)
- Forbid <style> tags in SVG sanitizer to prevent CSS injection
- Fix duplicate export in packages/common/src/index.ts
Thomas Camlong
ajnart
github-actions[bot]
Matt Van Horn