Mantine's Tabs keeps inactive panels mounted by default. On Mantine 9 with
React 19 that uses <Activity>, which preserves state and unmounts effects but
still builds the entire subtree - so a two-tab widget pays for both while
showing one.
health-monitoring both the system and the cluster monitoring view were built
media-missing both tabs built their full card list, each card with a
poster and a blurred backdrop
Two other Tabs users are deliberately left mounted: custom-api panels can hold
user component state, and the widget edit modal would reset half-filled forms.
media-missing also rendered every card into a scrolling tile with no lazy
loading, the same shape as media-releases. Its card height is fixed per density,
so contain-intrinsic-size is exact rather than a guess.
calendar fell back to https://placehold.co/400x400 when a poster failed to
load, so a self-hosted dashboard made a third-party request - once per failing
event - to draw a grey square, and showed nothing at all on a host without
internet access. Same grey square, inline.
The rss feed handler logged the whole serialised feed entry through a raw
console.debug, which no log level could suppress, and reported a match count
that was really the regex capture-group count.
Adds scripts/benchmarks/widget-inventory.mjs and docs/perf/WIDGET-AUDIT.md, a
verdict for all 50 widgets.
forwardImageAsync awaited response.arrayBuffer(), so the whole image sat in
server memory before anything was sent to the client. A media board asks for
dozens of posters and backdrops at once, so that is dozens of full images held
simultaneously - and a client that gives up midway still cost a complete
server-side download.
Forwarding response.body instead lets backpressure reach upstream. Measured
with 30 concurrent 2 MiB images and deliberately slow clients, one fresh
process per arm because the allocator does not return freed spans:
buffered +116.0 MiB, +117.8 MiB peak RSS
streamed +74.2 MiB, +78.8 MiB peak RSS
-36% peak RSS on that load, and it scales with concurrency times image size.
Content-Length is now forwarded too, so clients still get a progress bar.
fetchWithTrustedCertificatesAsync has 125 call sites and none of them pass a
dispatcher, so every HTTP request Homarr makes - every integration poll, every
proxied image - built a brand new EnvHttpProxyAgent. An agent owns its
connection pool and its TLS session cache, so a fresh one per request means a
full TLS handshake every time and no keep-alive reuse at all.
Measured against a local TLS server, 60 sequential requests:
fresh agent per request 5.11 ms/req 60 TLS handshakes
one shared agent 1.69 ms/req 2 TLS handshakes
3.0x faster and 30x fewer handshakes on loopback; over a real network the gap
is larger, since that is where handshake round-trips actually cost something.
Agent construction itself is cheap (0.004 ms, 1.7 KiB) because undici builds
the TLS store lazily at connect time - so the cost was never construction, it
was the handshake that a fresh pool forces.
The trust material is still read on every call and the cached agents are keyed
on a hash of it, so adding or removing a certificate - including by dropping a
file into the mounted folder rather than going through the UI - takes effect on
the very next request, exactly as before. Superseded agents are closed so their
idle sockets are released.
This branch has repeatedly named a 16.10 MiB subtree of 460 files, reached by every API
route, as the one remaining server-side lever with real headroom, attributing it to the tRPC
router importing every integration eagerly. Both halves are wrong.
The integration creator was already fully lazy: every kind in creator.ts is
`() => import(...)`, so no integration class loads until one is constructed. The earlier
barrel diet had done this properly - the claim was never checked against the code.
The subtree was an artefact of route preloading, measured before preloadEntriesOnStart:
false shipped, when Next loaded all 74 routes' module graphs at startup. Re-measured on the
current build, boot is 706 files and 10.56 MiB:
3.73 MiB 28 files .next/server/chunks (the app's own boot graph)
2.16 MiB 14 files better-sqlite3 (2.1 MiB of it the native binary)
1.75 MiB 359 files next/dist
1.58 MiB 25 files next/dist/compiled
0.45 MiB 77 files mysql2 <- known, structurally blocked
0.07 MiB 13 files pg <- same
There is no single large server lever left. What remains is diffuse: a full multi-tab session
loads 15.44 MiB of app chunks across 241 files, and both the ~41 MiB of retained source and
the ~38 MiB of compiler zones scale with that. Reducing it means shipping less application
JavaScript per route, which is a broad refactor with no single target.
The off-screen work was committed unmeasured; this closes that debt. Measured on two
containers restored from identical data:
image requests 138, 135 -> 22, 22 (-84%)
image bytes 12.27 MiB -> 0.03 MiB
layout objects 11611, 7836 -> 3694, 3705 (-53%)
A 99.8% drop in image bytes is not believable, so it was checked instead of reported. Two
checks were bad instruments before one worked:
- Counting decoded images via getBoundingClientRect() reported 80/80 loaded in BOTH arms.
Reading geometry forces layout, which forces content-visibility subtrees to render, which
triggers the very image loads being tested for. The instrument caused the result.
- Screenshots came back byte-identical in both arms, because this environment shows an
onboarding overlay and media-requests renders blank with unreachable integrations. Good
evidence of no visual difference; useless as evidence that images render.
- Counting img.complete && naturalWidth > 0 reads no geometry and gave the answer: 80/80
decoded in both arms, 138 responses before against 25 after.
So nothing is visually broken. The 80 imgs are integration icons sharing URLs, deduplicated to
~25 unique fetches in both arms; the ~113 avoided requests are other widgets' off-screen
artwork, chiefly media-requests' full-size per-card backdrops, which is also where the 12 MiB
came from.
Stated limit: media-requests renders blank in this environment, so its visible-card behaviour
after the change is unverified. Media-releases is verified at 80/80 decoded.
Replaces the earlier version with what was actually measured, and closes out all 54
findings from the widget audit rather than leaving categories open:
overlay-per-row (12) 5 fixed; 7 are functional tooltips with real labels
index-key (11) all safe - static text splits, or stateless rows from
user-configured JSONPath data with no stable id
nested-component (2) both fixed
inline-style-per-row (29) left alone; a fresh object per row only defeats memoisation
if the child is memoised, and these are not
Measured, deterministic per-widget counts:
mediaReleases 340 comp / 8387 fib -> 170 / 8057
calendar 171 comp / 1195 fib -> 75 / 907
beszelSystemTable 36 comp / 1286 fib -> 18 / 1230
Records two metrics that turned out unusable - Chrome's board-level JSEventListeners read
7905 to 17114 across six runs of identical code, and its Nodes counter includes detached
nodes - and one claim withdrawn: the media-releases tooltip fix measured zero, the win for
that widget came entirely from the shared OverflowBadge fix.
Last of the two nested-component findings from the widget audit. `Handle` was declared in the
render body and passed to a child as `handle={Handle}` - a component *type*. A type that
changes identity makes React unmount and remount the subtree instead of updating it, and this
component re-renders on every frame of a drag as dnd-kit updates `transform`, so the handle was
being torn down and rebuilt continuously while being dragged.
Wrapped in useMemo keyed on the two things it closes over.
Measured result of the whole overlay/remount sweep, per widget, deterministic component and
fiber counts on identical restored data:
widget before after
mediaReleases 340 comp / 8387 fib 170 / 8057 (-50% components)
calendar 171 comp / 1195 fib 75 / 907 (-56% components, -24% fibers)
beszelSystemTable 36 comp / 1286 fib 18 / 1230 (-50% components)
downloads / mediaRequests / app unchanged - their overlays are functional
Two corrections to earlier claims in this branch:
- The media-releases win came from the shared OverflowBadge fix, NOT from the tooltip change.
Measured on the build with only the tooltip fix, mediaReleases was still at 340; it dropped
to 170 once OverflowBadge stopped mounting a Popover when nothing overflows. The tooltip fix
remains zero on this board because every release has a description.
- The board-level "event listeners" metric is unusable. Across six runs of identical code it
read 7905, 8000, 8166, 8179, 10360 and 17114. Chrome's Nodes counter has the same problem
(23072 on one run against ~8850 on the rest) because it counts detached nodes awaiting
collection, while "elements under body" stayed at 6691-6726 throughout. Only the per-widget
component and fiber counts are stable enough to compare, so those are what is reported.
Two more instances of the same Mantine behaviour, found by searching for overlay components
that take a `disabled` prop rather than by measuring another board.
OverflowBadge (packages/ui) mounted a Popover and a Popover.Dropdown unconditionally - both
when `disablePopover` was set and, more importantly, when nothing overflowed at all, which is
the common case. It is shared UI, used by media-releases, releases, the docker table and the
widget inputs, so the cost repeated per row wherever a list renders it. The rendered markup is
unchanged: a Popover.Target only forwards a ref, so returning the badges directly is
visually identical.
beszel disk-usage mounted a HoverCard per table row with `disabled={filesystems.length === 0}`,
and already fell back to a plain progress bar inside its own target for that case - so for a
system reporting no extra filesystems the entire overlay existed to render nothing. Hoisted
that branch out; the now-unreachable ternary and the dead `disabled` prop went with it.
Measured result of this class of fix, on the calendar which had the worst ratio:
interactive components 171 -> 75 (-56%)
fibers 1195 -> 907 (-24%)
Honest note on the media-releases tooltip fix committed earlier: it measured **zero** on the
test board. Every release there has a description and the option is on, so all 80 tooltips are
legitimately enabled and none are skipped. It only helps boards where descriptions are absent
or the option is off. The mechanism is right; this board simply is not the case it helps.
Also confirmed NOT worth changing: downloads mounts 30 Tooltips but all three per row are
functional (integration name, hover detail, progress), so there is nothing disabled to skip.
Replacing them with native title attributes would change the UX, not just the cost.
Third instance of the same pattern, and the worst ratio yet. Mantine mounts a HoverCard's
Popover machinery - Popover, PopoverTarget and PopoverDropdown - even when it is disabled.
A month grid is around 42 cells and `disabled` is `isEditMode || eventsForDate.length === 0`,
so on a typical month most days mounted three components for a card that can never open, and
in edit mode every single day did.
Measured on a real board: 42 Popovers, 42 PopoverDropdowns and 42 PopoverTargets - 126 of
this widget's 171 listener-attaching components - in a widget with only 251 DOM nodes. That
is a far worse components-per-node ratio than media-releases, which had 340 across 2,951
nodes.
Returning the day cell directly when disabled is behaviour-identical: a disabled HoverCard
never opens, so nothing observable is lost.
This is now the third place this exact Mantine behaviour has cost real work - after the
shared board context menu and the media-release tooltips. The general rule: Mantine overlay
components mount their full machinery regardless of `disabled`, so `disabled` is a
visual-suppression prop, not a cost-avoidance one. Deciding before rendering is what avoids
the cost.
A static audit of every widget (scripts/benchmarks/audit-widget-render.mjs, added here)
found this: PercentCell was declared inside the widget body, which makes it a new component
type on every render. React cannot match a new type to the previous one, so it unmounted and
remounted every cell's subtree - Group, Text and Progress - instead of updating it. The
widget re-renders about once a second from its live Beszel subscription, and the cell is used
by three columns, so that was three subtrees per system torn down and rebuilt every second.
Hoisted to module scope and wrapped in memo. `size` is now passed as a prop; it was the only
thing the nested version closed over.
The audit covers all widget .tsx files and ranks findings by whether they sit inside a
.map(), since those multiply by row count:
29 inline-style-per-row
12 overlay-per-row
11 index-key
2 nested-component
Deliberately not fixing the other 52 yet. The 29 inline styles are the least likely to
matter - a fresh object per row only defeats memoisation if the child is memoised, and mostly
these are not - so changing 29 call sites would be churn without a measurement to justify it.
The 11 index keys are only harmful where a list reorders, which needs checking per widget. The
remaining overlay-per-row sites are the same pattern already fixed twice (context menus, then
media-release tooltips) and are the most likely to be worth it: calendar mounts 42 Popovers
and downloads 30 Tooltips, both confirmed at runtime.
Not measured: the mechanism is certain but the size depends on how often the widget renders
and how many systems are in the table.
Attributing listener-attaching components per widget found the same anti-pattern the
shared board context menu already fixed once, still present in list widgets:
interact nodes fibers kind / which components
340 2951 8387 mediaReleases
UnstyledButton x90 TooltipFloating x80 Popover x80
PopoverDropdown x80 PopoverTarget x10
171 251 1195 calendar
UnstyledButton x45 Popover x42 PopoverDropdown x42
PopoverTarget x42
46 264 898 downloads
Tooltip x30
Mantine mounts a TooltipFloating's Popover machinery even when the tooltip is disabled, so
wrapping every card unconditionally produced 80 TooltipFloating, 80 Popover and 80
PopoverDropdown components - 240 of this widget's 340 listener-attaching components - for
tooltips that only show on hover, on cards that are 91% off-screen.
Deciding before rendering means a card with no description, or with the option off, mounts
none of it. Behaviour is unchanged for cards that do have a description.
Also extends measure-widget-cost.mts to count these components per widget, which is what
found it. The board's 8,165 event listeners were never attributable before; now they are.
Not yet measured: the listener reduction depends on how many releases carry a description,
which is integration data. The mechanism is certain (the components are simply not created)
but the size of the win on a given board is not, and calendar's 42 Popovers and downloads'
30 Tooltips are untouched. Tracked in docs/perf/NEXT-STEPS.md.
Four commits now apply the off-screen row treatment; only the first was A/B'd. Records
which is which, and warns how to read a result for the three widgets that do not overdraw
on the test backup - a measurement there will correctly show no change, because the benefit
only exists once an integration returns more rows than the tile can show.
Also records two things that look like gaps and are not:
- content-visibility was applied to five of the 26 ScrollArea widgets on purpose. It
costs containment work and can be wrong on self-measuring content such as charts and
the notebook, so the rule was: apply where list length is decided by remote data.
- React cache() is server-only. It already wraps getBoardByNameAsync, getHomeBoardAsync,
getRscServerSettingsAsync and getQueryClient; it does nothing for client widget
rendering, where react-query does the caching and its gcTime was already verified at
5 minutes rather than the 24 hours the original fix catalogue assumed.
Extracts the media-releases fix into packages/widgets/src/common/offscreen-rows.ts and
applies it to the remaining widgets whose list length is decided by remote data rather
than by the tile: rssFeed, notifications and the calendar event list. Images in those
lists also get loading="lazy" - the rss cards carry two each, a backdrop and a poster.
The measured basis, from media-releases where the pattern was found:
image requests 135 -> 55 (-59%)
layout objects 8346 -> 5117 (-41%)
Scope was chosen deliberately rather than applied everywhere. 26 widgets use ScrollArea,
but on the measured board only two rendered more than their tile could show, and
content-visibility is not free: it costs a little containment work, and on content that
measures itself - charts, editors - skipping layout can be actively wrong. The widgets
changed here are the ones where overdraw is a function of how much data an integration
returns, so a user with a busier instance hits it even though this board does not. Chart
and fixed-size widgets are deliberately left alone.
Honest scope note: unlike the media-releases change, these three are NOT measured. On the
test board rssFeed renders 7 nodes and the calendar's 7 rows fit their tile, so there is
nothing to improve here to observe - the benefit only appears with more data than this
backup has. They are typechecked and lint-clean. docs/perf/NEXT-STEPS.md tracks them
alongside the media-requests change as pending verification.
A duplicate `style` prop on the notifications Card was caught by typecheck and merged
rather than left to override the existing colour and text-decoration styles.
Records where this stopped so it can be picked up cleanly:
1. c526dd2b7 (media-requests content-visibility) is typechecked but NOT measured - the
only performance claim on this branch not A/B'd before commit. Verify with
measure-board-paint.mts, three repeats; revert if image requests do not fall.
2. The one lever with headroom left is the tRPC router's eager integration imports - a
16.10 MiB subtree of 460 files reached by every API route, which pays twice because
both the ~41 MiB of retained source and the ~38 MiB of V8 compiler zones scale with
how much JavaScript the server compiles. Every allocator and V8-flag avenue is closed.
3. Deferred with prizes attached: ASCII-escaping bundles (9.45 MiB, needs its own PR),
lazy mysql2/pg (21.4 MiB, structurally blocked), beszelSystemStats chart SVG.
4. The proposed set for the clean PR, six changes producing -26% peak anon and -52% at
idle for +18% CPU - with the gRPC patch flagged as hygiene rather than a measured win,
and the +30% board-load figure flagged as not established.
Second instance of the pattern fixed in mediaReleases. The overdraw tool, corrected to
find row containers by child count rather than by uniform height, found only one other
widget genuinely rendering more than its tile can show:
nodes rows vis offscr imgs tile content kind
2951 80 7 2693 80 539 7206 mediaReleases (already fixed)
393 20 6 275 60 405 1512 mediaRequests-requestList
1,512 px of cards in a 405 px tile, and 20 cards carrying 60 images between them - each
card renders a full-tile backdrop plus a poster. Same treatment: content-visibility with
contain-intrinsic-size on the card, loading="lazy" on both images.
IMPORTANT: this is typechecked but **not measured**. Every other performance claim in this
branch was verified by an A/B before being committed; this one was not, because the session
ended first. It is committed so the work is not lost, and it is the first thing to verify
next - the same measurement used for mediaReleases applies:
STRESS_BASE_URL=http://127.0.0.1:PORT node --experimental-strip-types \
scripts/benchmarks/measure-board-paint.mts
Expect image requests to fall; do not expect JS heap to move, since decoded bitmaps live
outside JSHeapUsedSize.
Every other widget on the board fits its tile and needs nothing. The tool's "visible"
column misreads chart widgets (beszelSystemStats reports 28 rows / 0 visible because its
container is an SVG whose groups have no layout height) - content height versus tile height
is the trustworthy signal, and by that measure only these two overdraw.
PHASE2-FINDINGS.md had grown past 1,200 lines mixing server memory, client rendering,
methodology and a long list of rejected experiments, which made it unusable as a
reference. Split under docs/perf/:
README.md headline results and the change-by-change ratio table
SERVER-MEMORY.md what the container's RAM is made of, layer by layer
CLIENT-PERFORMANCE.md dashboard cost in Chrome: DOM, fibers, images, paint
REJECTED.md ideas implemented, measured and abandoned, with numbers
METHODOLOGY.md the tools, and the measurement traps that gave wrong answers
PHASE2-FINDINGS.md is now an index so existing links keep working.
METHODOLOGY.md is the part worth keeping: every wrong number in this investigation came
from an instrument rather than the application, and each trap is written down - snapshots
cannot explain peaks, V8 truncates snapshot strings to ~1 KiB, snapshot objects are not
live handles, measuring without a forced GC measures V8's growth, taking a CDP snapshot
added 134 MiB to the process being measured, per-component React attribution silently
returns nothing on a production build, a fixed port can measure a leftover container, and
a server-side change that moves a client-side metric means the workload changed.
Per-widget attribution of the dashboard found one widget responsible for 44% of the
board's DOM: mediaReleases at 2,951 nodes and 8,387 fibers, including 1,935 Mantine
Box components, against a board total of 9,293 nodes.
The cause is that it maps over every release the integrations return, with no cap.
Measured on a real board: the tile is 539 px tall and its content is 7,206 px, so of
80 rendered cards only 7 are visible. The other 73 cost DOM, fibers, and - most
expensively - a poster and a CSS backdrop image each, all fetched and decoded for
pixels nobody can see. Decoded bitmaps are far larger in memory than their files.
Fixed with content-visibility: auto plus contain-intrinsic-size on each card, so the
browser skips layout, paint, raster and background-image fetch for off-screen cards,
and loading="lazy" on the poster. Nothing leaves the DOM, so scrolling and in-page
search are unchanged, and the intrinsic size keeps the scrollbar from jumping.
Measured over three repeats per side on identical restored data:
image requests 135, 135, 135 -> 55, 55, 56 (-59%, deterministic)
image bytes 12.27 MiB -> 8.41 MiB (-31%)
layout objects 8346, 7815, 7815 -> 5117, 4696, 4636 (-41%)
Two things this does NOT show, stated because a single run suggested otherwise:
- JS heap did not clearly improve. Across four runs it was 46-88 MiB before against
74-86 MiB after; the before spread is 2x, so no claim either way. This is expected:
decoded image memory lives outside JSHeapUsedSize, so the actual saving from 81
fewer image fetches is invisible to that metric.
- Task duration and time-to-settled overlap between arms and support no conclusion.
Adds the tools this needed: measure-widget-cost.mts (per-widget DOM nodes, fibers and
heaviest components), measure-board-paint.mts (image requests, bytes, style/layout/task
time, layout objects), measure-widget-overdraw.mjs (rendered versus visible rows).
Closes the largest unexplained item in the peak breakdown, then closes it off as a lever.
Attribution: peak_malloced_memory goes 4.5 -> 42.1 MiB the moment a board's modules
compile, while current malloced_memory stays at 1.0 - memory V8 takes to compile, frees
back to malloc, and musl never returns to the kernel. Confirmed by re-reading the
--jitless runs, where the spike never occurs at all (4.5 MiB at every stage).
--jitless is unusable because it under-renders the page, but V8 can cap the optimisation
tier instead of removing codegen. --max-opt is rejected by NODE_OPTIONS and accepted on
the command line, so it was wired behind HOMARR_MAX_OPT and A/B'd on one image:
peak anon 276.0 -> 282.8 MiB (+2%)
boot idle 64.0 -> 72.3 MiB (+13%)
board loaded 186.1 -> 179.2 MiB (-4%)
4 tabs 270.8 -> 238.5 MiB (-12%)
after soak 199.0 -> 214.3 MiB (+8%)
CPU 28.1 -> 27.8 s (unchanged)
Null result. Render integrity was verified explicitly this time - listeners 7,784 in all
four runs, DOM nodes within 40 - so unlike jitless the page rendered fully and the
comparison is valid.
The mechanism explains it: measured under --max-opt=2, peak_malloced_memory at board load
is 42.1 MiB, identical to Turbofan. Maglev's zones are just as large. Capping the tier does
not reduce zone memory; only not compiling does, and that breaks rendering.
So this 38 MiB is not reachable through V8 configuration. It scales with how much
JavaScript V8 must compile, which returns it to the single remaining server-side lever:
ship less server code. Reverted.
Two things this settles.
First, the end-to-end number. Every Phase 2 result so far was an incremental A/B against
the build immediately before it, which cannot be summed. Rebuilt the merge-base
(06724cfeb) from a clean worktree and ran 2+2 alternated against HEAD, reporting cgroup
anonymous memory - the figure that counts against a container limit and cannot be
reclaimed:
stage before after
peak anon 376.6 -> 279.3 MiB (-26%)
boot idle 130.5 -> 62.4 MiB (-52%)
after sign-in 247.4 -> 142.9 MiB (-42%)
dashboard 293.4 -> 170.4 MiB (-42%)
4 tabs 366.5 -> 258.2 MiB (-30%)
after soak 323.7 -> 200.8 MiB (-38%)
peak node RSS 430.7 -> 336.2 MiB (-22%)
No before/after range overlaps at any stage. Costs: CPU 23.5 -> 27.7s (+18%, solid,
non-overlapping). Median board load reads +30% but is NOT established - individual loads
were [506,383,1233] and [848,431,574] before against [780,808,515] and [621,579,1004]
after, heavily overlapping with no cold-then-warm pattern, so six samples per side cannot
separate a real regression from the variance.
compare-stress-runs.mjs now leads with anonMiB, which the harness recorded all along but
the report was ignoring in favour of memory.current - a figure that includes page cache
and swings 60+ MiB between runs of a byte-identical image.
Second, jemalloc. The largest attributed-but-unfixed item is ~38 MiB of V8 zone memory
that musl frees to its own pools and never returns to the kernel, so an allocator that
decays spans back to the OS looked like the answer. Tested on the byte-identical image
with only HOMARR_MALLOC differing, asserting the "Using jemalloc" log line:
boot idle 62.4 -> 80.3 MiB (+29%)
after soak 215.5 -> 279.6 MiB (+30%)
4 tabs 259.9 -> 319.6 MiB (+23%)
CPU 28.0 -> 24.5 s (-13%)
The hypothesis was backwards. jemalloc keeps more freed memory on purpose - thread caches
plus time-based dirty-page decay - and spends it on speed. 13% cheaper CPU, 18-30% more
resident memory. Reverted, and it closes off "swap the allocator" as a RAM direction:
musl is already the frugal choice here.
Second correction in the same area. Having found that three Beszel widgets account for
82% of idle DOM churn, the obvious fix was to batch their subscription updates instead
of calling setState per record. Implemented, built, measured:
DOM mutations / 45s 733 -> 748
React commits / 45s 55 -> 55
Identical. Beszel emits roughly one record per second and the flush interval was also
one second, so each flush batched exactly one record. Coalescing at the arrival rate
does nothing - arithmetic that should have come before the implementation.
That also exposes a unit error in the previous commit's numbers: DOM mutations are not
re-renders. The real rates are 1.22 React commits/sec for the whole board and 13.3 DOM
mutations per commit, because a chart redraw touches many nodes. The stats widget's
"10.4 mutations/sec" is ~0.8 renders/sec - three live-monitoring widgets each refreshing
about once a second, which is what they exist to do.
So 73 commits/min is not pathological, it is the feature working, and there is no defect
to fix here. Reverted rather than shipped: keeping speculative complexity that measured
zero - an extra timer plus unmount and system-change teardown - is the trade this work
rejects everywhere else.
What survives is narrower: idle churn is widget-driven rather than app-shell driven, and
legitimate. The 650 MB browser heap therefore comes from interaction and multi-tab use,
not idle churn, and remains unexplained.
Corrects the largest wrong claim in this work. Earlier rounds reported that ~80% of
idle React commits entered at Next's internal HeadManagerContext and that "the App
Router root re-renders about once a second", and named it the highest-value next step.
Both are false. Measuring the same idle churn against boards of different sizes:
board items idle commits/min
hey 0 0.0
footbol 3 0.0
default 28 76.0
A 3-item board commits zero times per minute. If the router re-rendered on a timer,
every board would churn regardless of its contents.
The original attribution was a tooling artefact. measure-rerenders.mts decides a fiber
re-rendered from fiber.actualDuration, which only a React *profiling* build populates.
On a production build every fiber fails that test and both attribution tables come back
empty - and an empty table reads as "nothing re-rendered" rather than "this build cannot
tell you". The harness now says which case it is, and waits for the board container to be
attached rather than visible so a zero-item board can be measured at all - that control
is what disproved the claim.
Adds measure-widget-churn.mts, which attributes churn on a production build via DOM
mutations keyed to the nearest [data-type="item"] ancestor. On the 28-item board, idle
for 45s:
470 beszelSystemStats 64%
77 beszelSystemTable 11%
52 beszelSystemGrid 7%
22 dockerContainers 3%
93 (outside any widget) 13%
Three Beszel widgets are 82% of all idle churn, one of them 64% on its own at 10.4
mutations per second.
Also logs what Next.js itself costs, since it was assumed to be the problem: its loaded
source is a flat 3.47 MiB (364 framework files + 32 vendored) and does not grow at all
between boot and a fully loaded multi-tab session, while Homarr's own chunks grow from
3.73 to 15.44 MiB. Next is 16.1 MiB on disk; the largest things it vendors are its
next-server runtime (1.60), @edge-runtime (0.85), edge-runtime (0.51) and compression
(0.30). The framework is not where this application's memory goes.
Closes the last unexplained part of the peak footprint. The 118.8 MiB 'other
anonymous' smaps bucket is now mostly accounted for: V8 large-object space 34.5,
external buffers 24.6, and ~38 MiB of V8 parser/compiler zone memory that was freed
back to malloc but never returned to the kernel by musl - peak_malloced_memory goes
4.6 -> 42.2 MiB at board load while current malloced_memory stays at 1.0.
Two theories tested and rejected rather than asserted:
- 'Fewer loaded files means less compiler zone memory, so shipping less server JS
pays twice.' Wrong. peak_malloced_memory is identical (42.1-42.5) across the
preloadEntriesOnStart A/B despite 310 fewer files. The spike is a fixed cost of
the board route, not a per-file one. Claim withdrawn.
- Orphaned zlib streams. Next documents this leak class and the fix is absent from
this build, and a peak snapshot showed 15 Gzip objects still present at idle -
which read as accumulation. getActiveResourcesInfo() reports Zlib=0 both at
baseline and after 300 deliberately aborted gzip requests, RSS flat. Snapshot
objects are not live native handles, and nginx buffers responses so Next never
sees the mid-response disconnect that orphans a stream.
Adds a /handles endpoint to the probe, which is what settled the second one.
Records the Phase 2 peak work:
- Corrects the earlier claim that Next preloading all 74 route graphs is
'framework behaviour, not something this codebase sets'. Next documents
experimental.preloadEntriesOnStart for exactly this, found by reading its
memory-usage guide rather than by inference. Includes the deterministic
per-stage table: 310 files and 6.3 MiB of source stay unloaded even after a
full multi-tab session, so the cost is not deferred, it is never paid.
- Rejects --jitless with evidence. Code space really does go 11.6 -> 0.0 MiB and
peak RSS looked 24% better, but the flag was applied to the server while the
browser's listener count fell 80% and board load got faster, which is
impossible. The page under-rendered; the win was an app doing less work. The
tell generalises: a server-side change moving a client-side metric means the
workload changed and the comparison is void.
- Notes that --optimize-for-size, --no-opt and --lite-mode are rejected outright
by Node in NODE_OPTIONS, and that the rest of Next's memory guide covers
build-time memory under Webpack, which does not apply to this Turbopack build.
Next loads the module graph for all 74 app routes when the server starts. Measured
here: 1,114 files and 24 MiB of source resident before the first request, including
one shared 16 MiB subtree behind the API routes that reaches every integration
client. A dashboard serves a handful of routes per session.
`experimental.preloadEntriesOnStart: false` is Next's documented switch for this.
An earlier round of this work recorded route preloading as "framework behaviour, not
something this codebase sets" - that was wrong, and the findings doc is corrected.
Deterministic per-stage measurements, identical across both runs on each side:
stage source loaded files heapUsed
boot idle 24.0 -> 10.6 MiB 1114 -> 706 70.7 -> 44.8 (-37%)
post-login 27.9 -> 19.3 MiB 1215 -> 884 123.8 -> 109.8
board loaded 29.0 -> 21.7 MiB 1323 -> 1006 (run-to-run noise)
multi-tab 29.2 -> 22.9 MiB 1325 -> 1015 139.7 -> 127.8
post-soak 29.2 -> 22.9 MiB 1325 -> 1015 142.5 -> 127.4
The important part is the last two rows: 310 files and 6.3 MiB of source stay
unloaded even after a full multi-tab session, so this is not merely deferred cost -
routes the session never visits are never paid for at all. Live heap is lower at
every stage, by 26 MiB at idle and 12-15 MiB under load and at rest.
Peak node RSS does not move (350.3 -> 346.1, spreads overlap). V8 sizes its heap
against available headroom, so a smaller live set does not shrink peak RSS; what
improves is idle footprint and how much room is left before the heap cap. Median
board load is +3% with overlapping spreads, so the first-hit cost this trades for
did not show at this workload.
Full E2E suite passes 8/8 against the built image.
dockerode/lib/docker.js required './session' at module scope, which pulls in
@grpc/grpc-js and @grpc/proto-loader - 112 files. Its single call site is inside
Docker.prototype.buildImage, the BuildKit path. Homarr lists, starts and stops
containers and never builds images, so every install with a Docker widget paid for
a gRPC stack it cannot reach.
Measured with a forced GC, in context rather than in a bare process: requiring
dockerode goes from +26.6 MiB RSS / 5.2 MiB heap to +20.6 MiB / 2.7 MiB, with grpc
and protobufjs files going 63 and 49 to zero.
The bare-process figure was +24.1 MiB, and quoting that would have overstated the
win by 4x - most of it overlaps with what dockerode loads anyway. The patch comment
records the in-context number.
mysql2 (77 files) and pg (13 files) load on every SQLite install because
db/drivers/index.ts imports all three wrappers statically. Re-measured with a
forced GC between readings, the prize is 21.4 MiB RSS - not the ~30 MiB the naive
sum implied, and drizzle-orm's 124.3 MiB in the require-cost sweep was almost
entirely that same no-GC artefact.
Three attempts, each caught by booting the real image rather than by typecheck:
createRequire(import.meta.url) is undefined in the CommonJS migrate.cjs bundle and
broke startup for every dialect; a dynamic require resolves to a .cjs build Next
never traces; and leaving drizzle's dialect static defeats the purpose because
drizzle-orm/mysql2 itself imports mysql2.
The two remaining routes are worse than the problem - a second copy of drizzle's
core alongside the ESM one, or hardcoding drizzle's internal file layout so an
upgrade breaks startup for MySQL and Postgres. Backed out; the structural blocker
is that createDb must stay synchronous.
CodeQL flagged `key.replace("|", ...)` as js/incomplete-sanitization (high): it
replaces only the first occurrence. Here that was intentional - the key is
`functionName|url:line` and only the first separator matters - but written as a
replace it reads as a broken sanitiser, and it would mangle any URL containing a
pipe. Splitting on indexOf says what it means. Path trimming now uses replaceAll,
which is what was meant there too.
Profiling at the memory high-water mark - not at idle, which is what every earlier
round measured - found the largest single retainer in every snapshot was one V8
Context holding a 16.00 MiB ArrayBuffer. Expanding it through the dominator tree
named it: ssh2's crypto module, whose POLY1305_WASM_MODULE instantiates a
WebAssembly memory for ChaCha20-Poly1305 as soon as it is required.
Homarr has no SSH feature and does not depend on ssh2. The chain is
dockerode -> docker-modem -> ssh2, and docker-modem/lib/modem.js requires './ssh'
at module scope while using it in exactly one place, already guarded by
`protocol === 'ssh'`. A pnpm patch moves the require inside that branch. SSH-based
Docker hosts still work - require() is cached, so they pay it once - and every
socket-based install, which is everyone else, never loads it.
Measured over four alternated runs on a restored real backup, external memory:
before after
board loaded 25.4, 25.1 8.3, 11.2
nav-01 28.5, 25.4 10.1, 8.6
multi-tab 31.1, 35.8 13.0, 10.0
post-soak 21.4, 21.5 5.5, 7.2
Consistently -15 to -18 MiB, ranges non-overlapping, and 30 fewer files loaded
(1470 -> 1440).
Peak RSS does NOT improve, and that is worth stating plainly: V8 sizes its heap
against available headroom, so freeing native memory lets it grow further before
collecting. What changes is the composition - a fixed 16 MiB of unusable WASM
buffer becomes headroom the app can actually use, which matters under a container
memory limit but does not move the peak number.
Also adds the instruments this needed. A heap snapshot can never explain a peak,
because V8 runs a full GC before serialising one and reports only survivors:
- memory-probe.cjs gains in-process allocation sampling, on-demand snapshots via
v8.writeHeapSnapshot, an smaps endpoint, and require-graph recording.
- analyze-allocations.mjs diffs two stages to isolate what one stage allocated.
- analyze-smaps-stages.mjs splits each stage's RSS into JS heap, JIT, native and
file-backed pages.
- analyze-heap-dominators.mjs can now expand the largest retainer, which is what
identified ssh2. A plain graph walk escapes through __proto__ and is useless.
One trap recorded in the findings: taking the peak snapshot over CDP delivers it as
JS string chunks and added ~134 MiB of ArrayBuffers to the process being measured,
producing a fake 693 MiB reading. v8.writeHeapSnapshot streams from C++ instead, and
the probe now reports its own observerCostMiB so the cost of looking is visible.
The live site has >100k events for one event name, so the paged path logs that its
counts are a lower bound. The previous code had the identical 100k ceiling - the
46.3 MiB body was that ceiling being hit - but truncated in silence. The ceiling is
unchanged; it is now visible.
A flat module cache says 24 MiB of source is resident before the first request, but not
what pulled any of it in. Recording parent -> child as the CommonJS loader resolves it
answers that, and the answer redirects the remaining 'ship less server JS' work:
- Next loads all 74 app route bundles at startup, not lazily.
- One 16.10 MiB / 460-file subtree is reached by the tRPC/API routes AND by
instrumentation, so instrumentation's exclusive cost is 0 bytes - deferring it would
free nothing, because Next preloads /api/[...trpc] anyway.
- Only 1.10 MiB is common to every route, so this is the API surface pulling in every
integration, not a universal barrel.
Profiling the container end to end found one allocation site responsible for
most of the peak: a single 46.3 MiB JSON body, assembled from 3,946 socket
chunks, inside undici's fullyReadBody. That function holds the chunk list and
the joined copy at the same instant, so a body of size N costs about 2N.
The source was two Umami calls asking for pageSize=100000, with
getMultiEventTimeSeriesAsync running them for every tracked event name
concurrently via Promise.all. getEventNamesAsync was downloading every raw
event of the last 30 days purely to collect a handful of distinct name strings.
- getEventNamesAsync now reads the aggregated /metrics?type=event endpoint.
- getWebsiteEventTimeSeriesAsync pages 5,000 records at a time, folding each
page into the bucket map before requesting the next.
- getMultiEventTimeSeriesAsync runs series sequentially.
Instances that return aggregated {x,y} buckets still answer in one request, an
instance that accepts pageSize but ignores page is detected rather than
double-counted, and hitting the page ceiling logs instead of silently
undercounting.
Alternated runs, 2 per side, on a restored real backup:
peak container 498.9 -> 398.5 MiB (-20%)
peak node RSS 452.4 -> 353.3 MiB (-22%)
peak during 4 tabs 498.5 -> 356.3 MiB (-29%)
peak arrayBuffers 95.8 -> 6.6 MiB (-93%)
largest allocation 46.4 -> 2.4 MiB (-95%)
Costs +4% CPU from the extra round trips; board load latency unchanged.
Post-soak steady state and boot idle are unchanged within run-to-run spread.
Adds the attribution tooling this needed: attribute-memory.mts reconciles
cgroup -> processes -> address-space mappings -> V8 spaces -> loaded code,
memory-probe.cjs is an injectable diagnostics preload (never shipped) that
tracks buffer and fetch allocation sites, analyze-heap-dominators.mjs computes
real retained sizes, and compare-stress-runs.mjs reports alternated runs with
their spread.
Also corrects an earlier claim in PHASE2-FINDINGS.md: the reported "7.86 MiB of
duplicate chunk sources" was an artefact of V8 truncating string contents to
~1 KiB in heap snapshots. There are zero byte-identical duplicates among all
3,055 server JS files. The real doubling is character width - one codepoint
above U+00FF forces a whole source string to two bytes per character, which
costs 9.45 MiB across 80 loaded files.
Adds 6 tests; this integration had none.
Tested the hypothesis that cached base64 images were bloating TanStack Query and
Redis. They are not, measured on a loaded 4-tab session:
base64 data-URLs, server heap 0.01 MiB (2 strings)
base64 blobs, browser heap 0.01 MiB (7 strings)
Redis total 2.17 MB
image bytes in Redis none - image-proxy stores URL + headers only
What the server heap is actually made of (141 MiB used): 41.4 MiB is bundled
JavaScript source retained as strings, ~29% of the heap. The memory is the code, not
cached data — which is why every cache-tuning idea in the fix catalog measured at
zero, and why serverExternalPackages was the only lever that moved idle memory.
7.86 MiB of that is literal duplicate copies: the same module inlined into several
Turbopack chunks and retained once per chunk. @ioredis/commands' command table was in
10 chunks and resident 6 times (2.07 MiB wasted), so it is now external — ioredis is
already external and owns it as a direct dependency, so it stays traced.
Recorded as bundle hygiene, NOT a memory win: idle 89.7 -> 89.4 MiB is inside noise,
and board-loaded/peak deltas were smaller than the run-to-run spread (one "before" run
read 220.6 MiB against 169-173 for identical code).
got and @octokit/request-error duplicate just as badly but were deliberately left
bundled: they are only transitive dependencies, and a check inside the image showed
got is not traced into node_modules, so externalising it would have turned a bundled
module into a require that resolves to nothing — a possible MODULE_NOT_FOUND for zero
measured gain.
analyze-heap-strings.mjs buckets a snapshot's strings by what they are, which is what
made "37 MiB of strings" actionable instead of just alarming.
Verified: 28/28 widgets render, auth control passes, zero module-not-found in the
container log, typecheck clean. pnpm test 1794/1795 with two different Testcontainers
flakes across runs (pi-hole, sabnzbd); both pass 18/18 in isolation.
All previous numbers were spread across rounds with different instruments and
different workloads. This is one instrument, one workload, alternated, 2 runs per
side, merge-base vs HEAD:
server idle 141.9 -> 89.7 MiB (-37%)
server loaded 257.3 -> 173.0 MiB (-33%)
server peak 705.2 -> 568.4 MiB (-19%)
after idle soak 332.0 -> 232.5 MiB (-30%)
JS event listeners 10,126 -> 7,774 (-23%)
React fibers 5,224 -> 3,588 (-31%)
DOM nodes 9,175 -> 8,284 (-10%)
browser JS heap 55.2 -> 51.1 MiB (-7%)
image 410 -> 398 MB (-3%)
container CPU 22.8 -> 22.8 s unchanged
Recorded prominently rather than buried: under *sustained* 4-tab load the two builds
converge, 368 vs 364 MiB. The wins are idle, steady state and peak. That is why
hammering the app with five tabs did not feel 30% lighter — at saturation the
footprint is live widget data and render churn, which this PR does not change.
Also documents the gaps, since "tested" without them would be misleading: no
multi-hour soak (the #5301 reports are about days), single user, SQLite only, the
DevTools Profiler tab confirmed at bundle level rather than in the extension, and
the App Router re-rendering ~1x/second still unfixed.
verify-context-menu.mts now also covers touch long-press, and no longer asserts on a
fiber total — that counter swung 7,454 to 16,149 between runs of the same build, so
it was noise. probe-fiber-tree.mts is the stable counter and is what the -31% above
comes from.
measure-rerenders.mts counts React commits on an untouched board and attributes each
to the shallowest component that re-rendered, i.e. where the update entered the tree.
What it found: an idle board commits 73-96 times per minute, re-rendering
@mantine/core/Box over 30,000 times in 60s. That is the churn behind the 650 MB
browser peak — transient allocation rather than retention. About 80% of those commits
enter at the same anonymous depth-2 provider, props [value, children], value {appDir}:
Next.js's internal HeadManagerContext, whose value is a fresh object literal on every
App Router render, so each render re-renders the whole app beneath it. The rest enter
at __next_metadata_boundary__ / <title>, pointing the same way.
The trigger is inside Next's router/metadata handling — application code has no
polling router.refresh() — so nothing here fixes it. It is the next thing to chase and
it dominates everything downstream of it.
useTimeAgo called setState on a 1s interval even when the label was unchanged ("3
minutes ago" holds for a full minute), re-rendering its subtree ~60x/minute per
consumer for an identical string. It now compares before setting. Recorded as a
correctness fix and NOT a measured win: idle commits stayed inside their 73-96 range
afterwards, because the app-root churn dominates.
pnpm test 1794/1795 (pre-existing pi-hole Testcontainers flake); typecheck clean
across 37 packages; no new lint errors.
DevTools still reported "Profiling not supported" because the build was never a
profiling one, despite three settings appearing to enable it. Next applies the React
swap through a webpack alias — getReactProfilingInProduction, mapping
'react-dom/client$' to 'react-dom/profiling' — that Turbopack never sees. A Turbopack
resolveAlias does not catch it either: the vendored entry requires
./cjs/react-dom-client.production.js relatively, not through a bare specifier.
The Dockerfile now performs that same swap on the vendored entry when
HOMARR_PROFILING=true, guarded so it fails the build loudly if React's file layout
changes rather than silently producing a normal build again.
Added `pnpm docker:profiling`, which builds and then verifies, because
`docker build . -t homarr:performance` alone yields a non-profiling image and that is
almost certainly what was being run.
verify-react-profiling.mts checks the bundle rather than the runtime: 2 client chunks
carry React's profiling renderer with the flag, 0 without. It replaces a Playwright
probe that could not tell the two apart — React sends a reduced inject payload to a
synthetic devtools hook, byte-identical for both builds, so that probe reported a
false negative for profiling and production alike. The bundle marker discriminates;
confirm the Profiler tab itself with the real extension.
Mantine's Menu mounts Popover + PopoverDropdown + several context providers even
while closed, so a menu per widget was ~13 fibers each for something almost never
opened. WidgetContextMenuProvider now mounts a single menu for the board, anchored
to the cursor; items keep only a contextmenu/long-press listener.
The dropdown takes the section's gridstack ref as a prop rather than reading
SectionContext — the shared menu lives outside any section, and reading it from
context there crashed the board with "useSectionContext must be used within a
SectionContext".
Verified against both builds with scripts/benchmarks/verify-context-menu.mts
(opens on right-click, shows actions, closes on Escape, re-targets to a second
widget):
idle fibers 18,670 -> 16,278 (-12.8%)
fibers after use +4,191 -> +4
JS event listeners 10,128 -> 7,774 (-23%)
DOM nodes 9,189 -> 8,287 (-10%)
browser JS heap 54.1 -> 52.7 MiB
median board load 520 -> 479 ms
container CPU 20.4 -> 20.0 s
Two runs per side, four tabs across two boards. Server memory is unchanged, as
expected for a client-only change. The listener reduction is the point: the 650 MB
session had ~88k listeners and this removes a per-widget source of them.
The gate is validated by passing against unmodified code first. An earlier version
used Playwright's click({ button: "right" }), which emits no contextmenu event
here and so failed shipped code identically — it wrongly condemned a previous
attempt at this optimisation. A gate never run against the un-optimised build
proves only that something changed, not that it still works.
pnpm test 1794/1795 (pre-existing pi-hole Testcontainers flake, 9/9 in isolation);
typecheck clean across 37 packages; 0 lint errors.
probe-fiber-tree.mts walks the live React tree and reports which components are
mounted, what DOM they render and their props. Heap snapshots only give minified
names like `y`, and source maps do not help because the snapshot records runtime
names — props shape plus rendered DOM identifies a component reliably. It injects
a DevTools hook stub before load so React registers every root; climbing from one
DOM node's fiber missed most of the tree (2,093 fibers vs 3,918).
What it found, on a 28-widget board:
- The most-mounted component is @mantine/core/Box (513). It is the primitive every
Mantine component renders through, so it is a symptom of how much is mounted
rather than a bug. Proportions match the 650 MB session snapshot (13% vs 17%),
so the minified `y` there is almost certainly Box.
- 71 Menu -> Popover -> PopoverDropdown triplets are mounted for context menus that
are almost never opened. Isolated with the existing enableRightClickOnWidgets
flag: 3,918 fibers with them, 3,554 without — 364 fibers, 9%, scaling with item
count. The dropdown contents are already lazy; the wrappers are not.
A fix that deferred the Menu until first right-click was reverted: it broke
right-click and inflated the tree to 20,107 fibers, because the replayed
contextmenu event re-entered the arming handler. Doing it properly needs one
board-level menu instead of one per item. The Playwright check that caught this is
in the findings — a fiber count alone would have read as a win.
Also records that the demo account had completed_board_tour = 0, so earlier rounds
measured the first-time-user path; completing it moves the tree by 38 fibers, so
the published numbers stand.
The container-only harness was measuring the wrong thing. Real usage — several
tabs, several boards, clicking into widgets — reaches ~880 MB server and a
650 MB *browser* heap, and the browser was never measured at all.
Profiling build: `docker build --build-arg HOMARR_PROFILING=true . -t
homarr:performance`. React's production bundle strips the hooks DevTools needs,
hence "Profiling not supported". Note reactProductionProfiling in next.config is
read only by the webpack path; under Turbopack it comes from `next build
--profile`, and turbo strips any env var not declared in the task's `env`, so all
three had to line up before the flag actually took effect — verified by markers
present only in React's profiling bundle, and static chunks going 20 -> 84 MB
with source maps.
inspect-container.mts attaches to an already-running container (SIGUSR1 opens
Node's inspector — no rebuild, no restart), reports cgroup/per-process/V8 memory
and can pull a heap snapshot out over CDP. The inspected context has no require
and no dynamic import, so it reads globals and uses CDP domains instead.
analyze-heapsnapshot / analyze-react-fibers / analyze-heap-retainers summarise a
snapshot too large for DevTools to open.
Harness now opens STRESS_BOARDS across STRESS_TABS_PER_BOARD tabs, clicks into
widgets, and records browser heap, DOM nodes and listener count via CDP.
Performance.getMetrics silently returns nothing unless the domain is enabled,
which is why the first run reported a 0 MiB client heap.
Headline moves from single-sample -36% to repetition-backed -34% idle /
-35% loaded (2 before, 3 after, groups non-overlapping).
Round 3 (API-docs SPA lazy-loaded, MCP SDK external) is recorded as NOT a
memory win: every figure lands inside the previous build's spread, so it is
credited only with the deterministic -3 MB image.
Also records the measured cost of the levers not pursued: in-image Redis is
8.1 MiB RSS (the catalog guessed 10-40 MB) and nginx 9.2 MiB RSS / 3.1 PSS,
so neither justifies its deployment risk. next-server is 225.8 of ~236 MiB
at board-loaded, so anything further has to come out of that heap.
@scalar/api-reference-react is a documentation SPA reachable only from one
admin tab, and it renders nothing usable on the server, yet it was compiled
into the server graph for an SSR pass whose output is discarded. It is a
client component already, so next/dynamic with ssr: false is allowed in
place and needs no restructuring. @modelcontextprotocol/sdk is only reachable
through /api/mcp/[transport], so it moves to serverExternalPackages.
Honest accounting: this is an image-size and architecture change, not a
measured memory win. Idle landed at 89.4 MiB against 90.3/91.1/100.5 for the
previous build — the better end of that spread, but inside it, so on one
sample it cannot be claimed as a reduction. What is deterministic is image
size, 401 -> 398 MB. Kept because a docs SPA should not be server-rendered
regardless of what the memory number does.
pnpm typecheck and lint clean; pnpm test 1794/1795 with the known
Testcontainers pi-hole flake, which passes 9/9 in isolation.
Adds the GC-tuning and wider-unbundling numbers, states the CPU and image-size
costs alongside the memory wins, and documents the three assertions that now
prove the measurement runs behind a real session rather than a public page.
Two further wins, both measured on a restored 28-widget board behind a real
demo session (scripts/benchmarks/stress-restore.mts).
--max-semi-space-size=4 in run.sh. --max-old-space-size only bounds the old
generation; V8 also sizes the young generation generously (16 MB per
semi-space) and that slack is pure resident memory. 4 MB cut idle anonymous
memory 120.0 -> 97.2 MiB and peak 710.6 -> 560.2 MiB. It costs CPU, as any
GC-space reduction does — 15.8 -> 17.1 CPU-seconds over the run, ~8% — with
no board-load latency regression (median 594 -> 570 ms). Overridable via
HOMARR_MAX_SEMI_SPACE_SIZE.
serverExternalPackages for the remaining server-only dependencies. Bundling
a package inlines its transitive graph into a chunk that is compiled at boot
even when the code path never runs; leaving it external makes it a lazy
require. Adding winston, drizzle-orm, ical.js, jszip, ldapts, node-unifi,
@kubernetes/client-node and linkedom took idle to 90.3 MiB and board-loaded
209.6 -> 164.6 MiB, with slightly lower CPU (16.3s) and latency (549 ms).
The trade-off is image size: 388 -> 401 MB, because those packages now ship
in node_modules instead of being tree-shaken into chunks.
Cumulative vs the merge-base: idle 141.7 -> 90.3 MiB (-36%), board loaded
257.8 -> 164.6 MiB (-36%), under load 392.7 -> 302.5 MiB (-23%).
The harness now proves the session rather than inferring it: it asserts an
anonymous request to the private board is denied, requires the Auth.js
session cookie after sign-in, and reports whether it had to modify the
restored data at all (now "none"). It also records cgroup CPU seconds and
board-load latency, so a memory win bought with CPU cannot hide.
pnpm test 144/144 files, 1795 tests; typecheck and lint clean.
Removing the three zero-effect changes cost nothing: idle 120.0 MiB,
board loaded 229.5, after stress 357.0 — all within noise of the
untrimmed branch, with the lowest peak of any config measured.
Ablation (PHASE2-FINDINGS.md) attributed the memory win to the
serverExternalPackages line and the integrations barrel diet. These three
measured at exactly zero and were most of the file churn, so they go:
next-cache-handler: orphaned. cacheHandlers and cacheComponents are both
already gone from next.config.ts and nothing imported the package — it
was reachable only through a stale dependency entry.
cache-tags / cache-invalidation: there is no "use cache", cacheTag,
cacheLife or unstable_cache anywhere in app source, so revalidateTag was
firing on tags nothing registers, with no handler to receive them, inside
a catch that swallowed the resulting out-of-scope error. Removing the
calls also removed the SELECTs that existed only to supply a board name
or member list to them, so board.ts, group.ts, user.ts, serverSettings.ts,
section-router.ts and change-search-preferences.ts return to the
merge-base exactly. The live in-process integration data cache is
untouched.
Redis 6 -> 2 client consolidation: no measurable effect at idle (139.3 vs
139.2 MiB) or under load. Reverted, but the pubSub:last:* TTL is kept —
that one is an unbounded-growth fix, not a consolidation.
23 files, -883 lines. Diff vs merge-base: 100 files/3336 insertions ->
82 files/2484. pnpm test 144/144 files, 1795 tests, 0 failures;
typecheck and lint clean; app and Docker image build.
Measured on a populated instance (real backup restored through the
onboarding UI, 28-widget board open) rather than an empty container, and
compared on anonymous memory because memory.current swung 192-261 MiB
for the same image once page cache was counted.
Branch vs merge-base: -24% idle, -11% board loaded, -8% under load.
Cumulative ablation from the baseline image shows where that comes from:
adding mysql2/pg to serverExternalPackages — one line — accounts for the
entire idle win (-21 MiB), because the driver barrel statically imports
all three drivers and bundling inlined mysql2's whole graph into the
boot-loaded server chunks (108 MB of chunks vs 95 MB, and mysql2 absent
from node_modules because it had been inlined). The integrations barrel
diet carries the under-load win but moves idle by 0. Redis 6->2 clients
and the V8 heap cap both move idle by 0 — the cap is a drift ceiling,
not a reduction, since removing it entirely (cap4096) matched the branch
at every stage.
The cache-invalidation layer and next-cache-handler package measure at
exactly zero: no "use cache", cacheTag, cacheLife or unstable_cache
anywhere in app source, and cacheHandlers/cacheComponents are already
gone from next.config.ts, so 66 revalidateTag call sites across ~30
routers are no-ops. Keep/drop decision left open.
Two fixes found by running the harness against a real pair of images:
Login submitted before the form hydrated, so the click was a no-op that
looked identical to bad credentials. Load the page at networkidle and
retry the submit, surfacing any page alert if all attempts fail — the
same race already handled for the onboarding restore toggle.
memory.current counts reclaimable page cache, which moved the "same"
baseline between 192 and 261 MiB across runs. Anonymous memory is now
the headline metric (idle was 144 MiB anon vs 77 MiB of page cache), so
an A/B is not decided by unrelated disk I/O.
Sign-in redirects client-side, so page.waitForURL never reached its
implicit "load" state and every run failed after the restore step even
though authentication had succeeded. Poll window.location instead.
STRESS_NODE_OPTIONS injects NODE_OPTIONS directly, which is what makes
the V8 heap cap attributable: the merge-base run.sh does not read
HOMARR_MAX_OLD_SPACE_SIZE, so the cap can only be A/B'd against that
image by passing the flag in.
Thomas Camlong
ajnart