Commit Graph
39 Commits
Author SHA1 Message Date
Thomas Camlong bcea4733b9 perf(nextjs): stop preloading every route's module graph at startup
Next loads the module graph for all 74 app routes when the server starts. Measured
here: 1,114 files and 24 MiB of source resident before the first request, including
one shared 16 MiB subtree behind the API routes that reaches every integration
client. A dashboard serves a handful of routes per session.

`experimental.preloadEntriesOnStart: false` is Next's documented switch for this.
An earlier round of this work recorded route preloading as "framework behaviour, not
something this codebase sets" - that was wrong, and the findings doc is corrected.

Deterministic per-stage measurements, identical across both runs on each side:

  stage            source loaded      files        heapUsed
  boot idle       24.0 -> 10.6 MiB   1114 -> 706   70.7 -> 44.8  (-37%)
  post-login      27.9 -> 19.3 MiB   1215 -> 884   123.8 -> 109.8
  board loaded    29.0 -> 21.7 MiB   1323 -> 1006  (run-to-run noise)
  multi-tab       29.2 -> 22.9 MiB   1325 -> 1015  139.7 -> 127.8
  post-soak       29.2 -> 22.9 MiB   1325 -> 1015  142.5 -> 127.4

The important part is the last two rows: 310 files and 6.3 MiB of source stay
unloaded even after a full multi-tab session, so this is not merely deferred cost -
routes the session never visits are never paid for at all. Live heap is lower at
every stage, by 26 MiB at idle and 12-15 MiB under load and at rest.

Peak node RSS does not move (350.3 -> 346.1, spreads overlap). V8 sizes its heap
against available headroom, so a smaller live set does not shrink peak RSS; what
improves is idle footprint and how much room is left before the heap cap. Median
board load is +3% with overlapping spreads, so the first-hit cost this trades for
did not show at this workload.

Full E2E suite passes 8/8 against the built image.
2026-08-10 09:07:16 +02:00
Thomas Camlong f707f64333 perf: externalise @ioredis/commands, and identify what actually holds the memory
Tested the hypothesis that cached base64 images were bloating TanStack Query and
Redis. They are not, measured on a loaded 4-tab session:

  base64 data-URLs, server heap   0.01 MiB (2 strings)
  base64 blobs, browser heap      0.01 MiB (7 strings)
  Redis total                     2.17 MB
  image bytes in Redis            none - image-proxy stores URL + headers only

What the server heap is actually made of (141 MiB used): 41.4 MiB is bundled
JavaScript source retained as strings, ~29% of the heap. The memory is the code, not
cached data — which is why every cache-tuning idea in the fix catalog measured at
zero, and why serverExternalPackages was the only lever that moved idle memory.

7.86 MiB of that is literal duplicate copies: the same module inlined into several
Turbopack chunks and retained once per chunk. @ioredis/commands' command table was in
10 chunks and resident 6 times (2.07 MiB wasted), so it is now external — ioredis is
already external and owns it as a direct dependency, so it stays traced.

Recorded as bundle hygiene, NOT a memory win: idle 89.7 -> 89.4 MiB is inside noise,
and board-loaded/peak deltas were smaller than the run-to-run spread (one "before" run
read 220.6 MiB against 169-173 for identical code).

got and @octokit/request-error duplicate just as badly but were deliberately left
bundled: they are only transitive dependencies, and a check inside the image showed
got is not traced into node_modules, so externalising it would have turned a bundled
module into a require that resolves to nothing — a possible MODULE_NOT_FOUND for zero
measured gain.

analyze-heap-strings.mjs buckets a snapshot's strings by what they are, which is what
made "37 MiB of strings" actionable instead of just alarming.

Verified: 28/28 widgets render, auth control passes, zero module-not-found in the
container log, typecheck clean. pnpm test 1794/1795 with two different Testcontainers
flakes across runs (pi-hole, sabnzbd); both pass 18/18 in isolation.
2026-08-09 16:07:26 +02:00
ajnart 9bebc28102 style: auto-fix lint and format 2026-08-09 11:40:56 +00:00
Thomas Camlong 605431bc3f fix(perf): actually produce a React profiling build
DevTools still reported "Profiling not supported" because the build was never a
profiling one, despite three settings appearing to enable it. Next applies the React
swap through a webpack alias — getReactProfilingInProduction, mapping
'react-dom/client$' to 'react-dom/profiling' — that Turbopack never sees. A Turbopack
resolveAlias does not catch it either: the vendored entry requires
./cjs/react-dom-client.production.js relatively, not through a bare specifier.

The Dockerfile now performs that same swap on the vendored entry when
HOMARR_PROFILING=true, guarded so it fails the build loudly if React's file layout
changes rather than silently producing a normal build again.

Added `pnpm docker:profiling`, which builds and then verifies, because
`docker build . -t homarr:performance` alone yields a non-profiling image and that is
almost certainly what was being run.

verify-react-profiling.mts checks the bundle rather than the runtime: 2 client chunks
carry React's profiling renderer with the flag, 0 without. It replaces a Playwright
probe that could not tell the two apart — React sends a reduced inject payload to a
synthetic devtools hook, byte-identical for both builds, so that probe reported a
false negative for profiling and production alike. The bundle marker discriminates;
confirm the Profiler tab itself with the real extension.
2026-08-09 13:39:43 +02:00
Thomas Camlong 8f18b17207 test(perf): profiling build, container introspection, client-side measurement
The container-only harness was measuring the wrong thing. Real usage — several
tabs, several boards, clicking into widgets — reaches ~880 MB server and a
650 MB *browser* heap, and the browser was never measured at all.

Profiling build: `docker build --build-arg HOMARR_PROFILING=true . -t
homarr:performance`. React's production bundle strips the hooks DevTools needs,
hence "Profiling not supported". Note reactProductionProfiling in next.config is
read only by the webpack path; under Turbopack it comes from `next build
--profile`, and turbo strips any env var not declared in the task's `env`, so all
three had to line up before the flag actually took effect — verified by markers
present only in React's profiling bundle, and static chunks going 20 -> 84 MB
with source maps.

inspect-container.mts attaches to an already-running container (SIGUSR1 opens
Node's inspector — no rebuild, no restart), reports cgroup/per-process/V8 memory
and can pull a heap snapshot out over CDP. The inspected context has no require
and no dynamic import, so it reads globals and uses CDP domains instead.

analyze-heapsnapshot / analyze-react-fibers / analyze-heap-retainers summarise a
snapshot too large for DevTools to open.

Harness now opens STRESS_BOARDS across STRESS_TABS_PER_BOARD tabs, clicks into
widgets, and records browser heap, DOM nodes and listener count via CDP.
Performance.getMetrics silently returns nothing unless the domain is enabled,
which is why the first run reported a 0 MiB client heap.
2026-08-09 08:21:57 +02:00
Thomas Camlong 61d9faffc5 perf: keep the API-docs SPA and MCP SDK out of the server graph
@scalar/api-reference-react is a documentation SPA reachable only from one
admin tab, and it renders nothing usable on the server, yet it was compiled
into the server graph for an SSR pass whose output is discarded. It is a
client component already, so next/dynamic with ssr: false is allowed in
place and needs no restructuring. @modelcontextprotocol/sdk is only reachable
through /api/mcp/[transport], so it moves to serverExternalPackages.

Honest accounting: this is an image-size and architecture change, not a
measured memory win. Idle landed at 89.4 MiB against 90.3/91.1/100.5 for the
previous build — the better end of that spread, but inside it, so on one
sample it cannot be claimed as a reduction. What is deterministic is image
size, 401 -> 398 MB. Kept because a docs SPA should not be server-rendered
regardless of what the memory number does.

pnpm typecheck and lint clean; pnpm test 1794/1795 with the known
Testcontainers pi-hole flake, which passes 9/9 in isolation.
2026-08-09 03:20:28 +02:00
Thomas Camlong 79ea987761 perf: bound V8's young generation and unbundle server-only deps
Two further wins, both measured on a restored 28-widget board behind a real
demo session (scripts/benchmarks/stress-restore.mts).

--max-semi-space-size=4 in run.sh. --max-old-space-size only bounds the old
generation; V8 also sizes the young generation generously (16 MB per
semi-space) and that slack is pure resident memory. 4 MB cut idle anonymous
memory 120.0 -> 97.2 MiB and peak 710.6 -> 560.2 MiB. It costs CPU, as any
GC-space reduction does — 15.8 -> 17.1 CPU-seconds over the run, ~8% — with
no board-load latency regression (median 594 -> 570 ms). Overridable via
HOMARR_MAX_SEMI_SPACE_SIZE.

serverExternalPackages for the remaining server-only dependencies. Bundling
a package inlines its transitive graph into a chunk that is compiled at boot
even when the code path never runs; leaving it external makes it a lazy
require. Adding winston, drizzle-orm, ical.js, jszip, ldapts, node-unifi,
@kubernetes/client-node and linkedom took idle to 90.3 MiB and board-loaded
209.6 -> 164.6 MiB, with slightly lower CPU (16.3s) and latency (549 ms).
The trade-off is image size: 388 -> 401 MB, because those packages now ship
in node_modules instead of being tree-shaken into chunks.

Cumulative vs the merge-base: idle 141.7 -> 90.3 MiB (-36%), board loaded
257.8 -> 164.6 MiB (-36%), under load 392.7 -> 302.5 MiB (-23%).

The harness now proves the session rather than inferring it: it asserts an
anonymous request to the private board is denied, requires the Auth.js
session cookie after sign-in, and reports whether it had to modify the
restored data at all (now "none"). It also records cgroup CPU seconds and
board-load latency, so a memory win bought with CPU cannot hide.

pnpm test 144/144 files, 1795 tests; typecheck and lint clean.
2026-08-09 01:43:06 +02:00
Thomas Camlong b1713a9820 fix: revert Suspense layout wrappers causing E2E hydration errors
Revert layout.tsx, _layout-creator.tsx, manage/layout.tsx,
_theme.tsx, _creator.tsx, mantine.tsx, and login/page.tsx
to origin/dev — the Suspense provider splits caused
"EditMode is required", stream close, and React #419 errors.

Also remove cacheComponents/cacheHandlers from next.config.ts
(incompatible with auth/i18n/dynamic rendering) and fix
Boolean() truthy bug in Redis connection.
2026-08-08 12:40:48 +02:00
Thomas Camlong 6692acbd33 chore: strip formatting noise, delete dead routes, fix CodeRabbit issues
- Remove 179 format-only file changes (docs, migrations, plex, patchmon, CSS)
- Delete .cursor/goal files from git tracking
- Delete 4 dead /api/about routes (unused, data loaded from static JSON)
- Fix resolveAlias to use absolute paths (CodeRabbit)
- Fix Tailwind classes in Mantine app layout fallback (CodeRabbit)
- Return stale data during cache revalidation (CodeRabbit)
2026-08-08 08:45:16 +02:00
Thomas Camlong 345a4be4b0 fix: CI build errors + simplify cache handler and data cache
- Remove `export const dynamic` from 4 routes (incompatible with cacheComponents)
- Static import for redis-handler (Turbopack can't resolve relative dynamic imports)
- Simplify redis-handler: Buffer.concat, NonNullable, single connect promise
- Inline single-use helpers in integration-data-cache
- Non-greedy SVG regex to prevent ReDoS
- Inline getDevelopmentServiceAliases in next.config
- Remove restating comments
2026-08-08 00:19:27 +02:00
Thomas Camlong 018f98e318 perf: implement Phase 1 homepage diet via Next.js Cache Components
P1-0: V8 heap cap (HOMARR_MAX_OLD_SPACE_SIZE, default 512MB)
P1-1: Module graph cuts
  - Dynamic integration imports (55 static → lazy loaders)
  - Conditional DB driver loading (only configured driver loaded)
  - Replace jsdom with linkedom for SVG sanitization
  - Delete eager widget barrel (55 import * as → 0)
  - Extract createRetryableLoader to @homarr/common
P1-2: Enable cacheComponents + restructure root layout
  - Suspense boundaries around session-dependent providers
  - ColorSchemeScript replaces server-side cookie reads
  - Board/manage/auth layouts wrapped in Suspense
P1-3: Cache read paths
  - Cache tag registry + invalidation helpers
  - Cache invalidation wired into all mutation routers
  - Translation deep-merge memoized per locale
P1-4: Prerendered shells + loading.tsx files
P1-5: Shared integration data cache (60s TTL, stale-while-revalidate)
P1-6: Redis-backed Next.js cache handler (@homarr/next-cache-handler)
P1-7: Redis optional (memory channel fallback), TTL on pubSub:last:*,
      Redis clients consolidated from 6 to 2
2026-08-07 18:59:47 +02:00
Thomas Camlong de801f3f00 perf: optimize dashboard loading and builds 2026-08-02 15:58:16 +02:00
ajnart 4ca0b747bb chore(next): upgrade preview and isolate dev services 2026-07-29 06:47:07 +02:00
Thomasandhomarr-renovate[bot] 4d42eb65e4 chore(deps): mega dependency update (#6212)
Co-authored-by: homarr-renovate[bot] <158783068+homarr-renovate[bot]@users.noreply.github.com>
2026-07-06 10:05:51 +02:00
ajnart 60a69e31ef fix: revert production source maps to restore v1.67.0 memory baseline
CodeRabbit review addressed:
- Removed the env-var gating (simpler: just remove source maps entirely)
- CI health check now fails fast if container never becomes healthy

Changes vs origin/main:
- next.config.ts: remove productionBrowserSourceMaps and serverSourceMaps
  (they default to false, restoring pre-v1.68.0 behavior)
- Dockerfile: remove .map copy step and NODE_OPTIONS=--enable-source-maps
- CI: add memory-regression job (<700MB guardrail)

Closes #6123
2026-06-29 15:13:08 +02:00
ajnart 57ef0af4a2 fix: make source maps opt-in to restore v1.67.0 memory baseline
Production source maps (--enable-source-maps) caused a ~50% memory increase
(600MB to 1000MB) in v1.68.0. Node.js loads all .map files into memory when
this flag is enabled, and Next.js server chunks generate hundreds of map files.

Changes:
- Guard productionBrowserSourceMaps and serverSourceMaps behind
  HOMARR_ENABLE_SOURCE_MAPS env var (default off)
- Guard .map copy step and NODE_OPTIONS in Dockerfile behind
  ENABLE_SOURCE_MAPS build arg (default false)
- run.sh checks HOMARR_ENABLE_SOURCE_MAPS before setting --enable-source-maps
- Add Docker-based memory regression CI job (<700MB threshold)

Closes #6123
2026-06-29 10:08:52 +02:00
Matt Van Horn 89380f81dd feat: ship source maps for readable production stack traces (#6031) 2026-06-22 15:22:54 +02:00
Thomas a4db9cf0dc feat(beszel): add Beszel integration with system monitoring widgets (#5900) 2026-06-10 18:27:05 +02:00
Thomas 5d1f268438 feat: SQLite backup & restore with WASM preview (#5887) 2026-06-10 18:26:17 +02:00
Meier LukasandThomas a9855fe8d3 chore(deps): remove deprecated million lint (#5671)
* chore(deps): remove deprecated million lint

* fix: install now missing playwright dependency

* fix: broken lockfile

* fix: broken lockfile

---------

Co-authored-by: Thomas <thomas@ajnart.fr>
2026-05-18 21:42:02 +02:00
Meier Lukas 6feb37dfd4 Merge branch 'main' into next 2026-05-08 23:46:21 +02:00
sharpsalatandsharpsalat 277a509f49 feat: add pwa support (#5530)
* feat: add pwa support

* fix: deepsource warning

* fix: format warning

* fix: lint error

---------

Co-authored-by: sharpsalat <deniska.xrenov23@yandex.ru>
2026-05-07 21:53:17 +02:00
Thomas 5ee84f8cf4 perf: (next branch) reduce idle memory on startup (#5637) 2026-05-06 21:39:26 +00:00
Meier Lukas 298a96054e refactor(db): move to core package (#4589) 2025-12-17 08:59:52 +01:00
Meier Lukas d348abfe4a refactor(logs): move to core package (#4586) 2025-12-16 23:37:44 +01:00
Meier Lukas 10572ca962 fix: disable react-compiler due to issue with mantine-react-table data updates (#4593) 2025-12-05 19:10:23 +01:00
Manuel aaa23f3732 Merge commit from fork
* fix: sanitize user-media svg api endpoint using isomorphic dompurify

* fix: add iframe sandbox to prevent priviledge escalation
2025-11-14 18:00:15 +01:00
homarr-renovate[bot]andMeier Lukas 6ce23a6e97 fix(deps): update nextjs monorepo to v16 (major) (#4363)
Co-authored-by: homarr-renovate[bot] <158783068+homarr-renovate[bot]@users.noreply.github.com>
Co-authored-by: Meier Lukas <meierschlumpf@gmail.com>
2025-11-04 21:26:44 +01:00
Meier Lukas de33439b22 fix(video-widget): hls videos not working (#4015) 2025-09-06 12:29:30 +02:00
Meier Lukas c59cdeda47 fix: csp prevents external videos (#3372) 2025-06-10 22:14:43 +02:00
Meier Lukas 1b3770aafb fix: module not found sshcrypto.node during development (#3349) 2025-06-07 16:19:09 +02:00
Meier Lukas c7a21b1a5f fix: csp prevents external stylesheets (#3337) 2025-06-06 19:58:37 +02:00
Meier Lukas 05e159f566 fix: csp prevents fonts to load (#3225) 2025-05-25 16:39:20 +02:00
09af0a76f6 feat: allow custom protocols for app href (#2575)
Co-authored-by: Manuel <30572287+manuel-rw@users.noreply.github.com>
Co-authored-by: Meier Lukas <meierschlumpf@gmail.com>
2025-05-23 20:19:59 +00:00
homarr-renovate[bot]andMeier Lukas f55d8a9c2e fix(deps): update dependency next-intl to v4 (#2580)
* fix(deps): update dependency next-intl to v4

* fix: typecheck issue

* refactor: implement improvements for next-intl v4

* fix: typecheck issues

* fix: typecheck issue

---------

Co-authored-by: homarr-renovate[bot] <158783068+homarr-renovate[bot]@users.noreply.github.com>
Co-authored-by: Meier Lukas <meierschlumpf@gmail.com>
2025-03-12 18:37:43 +01:00
Meier Lukas 67d48e11d7 feat(logging): add log level env variable (#2299) 2025-02-18 22:54:15 +01:00
Meier Lukas 73f7d885cd fix: docker hosts and ports env variable wrongly used (#2050)
* fix: docker hosts and ports env variable wrongly used

* fix: ci issues
2025-01-22 20:43:54 +01:00
Meier Lukas 1fd3fd8dfb refactor: env validation typescript and common package (#1912) 2025-01-14 19:03:38 +01:00
Meier Lukas 6a68ccfee4 chore(config): change next-config from mjs to typescript (#1861) 2025-01-06 19:59:25 +01:00