Commit Graph
52 Commits
Author SHA1 Message Date
Thomas Camlong 63e65bbc6c perf(common): stop useTimeAgo re-rendering every second, add re-render profiler
measure-rerenders.mts counts React commits on an untouched board and attributes each
to the shallowest component that re-rendered, i.e. where the update entered the tree.

What it found: an idle board commits 73-96 times per minute, re-rendering
@mantine/core/Box over 30,000 times in 60s. That is the churn behind the 650 MB
browser peak — transient allocation rather than retention. About 80% of those commits
enter at the same anonymous depth-2 provider, props [value, children], value {appDir}:
Next.js's internal HeadManagerContext, whose value is a fresh object literal on every
App Router render, so each render re-renders the whole app beneath it. The rest enter
at __next_metadata_boundary__ / <title>, pointing the same way.

The trigger is inside Next's router/metadata handling — application code has no
polling router.refresh() — so nothing here fixes it. It is the next thing to chase and
it dominates everything downstream of it.

useTimeAgo called setState on a 1s interval even when the label was unchanged ("3
minutes ago" holds for a full minute), re-rendering its subtree ~60x/minute per
consumer for an identical string. It now compares before setting. Recorded as a
correctness fix and NOT a measured win: idle commits stayed inside their 73-96 range
afterwards, because the app-root churn dominates.

pnpm test 1794/1795 (pre-existing pi-hole Testcontainers flake); typecheck clean
across 37 packages; no new lint errors.
2026-08-09 15:18:38 +02:00
Thomas Camlong 605431bc3f fix(perf): actually produce a React profiling build
DevTools still reported "Profiling not supported" because the build was never a
profiling one, despite three settings appearing to enable it. Next applies the React
swap through a webpack alias — getReactProfilingInProduction, mapping
'react-dom/client$' to 'react-dom/profiling' — that Turbopack never sees. A Turbopack
resolveAlias does not catch it either: the vendored entry requires
./cjs/react-dom-client.production.js relatively, not through a bare specifier.

The Dockerfile now performs that same swap on the vendored entry when
HOMARR_PROFILING=true, guarded so it fails the build loudly if React's file layout
changes rather than silently producing a normal build again.

Added `pnpm docker:profiling`, which builds and then verifies, because
`docker build . -t homarr:performance` alone yields a non-profiling image and that is
almost certainly what was being run.

verify-react-profiling.mts checks the bundle rather than the runtime: 2 client chunks
carry React's profiling renderer with the flag, 0 without. It replaces a Playwright
probe that could not tell the two apart — React sends a reduced inject payload to a
synthetic devtools hook, byte-identical for both builds, so that probe reported a
false negative for profiling and production alike. The bundle marker discriminates;
confirm the Profiler tab itself with the real extension.
2026-08-09 13:39:43 +02:00
Thomas Camlong b4be884dfc perf(board): one shared context menu instead of one per widget
Mantine's Menu mounts Popover + PopoverDropdown + several context providers even
while closed, so a menu per widget was ~13 fibers each for something almost never
opened. WidgetContextMenuProvider now mounts a single menu for the board, anchored
to the cursor; items keep only a contextmenu/long-press listener.

The dropdown takes the section's gridstack ref as a prop rather than reading
SectionContext — the shared menu lives outside any section, and reading it from
context there crashed the board with "useSectionContext must be used within a
SectionContext".

Verified against both builds with scripts/benchmarks/verify-context-menu.mts
(opens on right-click, shows actions, closes on Escape, re-targets to a second
widget):

  idle fibers          18,670 -> 16,278  (-12.8%)
  fibers after use     +4,191 -> +4
  JS event listeners   10,128 -> 7,774   (-23%)
  DOM nodes             9,189 -> 8,287   (-10%)
  browser JS heap       54.1  -> 52.7 MiB
  median board load      520  -> 479 ms
  container CPU         20.4  -> 20.0 s

Two runs per side, four tabs across two boards. Server memory is unchanged, as
expected for a client-only change. The listener reduction is the point: the 650 MB
session had ~88k listeners and this removes a per-widget source of them.

The gate is validated by passing against unmodified code first. An earlier version
used Playwright's click({ button: "right" }), which emits no contextmenu event
here and so failed shipped code identically — it wrongly condemned a previous
attempt at this optimisation. A gate never run against the un-optimised build
proves only that something changed, not that it still works.

pnpm test 1794/1795 (pre-existing pi-hole Testcontainers flake, 9/9 in isolation);
typecheck clean across 37 packages; 0 lint errors.
2026-08-09 11:05:03 +02:00
Thomas Camlong fd7bcb2231 test(perf): fiber-tree probe, and the client-side diagnosis it produced
probe-fiber-tree.mts walks the live React tree and reports which components are
mounted, what DOM they render and their props. Heap snapshots only give minified
names like `y`, and source maps do not help because the snapshot records runtime
names — props shape plus rendered DOM identifies a component reliably. It injects
a DevTools hook stub before load so React registers every root; climbing from one
DOM node's fiber missed most of the tree (2,093 fibers vs 3,918).

What it found, on a 28-widget board:

- The most-mounted component is @mantine/core/Box (513). It is the primitive every
  Mantine component renders through, so it is a symptom of how much is mounted
  rather than a bug. Proportions match the 650 MB session snapshot (13% vs 17%),
  so the minified `y` there is almost certainly Box.
- 71 Menu -> Popover -> PopoverDropdown triplets are mounted for context menus that
  are almost never opened. Isolated with the existing enableRightClickOnWidgets
  flag: 3,918 fibers with them, 3,554 without — 364 fibers, 9%, scaling with item
  count. The dropdown contents are already lazy; the wrappers are not.

A fix that deferred the Menu until first right-click was reverted: it broke
right-click and inflated the tree to 20,107 fibers, because the replayed
contextmenu event re-entered the arming handler. Doing it properly needs one
board-level menu instead of one per item. The Playwright check that caught this is
in the findings — a fiber count alone would have read as a win.

Also records that the demo account had completed_board_tour = 0, so earlier rounds
measured the first-time-user path; completing it moves the tree by 38 fibers, so
the published numbers stand.
2026-08-09 10:37:33 +02:00
Thomas Camlong 8f18b17207 test(perf): profiling build, container introspection, client-side measurement
The container-only harness was measuring the wrong thing. Real usage — several
tabs, several boards, clicking into widgets — reaches ~880 MB server and a
650 MB *browser* heap, and the browser was never measured at all.

Profiling build: `docker build --build-arg HOMARR_PROFILING=true . -t
homarr:performance`. React's production bundle strips the hooks DevTools needs,
hence "Profiling not supported". Note reactProductionProfiling in next.config is
read only by the webpack path; under Turbopack it comes from `next build
--profile`, and turbo strips any env var not declared in the task's `env`, so all
three had to line up before the flag actually took effect — verified by markers
present only in React's profiling bundle, and static chunks going 20 -> 84 MB
with source maps.

inspect-container.mts attaches to an already-running container (SIGUSR1 opens
Node's inspector — no rebuild, no restart), reports cgroup/per-process/V8 memory
and can pull a heap snapshot out over CDP. The inspected context has no require
and no dynamic import, so it reads globals and uses CDP domains instead.

analyze-heapsnapshot / analyze-react-fibers / analyze-heap-retainers summarise a
snapshot too large for DevTools to open.

Harness now opens STRESS_BOARDS across STRESS_TABS_PER_BOARD tabs, clicks into
widgets, and records browser heap, DOM nodes and listener count via CDP.
Performance.getMetrics silently returns nothing unless the domain is enabled,
which is why the first run reported a 0 MiB client heap.
2026-08-09 08:21:57 +02:00
Thomas Camlong 79ea987761 perf: bound V8's young generation and unbundle server-only deps
Two further wins, both measured on a restored 28-widget board behind a real
demo session (scripts/benchmarks/stress-restore.mts).

--max-semi-space-size=4 in run.sh. --max-old-space-size only bounds the old
generation; V8 also sizes the young generation generously (16 MB per
semi-space) and that slack is pure resident memory. 4 MB cut idle anonymous
memory 120.0 -> 97.2 MiB and peak 710.6 -> 560.2 MiB. It costs CPU, as any
GC-space reduction does — 15.8 -> 17.1 CPU-seconds over the run, ~8% — with
no board-load latency regression (median 594 -> 570 ms). Overridable via
HOMARR_MAX_SEMI_SPACE_SIZE.

serverExternalPackages for the remaining server-only dependencies. Bundling
a package inlines its transitive graph into a chunk that is compiled at boot
even when the code path never runs; leaving it external makes it a lazy
require. Adding winston, drizzle-orm, ical.js, jszip, ldapts, node-unifi,
@kubernetes/client-node and linkedom took idle to 90.3 MiB and board-loaded
209.6 -> 164.6 MiB, with slightly lower CPU (16.3s) and latency (549 ms).
The trade-off is image size: 388 -> 401 MB, because those packages now ship
in node_modules instead of being tree-shaken into chunks.

Cumulative vs the merge-base: idle 141.7 -> 90.3 MiB (-36%), board loaded
257.8 -> 164.6 MiB (-36%), under load 392.7 -> 302.5 MiB (-23%).

The harness now proves the session rather than inferring it: it asserts an
anonymous request to the private board is denied, requires the Auth.js
session cookie after sign-in, and reports whether it had to modify the
restored data at all (now "none"). It also records cgroup CPU seconds and
board-load latency, so a memory win bought with CPU cannot hide.

pnpm test 144/144 files, 1795 tests; typecheck and lint clean.
2026-08-09 01:43:06 +02:00
Thomas Camlong fb80b3882d test(benchmarks): compare on anon memory, retry pre-hydration login clicks
Two fixes found by running the harness against a real pair of images:

Login submitted before the form hydrated, so the click was a no-op that
looked identical to bad credentials. Load the page at networkidle and
retry the submit, surfacing any page alert if all attempts fail — the
same race already handled for the onboarding restore toggle.

memory.current counts reclaimable page cache, which moved the "same"
baseline between 192 and 261 MiB across runs. Anonymous memory is now
the headline metric (idle was 144 MiB anon vs 77 MiB of page cache), so
an A/B is not decided by unrelated disk I/O.
2026-08-09 00:44:50 +02:00
Thomas Camlong 6363d96a1b test(benchmarks): fix post-login wait, allow V8 flag injection
Sign-in redirects client-side, so page.waitForURL never reached its
implicit "load" state and every run failed after the restore step even
though authentication had succeeded. Poll window.location instead.

STRESS_NODE_OPTIONS injects NODE_OPTIONS directly, which is what makes
the V8 heap cap attributable: the merge-base run.sh does not read
HOMARR_MAX_OLD_SPACE_SIZE, so the cap can only be A/B'd against that
image by passing the flag in.
2026-08-09 00:44:50 +02:00
Thomas Camlong 0d19968131 test(benchmarks): add populated-dashboard memory stress benchmark
Container memory was only ever measured on an empty install, which hides
the cost that actually matters: a real board with real widgets and
integrations loaded.

stress-restore.mts boots a fresh container, restores a supplied backup
through the onboarding restore UI (including the "I understand"
confirmation), signs in, opens the busiest board, waits for every widget
to mount, then hammers navigation and soaks idle while sampling cgroup
v2 memory at each stage.

The backup is passed by path via STRESS_BACKUP_ZIP and never enters the
repo — it carries real integration secrets and an encryption key.
.gitignore now also refuses homarr-backup*.zip and loose *.sqlite.

Baseline reading (merge-base image, 28-item board, all widgets ready):
223 MiB idle -> 448 MiB board loaded -> 688 MiB peak.
2026-08-09 00:44:50 +02:00
Thomas Camlong abec79abe8 fix: address CodeRabbit review — typed mocks, robust benchmark script
- Replace `as any` casts in integration-test-connection.spec.ts with
  Pick<Integration, "testConnectionAsync"> + mockResolvedValue
- build-compare.sh: split local declarations (SC2318), clean .next
  between builds with --force, add EXIT trap for safe stash/cleanup
2026-08-08 13:00:42 +02:00
Thomas Camlong e036cd86d8 perf: move dependencies to build-time static JSON, add build benchmark
- Move getDependenciesAsync filesystem scan to scripts/dump-dependencies.mts
  (runs at build time, outputs static-data/dependencies.json)
- About page now imports dependencies.json like other static data
  (eliminates runtime glob of all package.json files)
- Delete dead getDependenciesAsync + parseDependencyCatalogsAsync from
  package-reader.ts (only getPackageVersion remains)
- Add scripts/benchmarks/build-compare.sh for dev vs branch comparison
2026-08-08 08:58:54 +02:00
Thomas Camlong 6692acbd33 chore: strip formatting noise, delete dead routes, fix CodeRabbit issues
- Remove 179 format-only file changes (docs, migrations, plex, patchmon, CSS)
- Delete .cursor/goal files from git tracking
- Delete 4 dead /api/about routes (unused, data loaded from static JSON)
- Fix resolveAlias to use absolute paths (CodeRabbit)
- Fix Tailwind classes in Mantine app layout fallback (CodeRabbit)
- Return stale data during cache revalidation (CodeRabbit)
2026-08-08 08:45:16 +02:00
Thomas Camlong 018f98e318 perf: implement Phase 1 homepage diet via Next.js Cache Components
P1-0: V8 heap cap (HOMARR_MAX_OLD_SPACE_SIZE, default 512MB)
P1-1: Module graph cuts
  - Dynamic integration imports (55 static → lazy loaders)
  - Conditional DB driver loading (only configured driver loaded)
  - Replace jsdom with linkedom for SVG sanitization
  - Delete eager widget barrel (55 import * as → 0)
  - Extract createRetryableLoader to @homarr/common
P1-2: Enable cacheComponents + restructure root layout
  - Suspense boundaries around session-dependent providers
  - ColorSchemeScript replaces server-side cookie reads
  - Board/manage/auth layouts wrapped in Suspense
P1-3: Cache read paths
  - Cache tag registry + invalidation helpers
  - Cache invalidation wired into all mutation routers
  - Translation deep-merge memoized per locale
P1-4: Prerendered shells + loading.tsx files
P1-5: Shared integration data cache (60s TTL, stale-while-revalidate)
P1-6: Redis-backed Next.js cache handler (@homarr/next-cache-handler)
P1-7: Redis optional (memory channel fallback), TTL on pubSub:last:*,
      Redis clients consolidated from 6 to 2
2026-08-07 18:59:47 +02:00
Thomas Camlong 3f9fb215c6 fix: avoid ambiguous benchmark regex 2026-08-03 14:27:21 +02:00
Thomas Camlong de801f3f00 perf: optimize dashboard loading and builds 2026-08-02 15:58:16 +02:00
ajnart 0b9c4d6375 fix: address performance review feedback 2026-07-29 06:50:13 +02:00
ajnart 440626ba00 test(performance): add authenticated local benchmarks 2026-07-29 06:47:07 +02:00
Thomas ba7718bde5 feat: rework the ci (#6296) 2026-07-13 18:32:48 +02:00
Thomas 5d1f268438 feat: SQLite backup & restore with WASM preview (#5887) 2026-06-10 18:26:17 +02:00
605af19cd8 refactor: replace ESLint + Prettier with OXC (oxlint + oxfmt) (#5741)
Co-authored-by: Meier Lukas <meierschlumpf@gmail.com>
Co-authored-by: Meierschlumpf <63781622+Meierschlumpf@users.noreply.github.com>
2026-05-27 09:37:44 +02:00
Meier Lukas 6feb37dfd4 Merge branch 'main' into next 2026-05-08 23:46:21 +02:00
Borislaw SotnikowandMeier Lukas 4d30a2e8c4 feat(installations): support secrets file for all environment variables (#5125)
Co-authored-by: Meier Lukas <meierschlumpf@gmail.com>
2026-05-08 17:11:08 +02:00
Meier Lukas 7561462594 Merge branch 'main' into next 2026-05-01 22:30:33 +02:00
Meier LukasandCopilot a1aa5505a3 perf(memory): combine 3 node processes into one (#5600)
Co-authored-by: Copilot <copilot@github.com>
2026-05-01 16:35:11 +02:00
Meier Lukas 3d202d409f feat(about): add opencollective contributors (#5481) 2026-04-27 22:35:42 +02:00
Meier Lukas 9ba82133da feat(installations): support docker secrets file for encryption key (#4946) 2026-01-30 17:28:08 +01:00
Meier Lukas 298a96054e refactor(db): move to core package (#4589) 2025-12-17 08:59:52 +01:00
TobiasandMeier Lukas d084dfb470 fix: improve run script (#4609)
Co-authored-by: Meier Lukas <meierschlumpf@gmail.com>
2025-12-08 22:59:59 +01:00
Meier Lukas 6fb32a6061 ci: add on-release workflow to trigger docs-release, update bug-report-template and update security-policy (#3881) 2025-08-20 20:37:22 +02:00
ManuelandMeier Lukas 27bd93fa8a ci: update integrations in readme (#3740)
Co-authored-by: Meier Lukas <meierschlumpf@gmail.com>
2025-08-08 20:14:58 +02:00
Meier Lukas 8e960324bc feat(infra): add external redis (#3639) 2025-07-20 17:13:57 +02:00
Meier Lukas 9398dd983c feat(tasks): allow management of job intervals and disabling them (#3408) 2025-07-03 20:59:26 +02:00
Meier Lukas 6e03e5a40c fix: updated contributors contain unnecessary data (#3124) 2025-05-16 19:57:20 +02:00
Meier Lukas 6c71724e2e ci: add update-contributors workflow (#3014) 2025-05-12 18:01:27 +02:00
Meier Lukas a5e0e8c129 feat: add sigterm support (#2472) 2025-03-02 20:28:11 +01:00
Meier Lukas 8c36c3e36b feat(certificates): handle self signed certificates (#1951)
* wip: add page and loading of certificates in folder

* wip: add certificate addition and removal

* feat: add removal ui for certificates

* feat: migrate integrations to fetch or agent with trusted certificates

* fix: lock file issues

* fix: typecheck issue

* fix: inconsistent package versions

* chore: address pull request feedback

* fix: add missing navigation item and restrict access to page

* chore: address pull request feedback

* fix: inconsistent undici dependency version

* fix: inconsistent undici dependency version
2025-01-17 00:08:40 +01:00
Meier Lukas aeb681a858 feat(docker): add support for pgid and puid env variables (#1759) 2024-12-31 11:36:28 +01:00
Meier Lukas 56b57ad171 fix(docker): replace anonymous docker volume with env variable for encrypting secrets (#1809) 2024-12-31 11:30:29 +01:00
Meier Lukas 1baa9fc74c fix: docker statup permission issue (#1709) 2024-12-18 14:10:43 +01:00
Meier Lukas 15d47d0585 ci: generate auth secret in production (#1681)
* ci: generate auth secret in production

* refactor: remove no longer needed auth-secret from e2e test

* fix: remove static auth secret
2024-12-17 19:10:19 +01:00
Meier Lukas fe4dfc484b fix: adding volume /appdata removes created directories (#1445) 2024-11-07 19:48:45 +01:00
Meier Lukas 52b45d835d feat(docker): add encryption key generation for integration secrets (#1202)
* feat(docker): add encryption key generation for integration secrets

* chore: address pull request feedback

* fix: build failing

* fix: docker build fails because of error when encryption key not defined
2024-10-03 19:59:44 +02:00
Meier Lukas a30be7bc1e feat: add env variable to disable db migrations on startup (#1179) 2024-09-26 18:54:28 +02:00
Meier Lukas cd4619845b feat: add nginx proxy (#1073)
* feat: add nginx proxy

* fix: adjust path for websocket client to connect
2024-09-16 18:10:58 +02:00
Thomas Camlong ff3244d9ac Update run.sh (#498) 2024-05-16 21:26:07 +02:00
Meier Lukas fd44ee48f4 fix: rdb files can not be saved because of permission issues bug: redis can not save RDB files #318 (#472)
* fix: rdb files can not be saved because of permission issues bug: redis can not save RDB files #318

* refactor: move db and redis dumps to /appdata directory
2024-05-12 16:28:44 +02:00
homarr-renovate[bot]andMeier Lukas 1ce6fb0d0f fix(deps): update dependency drizzle-kit to ^0.21.1 (#462)
* fix(deps): update dependency drizzle-kit to ^0.21.0

* fix(deps): update dependency drizzle-kit to ^0.21.1

* fix: migrate to drizzle kit 0.21

* fix: format issues

* feat: add default env variables for db dialect and driver

* fix: issue with driver seems to be fixed

---------

Co-authored-by: homarr-renovate[bot] <158783068+homarr-renovate[bot]@users.noreply.github.com>
Co-authored-by: Meier Lukas <meierschlumpf@gmail.com>
2024-05-10 23:46:01 +02:00
Meier Lukas ad1796dc02 refactor: add dev directory to organize dev files #398 (#408)
* refactor: add dev directory to organize dev files #398

* chore: address pull request feedback
2024-05-01 19:58:58 +02:00
Meier Lukas 036925bf78 feat: add user groups (#376)
* feat: add user groups

* wip: add unit tests

* wip: add more tests and normalized name for creation and update

* test: add unit tests for group router

* fix: type issues, missing mysql schema, rename column creator_id to owner_id

* fix: lint and format issues

* fix: deepsource issues

* fix: forgot to add log message

* fix: build not working

* chore: address pull request feedback

* feat: add mysql migration and fix merge conflicts

* fix: format issue and test issue
2024-04-29 21:46:30 +02:00
Meier Lukas 669c6c8955 config: migrate from nestjs to tasks and websocket server in docker (#316)
* feat: make tasks script run in docker

* feat: make websocket server work in docker

* fix: format issue

* fix: broken lockfile

* fix: non matching typescript versions
2024-04-07 11:32:29 +02:00