Commit Graph
5430 Commits
Author SHA1 Message Date
Thomas Camlong b3f58cd8aa perf: drop the cache layers that measured at zero
Ablation (PHASE2-FINDINGS.md) attributed the memory win to the
serverExternalPackages line and the integrations barrel diet. These three
measured at exactly zero and were most of the file churn, so they go:

next-cache-handler: orphaned. cacheHandlers and cacheComponents are both
already gone from next.config.ts and nothing imported the package — it
was reachable only through a stale dependency entry.

cache-tags / cache-invalidation: there is no "use cache", cacheTag,
cacheLife or unstable_cache anywhere in app source, so revalidateTag was
firing on tags nothing registers, with no handler to receive them, inside
a catch that swallowed the resulting out-of-scope error. Removing the
calls also removed the SELECTs that existed only to supply a board name
or member list to them, so board.ts, group.ts, user.ts, serverSettings.ts,
section-router.ts and change-search-preferences.ts return to the
merge-base exactly. The live in-process integration data cache is
untouched.

Redis 6 -> 2 client consolidation: no measurable effect at idle (139.3 vs
139.2 MiB) or under load. Reverted, but the pubSub:last:* TTL is kept —
that one is an unbounded-growth fix, not a consolidation.

23 files, -883 lines. Diff vs merge-base: 100 files/3336 insertions ->
82 files/2484. pnpm test 144/144 files, 1795 tests, 0 failures;
typecheck and lint clean; app and Docker image build.
2026-08-09 00:59:41 +02:00
Thomas Camlong 84dcbd8847 docs(perf): attribute the Phase 1 memory win to specific changes
Measured on a populated instance (real backup restored through the
onboarding UI, 28-widget board open) rather than an empty container, and
compared on anonymous memory because memory.current swung 192-261 MiB
for the same image once page cache was counted.

Branch vs merge-base: -24% idle, -11% board loaded, -8% under load.

Cumulative ablation from the baseline image shows where that comes from:
adding mysql2/pg to serverExternalPackages — one line — accounts for the
entire idle win (-21 MiB), because the driver barrel statically imports
all three drivers and bundling inlined mysql2's whole graph into the
boot-loaded server chunks (108 MB of chunks vs 95 MB, and mysql2 absent
from node_modules because it had been inlined). The integrations barrel
diet carries the under-load win but moves idle by 0. Redis 6->2 clients
and the V8 heap cap both move idle by 0 — the cap is a drift ceiling,
not a reduction, since removing it entirely (cap4096) matched the branch
at every stage.

The cache-invalidation layer and next-cache-handler package measure at
exactly zero: no "use cache", cacheTag, cacheLife or unstable_cache
anywhere in app source, and cacheHandlers/cacheComponents are already
gone from next.config.ts, so 66 revalidateTag call sites across ~30
routers are no-ops. Keep/drop decision left open.
2026-08-09 00:44:50 +02:00
Thomas Camlong fb80b3882d test(benchmarks): compare on anon memory, retry pre-hydration login clicks
Two fixes found by running the harness against a real pair of images:

Login submitted before the form hydrated, so the click was a no-op that
looked identical to bad credentials. Load the page at networkidle and
retry the submit, surfacing any page alert if all attempts fail — the
same race already handled for the onboarding restore toggle.

memory.current counts reclaimable page cache, which moved the "same"
baseline between 192 and 261 MiB across runs. Anonymous memory is now
the headline metric (idle was 144 MiB anon vs 77 MiB of page cache), so
an A/B is not decided by unrelated disk I/O.
2026-08-09 00:44:50 +02:00
Thomas Camlong 6363d96a1b test(benchmarks): fix post-login wait, allow V8 flag injection
Sign-in redirects client-side, so page.waitForURL never reached its
implicit "load" state and every run failed after the restore step even
though authentication had succeeded. Poll window.location instead.

STRESS_NODE_OPTIONS injects NODE_OPTIONS directly, which is what makes
the V8 heap cap attributable: the merge-base run.sh does not read
HOMARR_MAX_OLD_SPACE_SIZE, so the cap can only be A/B'd against that
image by passing the flag in.
2026-08-09 00:44:50 +02:00
Thomas Camlong 0d19968131 test(benchmarks): add populated-dashboard memory stress benchmark
Container memory was only ever measured on an empty install, which hides
the cost that actually matters: a real board with real widgets and
integrations loaded.

stress-restore.mts boots a fresh container, restores a supplied backup
through the onboarding restore UI (including the "I understand"
confirmation), signs in, opens the busiest board, waits for every widget
to mount, then hammers navigation and soaks idle while sampling cgroup
v2 memory at each stage.

The backup is passed by path via STRESS_BACKUP_ZIP and never enters the
repo — it carries real integration secrets and an encryption key.
.gitignore now also refuses homarr-backup*.zip and loose *.sqlite.

Baseline reading (merge-base image, 28-item board, all widgets ready):
223 MiB idle -> 448 MiB board loaded -> 688 MiB peak.
2026-08-09 00:44:50 +02:00
ajnart 9187af8882 style: auto-fix lint and format 2026-08-08 11:02:04 +00:00
Thomas Camlong abec79abe8 fix: address CodeRabbit review — typed mocks, robust benchmark script
- Replace `as any` casts in integration-test-connection.spec.ts with
  Pick<Integration, "testConnectionAsync"> + mockResolvedValue
- build-compare.sh: split local declarations (SC2318), clean .next
  between builds with --force, add EXIT trap for safe stash/cleanup
2026-08-08 13:00:42 +02:00
Thomas Camlong b1713a9820 fix: revert Suspense layout wrappers causing E2E hydration errors
Revert layout.tsx, _layout-creator.tsx, manage/layout.tsx,
_theme.tsx, _creator.tsx, mantine.tsx, and login/page.tsx
to origin/dev — the Suspense provider splits caused
"EditMode is required", stream close, and React #419 errors.

Also remove cacheComponents/cacheHandlers from next.config.ts
(incompatible with auth/i18n/dynamic rendering) and fix
Boolean() truthy bug in Redis connection.
2026-08-08 12:40:48 +02:00
Thomas Camlong 3db87d781f fix: address CodeRabbit review issues
- redis-handler: add TLS_CA and USERNAME env support matching core client
- connection.ts: fix Boolean() truthy bug (CI="false" was disabling Redis)
- channel.ts: add in-memory lock for single-instance fallback mode
- channel-subscription-tracker: lazy-init subscriber client
- board.ts: invalidate user cache on board delete for home-board users
- .env.example: document REDIS_CACHE_DB vs DATABASE_INDEX collision risk
- CLI docs: fix docker exec command (remove /bin/bash prefix)
- Docker docs: fix docker rm -f and docker ps -a for stopped containers
2026-08-08 09:17:39 +02:00
Thomas Camlong 51467ded12 docs: remove obsolete X-Frame-Options ALLOW-FROM examples
ALLOW-FROM is not supported by any modern browser. The CSP
frame-ancestors directive is the correct replacement and was
already present in both examples.
2026-08-08 09:09:53 +02:00
Thomas Camlong 38fc9bfd01 chore: remove full-screen loading spinners from boards and manage 2026-08-08 09:09:19 +02:00
Thomas Camlong 9bc1fc397c chore: remove .cursor/goal files from git 2026-08-08 09:08:48 +02:00
Thomas Camlong 9596a83f60 fix: use any cast for mock integration in test-connection spec
The barrel no longer exports PiHoleIntegrationV6 after the dynamic
import refactor. The mock only needs testConnectionAsync, so a type
cast is sufficient.
2026-08-08 09:03:18 +02:00
Thomas Camlong e036cd86d8 perf: move dependencies to build-time static JSON, add build benchmark
- Move getDependenciesAsync filesystem scan to scripts/dump-dependencies.mts
  (runs at build time, outputs static-data/dependencies.json)
- About page now imports dependencies.json like other static data
  (eliminates runtime glob of all package.json files)
- Delete dead getDependenciesAsync + parseDependencyCatalogsAsync from
  package-reader.ts (only getPackageVersion remains)
- Add scripts/benchmarks/build-compare.sh for dev vs branch comparison
2026-08-08 08:58:54 +02:00
Thomas Camlong 6692acbd33 chore: strip formatting noise, delete dead routes, fix CodeRabbit issues
- Remove 179 format-only file changes (docs, migrations, plex, patchmon, CSS)
- Delete .cursor/goal files from git tracking
- Delete 4 dead /api/about routes (unused, data loaded from static JSON)
- Fix resolveAlias to use absolute paths (CodeRabbit)
- Fix Tailwind classes in Mantine app layout fallback (CodeRabbit)
- Return stale data during cache revalidation (CodeRabbit)
2026-08-08 08:45:16 +02:00
Thomas Camlong 95a7d56989 fix: pi-hole test import from barrel to direct path
The barrel no longer exports PiHoleIntegrationV5/V6 after the
dynamic integration import refactor.
2026-08-08 00:41:50 +02:00
Thomas Camlong 7bd5991076 ci: trigger CI run 2026-08-08 00:29:48 +02:00
Thomas Camlong 345a4be4b0 fix: CI build errors + simplify cache handler and data cache
- Remove `export const dynamic` from 4 routes (incompatible with cacheComponents)
- Static import for redis-handler (Turbopack can't resolve relative dynamic imports)
- Simplify redis-handler: Buffer.concat, NonNullable, single connect promise
- Inline single-use helpers in integration-data-cache
- Non-greedy SVG regex to prevent ReDoS
- Inline getDevelopmentServiceAliases in next.config
- Remove restating comments
2026-08-08 00:19:27 +02:00
Thomas Camlong 6f563640ac fix: remove as const from svgSanitizeOptions to fix DOMPurify type compat 2026-08-07 22:12:09 +02:00
Thomas Camlong 4270f2f2f1 fix: address CodeRabbit review — stream tee, cache guard, base64, layout fallback
- BoardLayoutFallback: use HomarrLogoWithTitle (no board context required)
- integration-data-cache: guard cache.set with placeholder identity to
  prevent stale write-back after invalidation + test
- next-cache-handler: memoize the promise (not resolved handler) to
  prevent concurrent Redis connections
- redis-handler: tee ReadableStream before consuming in set(), hand fresh
  branch to each get() caller
- redis-handler: base64-encode body instead of number[] (3-4x smaller)
2026-08-07 22:09:11 +02:00
ajnart 00855d4825 style: auto-fix lint and format 2026-08-07 19:44:19 +00:00
Thomas Camlong 39334569d6 fix: remove notifications from refetch-intervals to match widget definitions
The notifications widget has no refetchInterval in its definition,
so the static polling policy entry was causing a manifest test failure.
2026-08-07 21:42:08 +02:00
Thomas Camlong ffed73ba8f merge: incorporate latest dev changes into phase1-perf branch
Merge origin/dev to pick up service isolation (concurrently dev script,
turbopack aliases, instrumentation simplification), PostHog analytics,
color scheme, and other dev improvements while preserving all Phase 1
performance work (Suspense boundaries, cache components, Redis handler,
integration barrel diet, cache invalidation).
2026-08-07 21:36:48 +02:00
Thomas Camlong 90d9a50227 fix: correct Content-Type for content-detected SVG responses
When looksLikeSvg detects SVG content under a non-SVG MIME type,
set Content-Type to image/svg+xml so browsers render correctly
with X-Content-Type-Options: nosniff.
2026-08-07 21:25:07 +02:00
Thomas Camlong e836d07d4c fix: content-based SVG detection, guard tests, document REDIS_CACHE_DB
- Detect SVG by content (magic bytes) in addition to MIME type to prevent
  sanitization bypass via mislabeled uploads
- Add guard test verifying DOMPurify actually strips dirty payloads
- Add looksLikeSvg detection test
- Document REDIS_CACHE_DB in .env.example
2026-08-07 21:22:25 +02:00
Thomas Camlong 33293ba1cf refactor: simplify Phase 1 infrastructure code
- cache-invalidation: extract isExpectedRevalidateError helper
- integration-data-cache: flatten early-return chain, extract eviction
- redis-handler: extract StoredEntry type, readStreamBody, ensureConnected
- svg-purify: single getOwnPropertyDescriptor lookup per prop
- session-scoped-providers: simplify auth flow
- retryable-loader: use function declarations
- translation/request: replace spread-in-reduce with for loop
2026-08-07 21:04:42 +02:00
Thomas Camlong b3b2d5a022 fix: SVG regex prefers </svg> terminator, add self-closing children test
Use greedy <svg>...</svg> match preferring closing tag over inner />
to prevent truncation at first self-closing child element.
Self-closing root <svg .../> handled as separate alternative.
Added test with <path/> + <circle/> children asserting well-formed output.
2026-08-07 21:00:55 +02:00
Thomas Camlong 67ac2bbc7c fix: Redis expire guard, transferOwnership prev-owner, SVG regex
- Redis get() rejects legacy expire=0 entries and cleans them up
- transferOwnership invalidates previous owner in addition to new
- SVG regex uses non-greedy match and handles self-closing <svg />
- SVG fallback returns empty string instead of HTML wrapper
- Memory channel comment: FIFO eviction, not timestamp-based
2026-08-07 20:57:07 +02:00
Thomas Camlong fbb2eddd87 fix: concurrent stale race, Redis expire leak, missing invalidation, UI Suspense
- Fix integration data cache: set pending synchronously before creating
  fetch to prevent concurrent stale-revalidation from spawning duplicate
  upstream calls
- Redis cache handler: skip persisting entries with expire=0 (dynamic
  entries should not be stored in Redis)
- Add missing invalidation: user.changePassword, group.updateGroup,
  group.transferOwnership
- Move SearchEngineOptimization inside Suspense in root layout
- Add Suspense fallback loader to root layout
- Move searchParams resolution inside Suspense in login page
2026-08-07 20:53:36 +02:00
Thomas Camlong 38ac8f3075 fix: concurrent cache miss, SVG output, narrowed error handling, format
- Fix integration data cache: check pending promise before TTL to
  prevent returning undefined on concurrent cold-cache access
- Add concurrent-miss test (two parallel callers, one fetch)
- SVG sanitizer: extract <svg> from DOMPurify output to strip
  residual HTML wrapper
- Narrow safeRevalidateTag catch: log unexpected errors, silence only
  known "outside request scope" messages
- Run oxfmt across all packages (206 files formatted)
2026-08-07 20:48:17 +02:00
Thomas Camlong b81b90875f fix: revert DB driver to static imports, fix test regressions
- Revert createRequire() for DB drivers — require() can't resolve .ts
  files in Vitest, breaking 49 test suites
- Make cache invalidation fail-safe via try-catch around revalidateTag
  (throws outside Next.js context: tests, tasks service, websocket)
- Fix 6 integration test imports: barrel no longer exports concrete
  classes, point tests at direct source paths
- Fix Redis cache handler lint errors: remove non-null assertions,
  suppress __NEXT_BUILD_ID underscore warning
2026-08-07 20:43:07 +02:00
Thomas 06724cfeb2 ci: checkout repo in bot auto-merge workflows (#6535)
* ci: drop --delete-branch from reconcile auto-merge call

The reconcile job runs without a git checkout, so gh pr merge --delete-branch
fails with 'not a git repository' before enabling auto-merge. Auto-merge
itself needs no local git state; branch cleanup is already handled by the
PR-creating workflows.

* ci: checkout repo in bot auto-merge workflows

gh pr merge fails with 'not a git repository' when no checkout is present,
which prevented the reconcile job from enabling auto-merge. Restore the
checkout the previous approval workflow relied on.
2026-08-07 20:31:56 +02:00
homarr-update-contributors[bot] 131a5558b5 chore: update contributors (#6527) 2026-08-07 20:26:34 +02:00
Thomas b32ad82850 perf: cut cold route compilation and unblock preview images (#6441) 2026-08-07 20:24:26 +02:00
Thomas 622e1dcfa5 ci: drop --delete-branch from reconcile auto-merge call (#6534)
The reconcile job runs without a git checkout, so gh pr merge --delete-branch
fails with 'not a git repository' before enabling auto-merge. Auto-merge
itself needs no local git state; branch cleanup is already handled by the
PR-creating workflows.
2026-08-07 20:12:37 +02:00
homarr-releases[bot] 9674bde758 chore: update bug report template (#6533)
Co-authored-by: Releases Homarr <175486441+homarr-releases[bot]@users.noreply.github.com>
2026-08-07 18:08:24 +00:00
Thomas b479ada14f ci: auto-approve and auto-merge trusted bot pull requests (#6531)
* ci: auto-approve and auto-merge trusted bot pull requests

Replace the approve-only automatic-approval workflow with a combined
approve + enable-auto-merge workflow that also re-runs on every push,
plus a scheduled reconcile job that re-enables auto-merge for prepared
bot PRs if GitHub ever disables it. Bot PRs are gated to the known bot
actors, same-repo head, and base branch dev.

* ci: match bot authors by app slug in reconcile filter

gh pr list reports pull request authors of GitHub Apps as app/<slug>;
adjust the allowlist so the reconcile job actually matches bot PRs.
2026-08-07 20:05:46 +02:00
Releases Homarr bff3383ac9 chore(release): 1.74.0 [skip ci] v1.74.0 2026-08-07 17:41:07 +00:00
homarr-releases[bot] d865733838 chore(release): automatic release v1.74.0 2026-08-07 17:39:49 +00:00
Thomas Camlong 67fcd69eb8 fix: inverted tag expiration in Redis handler + saveItemOptions board name
- Fix critical bug: tag expiration comparison was inverted (< should be >)
  in Redis cache handler, which would prevent revalidateTag from clearing
  stale entries
- Pass board name to invalidateBoardCache in saveItemOptions
- Add clarifying comment about two-tier invalidation helpers
2026-08-07 19:30:58 +02:00
Thomas Camlong aeabfd3521 fix: requestMedia dual-layer cache invalidation
Mirror answerRequest pattern: invalidate both module-level integration
data cache and request-handler caches for media request list/stats.
2026-08-07 19:27:41 +02:00
Thomas Camlong 37930e44cc perf: clean integration barrel — remove 46 concrete class re-exports
- Remove all concrete integration class re-exports from barrel
- Replace instanceof check with kind check in health-monitoring
- Document cache invalidation as preparatory (cacheTag follow-up)
2026-08-07 19:22:32 +02:00
Thomas Camlong 451dc3ce39 fix: address review #4 — board name invalidation, SVG tests, test gaps
- Pass board.name to all invalidateBoardCache calls (permission mutations)
- Align integration create invalidation with update/delete
- Add initial-fetch-failure test for integration-data-cache
- Expand SVG XSS test corpus (javascript:, xlink:href, @import, onbegin)
- Forbid <style> tags in SVG sanitizer to prevent CSS injection
- Fix duplicate export in packages/common/src/index.ts
2026-08-07 19:18:18 +02:00
Thomas Camlong 4361c8dbe2 fix: address Bugbot review #3 — mutation invalidation gaps
- Dual-layer cache invalidation on interact mutations
  (dns-hole, media-requests, smart-home, indexer-manager, anchor-notes)
- importOldmarrConfig: invalidate created boards
- createBoard: invalidateUserCache when auto-setting homeBoardId
- group.savePositions: invalidate user caches for affected members
- integration.requestMedia: invalidate integration data cache
- user.delete: invalidate user cache
2026-08-07 19:14:02 +02:00
Thomas Camlong 0f8362c3da fix: address Bugbot review #2 findings
- Invalidate integration data cache from widget interact mutations
  (downloads, dns-hole, media-requests, indexer-manager, smart-home, anchor-notes)
- Redis handler getExpiration: fetch missing tags from Redis (multi-instance coherence)
- saveItemOptions: invalidate board cache after saving
- Remove unused cacheTags.app
- Fix integration data cache comment accuracy
2026-08-07 19:08:57 +02:00
Thomas Camlong 6b45aca5d5 fix: address Bugbot review findings
- Skip module-level integration cache for multi-instance (REDIS_IS_EXTERNAL)
- Add invalidateBoardCache to permission mutations (prevents future leak)
- Remove unused cacheTags.integrationData
- Replace KEYS with SCAN in Redis cache handler refreshTags()
- Document REDIS_CACHE_DB env var
2026-08-07 19:04:53 +02:00
Thomas Camlong 018f98e318 perf: implement Phase 1 homepage diet via Next.js Cache Components
P1-0: V8 heap cap (HOMARR_MAX_OLD_SPACE_SIZE, default 512MB)
P1-1: Module graph cuts
  - Dynamic integration imports (55 static → lazy loaders)
  - Conditional DB driver loading (only configured driver loaded)
  - Replace jsdom with linkedom for SVG sanitization
  - Delete eager widget barrel (55 import * as → 0)
  - Extract createRetryableLoader to @homarr/common
P1-2: Enable cacheComponents + restructure root layout
  - Suspense boundaries around session-dependent providers
  - ColorSchemeScript replaces server-side cookie reads
  - Board/manage/auth layouts wrapped in Suspense
P1-3: Cache read paths
  - Cache tag registry + invalidation helpers
  - Cache invalidation wired into all mutation routers
  - Translation deep-merge memoized per locale
P1-4: Prerendered shells + loading.tsx files
P1-5: Shared integration data cache (60s TTL, stale-while-revalidate)
P1-6: Redis-backed Next.js cache handler (@homarr/next-cache-handler)
P1-7: Redis optional (memory channel fallback), TTL on pubSub:last:*,
      Redis clients consolidated from 6 to 2
2026-08-07 18:59:47 +02:00
homarr-crowdin[bot] 85fcea3837 chore(lang): updated translations from crowdin (#6500) 2026-08-07 18:22:43 +02:00
Thomas f76a0f240a feat(apps): allow custom URI schemes in application links (#6521) 2026-08-06 21:52:56 +02:00
Thomas 637f03e107 docs(settings): update analytics docs to PostHog (#6525) 2026-08-06 21:47:04 +02:00