feat!: prepare Homarr v2 release (#6746)

This commit is contained in:
Thomas authored and GitHub committed 2026-08-30 22:54:33 +02:00
1 parent 90c7493f15
commit 38f4b0e1f8
52 files changed
+3446 -4480

No files matched your search

@@ -110,60 +110,3 @@ Agents can install existing widgets without regenerating them:
6. Test the installed definition, then use `board_addItem` with kind `customApi` and the returned definition ID.
Homarr encrypts the plaintext immediately. Homarr does not return the plaintext.
## Evaluate the copied prompt
Homarr maintainers can run the same self-contained prompt that the workbench copies through the live generation-quality suite:
```sh
OPENROUTER_API_KEY=... pnpm --filter @homarr/custom-widgets test:ai
```
The manifest suite covers nine complex and public-API prompts. Its deterministic fixture-backed cases include a fake
homelab health API, CoinMarketCap's keyless public API, Bored, Agify, and a multi-request Pokédex. Before the configured
model judges a result, the candidate must pass Homarr's real clipboard parser, canonical schema, and deterministic
checks for documented endpoints, authentication, request triggers, query bindings, response fields, and essential
runtime components.
To exercise the real Assistant lifecycle instead of accepting a returned JSON block, run:
```sh
OPENROUTER_API_KEY=... pnpm --filter @homarr/custom-widgets test:assistant-ai
```
The same suites can target the Homarr provider with a Workshop PocketBase token instead of an OpenRouter key:
```sh
AI_PROVIDER_BASE_URL=https://homarr.dev/api/ai/v1 \
AI_PROVIDER_API_KEY=<workshop-token> \
AI_PROVIDER_MODEL=homarr/model \
pnpm --filter @homarr/custom-widgets test:assistant-ai
```
`AI_PROVIDER_JUDGE_MODEL` can select a separate judge when the configured endpoint offers one. The Homarr provider
offers exactly one model, so generation and review both use `homarr/model`.
The assistant test exposes the same lazy skill, live schema, component catalog, targeted component references,
validation, preview-query, and create-from-preview tools as the product. It refuses to save a definition that was not
validated exactly or whose preview queries were not all tested. Multi-request fixtures return a different payload for
each documented path and require manual-query parameters. Use `--case=pokedex`, `--case=coinmarketcap-keyless`,
`--case=bored-activity`, or `--case=agify-name` for a focused run.
Both suites use the configured provider models for generation and harsh review. They save tool traces, validated
manifests, raw reviews, and detailed category scores under
`packages/custom-widgets/.ai-evaluations/`. These live tests are maintainer-invoked rather than CI-required because
they consume a configured provider account and exercise a nondeterministic model. The repository's ordinary tests
cover the lifecycle enforcement, fixture routing, score calculation, and failure behavior without network access.
Each scenario stops as soon as it passes and has a hard limit of ten generation attempts. Override the endpoint,
models, or per-scenario limit with `AI_PROVIDER_BASE_URL`, `AI_PROVIDER_API_KEY`, `AI_PROVIDER_MODEL`,
`AI_PROVIDER_JUDGE_MODEL`, and
`CUSTOM_WIDGET_AI_MAX_LOOPS`. Generated artifacts are local and ignored by Git; promote a reviewed, validated result
into the bundled-widget source deliberately.
The reference API check uses a real PokeAPI list/detail request and a read-only Portainer container query when `PORTAINER_BASE_URL` and `PORTAINER_API_KEY` are set. Use `pnpm --filter @homarr/custom-widgets test:reference-apis -- --pokeapi-only` for the public PokeAPI check alone. To exercise start, restart, and stop against a disposable container, also set `PORTAINER_TEST_CONTAINER_ID` and the explicit safeguard `PORTAINER_ALLOW_MUTATIONS=1`; the script restores the container's original running state.
Use `pnpm --filter @homarr/custom-widgets test:reference-apis -- --public-ai-apis-only` to verify the live Bored and
CoinMarketCap response shapes used by the assistant fixtures. The same command also verifies Agify when
`AGIFY_API_KEY` is configured; otherwise it reports the authenticated check as skipped without weakening the
deterministic Agify fixture tests.