feat!: prepare Homarr v2 release (#6746)
This commit is contained in:
1 parent
90c7493f15
commit
38f4b0e1f8
52 files changed
+3446
-4480
No files matched your search
@@ -110,60 +110,3 @@ Agents can install existing widgets without regenerating them:
|
||||
6. Test the installed definition, then use `board_addItem` with kind `customApi` and the returned definition ID.
|
||||
|
||||
Homarr encrypts the plaintext immediately. Homarr does not return the plaintext.
|
||||
|
||||
## Evaluate the copied prompt
|
||||
|
||||
Homarr maintainers can run the same self-contained prompt that the workbench copies through the live generation-quality suite:
|
||||
|
||||
```sh
|
||||
OPENROUTER_API_KEY=... pnpm --filter @homarr/custom-widgets test:ai
|
||||
```
|
||||
|
||||
The manifest suite covers nine complex and public-API prompts. Its deterministic fixture-backed cases include a fake
|
||||
homelab health API, CoinMarketCap's keyless public API, Bored, Agify, and a multi-request Pokédex. Before the configured
|
||||
model judges a result, the candidate must pass Homarr's real clipboard parser, canonical schema, and deterministic
|
||||
checks for documented endpoints, authentication, request triggers, query bindings, response fields, and essential
|
||||
runtime components.
|
||||
|
||||
To exercise the real Assistant lifecycle instead of accepting a returned JSON block, run:
|
||||
|
||||
```sh
|
||||
OPENROUTER_API_KEY=... pnpm --filter @homarr/custom-widgets test:assistant-ai
|
||||
```
|
||||
|
||||
The same suites can target the Homarr provider with a Workshop PocketBase token instead of an OpenRouter key:
|
||||
|
||||
```sh
|
||||
AI_PROVIDER_BASE_URL=https://homarr.dev/api/ai/v1 \
|
||||
AI_PROVIDER_API_KEY=<workshop-token> \
|
||||
AI_PROVIDER_MODEL=homarr/model \
|
||||
pnpm --filter @homarr/custom-widgets test:assistant-ai
|
||||
```
|
||||
|
||||
`AI_PROVIDER_JUDGE_MODEL` can select a separate judge when the configured endpoint offers one. The Homarr provider
|
||||
offers exactly one model, so generation and review both use `homarr/model`.
|
||||
|
||||
The assistant test exposes the same lazy skill, live schema, component catalog, targeted component references,
|
||||
validation, preview-query, and create-from-preview tools as the product. It refuses to save a definition that was not
|
||||
validated exactly or whose preview queries were not all tested. Multi-request fixtures return a different payload for
|
||||
each documented path and require manual-query parameters. Use `--case=pokedex`, `--case=coinmarketcap-keyless`,
|
||||
`--case=bored-activity`, or `--case=agify-name` for a focused run.
|
||||
|
||||
Both suites use the configured provider models for generation and harsh review. They save tool traces, validated
|
||||
manifests, raw reviews, and detailed category scores under
|
||||
`packages/custom-widgets/.ai-evaluations/`. These live tests are maintainer-invoked rather than CI-required because
|
||||
they consume a configured provider account and exercise a nondeterministic model. The repository's ordinary tests
|
||||
cover the lifecycle enforcement, fixture routing, score calculation, and failure behavior without network access.
|
||||
|
||||
Each scenario stops as soon as it passes and has a hard limit of ten generation attempts. Override the endpoint,
|
||||
models, or per-scenario limit with `AI_PROVIDER_BASE_URL`, `AI_PROVIDER_API_KEY`, `AI_PROVIDER_MODEL`,
|
||||
`AI_PROVIDER_JUDGE_MODEL`, and
|
||||
`CUSTOM_WIDGET_AI_MAX_LOOPS`. Generated artifacts are local and ignored by Git; promote a reviewed, validated result
|
||||
into the bundled-widget source deliberately.
|
||||
|
||||
The reference API check uses a real PokeAPI list/detail request and a read-only Portainer container query when `PORTAINER_BASE_URL` and `PORTAINER_API_KEY` are set. Use `pnpm --filter @homarr/custom-widgets test:reference-apis -- --pokeapi-only` for the public PokeAPI check alone. To exercise start, restart, and stop against a disposable container, also set `PORTAINER_TEST_CONTAINER_ID` and the explicit safeguard `PORTAINER_ALLOW_MUTATIONS=1`; the script restores the container's original running state.
|
||||
|
||||
Use `pnpm --filter @homarr/custom-widgets test:reference-apis -- --public-ai-apis-only` to verify the live Bored and
|
||||
CoinMarketCap response shapes used by the assistant fixtures. The same command also verifies Agify when
|
||||
`AGIFY_API_KEY` is configured; otherwise it reports the authenticated check as skipped without weakening the
|
||||
deterministic Agify fixture tests.
|
||||
Reference in new issue
Block a user