feat!: prepare Homarr v2 release (#6746)
This commit is contained in:
Thomas authored and 
GitHub committed
1 parent
90c7493f15
commit
38f4b0e1f8
52 files changed
+3446
-4480
No files matched your search
@@ -110,60 +110,3 @@ Agents can install existing widgets without regenerating them:
|
||||
6. Test the installed definition, then use `board_addItem` with kind `customApi` and the returned definition ID.
|
||||
|
||||
Homarr encrypts the plaintext immediately. Homarr does not return the plaintext.
|
||||
|
||||
## Evaluate the copied prompt
|
||||
|
||||
Homarr maintainers can run the same self-contained prompt that the workbench copies through the live generation-quality suite:
|
||||
|
||||
```sh
|
||||
OPENROUTER_API_KEY=... pnpm --filter @homarr/custom-widgets test:ai
|
||||
```
|
||||
|
||||
The manifest suite covers nine complex and public-API prompts. Its deterministic fixture-backed cases include a fake
|
||||
homelab health API, CoinMarketCap's keyless public API, Bored, Agify, and a multi-request Pokédex. Before the configured
|
||||
model judges a result, the candidate must pass Homarr's real clipboard parser, canonical schema, and deterministic
|
||||
checks for documented endpoints, authentication, request triggers, query bindings, response fields, and essential
|
||||
runtime components.
|
||||
|
||||
To exercise the real Assistant lifecycle instead of accepting a returned JSON block, run:
|
||||
|
||||
```sh
|
||||
OPENROUTER_API_KEY=... pnpm --filter @homarr/custom-widgets test:assistant-ai
|
||||
```
|
||||
|
||||
The same suites can target the Homarr provider with a Workshop PocketBase token instead of an OpenRouter key:
|
||||
|
||||
```sh
|
||||
AI_PROVIDER_BASE_URL=https://homarr.dev/api/ai/v1 \
|
||||
AI_PROVIDER_API_KEY=<workshop-token> \
|
||||
AI_PROVIDER_MODEL=homarr/model \
|
||||
pnpm --filter @homarr/custom-widgets test:assistant-ai
|
||||
```
|
||||
|
||||
`AI_PROVIDER_JUDGE_MODEL` can select a separate judge when the configured endpoint offers one. The Homarr provider
|
||||
offers exactly one model, so generation and review both use `homarr/model`.
|
||||
|
||||
The assistant test exposes the same lazy skill, live schema, component catalog, targeted component references,
|
||||
validation, preview-query, and create-from-preview tools as the product. It refuses to save a definition that was not
|
||||
validated exactly or whose preview queries were not all tested. Multi-request fixtures return a different payload for
|
||||
each documented path and require manual-query parameters. Use `--case=pokedex`, `--case=coinmarketcap-keyless`,
|
||||
`--case=bored-activity`, or `--case=agify-name` for a focused run.
|
||||
|
||||
Both suites use the configured provider models for generation and harsh review. They save tool traces, validated
|
||||
manifests, raw reviews, and detailed category scores under
|
||||
`packages/custom-widgets/.ai-evaluations/`. These live tests are maintainer-invoked rather than CI-required because
|
||||
they consume a configured provider account and exercise a nondeterministic model. The repository's ordinary tests
|
||||
cover the lifecycle enforcement, fixture routing, score calculation, and failure behavior without network access.
|
||||
|
||||
Each scenario stops as soon as it passes and has a hard limit of ten generation attempts. Override the endpoint,
|
||||
models, or per-scenario limit with `AI_PROVIDER_BASE_URL`, `AI_PROVIDER_API_KEY`, `AI_PROVIDER_MODEL`,
|
||||
`AI_PROVIDER_JUDGE_MODEL`, and
|
||||
`CUSTOM_WIDGET_AI_MAX_LOOPS`. Generated artifacts are local and ignored by Git; promote a reviewed, validated result
|
||||
into the bundled-widget source deliberately.
|
||||
|
||||
The reference API check uses a real PokeAPI list/detail request and a read-only Portainer container query when `PORTAINER_BASE_URL` and `PORTAINER_API_KEY` are set. Use `pnpm --filter @homarr/custom-widgets test:reference-apis -- --pokeapi-only` for the public PokeAPI check alone. To exercise start, restart, and stop against a disposable container, also set `PORTAINER_TEST_CONTAINER_ID` and the explicit safeguard `PORTAINER_ALLOW_MUTATIONS=1`; the script restores the container's original running state.
|
||||
|
||||
Use `pnpm --filter @homarr/custom-widgets test:reference-apis -- --public-ai-apis-only` to verify the live Bored and
|
||||
CoinMarketCap response shapes used by the assistant fixtures. The same command also verifies Agify when
|
||||
`AGIFY_API_KEY` is configured; otherwise it reports the authenticated check as skipped without weakening the
|
||||
deterministic Agify fixture tests.
|
||||
@@ -21,7 +21,8 @@ export const completeManifest = `{
|
||||
|
||||
:::warning Experimental feature
|
||||
|
||||
This feature is experimental and will probably change. If you encounter bugs, need more capabilities, or find something confusing, tell us in the [official discussion thread](https://github.com/homarr-labs/homarr/discussions/categories/custom-widgets). Your feedback directly shapes what comes next. Happy hacking!
|
||||
This feature is experimental and may change. Share feedback in the
|
||||
[Custom Widgets discussion](https://github.com/homarr-labs/homarr/discussions/categories/custom-widgets).
|
||||
|
||||
:::
|
||||
|
||||
@@ -84,54 +85,35 @@ Mutations remain authenticated and enforce each request's view, modify, or full
|
||||
- [Use MCP and the skills.sh skill](./agent-authoring).
|
||||
- [Correct problems](./troubleshooting).
|
||||
|
||||
The workbench keeps one continuous layout with visual source, request, and option editors. Advanced raw JSON remains available for experts and AI paste. Invalid values stay inline and disable both preview and Save/Create until they are corrected.
|
||||
The workbench provides visual source, request, and option editors, with raw JSON available for advanced use. Invalid
|
||||
values remain marked and disable preview and saving until corrected.
|
||||
|
||||
Exports, imports, the official Workshop client, and MCP validate the same schema. PocketBase stores community
|
||||
submissions without interpreting their content, and Homarr validates them before installation. Widgets saved in the v1
|
||||
format remain visible to administrators with a **Migration required** badge after upgrading. Use **Copy migration
|
||||
prompt**, give the redacted prompt to an LLM, copy its single complete JSON block, then select **Paste migrated widget**
|
||||
from that widget's action menu. Homarr reads the `template` value into the JSX editor automatically. The prompt keeps safe
|
||||
request metadata but deliberately omits the v1 display configuration because legacy free-text and JSX can contain
|
||||
hard-coded credentials. Add a credential-free description of the widget's visible layout and behavior when sending the
|
||||
prompt.
|
||||
Exports, imports, Workshop, and MCP validate the same schema. Homarr also validates community content before
|
||||
installation.
|
||||
|
||||
Homarr keeps the original v1 definition and encrypted secret as a compatibility backup even after the validated v2
|
||||
replacement is created. The v2 record uses the same stable ID and takes precedence. The archived v1 copy stays hidden
|
||||
while the replacement exists; deleting the v2 record surfaces the v1 entry as migration-required again. Automatic
|
||||
legacy deletion is intentionally unavailable during v2 stabilization. Credentials are never included in prompts,
|
||||
logs, or exports.
|
||||
|
||||
For administrators adding widgets while editing a board, the Custom Widgets section includes **Import from
|
||||
Workshop**. Non-administrators do not see the Custom Widget picker or Workshop actions. Selecting **Install widget**
|
||||
opens a two-step modal flow: first review the Workshop entry and its complete source in the read-only editor, then
|
||||
confirm API sources and credentials. After import, the new definition appears in the picker. Its tile waits for the board
|
||||
to be saved before server requests run.
|
||||
Legacy definitions show a **Migration required** badge. Use **Copy migration prompt**, describe the visible layout
|
||||
without secrets, and import the returned v2 JSON with **Paste migrated widget**. Homarr preserves the v1 definition and
|
||||
encrypted secret for rollback; credentials never appear in prompts, logs, or exports.
|
||||
|
||||
## Import and Workshop
|
||||
|
||||
The install flow always links to the corresponding public Workshop entry under the configured Workshop origin. Import confirmation shows origins, authentication modes, network scopes, methods, actions, and permissions. Every source has a suggested URL, but private and loopback sources must be confirmed or replaced for the current Homarr installation. The same screen accepts API keys, tokens, usernames, and passwords; credentials remain optional at install time and missing values stay visible on the Custom Widgets page.
|
||||
|
||||
For example, installing **Tautulli Activity** asks for both the user's Tautulli URL and API key. A stable public source such as PokéAPI keeps its published URL without creating unnecessary setup work.
|
||||
Administrators can select **Import from Workshop** while adding a board item. Review the complete source and requested
|
||||
permissions, then confirm deployment-specific URLs and credentials. Private and loopback URLs must be confirmed or
|
||||
replaced for the current Homarr installation.
|
||||
|
||||
Workshop publishes technically valid widgets immediately; users can vote or report them, and Workshop moderators can
|
||||
delete submissions. The report form expands within the Workshop detail page instead of interrupting the review with a
|
||||
dialog. Community moderation is separate from Homarr administration and is not a security boundary.
|
||||
Publishers should replace personal private-network hosts with a clear service placeholder before publishing. Homarr
|
||||
performs definitive validation again at install time.
|
||||
delete submissions. Community moderation is separate from Homarr administration and is not a security boundary. Review
|
||||
the source before installation and replace personal private-network hosts before publishing.
|
||||
|
||||
Connected agents can search, inspect, and install Workshop widgets with `customWidget_workshopSearch`, `customWidget_workshopGet`, and `customWidget_workshopInstall`. After installation they can set a source URL and credentials with `customWidget_sourceConfigure`, or create a secure user-facing setup URL with `customWidget_configurationRequestUser`. Secret values are never returned through MCP.
|
||||
|
||||
These tools are only exposed to API keys owned by Homarr administrators. Opening and completing a source-configuration
|
||||
URL also requires the same authenticated administrator; the URL is short-lived, single-use, and redacted.
|
||||
Connected agents can also search and install Workshop widgets. See [agent authoring](./agent-authoring) for the secure
|
||||
configuration flow; secret values are never returned through MCP.
|
||||
|
||||
## Reliability and rollback
|
||||
|
||||
Custom Widget response caches and action invalidation use shared Redis state, so multiple Homarr replicas see the same
|
||||
versioned cache generation. Deployments must point every replica at the same Redis service.
|
||||
Multiple Homarr replicas must use the same Redis service so Custom Widget response caches invalidate consistently.
|
||||
|
||||
Before upgrading, back up the database and `SECRET_ENCRYPTION_KEY`; a v1 binary can still read preserved v1 tables
|
||||
during application rollback. Widgets created only in v2 may be unavailable to the rolled-back binary, but do not
|
||||
replace the v1 backup.
|
||||
Before upgrading, back up the database and `SECRET_ENCRYPTION_KEY`. A v1 rollback can read preserved v1 definitions,
|
||||
but widgets created only in v2 may be unavailable.
|
||||
|
||||
## Bundled examples
|
||||
|
||||
|
||||
Reference in new issue
Block a user