fix: harden v2 custom widgets and Workshop (#6480)

This commit is contained in:
Thomas authored and ajnart committed 2026-08-13 19:50:46 +02:00
1 parent f77c47b94b
commit 400127b45c
138 files changed
+5134 -1977

No files matched your search

@@ -4,6 +4,9 @@ An external agent creates one widget at a time with the Homarr MCP server. The a
Homarr does not create widget packs or drafts for multiple widgets.
The MCP API key must belong to a Homarr administrator. Non-administrator keys do not list Custom Widget tools, prompts,
or resources. The same administrator requirement applies to the matching HTTP authoring endpoints.
<a className="button button--primary" href="https://www.skills.sh/homarr-labs/homarr/homarr-custom-widget">
View the Custom Widget Skill on skills.sh
</a>
@@ -67,7 +70,10 @@ These interfaces do not return an archive URL or archive hash.
12. Update the saved definition if the final verification finds a problem.
13. Repeat the procedure for the next widget.
Each source-configuration URL applies to one definition or preview and one source. The user confirms the server URL, network scope, and required credential fields directly in Homarr. The agent receives only request status and never receives plaintext credentials. Preview-scoped values expire with that preview; definition-scoped values persist.
Each source-configuration URL applies to one definition or preview and one source. It is short-lived, single-use, and
can only be opened and completed by an authenticated Homarr administrator. The administrator confirms the server URL,
network scope, and required credential fields directly in Homarr. The agent receives only request status and never
receives plaintext credentials. Preview-scoped values expire with that preview; definition-scoped values persist.
Agents can install existing widgets without regenerating them:
@@ -88,8 +94,11 @@ Homarr maintainers can run the same self-contained prompt that the workbench cop
OPENROUTER_API_KEY=... pnpm --filter @homarr/custom-widgets test:ai
```
The suite uses DeepSeek V4 Pro to author five complex widgets and DeepSeek V4 Flash to judge only widgets that pass Homarr's real parser and canonical schema. It saves every response, validated `widget.json`, `widget.jsx`, and detailed category scores under `packages/custom-widgets/.ai-evaluations/`.
The suite uses DeepSeek V4 Pro to author Pokédex, Portainer, football, Jellyfin, and Home Assistant widgets. Before DeepSeek V4 Flash judges a result, it must pass Homarr's real clipboard parser, canonical schema, and deterministic scenario checks for the documented API routes, authentication, bindings, actions, and essential runtime components. It saves every response, validated widget manifest, and detailed category scores under `packages/custom-widgets/.ai-evaluations/`.
The run has a hard budget of ten generation attempts across all scenarios. Override the models or budget with `OPENROUTER_GENERATOR_MODEL`, `OPENROUTER_JUDGE_MODEL`, and `CUSTOM_WIDGET_AI_MAX_LOOPS`. Generated artifacts are local and ignored by Git; promote a reviewed, validated result into the bundled-widget source deliberately.
Each scenario stops as soon as it passes and has a hard limit of ten generation attempts. Override the models or
per-scenario limit with `OPENROUTER_GENERATOR_MODEL`, `OPENROUTER_JUDGE_MODEL`, and
`CUSTOM_WIDGET_AI_MAX_LOOPS`. Generated artifacts are local and ignored by Git; promote a reviewed, validated result
into the bundled-widget source deliberately.
The reference API check uses a real PokeAPI list/detail request and a read-only Portainer container query when `PORTAINER_BASE_URL` and `PORTAINER_API_KEY` are set. Use `pnpm --filter @homarr/custom-widgets test:reference-apis -- --pokeapi-only` for the public PokeAPI check alone. To exercise start, restart, and stop against a disposable container, also set `PORTAINER_TEST_CONTAINER_ID` and the explicit safeguard `PORTAINER_ALLOW_MUTATIONS=1`; the script restores the container's original running state.