fix: harden assistant authoring and UI

This commit is contained in:
ajnart authored and Thomas Camlong committed 2026-08-09 18:09:11 +02:00
1 parent fa0e55cc93
commit edc7b4d55b
51 files changed
+4118 -916

No files matched your search

+42 -22
View File
@@ -33,7 +33,9 @@ on a dashboard. By default it follows the current conversation. Widget settings
conversation; the conversation picker stays hidden until **Pinned conversation** is selected.
The Shift+A panel is non-modal: opening it focuses its composer, but it does not capture focus from the board.
Users can continue typing in an Assistant widget or interacting with other dashboard controls while the panel
is open.
is open. Every visible assistant surface keeps its own draft, attachment list, selection, and focus while sharing
the same persisted conversation. Switching conversations restores the draft that belongs to that surface and
conversation instead of copying text into another widget or the floating panel.
When the assistant is unavailable, its command-menu result remains visible but disabled and explains that
an administrator must configure and enable it.
@@ -121,13 +123,19 @@ Files returned by a compatible model can be downloaded directly from the message
Assistant responses render GitHub-flavored Markdown, including headings, links, lists, task lists, tables,
quotes, inline code, and fenced code blocks. Line breaks are preserved, and Homarr repairs common streamed
responses where a provider returns escaped line breaks or places a bold-label list on one line.
Completed fenced code blocks use language-aware syntax highlighting; a streaming block stays plain until its
content is stable. Safe same-origin links are prefetched and use client-side navigation, so opening a returned
preview, editor, board, or other Homarr item does not unload the assistant. External links keep their normal
external-browser behavior.
Fenced `mermaid` blocks render as diagrams in the current color scheme using Mermaid's strict security mode;
invalid diagrams fall back to their source instead of breaking the response.
When Homarr's icon search returns an image URL, visual result tables render that actual icon in a compact
image column instead of substituting an emoji. Assistant-provided Markdown images accept safe relative,
HTTP, and HTTPS sources and load without forwarding the Homarr page as the referrer.
Images up to 1 MB and text, Markdown, CSV, HTML, XML, or JSON documents up to 350 KB can be attached by
using the paperclip or dropping files onto the composer. A request can contain up to five attachments.
using the paperclip, dropping files onto the composer, or pasting an image with `Cmd/Ctrl + V`. Attachment
previews stay local to the composer where they were added and show upload state, file type, and a removable image
thumbnail or document tile. A request can contain up to five attachments.
Image understanding depends on the selected model; document text is extracted locally by Homarr and sent
as part of the prompt.
@@ -143,24 +151,30 @@ their description, context size and image-input capability, plus the thinking-le
stored on the conversation after a completed request. Thinking mode can be left on **Auto**, disabled, or set
from minimal through high; explicit levels are sent using the AI SDK's provider-neutral reasoning setting and
still depend on support from the chosen model and endpoint. Reasoning text is shown live when the provider
streams it. The **Thinking** disclosure can be opened or closed without interrupting the response.
streams it through assistant-ui's compact chain-of-thought disclosure. The **Thinking** disclosure renders
Markdown, can be opened or closed without interrupting the response, and keeps the user's collapsed state when
another message starts.
Each completed response includes a circular context-window meter. Select it to open request details when
the provider reports them: input, output, cached-input and reasoning tokens, used and remaining context,
time to first output, total duration, output speed, agent-step timing, and cost. OpenRouter-reported cost
is labeled as reported.
While the next response is still streaming, the meter keeps the last completed reading for the same model and
context capacity instead of briefly showing an incorrect zero. It switches to the new request when token usage
arrives; incompatible model capacities are never mixed.
OpenRouter responses request inline usage and routing metadata. When request details are opened, Homarr also
loads the server-side generation record for each model step and briefly polls while OpenRouter finishes
processing it. The details separate Homarr's end-to-end timing from OpenRouter's provider latency,
generation time, and provider throughput, and can include normalized and native token counts, moderation
latency, cache discount, fallback count, service tier, data region, BYOK status, routed provider, generation
identifier, exact cost, and upstream inference cost. Provider credentials remain server-side; the browser
can request only generation identifiers already stored on a conversation owned by the signed-in user and
signed by the Homarr server for that user and conversation. Successful generation lookups are cached
briefly so polling and multi-step requests do not repeatedly query OpenRouter for the same record.
A compact context control beside the model selector gives a quick glance at cumulative conversation tokens and
cost. Its popover keeps the latest complete context reading while a new response starts and lists every request
turn with its model, provider, input, output, cached-input and reasoning tokens, context use, duration, speed,
agent steps, routing details, and cost when reported. Closing and reopening the popover reads persisted
conversation metadata instead of resetting the display to zero.
Each assistant response places a small provider logo in the same stable action toolbar as copy, export,
feedback, and regenerate. Hovering the logo shows the provider, exact model, timing, throughput, and cost without
adding another metadata row to the transcript. Branch navigation occupies its own adjacent group, so switching
between response versions never pushes the response actions off-screen.
OpenRouter responses request inline usage and routing metadata. Opening detailed turn data also loads the
server-side generation record for its model steps and briefly polls while OpenRouter finishes processing it.
The details separate Homarr's end-to-end timing from OpenRouter's provider latency, generation time, and provider
throughput, and can include normalized and native token counts, moderation latency, cache discount, fallback count,
service tier, data region, BYOK status, routed provider, generation identifier, exact cost, and upstream inference
cost. Provider credentials remain server-side; the browser can request only generation identifiers already stored
on a conversation owned by the signed-in user and signed by the Homarr server for that user and conversation.
Successful generation lookups are cached briefly so polling and multi-step requests do not repeatedly query
OpenRouter for the same record.
When a provider omits cost but model discovery includes prompt and completion pricing, Homarr labels the
calculated value as estimated. Missing provider data is shown as not reported rather than inferred.
@@ -193,7 +207,12 @@ Custom-widget tools let an administrator list, inspect, create, and update defin
form's **Design with AI** action sends a focused request into the current conversation, where the assistant
can propose the matching mutation and use the same approval controls. Existing stored secret values are
reported only as configured/not configured and are preserved on update unless replacements are explicitly
provided.
provided. Before authoring, the assistant lazily loads the complete installed Custom Widget skill, including
all bundled schema, runtime, and security references. Multiline JSX uses `templateLines` to avoid fragile streamed
JSON escaping. The assistant validates the candidate, creates a preview, executes every returned preview query,
checks the actual HTTP status and response shape, and iterates before saving. Preview and saved-definition results
include safe Homarr paths that open in the assistant without a page reload. When placement is still a choice, the
assistant uses the structured question UI instead of ending with a prose question.
The shield control in the conversation header can enable automatic approvals for the current conversation.
While enabled, Homarr accepts every affirmative approval step as it arrives: protected mutations are approved,
@@ -207,8 +226,9 @@ When the assistant genuinely needs missing information or a choice, it can pause
question. Each proposed answer is visibly categorized as **Confirm**, **Decline**, or **Option**, so the effect
of automatic approval is unambiguous. Users can select one of the proposed answers or choose **Other** and
enter their own response. While a question is unanswered, its controls stay in a dedicated area immediately
above the composer instead of requiring the user to find the tool call in the transcript. The answered choice
then remains in the conversation history.
above the composer instead of requiring the user to find the tool call in the transcript. The question temporarily
uses the available conversation body, with its own scrolling when necessary, rather than squeezing a long option
list into a short dock beside part of the transcript. The answered choice then remains in the conversation history.
Submitting the answer resumes the same agent run automatically. The minimized activity bar shows the
question or approval that needs attention and opens directly back to the request. While an answer, app
form, or mutation approval is pending, the composer preserves its draft but does not replace the pending
@@ -17,7 +17,9 @@ Install the Agent Skill from the verified Homarr repository:
npx skills add https://github.com/homarr-labs/homarr --skill homarr-custom-widget
```
The portable skill gives concise authoring guidance. The Homarr MCP prompt gives connected agents a retrieval workflow: read the live schema, then fetch only the component documentation needed for the widget.
The portable skill gives concise authoring guidance. For connected agents, Homarr returns the complete `SKILL.md` and
all bundled skill references once in one lazy tool response. The Homarr MCP prompt then gives the agent a
retrieval workflow: read the live schema and fetch only the component documentation needed for the widget.
The workbench's **Copy AI prompt** action is separate. It creates a self-contained prompt below 12,000 characters for an ordinary ChatGPT or Claude conversation. It includes the lean contract, bindings, recommended components, and two polished examples. It never tells an offline model to call MCP tools.
@@ -37,38 +39,55 @@ Repository code cannot give this attribution.
The authoring-prompt tool description contains one sentence. The prompt contains the workflow instructions and resource links.
`GET /api/custom-widgets/skill` and the MCP tool return the concise skill discovery payload:
`GET /api/custom-widgets/skill` and `customWidget_getSkill` return a self-contained structured skill payload:
```ts
{
name: "homarr-custom-widget";
version: string;
skillMd: string;
skillMd: string; // Complete SKILL.md
references: Record<string, string>;
skillsShUrl: string;
sourceUrl: string;
installCommand: string;
}
```
The full installed component catalog stays available through `homarr://custom-widgets/components` and per-component resources. It is deliberately not embedded in the skill or clipboard prompt.
The `homarr://custom-widgets/skill` MCP resource returns the same `SKILL.md` and references once as bundled Markdown.
The component catalog is a compact index of names, categories, safety levels, shared prop names, policies, and example
IDs. Per-component results contain only component-specific props and a reference to shared prop documentation. Use
`customWidget_getSharedProps` once with only the shared prop names needed across the selected components. This avoids
repeating hundreds of global props in every tool result. The catalog is deliberately not embedded in the skill or
clipboard prompt; the skill response still contains the complete authoring, schema, runtime, and security references.
These interfaces do not return an archive URL or archive hash.
## Create a Custom Widget with MCP
1. Get the prompt, live schema, and component catalog.
2. Read the external API documentation.
3. Build one definition.
4. Validate the definition without saving it.
5. Create a preview session.
6. Test the queries and actions.
7. Open the preview URL with browser tools.
8. Inspect the redacted journal and revise the local candidate until it is valid.
9. Create the widget, including any source URLs and credentials the user already supplied.
10. Otherwise, ask the user to configure the saved definition with the ten-minute Homarr source-setup URL.
11. Create a final preview with the saved definition ID so Homarr loads its stored credentials, then test it again.
12. Update the saved definition if the final verification finds a problem.
13. Repeat the procedure for the next widget.
1. Load `customWidget_getSkill`, then get the prompt and live schema.
2. Call `customWidget_getComponentCatalog`, followed by `customWidget_getComponent` for only the relevant installed
components and `customWidget_getExample` when one catalog example matches the interaction pattern. If shared props
are needed, collect their catalog names and call `customWidget_getSharedProps` once for that set.
3. Read the external API documentation.
4. Build one definition, using `templateLines` for multiline JSX tool input.
5. Validate the definition without saving it and repair every reported issue.
6. Create a preview session.
7. Run `customWidget_previewQuery` for every query returned by the preview and inspect its actual status and data shape.
8. Test relevant actions in simulation mode.
9. Open the returned preview path with browser tools.
10. Inspect the redacted journal and revise the local candidate until validation, data, and rendering all work.
11. Create the widget, including any source URLs and credentials the user already supplied.
12. Open its returned management path, and use a structured question when the user must choose whether to place it on a board.
13. Otherwise, ask the user to configure the saved definition with the ten-minute Homarr source-setup URL.
14. Create a final preview with the saved definition ID so Homarr loads its stored credentials, then test it again.
15. Update the saved definition if the final verification finds a problem. `customWidget_update` also accepts
`templateLines` for multiline JSX without resetting omitted options.
16. Repeat the procedure for the next widget.
Successful validation returns a compact summary instead of echoing the full widget and template. Preview creation
returns its session, links, and query checklist without repeating the submitted definition. Keep the candidate locally
and reuse it for the next lifecycle step.
Each source-configuration URL applies to one definition or preview and one source. It is short-lived, single-use, and
can only be opened and completed by an authenticated Homarr administrator. The administrator confirms the server URL,
+33 -14
View File
@@ -195,21 +195,36 @@ An agent creates one `homarr-custom-widget-v2` definition at a time. Start with
The API key owner must be a Homarr administrator. The same rule applies to management, import, migration, preview,
secrets, publication, Workshop installation, and the HTTP authoring resources.
Then, read the live schema and component resources. These resources describe the installed Mantine components.
Load the installed skill before authoring. `customWidget_getSkill` returns the complete `SKILL.md` entrypoint and every
bundled reference in one response, so an MCP client does not need filesystem access or a second documentation fetch.
Then read the live schema and only the component resources required by the design. These resources describe the
installed Mantine components.
1. Call `customWidget_getAuthoringPrompt` and `customWidget_schema`.
2. Build one Custom Widget definition.
3. Call `customWidget_validate` without saving the definition.
4. Call `customWidget_previewCreate`.
5. Test named requests with `customWidget_previewQuery` and `customWidget_previewAction`.
6. Open the returned preview URL.
7. Examine `customWidget_previewJournal`.
8. Revise the local candidate and create another preview session until it is ready.
9. Call `customWidget_create`, supplying any credentials and deployment-specific source URLs the user already provided.
10. If the user must enter the source URL or credentials, call `customWidget_configurationRequestUser` with the new definition ID and wait for completion.
11. Create a final preview with that definition ID so Homarr loads its stored credentials, then test it again.
12. Use `customWidget_update` or `customWidget_templatePatch` for any final saved-definition corrections.
13. Repeat this procedure for the next Custom Widget.
1. Call `customWidget_getSkill`, `customWidget_getAuthoringPrompt`, `customWidget_schema`, and
`customWidget_getComponentCatalog`.
2. Fetch only the relevant installed component documentation with `customWidget_getComponent` and, when useful,
one interaction example with `customWidget_getExample`. Component results contain component-specific props; collect
any needed shared prop names from the compact catalog and fetch them together once with `customWidget_getSharedProps`.
3. Build one Custom Widget definition. Use `templateLines` for generated multiline JSX to avoid a fragile JSON-escaped
template string.
4. Call `customWidget_validate` without saving the definition and repair every returned issue path.
5. Call `customWidget_previewCreate`.
6. Call `customWidget_previewQuery` for every entry in the returned `queries` list. Supply the listed invocation
parameters for manual queries and inspect the real HTTP status and response shape before binding template fields.
7. Test relevant actions with `customWidget_previewAction` in simulation mode.
8. Open the returned `previewPath` or `previewUrl`.
9. Examine `customWidget_previewJournal`.
10. Revise the local candidate and create another preview session until validation, query data, actions, and rendering
are ready.
11. Call `customWidget_create`, supplying any credentials and deployment-specific source URLs the user already provided.
12. Open the returned `managementPath` to review the exact saved definition.
13. If the user must enter the source URL or credentials, call `customWidget_configurationRequestUser` with the new definition ID and wait for completion.
14. Create a final preview with that definition ID so Homarr loads its stored credentials, then test it again.
15. Use `customWidget_update` or `customWidget_templatePatch` for any final saved-definition corrections.
16. Repeat this procedure for the next Custom Widget.
Validation and preview creation return compact lifecycle metadata. They do not echo the submitted widget or duplicate
its template as both a string and line array; the agent should retain and reuse its local candidate.
The agent receives the configuration-request status. The agent does not receive plaintext credentials. A preview-scoped setup URL configures only that temporary preview; use a definition-scoped URL when the values must persist.
@@ -245,6 +260,10 @@ The Custom Widget tools are:
- `customWidget_workshopGet`
- `customWidget_workshopInstall`
- `customWidget_getAuthoringPrompt`
- `customWidget_getComponentCatalog`
- `customWidget_getComponent`
- `customWidget_getSharedProps`
- `customWidget_getExample`
- `customWidget_getSkill`
Custom Widget permissions are not assignable group capabilities. Administrator status is required for the complete
+6 -4
View File
@@ -41,17 +41,19 @@ active conversation silently: select **Open conversation** inside the widget bef
### Conversation features
- Create, switch, rename, and delete conversations.
- Send text and supported file or image attachments.
- Send text and supported file or image attachments by picker, drag and drop, or image paste.
- Mention permitted apps, integrations, boards, widgets, and tools with `@`.
- Use `/` shortcuts for common dashboard tasks.
- Review tool calls, approve mutations, and answer structured assistant questions inline.
- Answer pending structured questions from the dock above the composer without scrolling through the conversation.
- Enable conversation-scoped automatic approvals from the shield control to approve protected mutations, use presented form details, and choose answers categorized as affirmative without individual clicks.
- Edit user messages and copy, regenerate, branch, quote, export, or rate responses.
- Inspect Markdown, sources, generated files, tool results, reasoning, token use, context-window use, timing,
speed, provider routing, and cost.
- Inspect syntax-highlighted Markdown, sources, generated files, tool results, streaming chain-of-thought,
cumulative conversation usage, per-turn token and context use, timing, speed, provider routing, and cost.
- Change the model and reasoning effort for the active conversation.
- Refresh the current board from the conversation header. Approved mutations also refresh visible board data automatically.
The expanded assistant and board widget use the same persisted thread runtime. A response continues in the
background when the widget leaves the viewport or the expanded panel is minimized.
background when the widget leaves the viewport or the expanded panel is minimized. Each visible surface keeps an
independent draft, attachments, selection, and focus, so typing in a board widget cannot move or overwrite the
floating panel composer.