integrate assistant custom widget workflow

This commit is contained in:
ajnart committed 2026-08-03 15:22:08 +02:00
1 parent abc17e7c2a
commit 5a3022a073
11 files changed
+406 -161

No files matched your search

+10
View File
@@ -39,6 +39,10 @@ Homarr includes presets for OpenRouter, OpenAI, Anthropic Claude, Google Gemini,
DeepSeek, Together AI, Ollama, and LM Studio. Provider icons make the presets easy to identify. A custom
preset connects any other service that implements the OpenAI Chat Completions API.
The provider list also shows **Homarr — Coming soon** as a disabled option. It reserves a clear place for
Homarr's future managed assistant service without making an unavailable backend selectable or changing
the server-side provider configuration today.
The management page presents setup as three compact sections: provider and endpoint, credentials, then
model and availability. A status summary at the top shows whether the connection is ready, whether a key
is protected, which model is selected, and whether users can open the assistant.
@@ -136,6 +140,12 @@ validated input, result, and error state without flooding the conversation with
Common collection and widget results are summarized inline as compact responsive cards, while the full
structured result remains available in the tool card details.
Custom-widget tools let an administrator list, inspect, create, and update definitions. The custom-widget
form's **Design with AI** action sends a focused request into the current conversation, where the assistant
can propose the matching mutation and use the same approval controls. Existing stored secret values are
reported only as configured/not configured and are preserved on update unless replacements are explicitly
provided.
The shield control in the conversation header can enable automatic approvals for the current conversation.
While enabled, Homarr accepts every affirmative approval step as it arrives: protected mutations are approved,
presented app details and board changes are used, and structured questions with an explicitly categorized
@@ -50,9 +50,17 @@ To create a custom widget, navigate to the **Custom Widgets** section in the man
| **Action Button** | Clickable button executing the API request | Triggers: restart, clear cache |
| **Custom JSX** | Custom JSX template with Mantine v9 components | Full layout control |
### AI-Powered JSX Generation
### Design with AI
When editing or creating a custom widget with the **Custom JSX** display type, you'll find a **Copy Prompt** button. Clicking it copies a pre-formatted prompt describing your widget's name, API endpoint, and available Mantine components. Paste this into your preferred AI assistant (ChatGPT, Claude, Copilot, etc.) to generate a JSX template tailored to your data.
The **Design with AI** action is available while creating or editing a custom widget. Describe what the
widget should show and, when [Homarr Assistant](../assistant) is enabled, send the request directly from
the form. The assistant receives the widget schema, the current non-secret configuration, and the latest
test response. It can then propose and run the matching create or update tool after you approve the
mutation in the conversation.
Run **Test** first when possible so the prompt includes the endpoint's real response shape. Saved secret
values are never added to the prompt. **Copy to clipboard** remains available when Homarr Assistant is
disabled or when you prefer to use another AI service such as ChatGPT, Claude, or Gemini.
<details>
<summary>Example: Pokédex widget using Custom JSX</summary>
@@ -59,9 +59,10 @@ import { getAssistantConnectionState } from "./assistant-configuration-state";
type HeaderEntry = { id: number; name: string; value: string };
type CredentialFlow = "idle" | "replace" | "remove";
type AssistantProviderOption = AssistantProvider | "homarr";
const ProviderIcon = ({ providerId, size = 20 }: { providerId: AssistantProvider; size?: number }) => {
const iconUrl = assistantProviderPresets[providerId].iconUrl;
const ProviderIcon = ({ providerId, size = 20 }: { providerId: AssistantProviderOption; size?: number }) => {
const iconUrl = providerId === "homarr" ? "/logo/logo.png" : assistantProviderPresets[providerId].iconUrl;
return iconUrl ? (
<Image src={iconUrl} alt="" aria-hidden w={size} h={size} fit="contain" />
) : (
@@ -187,15 +188,29 @@ export const AssistantConfiguration = () => {
);
const providerOptions = useMemo(
() =>
(["hosted", "local", "custom"] satisfies AssistantProviderCategory[]).map((category) => ({
group: t(`provider.groups.${category}`),
items: assistantProviderIds
(["hosted", "local", "custom"] satisfies AssistantProviderCategory[]).map((category) => {
const items = assistantProviderIds
.filter((providerId) => assistantProviderPresets[providerId].category === category)
.map((providerId) => ({
value: providerId,
label: t(`provider.options.${providerId}.label`),
})),
})),
}));
return {
group: t(`provider.groups.${category}`),
items:
category === "hosted"
? [
{
value: "homarr",
label: t("provider.options.homarr.label"),
disabled: true,
},
...items,
]
: items,
};
}),
[t],
);
const selectedModel = connectionChanged ? undefined : models?.find((model) => model.id === modelId);
@@ -431,9 +446,16 @@ export const AssistantConfiguration = () => {
allowDeselect={false}
leftSection={<ProviderIcon providerId={provider} />}
renderOption={({ option }) => (
<Group gap="sm" wrap="nowrap">
<ProviderIcon providerId={option.value as AssistantProvider} />
<Text size="sm">{option.label}</Text>
<Group gap="sm" wrap="nowrap" justify="space-between" w="100%">
<Group gap="sm" wrap="nowrap">
<ProviderIcon providerId={option.value as AssistantProviderOption} />
<Text size="sm">{option.label}</Text>
</Group>
{option.value === "homarr" && (
<Badge size="xs" variant="light" color="gray">
{t("provider.comingSoon")}
</Badge>
)}
</Group>
)}
/>
@@ -0,0 +1,55 @@
import { describe, expect, test } from "vitest";
import { buildAiPrompt } from "./_copy-ai-prompt-button";
const schema = {
type: "object",
required: ["name", "url", "displayType", "displayConfig"],
};
describe("buildAiPrompt", () => {
test("asks Homarr Assistant to create a definition and includes the user's request", () => {
const prompt = buildAiPrompt(
schema,
'{"status":"online"}',
{
$schema: "homarr-custom-widget-v2",
name: "Status",
url: "https://example.com/status",
},
"Show a compact green status when the service is online.",
);
expect(prompt).toContain("`customWidget_create`");
expect(prompt).toContain("`secrets: []`");
expect(prompt).toContain("do not ask for typed confirmation");
expect(prompt).toContain('```json\n{"status":"online"}\n```');
expect(prompt).toContain('"name": "Status"');
expect(prompt).toContain("Show a compact green status when the service is online.");
});
test("asks Homarr Assistant to update the selected definition without replacing its secrets", () => {
const prompt = buildAiPrompt(
schema,
null,
{
name: "Existing widget",
authType: "bearer",
},
"Use a progress bar.",
"widget-definition-id",
);
expect(prompt).toContain("custom widget definition `widget-definition-id`");
expect(prompt).toContain("`customWidget_update`");
expect(prompt).toContain("Preserve configured secrets by omitting the `secrets` field");
expect(prompt).not.toContain('"secrets"');
});
test("keeps copy-to-clipboard prompts useful before a request is entered", () => {
const prompt = buildAiPrompt(schema);
expect(prompt).toContain("PASTE_YOUR_API_RESPONSE_HERE");
expect(prompt).toContain("Describe what you want the widget to show or change here.");
});
});
@@ -1,13 +1,16 @@
"use client";
import { Button, Popover, Stack, Text } from "@mantine/core";
import { Alert, Button, Group, Popover, Stack, Text, Textarea } from "@mantine/core";
import { useDisclosure } from "@mantine/hooks";
import { IconCopy, IconSparkles } from "@tabler/icons-react";
import { IconCopy, IconRobot, IconSparkles } from "@tabler/icons-react";
import { useEffect, useRef, useState } from "react";
import { clientApi } from "@homarr/api/client";
import { showErrorNotification, showSuccessNotification } from "@homarr/notifications";
import { useScopedI18n } from "@homarr/translation/client";
import { useOptionalHomarrAssistant } from "~/components/assistant/assistant-provider";
const PROMPT_HEADER = `You are helping configure a Homarr custom widget. Homarr is a self-hosted dashboard that can display data from any API endpoint as a widget.
## Your Task
@@ -267,14 +270,16 @@ PASTE_YOUR_API_RESPONSE_HERE
const PROMPT_FOOTER = `
## Your Request
Describe what you want the widget to show:
`;
function buildAiPrompt(
const PROMPT_REQUEST_PLACEHOLDER = "Describe what you want the widget to show or change here.";
export function buildAiPrompt(
jsonSchema: unknown,
rawResponse?: string | null,
currentConfig?: Record<string, unknown> | null,
request?: string,
definitionId?: string,
) {
const schemaStr = JSON.stringify(jsonSchema, null, 2);
const responseSection = rawResponse
@@ -285,22 +290,57 @@ function buildAiPrompt(
? `\n## Current Widget Configuration\nThe widget is currently configured as follows. Use this as a starting point and modify based on the user's request:\n\n\`\`\`json\n${JSON.stringify(currentConfig, null, 2)}\n\`\`\`\n`
: "";
return PROMPT_HEADER + schemaStr + PROMPT_RULES + responseSection + configSection + PROMPT_FOOTER;
const homarrActionSection = definitionId
? `\n## Homarr Assistant Action\nThis is custom widget definition \`${definitionId}\`. If the customWidget tools are available, validate the proposed configuration and immediately call \`customWidget_update\` with this ID and the changed fields. Calling the tool is the required result and takes priority over the JSON-only output instruction above; do not ask for typed confirmation because Homarr will show its approval UI. Preserve configured secrets by omitting the \`secrets\` field. If tools are unavailable, return the JSON configuration instead.\n`
: `\n## Homarr Assistant Action\nIf the customWidget tools are available, validate the proposed configuration and immediately call \`customWidget_create\` with \`secrets: []\`. Calling the tool is the required result and takes priority over the JSON-only output instruction above; do not ask for typed confirmation because Homarr will show its approval UI. If tools are unavailable, return the JSON configuration instead.\n`;
const requestText = request?.trim() || PROMPT_REQUEST_PLACEHOLDER;
return (
PROMPT_HEADER +
schemaStr +
PROMPT_RULES +
responseSection +
configSection +
homarrActionSection +
PROMPT_FOOTER +
requestText
);
}
interface CopyAiPromptButtonProps {
rawResponse?: string | null;
currentConfig?: Record<string, unknown> | null;
definitionId?: string;
}
export const CopyAiPromptButton = ({ rawResponse, currentConfig }: CopyAiPromptButtonProps) => {
export const CopyAiPromptButton = ({ rawResponse, currentConfig, definitionId }: CopyAiPromptButtonProps) => {
const t = useScopedI18n("customWidget");
const [opened, { open, close }] = useDisclosure(false);
const assistant = useOptionalHomarrAssistant();
const [opened, { toggle, close }] = useDisclosure(false);
const [request, setRequest] = useState("");
const requestInputRef = useRef<HTMLTextAreaElement>(null);
const { data: schema, isLoading } = clientApi.customWidget.schema.useQuery();
useEffect(() => {
if (opened) requestInputRef.current?.focus();
}, [opened]);
const getPrompt = () => (schema ? buildAiPrompt(schema, rawResponse, currentConfig, request, definitionId) : null);
const handleAssistant = () => {
const prompt = getPrompt();
if (!prompt || !assistant?.enabled || request.trim().length === 0) return;
if (!assistant.sendPrompt(prompt)) return;
close();
setRequest("");
showSuccessNotification({ title: t("action.aiPrompt"), message: t("notification.aiPromptSent") });
};
const handleCopy = async () => {
if (!schema) return;
const prompt = buildAiPrompt(schema, rawResponse, currentConfig);
const prompt = getPrompt();
if (!prompt) return;
try {
await navigator.clipboard.writeText(prompt);
close();
@@ -311,31 +351,70 @@ export const CopyAiPromptButton = ({ rawResponse, currentConfig }: CopyAiPromptB
};
return (
<Popover opened={opened} onClose={close} width={320} position="bottom" shadow="md" withinPortal>
<Popover opened={opened} onClose={close} width="target" position="bottom" shadow="md" withinPortal>
<Popover.Target>
<Button
type="button"
variant="light"
leftSection={<IconSparkles size={16} />}
onClick={open}
onClick={toggle}
loading={isLoading}
disabled={!schema}
fullWidth
size="sm"
>
{t("action.copyAiPrompt")}
{t("action.aiPrompt")}
</Button>
</Popover.Target>
<Popover.Dropdown>
<Stack gap="sm">
<Text size="sm">{t("notification.aiPromptDescription")}</Text>
<Textarea
ref={requestInputRef}
label={t("notification.aiPromptRequestLabel")}
placeholder={t("notification.aiPromptRequestPlaceholder")}
value={request}
onChange={(event) => setRequest(event.currentTarget.value)}
autosize
minRows={3}
maxRows={6}
onKeyDown={(event) => {
if (event.key === "Enter" && (event.metaKey || event.ctrlKey)) {
event.preventDefault();
handleAssistant();
}
}}
/>
{!rawResponse && (
<Text size="xs" c="dimmed" fs="italic">
{t("notification.aiPromptNoResponse")}
</Text>
)}
<Button leftSection={<IconCopy size={16} />} onClick={() => void handleCopy()} fullWidth>
{t("notification.aiPromptCopy")}
</Button>
{assistant?.enabled ? (
<Button
type="button"
leftSection={<IconRobot size={16} />}
onClick={handleAssistant}
disabled={request.trim().length === 0 || assistant.isRunning}
fullWidth
>
{assistant.isRunning ? t("notification.aiPromptAssistantBusy") : t("notification.aiPromptAssistant")}
</Button>
) : (
<Alert color="gray" icon={<IconRobot size={16} />}>
{t("notification.aiPromptAssistantUnavailable")}
</Alert>
)}
<Group justify="flex-end">
<Button
type="button"
variant="default"
leftSection={<IconCopy size={16} />}
onClick={() => void handleCopy()}
>
{t("notification.aiPromptCopy")}
</Button>
</Group>
</Stack>
</Popover.Dropdown>
</Popover>
@@ -688,6 +688,19 @@ export function CustomWidgetForm({ mode, initialValues, definitionId }: CustomWi
}, [previewRefreshSignal]);
const previewInput = getPreviewInput();
const aiPromptCurrentConfig = {
$schema: "homarr-custom-widget-v2",
name: form.values.name,
description: form.values.description || undefined,
iconUrl: form.values.iconUrl || undefined,
url: form.values.url,
authType: form.values.authType,
headerName: form.values.headerName || undefined,
method: form.values.method,
requestBody: form.values.requestBody || undefined,
displayType: form.values.displayType,
displayConfig: previewInput.displayConfig,
};
return (
<form onSubmit={handleSubmit}>
@@ -809,15 +822,22 @@ export function CustomWidgetForm({ mode, initialValues, definitionId }: CustomWi
</Fieldset>
<Box hiddenFrom="md">
<CustomWidgetPreview
getFormValues={getPreviewInput}
formValues={previewInput}
fetchResult={previewFetchResult}
cachedJson={previewJson}
onTest={() => void handlePreviewTest()}
isTesting={previewMutation.isPending}
testError={previewMutation.error?.message ?? null}
/>
<Stack gap="sm">
<CopyAiPromptButton
rawResponse={previewFetchResult?.rawResponse}
currentConfig={aiPromptCurrentConfig}
definitionId={definitionId}
/>
<CustomWidgetPreview
getFormValues={getPreviewInput}
formValues={previewInput}
fetchResult={previewFetchResult}
cachedJson={previewJson}
onTest={() => void handlePreviewTest()}
isTesting={previewMutation.isPending}
testError={previewMutation.error?.message ?? null}
/>
</Stack>
</Box>
<Group justify="end">
@@ -845,19 +865,8 @@ export function CustomWidgetForm({ mode, initialValues, definitionId }: CustomWi
</Button>
<CopyAiPromptButton
rawResponse={previewFetchResult?.rawResponse}
currentConfig={{
$schema: "homarr-custom-widget-v2",
name: form.values.name,
description: form.values.description,
iconUrl: form.values.iconUrl,
url: form.values.url,
authType: form.values.authType,
headerName: form.values.headerName,
method: form.values.method,
requestBody: form.values.requestBody,
displayType: form.values.displayType,
displayConfig: previewInput.displayConfig,
}}
currentConfig={aiPromptCurrentConfig}
definitionId={definitionId}
/>
<CustomWidgetPreview
getFormValues={getPreviewInput}