refactor: simplify widget and integration architecture
This commit is contained in:
1 parent
c988c35b29
commit
8f9efd0074
179 files changed
+1509
-4187
No files matched your search
@@ -8,206 +8,58 @@ tags:
|
||||
sidebar_position: 2
|
||||
---
|
||||
|
||||
Homarr is moving native widgets and integrations to vertical feature modules: the stable identity, lightweight metadata,
|
||||
client entry, server registration, cache policy, and documentation source live beside the implementation. Generated
|
||||
files preserve static imports and package boundaries for Next.js, tRPC, and Docusaurus.
|
||||
Homarr keeps feature registration explicit and typed. There is no generator or generated registry to run after a
|
||||
change. A new feature is available when its typed metadata and lazy loader are registered.
|
||||
|
||||
Registration means availability. There is no separate feature-enable list or runtime kill switch.
|
||||
## Add a widget
|
||||
|
||||
## Current status
|
||||
Keep the UI, options, helpers, and tests together in `packages/widgets/src/<widget>/`. The usual folder only needs an
|
||||
`index.ts` definition and a lazily imported `component.tsx`; complex widgets can add local files as needed.
|
||||
|
||||
Weather and Downloads are the reference widget modules. Their folders now own registration and canonical docs:
|
||||
1. Add the stable kind to `packages/definitions/src/widget.ts` and its documentation slug to the direct typed map in
|
||||
`packages/definitions/src/docs/widget-doc-slugs.ts`.
|
||||
2. Export `definition` and `componentLoader` from `packages/widgets/src/<widget>/index.ts` with
|
||||
`createWidgetDefinition`. The local definition owns its icon, query keys, refresh interval, options, matchers, and
|
||||
errors. Keep the component behind `withDynamicImport` so adding a widget does not load it on every dashboard.
|
||||
3. For an integration-backed widget, add its integration kinds and selection rules once to
|
||||
`packages/definitions/src/widget-integration-map.ts`, then spread `getWidgetIntegrationConfig(kind)` into the local
|
||||
definition. This server-safe config is also the API authorization boundary, avoiding a package cycle without
|
||||
duplicating widget metadata.
|
||||
4. Register the module in the explicit lazy-loader and type-only maps in `packages/widgets/src/registry.ts`, then add
|
||||
its icon to `packages/ui/src/widget-icons.ts`. The two registry entries preserve exact option/component inference;
|
||||
keep them in the same order so drift is obvious in review.
|
||||
5. If the widget needs server data, add its tRPC router under `packages/api/src/router/widgets/` and add its explicit
|
||||
lazy import to `packages/api/src/router/widgets/index.ts`. Keep upstream requests and Redis response caching in
|
||||
`@homarr/request-handler`. Use the widget integration middleware for integration-backed procedures.
|
||||
6. Add canonical documentation in `apps/docs/docs/widgets/<slug>/`.
|
||||
|
||||
```text
|
||||
packages/widgets/src/weather/
|
||||
├── module.ts # authored manifest and client/server data policy
|
||||
├── index.ts # options and presentation-specific definition
|
||||
├── component.tsx # client UI
|
||||
├── module.generated.ts # generated icon projection; do not edit
|
||||
└── docs/ # canonical MDX, metadata, and assets
|
||||
```
|
||||
Weather is the small integration-free example. Downloads shows integration selection, polling, server caching,
|
||||
mutations, partial upstream failures, and a larger options surface.
|
||||
|
||||
The remaining widgets still use the compatibility registries and can be migrated one folder at a time without changing
|
||||
their persisted `WidgetKind` or tRPC paths. The generated and legacy entries are merged behind the same runtime loaders.
|
||||
## Add an integration
|
||||
|
||||
Beszel is the reference integration module. Its folder owns the manifest, implementation, capability helpers, tests,
|
||||
and canonical docs. The generator projects its metadata into definitions and its constructor into the lazy integration
|
||||
factory without eagerly importing the implementation.
|
||||
Keep the HTTP client and its capability-specific types in `packages/integrations/src/<integration>/`.
|
||||
|
||||
The server router implementation is not physically inside these first two widget folders yet. `@homarr/widgets`
|
||||
currently imports `@homarr/api/client`; making `@homarr/api` import widget server files would create a package cycle and
|
||||
risk shipping server code to the browser. The module already owns the lazy router registration, and the neutral server
|
||||
operation types are the cycle-breaking seam for the next migration stage.
|
||||
1. Add the typed definition to `packages/definitions/src/integration.ts`, including credential alternatives,
|
||||
categories, documentation slug, ports, Docker aliases, and onboarding metadata when relevant.
|
||||
2. Add an explicit lazy creator to `packages/integrations/src/base/creator.ts`. The literal import keeps the
|
||||
implementation out of the initial server bundle and makes the registration easy to find.
|
||||
3. Implement only the interfaces required by the widgets that use the service.
|
||||
4. Add canonical documentation in `apps/docs/docs/integrations/<slug>/`. Its `index.ts` must satisfy
|
||||
`IntegrationDefinition` so invalid metadata fails type checking.
|
||||
|
||||
## The module manifest
|
||||
Beszel is a compact example with several widgets sharing one integration client.
|
||||
|
||||
`module.ts` is pure and server-safe. It must not import React, tRPC, an integration implementation, secrets, or a request
|
||||
handler. It describes import locations as strings so the generator can emit literal imports into the correct package.
|
||||
## Data and errors
|
||||
|
||||
```ts
|
||||
import { defineWidgetModule } from "@homarr/definitions";
|
||||
- TanStack Query owns browser caching and refresh behavior. Redis owns shared server response caching.
|
||||
- Cache raw service responses, never authorization decisions or decrypted credentials. Integration-backed cache keys
|
||||
include the integration identity and are invalidated after mutations.
|
||||
- Keep validation and authorization on the server. Client-side checks improve UX but do not replace tRPC guards.
|
||||
- Use the shared logger with safe identifiers, operation names, duration, and the original error cause. Never log
|
||||
secrets or full upstream payloads.
|
||||
- Multi-integration reads should return successful providers when one service fails and surface a clear error when all
|
||||
providers fail.
|
||||
|
||||
export default defineWidgetModule({
|
||||
kind: "example",
|
||||
icon: "IconSparkles",
|
||||
clientEntry: ".",
|
||||
documentation: {
|
||||
slug: "example",
|
||||
sourceDirectory: "docs",
|
||||
},
|
||||
integration: {
|
||||
categories: ["downloadClient"],
|
||||
serverMaxIntegrations: 1,
|
||||
},
|
||||
operations: [
|
||||
{
|
||||
name: "getData",
|
||||
path: ["widget", "example", "getData"],
|
||||
client: {
|
||||
refetchIntervalSeconds: 30,
|
||||
persist: true,
|
||||
},
|
||||
serverCache: {
|
||||
namespace: "example:data",
|
||||
ttlMs: 30_000,
|
||||
scope: "integration",
|
||||
},
|
||||
},
|
||||
],
|
||||
routers: [
|
||||
{
|
||||
namespace: "example",
|
||||
module: "./example",
|
||||
exportName: "exampleRouter",
|
||||
},
|
||||
],
|
||||
});
|
||||
```
|
||||
|
||||
Run the synchronizer after adding or changing a module:
|
||||
|
||||
```sh
|
||||
pnpm widgets:new example --icon IconSparkles
|
||||
pnpm widgets:sync
|
||||
pnpm widgets:sync:check
|
||||
```
|
||||
|
||||
`widgets:new` creates a minimal client-only folder and immediately runs the synchronizer. Replace the empty component,
|
||||
add options, and extend the manifest with operations or router registrations as needed; it never edits a hand-written
|
||||
central feature list.
|
||||
|
||||
The check command is read-only and reports drift. The synchronizer rejects duplicate widget kinds, docs slugs, router
|
||||
namespaces with conflicting targets, operation names, and cache namespaces. It also verifies client entries, router
|
||||
modules, and local docs before producing anything.
|
||||
|
||||
It generates these compatibility projections:
|
||||
|
||||
| Consumer | Generated projection |
|
||||
| ---------------------- | -------------------------------------------------------------------- |
|
||||
| Definitions and policy | `packages/definitions/src/generated/widget-modules.ts` |
|
||||
| Client lazy loading | `packages/widgets/src/generated/widget-modules.ts` |
|
||||
| Lightweight icons | `packages/ui/src/generated/widget-icons.ts` |
|
||||
| Lazy tRPC routers | `packages/api/src/router/widgets/generated-registry.ts` |
|
||||
| Docusaurus | `apps/docs/docs/widgets/<slug>/` from the module's canonical `docs/` |
|
||||
|
||||
Generated client and server registries contain explicit import literals. Do not replace them with a variable dynamic
|
||||
import: explicit paths let Next.js and Turbopack discover chunks without eagerly loading every widget.
|
||||
|
||||
Use `pnpm features:sync` or `pnpm features:sync:check` when a change spans widgets and integrations.
|
||||
|
||||
## Client and server data caching
|
||||
|
||||
The operation is the shared identity for both cache layers:
|
||||
|
||||
- The tRPC path drives the TanStack Query key.
|
||||
- `refetchIntervalSeconds` configures the root Query Client without importing widget UI code.
|
||||
- `persist` controls session-scoped browser persistence. Persisted payloads are size-bounded, sanitized, and separated
|
||||
by the authenticated user and permission scope.
|
||||
- `serverCache` is imported by the request handler through `getWidgetServerCachePolicy`, so its namespace and TTL are
|
||||
not copied into another file.
|
||||
|
||||
Server reads use bounded in-process L1 caching and a shared Redis L2. Inputs are canonically serialized and SHA-256
|
||||
hashed; raw URLs, options, and secrets never appear in Redis keys or cache logs. Redis operations and lock waits are
|
||||
bounded, refreshes use token-safe distributed locks, and stale data can be retained for the declared error window.
|
||||
Redis failure degrades to local caching and produces structured logs with safe metadata and the original cause.
|
||||
|
||||
Integration-backed cache keys include the integration ID and its Redis generation. Mutations advance that generation,
|
||||
which invalidates all replicas without scanning keys. Integration-free widget handlers use a namespace generation;
|
||||
manual refreshes such as Docker inventory invalidation therefore clear both local and shared values. A zero TTL keeps
|
||||
only overlapping in-flight request deduplication and never stores a Redis payload, which is appropriate for realtime
|
||||
or subscription data.
|
||||
|
||||
Generation reads are process-cached for one second and concurrent misses are deduplicated, so a hot L1 hit does not add
|
||||
a Redis round trip. Redis failure backs off for five seconds instead of producing a request storm; local invalidations
|
||||
take effect immediately and other replicas converge within the one-second generation window.
|
||||
|
||||
TanStack Query remains the browser cache and Redis remains the server cache. A browser persistence hit never bypasses
|
||||
server authorization, and Redis never caches permission decisions or decrypted credentials.
|
||||
|
||||
## Server operations and tRPC
|
||||
|
||||
The neutral contract includes helpers for queries, mutations, subscriptions, and routes:
|
||||
|
||||
- `defineWidgetServerOperation`
|
||||
- `defineWidgetServerSubscription`
|
||||
- `defineWidgetServerRoute`
|
||||
- `defineWidgetServerModule`
|
||||
|
||||
They describe validated input, authorization intent, cache policy, an `AbortSignal`, and a framework-neutral resolver.
|
||||
The future API adapter will inject tRPC procedure builders, integration middleware, logging, and error normalization.
|
||||
This dependency direction lets a local `server.ts` be consumed by API code without the widget client importing API
|
||||
runtime code.
|
||||
|
||||
Until that adapter replaces the current client dependency, keep actual upstream calls in `@homarr/request-handler` and
|
||||
the tRPC compatibility router in `packages/api/src/router/widgets/`. Put the router registration in the local manifest;
|
||||
do not add it to the hand-written registry.
|
||||
|
||||
An integration-backed procedure must use the widget-aware middleware. It derives compatible kinds and maximum
|
||||
selection limits from the module/catalog, resolves only the requested records, decrypts secrets on the server, and
|
||||
checks direct user and group permissions before the procedure runs. Use `query` permission for reads and `interact`
|
||||
permission for mutations.
|
||||
|
||||
## Errors and logs
|
||||
|
||||
- Root tRPC instrumentation records the procedure path, operation type, user, duration, and outcome.
|
||||
- Integration middleware records safe requested/resolved IDs and kinds plus denial reasons; it never logs credentials.
|
||||
- Integration calls are timed and normalized into public errors while preserving a structured server-side cause.
|
||||
- Multi-integration reads use `settleIntegrationQueries` so one failed service does not erase successful providers.
|
||||
- Widget components read through the shared usable-data helper: stale data remains visible after a refresh error, while
|
||||
first-load failures reach the common widget error boundary and logs link.
|
||||
- Lazy client, router, and integration imports cache successful and in-flight loads, evict failed promises, and log
|
||||
load duration so a transient chunk failure can retry.
|
||||
|
||||
## Complex modules
|
||||
|
||||
The manifest is the common path, not a limit on implementation size.
|
||||
|
||||
- Beszel can group multiple stable widget kinds behind one router and keep live statistics on an SSE subscription with
|
||||
Redis payload caching disabled.
|
||||
- Assistant can use a route/stream adapter for chat while keeping client runtime state and tools in subfolders.
|
||||
- A proxy-only widget can expose a server route without inventing a polling query.
|
||||
- A widget can add custom query matchers, context actions, prefetching, or nested client components while retaining the
|
||||
same registration, permissions, logging, error, and cache contracts.
|
||||
|
||||
## Integration modules
|
||||
|
||||
Create an integration module with an explicit icon URL:
|
||||
|
||||
```sh
|
||||
pnpm integrations:new example --icon-url https://example.com/example.svg
|
||||
pnpm integrations:sync:check
|
||||
```
|
||||
|
||||
The scaffold creates a compileable connection test, pure `module.ts`, and canonical docs in
|
||||
`packages/integrations/src/example/`. Add capability interfaces and service methods beside that implementation, then
|
||||
connect them to widget request handlers. Declare real credential alternatives, categories, Docker discovery aliases,
|
||||
ports, and onboarding metadata in the local module instead of editing the central definitions and creator registries.
|
||||
|
||||
The integration synchronizer validates kinds, credential kinds, categories, ports, URLs, documentation, creator
|
||||
exports, and collisions. It generates the definitions projection, a typed lazy creator registry with literal imports,
|
||||
and Docusaurus copies whose edit links point back to the canonical module docs.
|
||||
|
||||
Beszel is migrated to this path. Other integrations still use the compatibility registry and can move independently
|
||||
without changing persisted `IntegrationKind` values or widget capability checks. Keep integrations server-only,
|
||||
implement only their capability interfaces, and use narrow schema/interface exports instead of the broad runtime
|
||||
barrel while migrating them.
|
||||
Registration files intentionally contain boring, explicit imports. This small amount of repetition preserves exact
|
||||
TypeScript inference, gives Next.js discoverable lazy chunks, and keeps Ctrl+click navigation reliable.
|
||||
@@ -1 +0,0 @@
|
||||
packages/integrations/src/beszel/docs
|
||||
@@ -1,7 +1,9 @@
|
||||
import type { IntegrationDefinition } from "@site/src/types";
|
||||
|
||||
export const beszelIntegration = {
|
||||
name: "Beszel",
|
||||
description:
|
||||
"Beszel is a lightweight server monitoring platform with Docker stats, historical data, and alert functions.",
|
||||
iconUrl: "https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons@master/svg/beszel.svg",
|
||||
path: "../../integrations/beszel",
|
||||
} as const;
|
||||
} satisfies IntegrationDefinition;
|
||||
@@ -1 +0,0 @@
|
||||
packages/widgets/src/downloads/docs
|
||||
@@ -1 +0,0 @@
|
||||
packages/widgets/src/weather/docs
|
||||
@@ -2,37 +2,8 @@ import type * as Preset from "@docusaurus/preset-classic";
|
||||
import type { Config } from "@docusaurus/types";
|
||||
import { themes as prismThemes } from "prism-react-renderer";
|
||||
import { resolveHomarrUrlConfig } from "@homarr/workshop/schema";
|
||||
import { generatedIntegrationDocumentationSources, generatedWidgetDocumentationSources } from "@homarr/definitions";
|
||||
const a11yEmoji = require("@fec/remark-a11y-emoji");
|
||||
|
||||
const githubDocsEditRoot = "https://github.com/homarr-labs/homarr/edit/dev";
|
||||
|
||||
const getDocumentationEditUrl = (docPath: string) => {
|
||||
const integrationPath = /^integrations\/([^/]+)\/(.+)$/u.exec(docPath);
|
||||
const integrationSlug = integrationPath?.[1];
|
||||
const integrationRelativePath = integrationPath?.[2];
|
||||
if (
|
||||
integrationSlug &&
|
||||
integrationRelativePath &&
|
||||
Object.hasOwn(generatedIntegrationDocumentationSources, integrationSlug)
|
||||
) {
|
||||
const source =
|
||||
generatedIntegrationDocumentationSources[
|
||||
integrationSlug as keyof typeof generatedIntegrationDocumentationSources
|
||||
];
|
||||
return `${githubDocsEditRoot}/${source}/${integrationRelativePath}`;
|
||||
}
|
||||
|
||||
const widgetPath = /^widgets\/([^/]+)\/(.+)$/u.exec(docPath);
|
||||
const slug = widgetPath?.[1];
|
||||
const relativePath = widgetPath?.[2];
|
||||
if (slug && relativePath && Object.hasOwn(generatedWidgetDocumentationSources, slug)) {
|
||||
const source = generatedWidgetDocumentationSources[slug as keyof typeof generatedWidgetDocumentationSources];
|
||||
return `${githubDocsEditRoot}/${source}/${relativePath}`;
|
||||
}
|
||||
return `${githubDocsEditRoot}/apps/docs/docs/${docPath}`;
|
||||
};
|
||||
|
||||
const deprecatedWorkshopUrl = process.env.WORKSHOP_URL;
|
||||
if (deprecatedWorkshopUrl && !process.env.WORKSHOP_API_URL) {
|
||||
console.warn("WORKSHOP_URL is deprecated; use WORKSHOP_API_URL instead.");
|
||||
@@ -94,7 +65,7 @@ const config: Config = {
|
||||
{
|
||||
docs: {
|
||||
sidebarPath: require.resolve("./sidebars.js"),
|
||||
editUrl: ({ docPath }) => getDocumentationEditUrl(docPath),
|
||||
editUrl: ({ docPath }) => `https://github.com/homarr-labs/homarr/edit/dev/apps/docs/docs/${docPath}`,
|
||||
remarkPlugins: [a11yEmoji],
|
||||
exclude: [],
|
||||
showLastUpdateAuthor: false,
|
||||
|
||||
@@ -2,13 +2,17 @@ import type { LoadContext, Plugin } from "@docusaurus/types";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
|
||||
import { integrationDocSlugs } from "@homarr/definitions/docs/integration-doc-slugs";
|
||||
import { widgetDocSlugs } from "@homarr/definitions/docs/widget-doc-slugs";
|
||||
import { getIntegrationDocumentationSlug, integrationKinds } from "@homarr/definitions/integration";
|
||||
|
||||
const slugMaps = {
|
||||
Integration: { slugs: integrationDocSlugs, folder: "integrations" },
|
||||
Widget: { slugs: widgetDocSlugs, folder: "widgets" },
|
||||
} as const;
|
||||
const documentationEntries = [
|
||||
{
|
||||
label: "Integration",
|
||||
folder: "integrations",
|
||||
entries: integrationKinds.map((kind) => [kind, getIntegrationDocumentationSlug(kind)] as const),
|
||||
},
|
||||
{ label: "Widget", folder: "widgets", entries: Object.entries(widgetDocSlugs) },
|
||||
] as const;
|
||||
|
||||
export default function validateDocsCoveragePlugin(context: LoadContext): Plugin {
|
||||
return {
|
||||
@@ -17,8 +21,8 @@ export default function validateDocsCoveragePlugin(context: LoadContext): Plugin
|
||||
const docsDir = path.resolve(context.siteDir, "docs");
|
||||
const missing: string[] = [];
|
||||
|
||||
for (const [label, { slugs, folder }] of Object.entries(slugMaps)) {
|
||||
for (const [kind, slug] of Object.entries(slugs)) {
|
||||
for (const { label, folder, entries } of documentationEntries) {
|
||||
for (const [kind, slug] of entries) {
|
||||
if (!slug) continue;
|
||||
if (!fs.existsSync(path.join(docsDir, folder, slug))) {
|
||||
missing.push(`${label} "${kind}" -> docs/${folder}/${slug}/`);
|
||||
|
||||
@@ -141,12 +141,14 @@ const ScopedTRPCReactProvider = ({
|
||||
refetchOnWindowFocus: false,
|
||||
refetchOnReconnect: false,
|
||||
});
|
||||
for (const { queryKey, intervalSeconds, staleTimeSeconds } of widgetQueryRefetchIntervals) {
|
||||
for (const queryDefaults of widgetQueryRefetchIntervals) {
|
||||
const policy: { refetchInterval?: number | false; staleTime?: number } = {};
|
||||
if (intervalSeconds === null) policy.refetchInterval = false;
|
||||
if (typeof intervalSeconds === "number") policy.refetchInterval = intervalSeconds * 1000;
|
||||
if (staleTimeSeconds !== undefined) policy.staleTime = staleTimeSeconds * 1000;
|
||||
client.setQueryDefaults(queryKey, policy);
|
||||
if (queryDefaults.intervalSeconds === null) policy.refetchInterval = false;
|
||||
if (typeof queryDefaults.intervalSeconds === "number") {
|
||||
policy.refetchInterval = queryDefaults.intervalSeconds * 1000;
|
||||
}
|
||||
if ("staleTimeSeconds" in queryDefaults) policy.staleTime = queryDefaults.staleTimeSeconds * 1000;
|
||||
client.setQueryDefaults(queryDefaults.queryKey, policy);
|
||||
}
|
||||
for (const { queryKey, ...policy } of dashboardSupportingQueryPolicies) {
|
||||
client.setQueryDefaults(queryKey, policy);
|
||||
|
||||
@@ -10,26 +10,25 @@ import { getI18n } from "@homarr/translation/server";
|
||||
|
||||
import { createMetaTitle } from "~/metadata";
|
||||
import { DynamicBreadcrumb } from "~/components/navigation/dynamic-breadcrumb";
|
||||
import { getMcpRuntimeAsync } from "~/app/api/mcp/_extract-tools";
|
||||
import { extractMcpTools } from "~/app/api/mcp/_extract-tools";
|
||||
import { ApiKeysManagement } from "./components/api-keys";
|
||||
import { ApiPageTabs } from "./components/api-page-tabs";
|
||||
import { ScalarApiReference } from "./components/scalar-api-reference";
|
||||
|
||||
import type { McpToolGroup } from "./components/api-page-tabs";
|
||||
|
||||
async function getMcpToolGroupsAsync(): Promise<McpToolGroup[]> {
|
||||
const { tools, procedureTypes } = await getMcpRuntimeAsync();
|
||||
function getMcpToolGroups(): McpToolGroup[] {
|
||||
const tools = extractMcpTools();
|
||||
const groups = new Map<string, McpToolGroup["tools"]>();
|
||||
for (const tool of tools) {
|
||||
const namespace = tool.pathInRouter[0] ?? "other";
|
||||
if (!groups.has(namespace)) {
|
||||
groups.set(namespace, []);
|
||||
}
|
||||
const procedureKey = tool.pathInRouter.join(".");
|
||||
groups.get(namespace)?.push({
|
||||
name: tool.name,
|
||||
description: tool.description,
|
||||
type: procedureTypes.get(procedureKey) ?? "query",
|
||||
type: tool.type,
|
||||
});
|
||||
}
|
||||
return Array.from(groups.entries()).map(([namespace, items]) => ({
|
||||
@@ -60,7 +59,7 @@ export default async function ApiPage() {
|
||||
headers(),
|
||||
api.apiKeys.getAll(),
|
||||
getI18n("management.page.tool.api.tab"),
|
||||
getMcpToolGroupsAsync(),
|
||||
getMcpToolGroups(),
|
||||
]);
|
||||
const baseUrl = extractBaseUrlFromHeaders(requestHeaders);
|
||||
const document = openApiDocument(baseUrl);
|
||||
|
||||
@@ -6,7 +6,7 @@ import type { ToolSet, UIMessage } from "ai";
|
||||
import { createUIMessageStream, createUIMessageStreamResponse, jsonSchema, stepCountIs, streamText, tool } from "ai";
|
||||
import { z } from "zod/v4";
|
||||
|
||||
import { createTRPCContext } from "@homarr/api/mcp";
|
||||
import { createTRPCContext, mcpRouter } from "@homarr/api/mcp";
|
||||
import {
|
||||
createAssistantGenerationAccessToken,
|
||||
getAssistantContextEntitiesAsync,
|
||||
@@ -16,7 +16,6 @@ import { auth } from "@homarr/auth/next";
|
||||
import { env } from "@homarr/common/env";
|
||||
import { decryptSecret } from "@homarr/common/server";
|
||||
import { createLogger } from "@homarr/core/infrastructure/logs";
|
||||
import { ErrorWithMetadata } from "@homarr/core/infrastructure/logs/error";
|
||||
import { and, eq } from "@homarr/db";
|
||||
import { db } from "@homarr/db";
|
||||
import { assistantConfigurations, assistantThreads } from "@homarr/db/schema";
|
||||
@@ -36,7 +35,7 @@ import type {
|
||||
AssistantWebSearchSource,
|
||||
} from "~/components/assistant/assistant-message-metadata";
|
||||
|
||||
import { getMcpRuntimeAsync } from "../../mcp/_extract-tools";
|
||||
import { extractMcpTools } from "../../mcp/_extract-tools";
|
||||
import { buildAssistantRequestContext, sanitizeAttachmentFilename } from "./assistant-chat-input";
|
||||
import { getAssistantModelLookupStatus } from "./assistant-model-lookup";
|
||||
import { convertAssistantMessagesToModelMessages } from "./assistant-message-conversion";
|
||||
@@ -493,24 +492,14 @@ export async function POST(request: Request) {
|
||||
}
|
||||
const canAuthorCustomWidgets = session.user.permissions.includes("admin");
|
||||
const customWidgetAuthoringContext = getCustomWidgetAuthoringContext(incomingMessages, canAuthorCustomWidgets);
|
||||
let mcpRuntime;
|
||||
try {
|
||||
mcpRuntime = await getMcpRuntimeAsync();
|
||||
} catch (error) {
|
||||
logger.error(new ErrorWithMetadata("Assistant tools are unavailable", { requestId }, { cause: error }));
|
||||
return Response.json(
|
||||
{ error: "Homarr tools could not be loaded. Check the Homarr logs and try again." },
|
||||
{ status: 503 },
|
||||
);
|
||||
}
|
||||
const caller = mcpRuntime.router.createCaller(context);
|
||||
const caller = mcpRouter.createCaller(context);
|
||||
const omittedCustomWidgetTools = new Set<string>(customWidgetAuthoringContext.omittedToolNames);
|
||||
const mcpTools = mcpRuntime.tools.filter((mcpTool) => !omittedCustomWidgetTools.has(mcpTool.name));
|
||||
const mcpTools = extractMcpTools().filter((mcpTool) => !omittedCustomWidgetTools.has(mcpTool.name));
|
||||
const customWidgetComponentDocumentBudget = createCustomWidgetComponentDocumentBudget();
|
||||
|
||||
const homarrTools = Object.fromEntries(
|
||||
mcpTools.map((mcpTool) => {
|
||||
const requiresApproval = mcpRuntime.procedureTypes.get(mcpTool.pathInRouter.join(".")) === "mutation";
|
||||
const requiresApproval = mcpTool.type === "mutation";
|
||||
return [
|
||||
mcpTool.name,
|
||||
tool({
|
||||
@@ -558,11 +547,9 @@ export async function POST(request: Request) {
|
||||
}),
|
||||
) satisfies ToolSet;
|
||||
const toolApproval = Object.fromEntries(
|
||||
mcpTools.flatMap((mcpTool) =>
|
||||
mcpRuntime.procedureTypes.get(mcpTool.pathInRouter.join(".")) === "mutation"
|
||||
? [[mcpTool.name, "user-approval" as const]]
|
||||
: [],
|
||||
),
|
||||
mcpTools
|
||||
.filter((mcpTool) => mcpTool.type === "mutation")
|
||||
.map((mcpTool) => [mcpTool.name, "user-approval" as const]),
|
||||
);
|
||||
|
||||
const frontendTools = Object.fromEntries(
|
||||
|
||||
@@ -16,6 +16,7 @@ const mocks = vi.hoisted(() => ({
|
||||
vi.mock("next/server", () => ({ userAgent: () => ({ ua: "MCP route test" }) }));
|
||||
vi.mock("@homarr/api/mcp", () => ({
|
||||
createTRPCContext: vi.fn(() => ({})),
|
||||
mcpRouter: { createCaller: vi.fn(() => ({ board: { getAllBoards: mocks.toolProcedure } })) },
|
||||
}));
|
||||
vi.mock("@homarr/auth/api-key", () => ({
|
||||
API_KEY_HEADER_NAME: "ApiKey",
|
||||
@@ -41,30 +42,29 @@ vi.mock("@homarr/custom-widgets/core", () => ({
|
||||
vi.mock("@homarr/db", () => ({ db: {} }));
|
||||
vi.mock("~/versions/package-reader", () => ({ getPackageVersion: () => "test-version" }));
|
||||
vi.mock("../_extract-tools", () => ({
|
||||
getMcpRuntimeAsync: async () => ({
|
||||
router: { createCaller: vi.fn(() => ({ board: { getAllBoards: mocks.toolProcedure } })) },
|
||||
procedureTypes: new Map([["board.getAllBoards", "query"]]),
|
||||
tools: [
|
||||
{
|
||||
name: "board_getAllBoards",
|
||||
description: "List boards",
|
||||
pathInRouter: ["board", "getAllBoards"],
|
||||
inputSchema: { type: "object", properties: {} },
|
||||
},
|
||||
{
|
||||
name: "customWidget_list",
|
||||
description: "List Custom Widgets",
|
||||
pathInRouter: ["customWidget", "list"],
|
||||
inputSchema: { type: "object", properties: {} },
|
||||
},
|
||||
{
|
||||
name: "customWidget_workshopSearch",
|
||||
description: "Search Workshop",
|
||||
pathInRouter: ["customWidget", "workshopSearch"],
|
||||
inputSchema: { type: "object", properties: {} },
|
||||
},
|
||||
],
|
||||
}),
|
||||
extractMcpTools: () => [
|
||||
{
|
||||
name: "board_getAllBoards",
|
||||
description: "List boards",
|
||||
type: "query",
|
||||
pathInRouter: ["board", "getAllBoards"],
|
||||
inputSchema: { type: "object", properties: {} },
|
||||
},
|
||||
{
|
||||
name: "customWidget_list",
|
||||
description: "List Custom Widgets",
|
||||
type: "query",
|
||||
pathInRouter: ["customWidget", "list"],
|
||||
inputSchema: { type: "object", properties: {} },
|
||||
},
|
||||
{
|
||||
name: "customWidget_workshopSearch",
|
||||
description: "Search Workshop",
|
||||
type: "query",
|
||||
pathInRouter: ["customWidget", "workshopSearch"],
|
||||
inputSchema: { type: "object", properties: {} },
|
||||
},
|
||||
],
|
||||
}));
|
||||
|
||||
import { POST } from "./route";
|
||||
|
||||
@@ -1,55 +1,11 @@
|
||||
import type { McpRouter, McpTool } from "@homarr/api/mcp";
|
||||
import { extractMcpToolsFromProcedures, getMcpRouterAsync } from "@homarr/api/mcp";
|
||||
import { createLogger } from "@homarr/core/infrastructure/logs";
|
||||
import { ErrorWithMetadata } from "@homarr/core/infrastructure/logs/error";
|
||||
import { extractMcpToolsFromProcedures, mcpRouter } from "@homarr/api/mcp";
|
||||
|
||||
const logger = createLogger({ module: "mcpTools" });
|
||||
let cache: ReturnType<typeof extractMcpToolsFromProcedures> | null = null;
|
||||
|
||||
export type McpProcedureType = "query" | "mutation";
|
||||
export function extractMcpTools() {
|
||||
if (cache) return cache;
|
||||
|
||||
export interface McpRuntime {
|
||||
router: McpRouter;
|
||||
tools: McpTool[];
|
||||
procedureTypes: ReadonlyMap<string, McpProcedureType>;
|
||||
}
|
||||
|
||||
const createMcpRuntimeAsync = async (): Promise<McpRuntime> => {
|
||||
const startedAt = Date.now();
|
||||
const router = await getMcpRouterAsync();
|
||||
const tools = extractMcpToolsFromProcedures(router);
|
||||
const procedures = router["_def"].procedures as unknown as Record<string, { ["_def"]?: { type?: McpProcedureType } }>;
|
||||
const procedureTypes = new Map<string, McpProcedureType>();
|
||||
|
||||
for (const [path, procedure] of Object.entries(procedures)) {
|
||||
const type = procedure["_def"]?.type;
|
||||
if (type === "query" || type === "mutation") procedureTypes.set(path, type);
|
||||
}
|
||||
|
||||
logger.info("Extracted MCP tools", {
|
||||
durationMs: Date.now() - startedAt,
|
||||
toolCount: tools.length,
|
||||
});
|
||||
return { router, tools, procedureTypes };
|
||||
};
|
||||
|
||||
let mcpRuntimePromise: Promise<McpRuntime> | null = null;
|
||||
|
||||
export async function getMcpRuntimeAsync(): Promise<McpRuntime> {
|
||||
if (mcpRuntimePromise) return await mcpRuntimePromise;
|
||||
|
||||
const loadPromise = createMcpRuntimeAsync();
|
||||
mcpRuntimePromise = loadPromise;
|
||||
|
||||
try {
|
||||
return await loadPromise;
|
||||
} catch (error) {
|
||||
if (mcpRuntimePromise === loadPromise) mcpRuntimePromise = null;
|
||||
logger.error(new ErrorWithMetadata("Failed to extract MCP tools", {}, { cause: error }));
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
export async function extractMcpTools(): Promise<McpTool[]> {
|
||||
const { tools } = await getMcpRuntimeAsync();
|
||||
return tools;
|
||||
cache = extractMcpToolsFromProcedures(mcpRouter);
|
||||
|
||||
return cache;
|
||||
}
|
||||
@@ -5,12 +5,11 @@ import { ResourceTemplate } from "@modelcontextprotocol/server";
|
||||
import { z } from "zod/v4";
|
||||
|
||||
import type { McpTool } from "@homarr/api/mcp";
|
||||
import { createTRPCContext } from "@homarr/api/mcp";
|
||||
import { createTRPCContext, mcpRouter } from "@homarr/api/mcp";
|
||||
import { API_KEY_HEADER_NAME, getSessionFromApiKeyAsync } from "@homarr/auth/api-key";
|
||||
import { extractBaseUrlFromHeaders } from "@homarr/common";
|
||||
import { ipAddressFromHeaders } from "@homarr/common/server";
|
||||
import { createLogger } from "@homarr/core/infrastructure/logs";
|
||||
import { ErrorWithMetadata } from "@homarr/core/infrastructure/logs/error";
|
||||
import { buildCustomWidgetMcpPrompt } from "@homarr/custom-widgets/authoring-prompt";
|
||||
import {
|
||||
getCustomWidgetComponent,
|
||||
@@ -23,7 +22,7 @@ import { db } from "@homarr/db";
|
||||
|
||||
import { getPackageVersion } from "~/versions/package-reader";
|
||||
import { getSafeAssistantToolError } from "../assistant/chat/assistant-tool-error";
|
||||
import { getMcpRuntimeAsync } from "./_extract-tools";
|
||||
import { extractMcpTools } from "./_extract-tools";
|
||||
import { createMcpProtocolHandler } from "./_protocol";
|
||||
|
||||
const logger = createLogger({ module: "mcpRoute" });
|
||||
@@ -81,8 +80,10 @@ function clearAuthFailures(ip: string | null) {
|
||||
authFailures.delete(ip ?? "unknown");
|
||||
}
|
||||
|
||||
const getVisibleTools = (tools: McpTool[], isAdmin: boolean) =>
|
||||
isAdmin ? tools : tools.filter((tool) => !tool.name.startsWith("customWidget_"));
|
||||
const getVisibleTools = (tools: McpTool[], isAdmin: boolean) => {
|
||||
if (isAdmin) return tools;
|
||||
return tools.filter((tool) => !tool.name.startsWith("customWidget_"));
|
||||
};
|
||||
|
||||
function registerCustomWidgetAuthoring(server: McpServer) {
|
||||
server.registerPrompt(
|
||||
@@ -322,23 +323,12 @@ export const handleMcpRequest = async (req: NextRequest) => {
|
||||
clearAuthFailures(ipAddress);
|
||||
logger.info("MCP request authenticated", { userId: session.user.id });
|
||||
|
||||
let mcpRuntime;
|
||||
try {
|
||||
mcpRuntime = await getMcpRuntimeAsync();
|
||||
} catch (error) {
|
||||
logger.error(new ErrorWithMetadata("MCP runtime is unavailable", {}, { cause: error }));
|
||||
return jsonErrorResponse(503, {
|
||||
error: "service_unavailable",
|
||||
hint: "The MCP tool registry could not be loaded. Check the Homarr logs and try again.",
|
||||
});
|
||||
}
|
||||
|
||||
const ctx = createTRPCContext({ session, headers: req.headers });
|
||||
const caller = mcpRuntime.router.createCaller(ctx);
|
||||
const caller = mcpRouter.createCaller(ctx);
|
||||
const isAdmin = session.user.permissions.includes("admin");
|
||||
const protocolHandler = createMcpProtocolHandler({
|
||||
caller,
|
||||
tools: getVisibleTools(mcpRuntime.tools, isAdmin),
|
||||
tools: getVisibleTools(extractMcpTools(), isAdmin),
|
||||
version: getPackageVersion(),
|
||||
instructions: SERVER_INSTRUCTIONS,
|
||||
configureServer: isAdmin ? registerCustomWidgetAuthoring : undefined,
|
||||
|
||||
@@ -50,6 +50,7 @@ const echoValidator = z.object({ message: z.string() });
|
||||
const echoTool: McpTool = {
|
||||
name: "echo",
|
||||
description: "Echo a message",
|
||||
type: "query",
|
||||
pathInRouter: ["echo"],
|
||||
inputValidator: echoValidator,
|
||||
inputSchema: z.toJSONSchema(echoValidator),
|
||||
|
||||
@@ -74,6 +74,7 @@ const unavailableMessageKeys = {
|
||||
|
||||
export const AssistantGate = ({ availability, children }: AssistantGateProps) => {
|
||||
const t = useI18n("assistant");
|
||||
|
||||
if (availability === "enabled") {
|
||||
return <EnabledAssistantRoot>{children}</EnabledAssistantRoot>;
|
||||
}
|
||||
|
||||
@@ -7,7 +7,7 @@ import { useI18n } from "@homarr/translation/client";
|
||||
|
||||
/**
|
||||
* Keeps the assistant discoverable in the spotlight when it is unavailable, without loading the
|
||||
* assistant runtime. Lives apart from `assistant-provider` so the unavailable path stays cheap.
|
||||
* assistant runtime. Lives apart from `assistant-provider` so the disabled path stays cheap.
|
||||
*/
|
||||
export const useRegisterAssistantSpotlightPlaceholder = (description: string) => {
|
||||
const t = useI18n("assistant");
|
||||
|
||||
@@ -63,14 +63,14 @@ describe("item actions create-item", () => {
|
||||
{
|
||||
layoutId,
|
||||
height: 1,
|
||||
width: 1,
|
||||
width: 2,
|
||||
...emptyPosition,
|
||||
sectionId: firstSectionId,
|
||||
},
|
||||
],
|
||||
}),
|
||||
);
|
||||
expect(emptyPositionSpy).toHaveBeenCalledWith([], layout.columnCount, undefined, { height: 1, width: 1 });
|
||||
expect(emptyPositionSpy).toHaveBeenCalledWith([], layout.columnCount, undefined, { height: 1, width: 2 });
|
||||
});
|
||||
test("should correctly pass containers and items to getFirstEmptyPosition", () => {
|
||||
// Arrange
|
||||
@@ -118,14 +118,14 @@ describe("item actions create-item", () => {
|
||||
expect(createdItem).toEqual(
|
||||
expect.objectContaining({
|
||||
kind: itemKind,
|
||||
layouts: [{ ...emptyPosition, height: 1, width: 1, sectionId: firstSectionId, layoutId }],
|
||||
layouts: [{ ...emptyPosition, height: 1, width: 2, sectionId: firstSectionId, layoutId }],
|
||||
}),
|
||||
);
|
||||
expect(spy).toHaveBeenCalledWith(
|
||||
[expect.objectContaining(itemAndSectionPosition), expect.objectContaining(itemAndSectionPosition)],
|
||||
layout.columnCount,
|
||||
undefined,
|
||||
{ height: 1, width: 1 },
|
||||
{ height: 1, width: 2 },
|
||||
);
|
||||
});
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import type { IntegrationKind, WidgetKind } from "@homarr/definitions";
|
||||
import { nativeFeatureCapabilities, widgetKinds } from "@homarr/definitions";
|
||||
import { widgetIntegrationConfigs, widgetKinds } from "@homarr/definitions";
|
||||
|
||||
export interface BoardRecipeRecommendation {
|
||||
widgetKind: WidgetKind;
|
||||
@@ -36,10 +36,10 @@ export const getBoardRecipeRecommendations = ({
|
||||
return widgetKinds
|
||||
.filter((widgetKind) => !existingKinds.has(widgetKind))
|
||||
.flatMap((widgetKind) => {
|
||||
const capability = nativeFeatureCapabilities[widgetKind as keyof typeof nativeFeatureCapabilities];
|
||||
const capability = widgetIntegrationConfigs[widgetKind as keyof typeof widgetIntegrationConfigs];
|
||||
if (!capability) return [];
|
||||
|
||||
const matches = capability.integrations.filter((integrationKind) => availableKinds.has(integrationKind));
|
||||
const matches = capability.supportedIntegrations.filter((integrationKind) => availableKinds.has(integrationKind));
|
||||
if (matches.length === 0) return [];
|
||||
|
||||
const preferredMatch = preferredIntegrationKind && matches.includes(preferredIntegrationKind);
|
||||
@@ -51,10 +51,10 @@ export const getBoardRecipeRecommendations = ({
|
||||
widgetKind,
|
||||
integrationKind,
|
||||
isNewlyAvailable: Boolean(preferredMatch),
|
||||
score: (preferredMatch ? 1_000 : 0) + matches.length * 10 - capability.integrations.length,
|
||||
score: (preferredMatch ? 1_000 : 0) + matches.length * 10 - capability.supportedIntegrations.length,
|
||||
},
|
||||
];
|
||||
})
|
||||
.toSorted((left, right) => right.score - left.score || left.widgetKind.localeCompare(right.widgetKind))
|
||||
.sort((left, right) => right.score - left.score || left.widgetKind.localeCompare(right.widgetKind))
|
||||
.slice(0, Math.max(0, limit));
|
||||
};
|
||||
@@ -91,7 +91,6 @@ const ItemSelectModalContent = ({
|
||||
initialSearch = "",
|
||||
}: ItemSelectModalContentProps) => {
|
||||
const [search, setSearch] = useState(initialSearch);
|
||||
const canBrowseCustomWidgets = isAdmin;
|
||||
const t = useI18n();
|
||||
const board = useRequiredBoard();
|
||||
const currentLayoutId = useCurrentLayout();
|
||||
@@ -480,7 +479,7 @@ const ItemSelectModalContent = ({
|
||||
/>
|
||||
))}
|
||||
|
||||
{canBrowseCustomWidgets && (
|
||||
{isAdmin && (
|
||||
<>
|
||||
<Divider
|
||||
label={
|
||||
@@ -535,7 +534,7 @@ const ItemSelectModalContent = ({
|
||||
)}
|
||||
</SimpleGrid>
|
||||
|
||||
{filteredItems.length === 0 && (!canBrowseCustomWidgets || filteredCustomWidgets.length === 0) && (
|
||||
{filteredItems.length === 0 && (!isAdmin || filteredCustomWidgets.length === 0) && (
|
||||
<Center p="xl">
|
||||
<Text c="dimmed">{t("common.noResults")}</Text>
|
||||
</Center>
|
||||
@@ -654,13 +653,12 @@ const ItemSelectModalFrame = ({
|
||||
const utils = clientApi.useUtils();
|
||||
const { data: session } = useSession();
|
||||
const isAdmin = session?.user.permissions.includes("admin") ?? false;
|
||||
const canBrowseCustomWidgets = isAdmin;
|
||||
const canCreateIntegration = session?.user.permissions.includes("integration-create") ?? false;
|
||||
const t = useI18n();
|
||||
const { data: integrationData } = clientApi.integration.all.useQuery();
|
||||
const { data: customWidgetDefs } = clientApi.customWidget.available.useQuery(
|
||||
{ boardId: innerProps.boardId },
|
||||
{ enabled: canBrowseCustomWidgets },
|
||||
{ enabled: isAdmin },
|
||||
);
|
||||
|
||||
return (
|
||||
|
||||
Reference in new issue
Block a user