Compare commits

..
Author SHA1 Message Date
Thomas Camlong acc11e8d17 fix(assistant): keep availability and authentication as separate states
The gate took a single boolean, so a signed-out visitor on an instance that
does have the assistant configured was told an administrator still needs to
configure it. The server cannot check availability without a session, so
collapsing the two states made it state something it could not know.

The layout now resolves one of enabled, unconfigured, unauthenticated or
error, which maps onto the four messages that already exist in the
translations. Dropping the client-side session check from the branch also
removes a possible remount once the session settles.
2026-08-07 18:18:53 +02:00
Thomas Camlong 09c56002d6 fix(db): name the custom widget secret foreign key within mysql limits
drizzle derives `custom_widget_secret_definition_id_custom_widget_definition_id_fk`,
which is 65 characters. MySQL caps identifiers at 64 and rejects it with
error 1059, so migration 0040 created the key under a shortened name. The
schema never declared that name, so every `migration:mysql:generate` run
re-proposed dropping and recreating the key under the long one, producing a
migration that cannot be applied.

Declaring the shortened name explicitly makes the schema match what is
actually in the database. Only the mysql schema changes; sqlite and
postgresql keep their derived names and need no migration.

The parity test matched keys across dialects by generated name, which no
longer holds, so it now matches on what a key links - its columns, target
table and target columns - and compares the referential actions. That also
means it reads keys through getTableConfig and so covers table-level
declarations, not just inline .references() ones.

Adds a guard test asserting no mysql table, foreign key or index name
exceeds 64 characters, so the next long pairing fails in CI instead of
producing an unappliable migration.
2026-08-07 18:14:23 +02:00
Thomas Camlong 7276bdad21 refactor(assistant): resolve availability through the existing endpoint
Drops the feature-flags module, the /api/features endpoint and its test in
favour of calling api.assistant.getAvailability() from the root layout, the
same way the layout already calls api.user.getById().

The gate only needs one server-resolved boolean, and getAvailability
already computes exactly that. The separate helper duplicated the enabled /
modelId / api-key-required rule, so the two copies could drift.

A general feature-flag surface is still worth having, but it belongs at
boot time across every optional feature rather than as a per-request
database read introduced alongside the assistant.
2026-08-07 17:59:51 +02:00
Thomas Camlong f25be7cf1c fix(api): make assistant message ordering iterative and cycle-safe
Addresses two review findings.

The depth-first walk recursed once per message. A conversation is a single
parent chain and getThread loads the whole thread, so a long conversation
could exhaust the call stack; a 30,000 message chain already threw
RangeError. It now uses an explicit stack, pushing siblings in reverse so
the incoming order is preserved.

The cycle fallback appended unvisited messages without walking their
children, so for child -> a, a -> b, b -> a the child came out ahead of its
parent. Each remaining component is now entered at the earliest member of
its cycle in the incoming order and traversed from there, which leaves only
the cycle's own back-edge violating parent-before-child - unavoidable by
definition - instead of arbitrary messages.
2026-08-07 17:49:39 +02:00
Thomas Camlong a4e1c3c60e perf(assistant): load the assistant runtime only when it is enabled
AssistantProvider was imported statically by the root layout, so
assistant-ui, the Lexical composer, the Markdown renderer and Mermaid were
downloaded and evaluated on every page. The existing getAvailability gate
only skipped rendering, and the disabled branch still imported the board
widget, which pulls the whole conversation surface. An instance with the
assistant switched off paid the full cost, and so did the login page.

Measured on two production builds, /auth/login went from 42 chunks /
3.81MB including an 820KB assistant-ui chunk to 36 chunks / 2.40MB with
none, and the board page from 55 / 5.35MB to 56 / 4.56MB with none.

Adds getFeatureFlagsAsync as one source of truth for instance-level
feature switches, exposed both to the root layout and through a small
GET /api/features endpoint. Resolving the flag on the server is what makes
this work: next/dynamic never references the heavy chunk when the feature
is off, while enabled instances still server-render it, so there is no
loading flash and no remount.
2026-08-07 17:30:27 +02:00
Thomas Camlong d82db8c6bd fix(api): return assistant messages with parents before children
getThread ordered purely by created_at, which is second-granular on MySQL
and SQLite. A user turn and the assistant reply routinely land in the same
second, so their relative order was whatever the storage engine returned.
The client rebuilds the conversation tree from parent_id, so a child ahead
of its parent corrupts the thread.

Adds an id tiebreaker for a deterministic database order, then a stable
topological pass that guarantees parents precede children while preserving
sibling order. Dangling parent references are treated as roots and cycles
are appended, so no message is ever dropped.
2026-08-07 17:30:16 +02:00
Thomas Camlong a4a53af79d fix(db): repair mysql migration metadata so drizzle-kit works again
0042_snapshot.json spelled one table's check-constraint key
`checkConstraints` instead of `checkConstraint`. drizzle-kit rejects the
whole folder over it and exits 0 with only a printed message, so
`migration:mysql:generate` has silently produced nothing since #6192.

Every mysql migration authored since then was therefore hand-written with
no snapshot, including 0044 and 0045. Without those snapshots the next
generate diffs against 0043 and re-emits the assistant tables.

Adds the two missing snapshots and a test covering the invariants: journal
and .sql files agree both ways, every entry has a snapshot, indexes are
gap-free, and snapshots agree on the key spelling. The same test would have
caught the v1.70.0 release that shipped 0042_add_widget_secret.sql without
a journal entry, leaving upgraded MySQL instances with no widget_secret
table.
2026-08-07 17:30:07 +02:00
Thomas Camlong 4acd6548b5 fix(db): store assistant messages as mediumtext on mysql
MySQL caps `text` at 65,535 bytes. A single base64 image attachment is
allowed up to ~1.34MB by the chat route, so appending such a message fails
with ER_DATA_TOO_LONG under the default strict mode. SQLite and PostgreSQL
are unaffected because their text types are effectively unbounded.

`mediumtext` holds 16MB, comfortably above the 2MB request cap.

The migration keeps the parenthesised default expression that every other
text default in this folder uses, because MySQL rejects a literal default
on TEXT columns.
2026-08-07 17:29:55 +02:00
Thomas Camlong da93adac14 ci: make e2e tests resilient to CI resource contention
E2E specs spin up Docker containers in parallel and intermittently time
out waiting for UI elements under shared-runner load. Limit parallel
workers to 2 and retry flaky failures twice so the container job can
complete and publish the PR preview image.
2026-08-07 16:28:33 +02:00
Thomas Camlong 926b7e2a5e test(e2e): wait for login form hydration before filling
The customJsx E2E test occasionally failed at login with the form
showing 'This field is required' on the username, because the submit
button was clicked while the form was still hydrating (it is disabled
until mounted) and the field values were reset by the re-mount. Wait
for the submit button to become enabled before filling, and assert the
username value stuck before submitting.
2026-08-07 15:49:21 +02:00
Thomas Camlong 17f08c6d14 ci: skip CI for autofix commits
The auto-fix workflow pushes a 'style: auto-fix lint and format' commit
after every user push, which triggers a new CI run that cancels the
in-progress run (concurrency cancel-in-progress) before the container
job can finish and publish the PR preview image. Mark the autofix commit
with [skip ci] so it does not restart CI, letting the user's run
complete and publish the image.
2026-08-07 15:25:03 +02:00
Thomas Camlong 32cad81e1c chore: remove unused assistant subtitle translation 2026-08-07 15:14:23 +02:00
Thomas Camlong eaacfdf7ce chore: simplify assistant header branding
Remove the 'Connected to this Homarr instance' subtitle (and the running
state text) from the assistant panel header, keeping only the robot icon
and title.
2026-08-07 15:07:26 +02:00
Thomas Camlong 96659fb543 chore: shrink demo assistant widget to 3x3 2026-08-07 15:04:24 +02:00
Thomas d625640051 feat: add assistant widget to demo dashboard (#6529) 2026-08-07 15:02:03 +02:00
Thomas b14756874e fix: handle missing message in assistant generation telemetry (#6528) 2026-08-07 14:59:59 +02:00
26 changed files with 6427 additions and 174 deletions

No files matched your search

+1 -1
View File
@@ -44,4 +44,4 @@ jobs:
- name: Commit fixes
uses: stefanzweifel/git-auto-commit-action@v7
with:
commit_message: "style: auto-fix lint and format"
commit_message: "style: auto-fix lint and format [skip ci]"
+13 -2
View File
@@ -26,7 +26,8 @@ import type { SupportedLanguage } from "@homarr/translation";
import { isLocaleRTL, isLocaleSupported } from "@homarr/translation";
import { Analytics } from "~/components/layout/analytics";
import { AssistantProvider } from "~/components/assistant/assistant-provider";
import type { AssistantAvailability } from "~/components/assistant/assistant-gate";
import { AssistantGate } from "~/components/assistant/assistant-gate";
import { CrowdinLiveTranslation } from "~/components/layout/crowdin-live-translation";
import { SearchEngineOptimization } from "~/components/layout/search-engine-optimization";
@@ -86,6 +87,16 @@ export default async function Layout(props: {
const session = await auth();
const user = session ? await api.user.getById({ userId: session.user.id }).catch(() => null) : null;
const serverSettings = await getServerSettingsAsync(db);
// Resolved on the server so the assistant runtime chunk is never referenced by a page unless the
// assistant is actually usable. Reuses the availability check the management page invalidates.
// The states stay distinct so a signed-out visitor is told to sign in rather than that the
// instance is unconfigured, which the server cannot know for them.
const assistantAvailability: AssistantAvailability = !session
? "unauthenticated"
: await api.assistant
.getAvailability()
.then((availability) => (availability.enabled ? ("enabled" as const) : ("unconfigured" as const)))
.catch(() => "error" as const);
const colorScheme = await getCurrentColorSchemeAsync();
const direction = isLocaleRTL((await props.params).locale) ? "rtl" : "ltr";
@@ -124,7 +135,7 @@ export default async function Layout(props: {
(innerProps) => <CustomMantineProvider {...innerProps} defaultColorScheme={colorScheme} />,
(innerProps) => <ModalProvider {...innerProps} />,
(innerProps) => <SpotlightProvider {...innerProps} />,
(innerProps) => <AssistantProvider {...innerProps} />,
(innerProps) => <AssistantGate availability={assistantAvailability} {...innerProps} />,
]);
const { locale } = await props.params;
@@ -9,7 +9,7 @@ import { clientApi } from "@homarr/api/client";
import { showErrorNotification, showSuccessNotification } from "@homarr/notifications";
import { useScopedI18n } from "@homarr/translation/client";
import { useOptionalHomarrAssistant } from "~/components/assistant/assistant-provider";
import { useOptionalHomarrAssistant } from "~/components/assistant/assistant-context";
const PROMPT_HEADER = `You are helping configure a Homarr custom widget. Homarr is a self-hosted dashboard that can display data from any API endpoint as a widget.
@@ -0,0 +1,56 @@
"use client";
import { createContext, useContext } from "react";
import type { AssistantReasoningMode, AssistantRuntimeModelOption } from "./assistant-preferences";
/**
* Context and hooks live in their own module so that consumers such as the user menu can read the
* assistant state without pulling in `assistant-provider`, which statically imports the
* assistant-ui runtime, the Lexical composer, the Markdown renderer and Mermaid. Importing those
* from a component that renders on every page would ship them to every page.
*/
export interface AssistantContextValue {
enabled: boolean;
unavailableDescription: string | null;
opened: boolean;
isRunning: boolean;
isRefreshing: boolean;
unreadCount: number;
open: () => void;
close: () => void;
toggle: () => void;
sendPrompt: (prompt: string) => boolean;
refreshCurrentView: () => Promise<void>;
}
export const AssistantContext = createContext<AssistantContextValue | null>(null);
export interface AssistantPreferencesContextValue {
defaultModelId: string | null;
modelId: string | null;
models: AssistantRuntimeModelOption[];
reasoning: AssistantReasoningMode;
isLoading: boolean;
setModelId: (modelId: string) => void;
setReasoning: (reasoning: AssistantReasoningMode) => void;
getRequestBody: () => { modelId?: string; reasoning: AssistantReasoningMode };
}
export const AssistantPreferencesContext = createContext<AssistantPreferencesContextValue | null>(null);
export const useHomarrAssistant = () => {
const value = useContext(AssistantContext);
if (!value) {
throw new Error("useHomarrAssistant must be used within AssistantProvider");
}
return value;
};
export const useOptionalHomarrAssistant = () => useContext(AssistantContext);
export const useAssistantPreferences = () => {
const value = useContext(AssistantPreferencesContext);
if (!value) throw new Error("useAssistantPreferences must be used within AssistantPreferencesProvider");
return value;
};
@@ -0,0 +1,79 @@
"use client";
import type { PropsWithChildren } from "react";
import { useMemo } from "react";
import dynamic from "next/dynamic";
import { useScopedI18n } from "@homarr/translation/client";
import { AssistantWidgetRendererProvider } from "@homarr/widgets";
import { AssistantContext } from "./assistant-context";
import { AssistantBoardWidgetLazy } from "./assistant-widget-lazy";
import { useRegisterAssistantSpotlightPlaceholder } from "./assistant-spotlight-placeholder";
/**
* The enabled runtime is the expensive half of the assistant (assistant-ui, Lexical, Markdown,
* Mermaid). Loading it through `next/dynamic` puts it in its own chunk that is only requested when
* this component actually renders. Because `enabled` is resolved on the server and handed down as a
* prop, an instance with the assistant switched off never renders it, so the chunk is never part of
* the page payload. SSR stays on so enabled instances render children in the initial HTML without a
* loading flash or a remount once the chunk arrives.
*/
const EnabledAssistantRoot = dynamic(() =>
import("./assistant-provider").then((module) => ({ default: module.EnabledAssistantRoot })),
);
const DisabledAssistant = ({ children, description }: PropsWithChildren<{ description: string }>) => {
useRegisterAssistantSpotlightPlaceholder(description);
const value = useMemo(
() => ({
enabled: false,
unavailableDescription: description,
opened: false,
isRunning: false,
isRefreshing: false,
unreadCount: 0,
open: () => undefined,
close: () => undefined,
toggle: () => undefined,
sendPrompt: () => false,
refreshCurrentView: () => Promise.resolve(),
}),
[description],
);
return (
<AssistantContext.Provider value={value}>
<AssistantWidgetRendererProvider renderer={AssistantBoardWidgetLazy}>{children}</AssistantWidgetRendererProvider>
</AssistantContext.Provider>
);
};
/**
* Why the assistant is or is not usable, resolved on the server.
*
* `unauthenticated` is kept apart from `unconfigured` because the server cannot check availability
* for a signed-out visitor, and telling them the instance is unconfigured would be a guess.
*/
export type AssistantAvailability = "enabled" | "unconfigured" | "unauthenticated" | "error";
interface AssistantGateProps extends PropsWithChildren {
availability: AssistantAvailability;
}
const unavailableMessageKeys = {
unauthenticated: "unavailable.signIn",
unconfigured: "unavailable.notConfigured",
error: "unavailable.error",
} as const;
export const AssistantGate = ({ availability, children }: AssistantGateProps) => {
const t = useScopedI18n("common.assistant");
if (availability === "enabled") {
return <EnabledAssistantRoot>{children}</EnabledAssistantRoot>;
}
return <DisabledAssistant description={t(unavailableMessageKeys[availability])}>{children}</DisabledAssistant>;
};
@@ -2454,9 +2454,6 @@ export const AssistantConversationSurface = ({
<Text fw={700} lh={1.1} lineClamp={1}>
{t("title")}
</Text>
<Text size="xs" c="dimmed" lineClamp={1}>
{isRunning ? t("activity.thinking") : t("subtitle")}
</Text>
</div>
</Group>
<Group className={classes.panelActions} gap={2} wrap="nowrap">
@@ -1,7 +1,7 @@
"use client";
import type { PropsWithChildren } from "react";
import { createContext, useCallback, useContext, useEffect, useMemo, useRef, useState } from "react";
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import { useRouter } from "next/navigation";
import type {
AttachmentAdapter,
@@ -17,7 +17,6 @@ import { useHotkeys } from "@mantine/hooks";
import { createAssistantStream } from "assistant-stream";
import { clientApi, fetchApi } from "@homarr/api/client";
import { useSession } from "@homarr/auth/client";
import { createId } from "@homarr/common";
import { hotkeys } from "@homarr/definitions";
import { showErrorNotification, showWarningNotification } from "@homarr/notifications";
@@ -30,6 +29,7 @@ import {
} from "@homarr/spotlight";
import { AssistantWidgetRendererProvider } from "@homarr/widgets";
import { AssistantContext, AssistantPreferencesContext, useAssistantPreferences } from "./assistant-context";
import { shouldAutomaticallyContinueAssistant } from "./assistant-auto-submit";
import { AssistantAutoApprovalProvider } from "./assistant-auto-approval";
import { createAssistantBrowserToolExecutors } from "./assistant-browser-tool-executors";
@@ -53,33 +53,6 @@ import { initialAssistantNotificationState, updateAssistantNotificationState } f
import { AssistantViewRefreshProvider, useAssistantViewRefresh } from "./assistant-view-refresh";
import { AssistantBoardWidget } from "./assistant-widget";
interface AssistantContextValue {
enabled: boolean;
unavailableDescription: string | null;
opened: boolean;
isRunning: boolean;
isRefreshing: boolean;
unreadCount: number;
open: () => void;
close: () => void;
toggle: () => void;
sendPrompt: (prompt: string) => boolean;
refreshCurrentView: () => Promise<void>;
}
const AssistantContext = createContext<AssistantContextValue | null>(null);
interface AssistantPreferencesContextValue {
defaultModelId: string | null;
modelId: string | null;
models: AssistantRuntimeModelOption[];
reasoning: AssistantReasoningMode;
isLoading: boolean;
setModelId: (modelId: string) => void;
setReasoning: (reasoning: AssistantReasoningMode) => void;
getRequestBody: () => { modelId?: string; reasoning: AssistantReasoningMode };
}
const AssistantPreferencesContext = createContext<AssistantPreferencesContextValue | null>(null);
const ignoreUnsupportedArchiveAction = () => Promise.resolve();
const assistantImageAttachmentTypes = ["image/gif", "image/jpeg", "image/png", "image/webp"];
const assistantDocumentAttachmentTypes = [
@@ -158,22 +131,6 @@ const createAssistantAttachmentAdapter = (allowImages: boolean): AttachmentAdapt
};
};
export const useHomarrAssistant = () => {
const value = useContext(AssistantContext);
if (!value) {
throw new Error("useHomarrAssistant must be used within AssistantProvider");
}
return value;
};
export const useOptionalHomarrAssistant = () => useContext(AssistantContext);
export const useAssistantPreferences = () => {
const value = useContext(AssistantPreferencesContext);
if (!value) throw new Error("useAssistantPreferences must be used within AssistantPreferencesProvider");
return value;
};
const AssistantPreferencesProvider = ({ children }: PropsWithChildren) => {
const { data, isLoading } = clientApi.assistant.getRuntimeOptions.useQuery(undefined, {
staleTime: 10 * 60_000,
@@ -766,79 +723,16 @@ const EnabledAssistantProvider = ({ children }: PropsWithChildren) => {
);
};
interface DisabledAssistantProviderProps extends PropsWithChildren {
description: string;
}
const DisabledAssistantProvider = ({ children, description }: DisabledAssistantProviderProps) => {
const t = useScopedI18n("common.assistant");
const spotlightItem = useMemo(
() => ({
id: "homarr-assistant",
name: t("spotlight"),
icon: "/logo/logo.png",
description,
unavailable: true,
alwaysVisible: true,
interaction: (_query: string) => ({ type: "none" as const }),
}),
[description, t],
);
useRegisterSpotlightContextResults("homarr-assistant", [spotlightItem], [spotlightItem]);
const value = useMemo(
() => ({
enabled: false,
unavailableDescription: description,
opened: false,
isRunning: false,
isRefreshing: false,
unreadCount: 0,
open: () => undefined,
close: () => undefined,
toggle: () => undefined,
sendPrompt: () => false,
refreshCurrentView: () => Promise.resolve(),
}),
[description],
);
return (
<AssistantContext.Provider value={value}>
<AssistantWidgetRendererProvider renderer={AssistantBoardWidget}>{children}</AssistantWidgetRendererProvider>
</AssistantContext.Provider>
);
};
export const AssistantProvider = ({ children }: PropsWithChildren) => {
const t = useScopedI18n("common.assistant");
const session = useSession();
const { data, isLoading, isError } = clientApi.assistant.getAvailability.useQuery(undefined, {
enabled: session.status === "authenticated",
staleTime: 60_000,
});
const enabled = Boolean(data?.enabled && session.status === "authenticated");
if (enabled) {
return (
<AssistantPreferencesProvider>
<AssistantViewRefreshProvider>
<AssistantRuntime>
<EnabledAssistantProvider>{children}</EnabledAssistantProvider>
</AssistantRuntime>
</AssistantViewRefreshProvider>
</AssistantPreferencesProvider>
);
}
const unavailableDescription =
session.status !== "authenticated"
? t("unavailable.signIn")
: isLoading
? t("unavailable.checking")
: isError
? t("unavailable.error")
: t("unavailable.notConfigured");
return <DisabledAssistantProvider description={unavailableDescription}>{children}</DisabledAssistantProvider>;
};
/**
* The enabled half of the assistant. Only rendered (and only downloaded) when the instance has the
* assistant configured and the visitor is signed in - see `assistant-gate`.
*/
export const EnabledAssistantRoot = ({ children }: PropsWithChildren) => (
<AssistantPreferencesProvider>
<AssistantViewRefreshProvider>
<AssistantRuntime>
<EnabledAssistantProvider>{children}</EnabledAssistantProvider>
</AssistantRuntime>
</AssistantViewRefreshProvider>
</AssistantPreferencesProvider>
);
@@ -0,0 +1,28 @@
"use client";
import { useMemo } from "react";
import { useRegisterSpotlightContextResults } from "@homarr/spotlight";
import { useScopedI18n } from "@homarr/translation/client";
/**
* Keeps the assistant discoverable in the spotlight when it is unavailable, without loading the
* assistant runtime. Lives apart from `assistant-provider` so the disabled path stays cheap.
*/
export const useRegisterAssistantSpotlightPlaceholder = (description: string) => {
const t = useScopedI18n("common.assistant");
const spotlightItem = useMemo(
() => ({
id: "homarr-assistant",
name: t("spotlight"),
icon: "/logo/logo.png",
description,
unavailable: true,
alwaysVisible: true,
interaction: (_query: string) => ({ type: "none" as const }),
}),
[description, t],
);
useRegisterSpotlightContextResults("homarr-assistant", [spotlightItem], [spotlightItem]);
};
@@ -0,0 +1,18 @@
"use client";
import dynamic from "next/dynamic";
import type { WidgetComponentProps } from "@homarr/widgets";
/**
* Boards register a renderer for the assistant widget even when no board uses one. Loading the
* renderer lazily keeps the conversation surface out of the board bundle until a board actually
* places an assistant widget.
*/
const LazyAssistantBoardWidget = dynamic(() =>
import("./assistant-widget").then((module) => ({ default: module.AssistantBoardWidget })),
);
export const AssistantBoardWidgetLazy = (props: WidgetComponentProps<"assistant">) => (
<LazyAssistantBoardWidget {...props} />
);
@@ -12,7 +12,7 @@ import type { WidgetComponentProps } from "@homarr/widgets";
import classes from "./assistant-panel.module.css";
import { AssistantConversationSurface } from "./assistant-panel";
import { getPendingAssistantAction } from "./assistant-pending-action";
import { useAssistantPreferences, useHomarrAssistant } from "./assistant-provider";
import { useAssistantPreferences, useHomarrAssistant } from "./assistant-context";
export const AssistantBoardWidget = (props: WidgetComponentProps<"assistant">) => {
const assistant = useHomarrAssistant();
@@ -24,7 +24,7 @@ import { useScopedI18n } from "@homarr/translation/client";
import { Link } from "@homarr/ui";
import { useAuthContext } from "~/app/[locale]/_client-providers/session";
import { useOptionalHomarrAssistant } from "./assistant/assistant-provider";
import { useOptionalHomarrAssistant } from "./assistant/assistant-context";
import { CurrentColorSchemeCombobox } from "./color-scheme/current-color-scheme-combobox";
import { CurrentLanguageCombobox } from "./language/current-language-combobox";
import { DockerQuickAccessModal } from "./layout/header/docker-quick-access-modal";
+6 -1
View File
@@ -43,9 +43,14 @@ describe("Custom JSX custom widgets", () => {
try {
await page.goto(`${baseUrl}/auth/login`);
// The submit button is disabled until the form finishes hydrating, so wait for it
// before filling to avoid a hydration re-mount resetting the field values.
const submitButton = page.locator("css=button[type='submit']");
await expect(submitButton).toBeEnabled({ timeout: 30_000 });
await page.getByLabel("Username").fill(adminCredentials.username);
await page.locator("#password").fill(adminCredentials.password);
await page.locator("css=button[type='submit']").click();
await expect(page.getByLabel("Username")).toHaveValue(adminCredentials.username);
await submitButton.click();
await expect(page).not.toHaveURL(/\/auth\/login/, { timeout: 30_000 });
await page.goto(`${baseUrl}/manage/custom-widgets/new`);
+1 -1
View File
@@ -36,7 +36,7 @@
"scripts:update-readme-integrations": "tsx ./scripts/update-integration-list.mts",
"start": "concurrently \"pnpm with-env node apps/tasks/tasks.cjs\" \"pnpm with-env node apps/websocket/wssServer.cjs\" \"pnpm -F nextjs start\"",
"test": "cross-env NODE_ENV=development CI=true vitest run --exclude e2e --coverage.enabled ",
"test:e2e": "cross-env NODE_ENV=development CI=true vitest e2e",
"test:e2e": "cross-env NODE_ENV=development CI=true vitest e2e --retry 2 --maxWorkers 2",
"test:ui": "cross-env NODE_ENV=development CI=true vitest --exclude e2e --ui --coverage.enabled",
"typecheck": "turbo typecheck",
"with-env": "dotenv -e .env --"
+101
View File
@@ -0,0 +1,101 @@
interface OrderableMessage {
id: string;
parentId: string | null;
}
/**
* Orders thread messages so that a parent always precedes its children.
*
* The database can only order by `created_at`, which is second-granular on MySQL and SQLite. Two
* messages written in the same second therefore tie, and the client rebuilds the conversation tree
* from `parent_id`, so a child arriving before its parent corrupts the thread. Sorting here keeps
* the relative order of siblings from the incoming (already deterministic) list.
*/
export const orderMessagesByParent = <TMessage extends OrderableMessage>(messages: TMessage[]): TMessage[] => {
if (messages.length <= 1) return messages;
const messagesById = new Map<string, TMessage>();
const incomingIndexById = new Map<string, number>();
messages.forEach((message, index) => {
messagesById.set(message.id, message);
incomingIndexById.set(message.id, index);
});
const childrenByParent = new Map<string | null, TMessage[]>();
const resolvedParentIds = new Map<string, string | null>();
for (const message of messages) {
// Treat a dangling parent reference as a root so the message is never dropped.
const parentId = message.parentId !== null && messagesById.has(message.parentId) ? message.parentId : null;
resolvedParentIds.set(message.id, parentId);
const siblings = childrenByParent.get(parentId);
if (siblings) siblings.push(message);
else childrenByParent.set(parentId, [message]);
}
const ordered: TMessage[] = [];
const visited = new Set<string>();
// A conversation is one long parent chain, so the traversal depth grows with the message count.
// An explicit stack keeps a long thread from exhausting the call stack.
const visitFrom = (start: TMessage) => {
const stack: TMessage[] = [start];
while (stack.length > 0) {
const message = stack.pop();
if (!message || visited.has(message.id)) continue;
visited.add(message.id);
ordered.push(message);
const children = childrenByParent.get(message.id);
if (!children) continue;
// Pushed in reverse so the stack pops siblings back in their incoming order.
for (let index = children.length - 1; index >= 0; index--) {
const child = children[index];
if (child) stack.push(child);
}
}
};
for (const root of childrenByParent.get(null) ?? []) {
visitFrom(root);
}
// Whatever is left belongs to a parent cycle, which by definition has no root to start from.
// Corrupted data should still come back complete and in a sensible order, so each cycle is
// broken at its earliest member in the incoming order and traversed from there. Only that one
// back-edge can violate parent-before-child; every other message in the component is fine.
const findCycleEntry = (start: TMessage) => {
const path: TMessage[] = [start];
const positionInPath = new Map<string, number>([[start.id, 0]]);
let current = start;
for (;;) {
const parentId = resolvedParentIds.get(current.id) ?? null;
if (parentId === null || visited.has(parentId)) return current;
const parent = messagesById.get(parentId);
if (!parent) return current;
const loopStart = positionInPath.get(parent.id);
if (loopStart !== undefined) {
// Closed the loop: the cycle is the tail of the walked path. Start from whichever of its
// members came first in the incoming list so the choice is deterministic.
return path.slice(loopStart).reduce((earliest, member) => {
const memberIndex = incomingIndexById.get(member.id) ?? 0;
const earliestIndex = incomingIndexById.get(earliest.id) ?? 0;
return memberIndex < earliestIndex ? member : earliest;
});
}
positionInPath.set(parent.id, path.length);
path.push(parent);
current = parent;
}
};
for (const message of messages) {
if (visited.has(message.id)) continue;
visitFrom(findCycleEntry(message));
}
return ordered;
};
+5 -2
View File
@@ -29,6 +29,7 @@ import {
import type { createTRPCContext } from "../trpc";
import { fetchOpenRouterGenerationTelemetryAsync } from "../assistant-generation-telemetry";
import { orderMessagesByParent } from "../assistant-message-order";
import { verifyAssistantGenerationAccessToken } from "../assistant-generation-access";
import { createTRPCRouter, isDemoMode, permissionRequiredProcedure, protectedProcedure } from "../trpc";
import { boardRouter } from "./board";
@@ -710,11 +711,13 @@ export const assistantRouter = createTRPCRouter({
const thread = await ownedThreadAsync(ctx.db, input.threadId, ctx.session.user.id);
const messages = await ctx.db.query.assistantMessages.findMany({
where: eq(assistantMessages.threadId, thread.id),
orderBy: assistantMessages.createdAt,
// `created_at` is only second-granular on MySQL and SQLite, so sibling messages written in the
// same second tie. The id tiebreaker makes the database order at least deterministic.
orderBy: [asc(assistantMessages.createdAt), asc(assistantMessages.id)],
});
return {
thread,
messages: messages.map((message) => ({
messages: orderMessagesByParent(messages).map((message) => ({
id: message.id,
parentId: message.parentId,
format: message.format,
@@ -0,0 +1,89 @@
import { describe, expect, test } from "vitest";
import { orderMessagesByParent } from "../assistant-message-order";
const message = (id: string, parentId: string | null) => ({ id, parentId });
describe("orderMessagesByParent", () => {
test("keeps an already correct order untouched", () => {
const messages = [message("a", null), message("b", "a"), message("c", "b")];
expect(orderMessagesByParent(messages).map((entry) => entry.id)).toEqual(["a", "b", "c"]);
});
test("moves a child that the database returned before its parent", () => {
// Happens when created_at ties on MySQL/SQLite, which store whole seconds only.
const messages = [message("c", "b"), message("a", null), message("b", "a")];
expect(orderMessagesByParent(messages).map((entry) => entry.id)).toEqual(["a", "b", "c"]);
});
test("preserves sibling order from the incoming list", () => {
const messages = [message("root", null), message("first", "root"), message("second", "root")];
expect(orderMessagesByParent(messages).map((entry) => entry.id)).toEqual(["root", "first", "second"]);
});
test("keeps branches grouped under their parent", () => {
const messages = [
message("root", null),
message("branch-a", "root"),
message("branch-b", "root"),
message("branch-a-child", "branch-a"),
];
expect(orderMessagesByParent(messages).map((entry) => entry.id)).toEqual([
"root",
"branch-a",
"branch-a-child",
"branch-b",
]);
});
test("treats a dangling parent reference as a root instead of dropping the message", () => {
const messages = [message("orphan", "deleted-parent"), message("root", null)];
expect(orderMessagesByParent(messages).map((entry) => entry.id)).toEqual(["orphan", "root"]);
});
test("does not lose messages that form a cycle", () => {
const messages = [message("a", "b"), message("b", "a")];
expect(
orderMessagesByParent(messages)
.map((entry) => entry.id)
.toSorted(),
).toEqual(["a", "b"]);
});
test("keeps a child of a cycle behind its parent", () => {
// `child` hangs off the cycle a -> b -> a. Only the cycle's own back-edge can break the
// parent-before-child contract; `child` must still come after `a`.
const messages = [message("child", "a"), message("a", "b"), message("b", "a")];
const ordered = orderMessagesByParent(messages).map((entry) => entry.id);
expect(ordered.toSorted()).toEqual(["a", "b", "child"]);
expect(ordered.indexOf("a")).toBeLessThan(ordered.indexOf("child"));
});
test("handles a chain deeper than the call stack", () => {
// `getThread` loads every message in a thread, and a conversation is one long parent chain, so
// a recursive walk would blow the stack on a long thread.
const depth = 50_000;
const messages = Array.from({ length: depth }, (_, index) =>
message(`m${index}`, index === 0 ? null : `m${index - 1}`),
);
const ordered = orderMessagesByParent(messages);
expect(ordered).toHaveLength(depth);
expect(ordered[0]?.id).toBe("m0");
expect(ordered[depth - 1]?.id).toBe(`m${depth - 1}`);
});
test("returns short lists unchanged", () => {
expect(orderMessagesByParent([])).toEqual([]);
expect(orderMessagesByParent([message("only", null)]).map((entry) => entry.id)).toEqual(["only"]);
});
});
@@ -26,7 +26,7 @@ CREATE TABLE `assistant_message` (
`thread_id` varchar(64) NOT NULL,
`parent_id` varchar(128),
`format` varchar(64) DEFAULT 'ai-sdk/v6' NOT NULL,
`content` text DEFAULT ('{"json": {}}') NOT NULL,
`content` mediumtext DEFAULT ('{"json": {}}') NOT NULL,
`created_at` timestamp DEFAULT (now()) NOT NULL,
CONSTRAINT `assistant_message_id` PRIMARY KEY(`id`)
);
@@ -2627,7 +2627,7 @@
}
},
"uniqueConstraints": {},
"checkConstraints": {}
"checkConstraint": {}
}
},
"views": {},
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+2
View File
@@ -512,6 +512,8 @@ const buildDemoWidgets = (appIds: string[]): DemoWidget[] => [
{ kind: "notebook", width: 4, height: 4, needsIntegration: false },
{ kind: "dockerContainers", width: 6, height: 2, needsIntegration: false },
{ kind: "weather", width: 2, height: 1, needsIntegration: false },
// Row 12: assistant
{ kind: "assistant", width: 3, height: 3, needsIntegration: false },
];
const seedDemoUserAsync = async (db: Database) => {
+14 -4
View File
@@ -6,8 +6,10 @@ import type { AnyMySqlColumn } from "drizzle-orm/mysql-core";
import {
boolean,
customType,
foreignKey,
index,
int,
mediumtext,
mysqlTable,
primaryKey,
smallint,
@@ -536,7 +538,9 @@ export const assistantMessages = mysqlTable(
.references(() => assistantThreads.id, { onDelete: "cascade" }),
parentId: varchar({ length: 128 }),
format: varchar({ length: 64 }).notNull().default("ai-sdk/v6"),
content: text().default(emptySuperJSON).notNull(),
// `text` only holds 64KB on MySQL, which a single base64 image attachment already exceeds.
// `mediumtext` holds 16MB, comfortably above the 2MB request cap enforced by the chat route.
content: mediumtext().default(emptySuperJSON).notNull(),
createdAt: timestamp().notNull().defaultNow(),
},
(message) => ({
@@ -594,14 +598,20 @@ export const customWidgetSecrets = mysqlTable(
updatedAt: timestamp()
.$onUpdateFn(() => new Date())
.notNull(),
definitionId: varchar({ length: 64 })
.notNull()
.references(() => customWidgetDefinitions.id, { onDelete: "cascade" }),
definitionId: varchar({ length: 64 }).notNull(),
},
(table) => ({
compoundKey: primaryKey({
columns: [table.definitionId, table.kind],
}),
// The name drizzle derives here is 65 characters, one over MySQL's 64-character identifier
// limit, so migration 0040 created the key under this shortened name. Declaring it explicitly
// keeps `drizzle-kit generate` from re-proposing an ALTER that MySQL rejects with error 1059.
definitionReference: foreignKey({
name: "cw_secret_definition_id_cw_definition_id_fk",
columns: [table.definitionId],
foreignColumns: [customWidgetDefinitions.id],
}).onDelete("cascade"),
}),
);
@@ -0,0 +1,98 @@
import fs from "fs";
import path from "path";
import { describe, expect, test } from "vitest";
/**
* Guards the migration folders against the failure modes that have actually shipped:
*
* - v1.70.0 shipped `0042_add_widget_secret.sql` for MySQL without a `_journal.json` entry, so the
* migrator never ran it and MySQL instances upgraded from that release had no `widget_secret`
* table.
* - The follow-up fix hand-wrote `0042_snapshot.json` with a `checkConstraints` key instead of
* `checkConstraint`, which made `drizzle-kit generate` reject the folder and silently exit 0 for
* every later MySQL migration.
*/
const dialects = ["mysql", "postgresql", "sqlite"] as const;
const migrationsRoot = path.join(__dirname, "..", "migrations");
interface Journal {
entries: { idx: number; tag: string }[];
}
const readDialect = (dialect: (typeof dialects)[number]) => {
const folder = path.join(migrationsRoot, dialect);
const journal = JSON.parse(fs.readFileSync(path.join(folder, "meta", "_journal.json"), "utf8")) as Journal;
const sqlFiles = fs
.readdirSync(folder)
.filter((file) => file.endsWith(".sql"))
.map((file) => file.replace(/\.sql$/u, ""));
const snapshots = fs
.readdirSync(path.join(folder, "meta"))
.filter((file) => /^\d+_snapshot\.json$/u.test(file))
.map((file) => file.split("_")[0]);
return { folder, journal, sqlFiles, snapshots };
};
describe.each(dialects)("%s migrations", (dialect) => {
test("every journal entry has a matching .sql file", () => {
const { journal, sqlFiles } = readDialect(dialect);
const missing = journal.entries.filter((entry) => !sqlFiles.includes(entry.tag)).map((entry) => entry.tag);
expect(missing).toEqual([]);
});
test("every .sql file is registered in the journal", () => {
const { journal, sqlFiles } = readDialect(dialect);
const tags = new Set(journal.entries.map((entry) => entry.tag));
const orphans = sqlFiles.filter((file) => !tags.has(file));
expect(orphans).toEqual([]);
});
test("every journal entry has a matching snapshot", () => {
const { journal, snapshots } = readDialect(dialect);
const missing = journal.entries
.filter((entry) => !snapshots.includes(String(entry.idx).padStart(4, "0")))
.map((entry) => entry.tag);
expect(missing).toEqual([]);
});
test("journal indexes are unique and gap-free", () => {
const { journal } = readDialect(dialect);
const indexes = journal.entries.map((entry) => entry.idx);
expect(indexes).toEqual(indexes.map((_, position) => position));
});
// The check-constraint key is spelled differently per snapshot format version (MySQL's v5 uses
// `checkConstraint`, the newer SQLite/Postgres formats use `checkConstraints`), so rather than
// hard-coding a spelling this asserts every snapshot in a dialect agrees with the others. A
// hand-written snapshot that gets it wrong stands out as the odd one and fails drizzle-kit.
test("all snapshots agree on the check-constraint key spelling", () => {
const { folder, snapshots } = readDialect(dialect);
const spellingsPerSnapshot = snapshots.map((snapshot) => {
const file = path.join(folder, "meta", `${snapshot}_snapshot.json`);
const parsed = JSON.parse(fs.readFileSync(file, "utf8")) as { tables?: Record<string, object> };
const spellings = new Set<string>();
for (const table of Object.values(parsed.tables ?? {})) {
if ("checkConstraint" in table) spellings.add("checkConstraint");
if ("checkConstraints" in table) spellings.add("checkConstraints");
}
return { snapshot, spellings: [...spellings].toSorted().join("+") };
});
const used = [...new Set(spellingsPerSnapshot.map((entry) => entry.spellings))].filter(
(spelling) => spelling.length > 0,
);
expect(used, `snapshots disagree: ${JSON.stringify(spellingsPerSnapshot.filter((e) => e.spellings))}`).toHaveLength(
1,
);
});
});
+18 -1
View File
@@ -1,9 +1,10 @@
import path from "path";
import { MySqlContainer } from "@testcontainers/mysql";
import { eq } from "drizzle-orm";
import { drizzle } from "drizzle-orm/mysql2";
import { migrate } from "drizzle-orm/mysql2/migrator";
import mysql from "mysql2";
import { describe, test } from "vitest";
import { describe, expect, test } from "vitest";
import { DB_CASING } from "@homarr/core/infrastructure/db/constants";
@@ -35,6 +36,22 @@ describe("Mysql Migration", () => {
// Check if users table exists
await database.query.users.findMany();
// A single base64 image attachment already exceeds the 64KB that `text` holds on MySQL, so
// `assistant_message.content` has to be `mediumtext`. Everything below 64KB passes either way.
const largeContent = "x".repeat(200_000);
await database.insert(mysqlSchema.users).values({ id: "size-check-user", name: "size-check" });
await database.insert(mysqlSchema.assistantThreads).values({ id: "size-check-thread", userId: "size-check-user" });
await database.insert(mysqlSchema.assistantMessages).values({
id: "size-check-message",
threadId: "size-check-thread",
parentId: null,
content: largeContent,
});
const [storedMessage] = await database.query.assistantMessages.findMany({
where: eq(mysqlSchema.assistantMessages.id, "size-check-message"),
});
expect(storedMessage?.content).toHaveLength(largeContent.length);
connection.end();
await mysqlContainer.stop();
}, 120_000);
+58 -32
View File
@@ -1,7 +1,10 @@
/* eslint-disable @typescript-eslint/no-non-null-assertion */
import { is } from "drizzle-orm";
import type { Column, InferSelectModel } from "drizzle-orm";
import { MySqlTable, getTableConfig as getMysqlTableConfig } from "drizzle-orm/mysql-core";
import type { ForeignKey as MysqlForeignKey, MySqlTableWithColumns } from "drizzle-orm/mysql-core";
import type { PgTableWithColumns, ForeignKey as PostgresqlForeignKey } from "drizzle-orm/pg-core";
import { SQLiteTable, getTableConfig as getSqliteTableConfig } from "drizzle-orm/sqlite-core";
import type { ForeignKey as SqliteForeignKey, SQLiteTableWithColumns } from "drizzle-orm/sqlite-core";
import { expect, expectTypeOf, test } from "vitest";
@@ -54,6 +57,21 @@ type FixedSqliteConfig = {
};
};
/** Identifies a foreign key by what it links rather than by its generated name. */
const referenceSignature = (foreignKey: {
reference: () => { columns: { name: string }[]; foreignTable: object; foreignColumns: { name: string }[] };
}) => {
const reference = foreignKey.reference();
const target = Object.keys(reference.foreignTable).sort().join("+");
return `${reference.columns
.map((column) => column.name)
.sort()
.join(",")}->${target}(${reference.foreignColumns
.map((column) => column.name)
.sort()
.join(",")})`;
};
test("schemas should match", () => {
expectTypeOf<SqliteTables>().toEqualTypeOf<MysqlTables>();
expectTypeOf<MysqlTables>().toEqualTypeOf<SqliteTables>();
@@ -88,56 +106,64 @@ test("schemas should match", () => {
});
const mysqlTable = mysqlTables[tableName];
const sqliteForeignKeys = sqliteTable[Symbol.for("drizzle:SQLiteInlineForeignKeys") as keyof typeof sqliteTable] as
| SqliteForeignKey[]
| undefined;
const mysqlForeignKeys = mysqlTable[Symbol.for("drizzle:MySqlInlineForeignKeys") as keyof typeof mysqlTable] as
| MysqlForeignKey[]
| undefined;
// Read through getTableConfig so keys declared at table level count too, not just inline
// `.references()` ones. MySQL needs a table-level declaration wherever drizzle's derived name
// would exceed its 64-character identifier limit.
if (!is(sqliteTable, SQLiteTable) || !is(mysqlTable, MySqlTable)) return;
if (!sqliteForeignKeys && !mysqlForeignKeys) return;
const sqliteForeignKeys = getSqliteTableConfig(sqliteTable).foreignKeys as SqliteForeignKey[];
const mysqlForeignKeys = getMysqlTableConfig(mysqlTable).foreignKeys as MysqlForeignKey[];
expect(mysqlForeignKeys, `mysql foreign key for ${tableName} to be defined`).toBeDefined();
expect(sqliteForeignKeys, `sqlite foreign key for ${tableName} to be defined`).toBeDefined();
if (sqliteForeignKeys.length === 0 && mysqlForeignKeys.length === 0) return;
expect(
sqliteForeignKeys!.length,
sqliteForeignKeys.length,
`expect number of foreign keys in table ${tableName} to be the same for both schemas`,
).toEqual(mysqlForeignKeys!.length);
).toEqual(mysqlForeignKeys.length);
sqliteForeignKeys?.forEach((sqliteForeignKey) => {
sqliteForeignKey.getName();
const mysqlForeignKey = mysqlForeignKeys!.find((key) => key.getName() === sqliteForeignKey.getName());
expect(
mysqlForeignKey,
`expect foreign key ${sqliteForeignKey.getName()} to be defined in mysql schema`,
).toBeDefined();
sqliteForeignKeys.forEach((sqliteForeignKey) => {
const describeKey = `${tableName}.${referenceSignature(sqliteForeignKey)}`;
// Matched on what the key actually does rather than on its generated name, because the name
// is allowed to differ per dialect.
const mysqlForeignKey = mysqlForeignKeys.find(
(key) => referenceSignature(key) === referenceSignature(sqliteForeignKey),
);
expect(mysqlForeignKey, `expect foreign key ${describeKey} to be defined in mysql schema`).toBeDefined();
expect(
sqliteForeignKey.onDelete,
`expect foreign key (${sqliteForeignKey.getName()}) onDelete to be the same for both schemas`,
`expect foreign key (${describeKey}) onDelete to be the same for both schemas`,
).toEqual(mysqlForeignKey!.onDelete);
expect(
sqliteForeignKey.onUpdate,
`expect foreign key (${sqliteForeignKey.getName()}) onUpdate to be the same for both schemas`,
`expect foreign key (${describeKey}) onUpdate to be the same for both schemas`,
).toEqual(mysqlForeignKey!.onUpdate);
sqliteForeignKey.reference().foreignColumns.forEach((column) => {
expect(
mysqlForeignKey!.reference().foreignColumns.map((column) => column.name),
`expect foreign key (${sqliteForeignKey.getName()}) columns to be the same for both schemas`,
).toContainEqual(column.name);
});
expect(
Object.keys(sqliteForeignKey.reference().foreignTable),
`expect foreign key (${sqliteForeignKey.getName()}) table to be the same for both schemas`,
).toEqual(Object.keys(mysqlForeignKey!.reference().foreignTable));
});
});
});
test("mysql identifiers stay within the 64 character limit", () => {
// MySQL rejects longer identifiers with error 1059, and drizzle derives foreign key names from
// the table and column names, so a long pairing silently produces a migration that cannot apply.
const tooLong: string[] = [];
objectEntries(mysqlSchema).forEach(([, table]) => {
if (!is(table, MySqlTable)) return;
const config = getMysqlTableConfig(table);
if (config.name.length > 64) tooLong.push(config.name);
for (const foreignKey of config.foreignKeys) {
if (foreignKey.getName().length > 64) tooLong.push(foreignKey.getName());
}
for (const index of config.indexes) {
if (index.config.name.length > 64) tooLong.push(index.config.name);
}
});
expect(tooLong).toEqual([]);
});
test("schemas should match for postgresql", () => {
expectTypeOf<SqliteTables>().toEqualTypeOf<PostgresqlTables>();
expectTypeOf<PostgresqlTables>().toEqualTypeOf<SqliteTables>();
-1
View File
@@ -1513,7 +1513,6 @@
},
"assistant": {
"title": "Homarr Assistant",
"subtitle": "Connected to this Homarr instance",
"open": "Open Homarr Assistant",
"close": "Close assistant",
"minimize": "Minimize assistant",