feat(open-webui): add image, knowledge and webpage attachments with retry

Extend the AI Chat widget with three ways to ground a conversation:

- Images: attach images to a message, sent inline as OpenAI multimodal
  content parts for vision-capable models. A hint warns when the selected
  model likely has no vision support.
- Knowledge bases: pick RAG collections exposed by /api/v1/knowledge/.
- Web pages: ingest a URL via /api/v1/retrieval/process/web.

Open WebUI does not run RAG from the chat-completions `files` parameter,
so attached collections (web pages and knowledge bases) are grounded by
retrieving the most relevant chunks at send time
(/api/v1/retrieval/query/collection) and injecting them as system context.

Also add a Retry action to re-run the last turn after a model failure, and
handle the paginated `{ items, total }` shape of the knowledge endpoint on
newer Open WebUI versions.
This commit is contained in:
Lok3sh-Kumar committed 2026-06-27 14:15:08 +05:30
1 parent 005c2e4cd1
commit f61e7539d7
8 files changed
+624 -30

No files matched your search

+19 -1
View File
@@ -1,7 +1,12 @@
import dayjs from "dayjs";
import { createIntegrationAsync } from "@homarr/integrations";
import type { OpenWebUiChat, OpenWebUiChatListItem, OpenWebUiModel } from "@homarr/integrations/types";
import type {
OpenWebUiChat,
OpenWebUiChatListItem,
OpenWebUiKnowledge,
OpenWebUiModel,
} from "@homarr/integrations/types";
import { createCachedIntegrationRequestHandler } from "./lib/cached-integration-request-handler";
@@ -18,6 +23,19 @@ export const openWebUiModelsRequestHandler = createCachedIntegrationRequestHandl
queryKey: "openWebUiModels",
});
export const openWebUiKnowledgeRequestHandler = createCachedIntegrationRequestHandler<
OpenWebUiKnowledge[],
"openWebUi",
Record<string, never>
>({
async requestAsync(integration) {
const integrationInstance = await createIntegrationAsync(integration);
return await integrationInstance.getKnowledgeAsync();
},
cacheDuration: dayjs.duration(5, "minutes"),
queryKey: "openWebUiKnowledge",
});
export const openWebUiChatsRequestHandler = createCachedIntegrationRequestHandler<
OpenWebUiChatListItem[],
"openWebUi",