- persistence: tag created/updated Open WebUI chats with a homarr-user:<id> tag
(via chat meta.tags) so conversations can be found back per Homarr user even
when an integration/account is shared
- voice recording: distinguish a denied/dismissed microphone permission from a
generic mic failure and show an actionable message
- widget: surface the models query loading and connection-error states in the
chat empty state instead of silently showing 'no models'
- tests: cover the final-SSE-frame flush, the default request timeout (and the
streaming opt-out), and multimodal chat parsing round-trip
Per ajnart's review (and CodeRabbit withdrawing its own MCP comment on the
predecessor PR), the Open WebUI chat isn't a meaningful MCP surface:
- remove the .meta({ mcp }) metadata from every openWebUi procedure
Remaining CodeRabbit comments from the predecessor PR that weren't carried over:
- processWeb: constrain the URL to http(s) and reject loopback/metadata hosts
(private LAN ranges stay allowed for self-hosters); it is also creator-gated
- uploadFile/transcribe: bound the base64 payload and validate filename/type
- docs: align the example URL with the integration default port (8080)
- new chat: clear the composer draft so it doesn't leak into the fresh chat
- web ingestion: keep the typed URL when ingestion fails
- persistence: fall back to the default title for image-only opening turns, and
don't persist a partial reply from a failed turn as finished history
- multimodal round-trip: make the native chat/history contracts accept the same
text-or-parts content union as completions, and flatten (not drop) non-text
nodes when rebuilding transcripts so attachment-backed chats aren't lost.
Image data URLs are still kept out of persisted history by design.
API / router (packages/api):
- open-webui: expose serializable procedures as MCP tools via .meta({ mcp })
and gate knowledge/files/notes reads behind protectedProcedure + interact so
anonymous board viewers can't enumerate private source content
- integration create: wrap integration + creator-permission + secret writes in
a single cross-driver transaction so a partial failure can't leave an
unusable integration
- integration test: scope the permission lookup to the created integration
Integration (packages/integrations):
- open-webui: log only safe metadata (not raw completion chunks) on parse
failure, flush the final buffered SSE frame before treating the stream as
truncated, and apply a default request timeout (streaming opts out)
Widget (packages/widgets):
- collapse knowledge rows by default to avoid an N+1 getKnowledgeFiles burst
- strip markdown images from assistant output to block untrusted fetches
- reconcile the selected model when the model list changes
- clear draft attachments when switching to a saved chat
- dedupe async note/chat attaches after the fetch resolves
- discard stale persist responses via a session token so a cleared chat can't
be rebound to an old Open WebUI chat id
- release the mic stream on mid-start failures and mark the recording cancelled
on unmount so partial audio isn't transcribed
Break the monolithic chat.tsx into focused hooks (use-attachments,
use-chat-stream, use-voice-recording, use-camera-capture) and
presentational components (chat-components), with shared types and
utils extracted to chat-types/chat-utils.
Review fixes applied on top:
- require auth for the streaming completion (protectedProcedure) so
anonymous board viewers cannot spend the owner's LLM credits
- rename image-only state to images and the picker query options object
to pickerQueryOptions for clarity
- extract AttachPopover, clearPending(), composerEmpty and
releaseAudioResources() to remove duplication
The knowledge detail endpoint returns `files: null` on recent Open WebUI
versions, so the expandable picker showed nothing to select. List the
files endpoint and filter by the base's collection id instead; each file
grounds via its own `file-{id}` collection.
- Voice messages: record from the microphone and transcribe to the input
via Open WebUI's /api/v1/audio/transcriptions (mic button in composer).
- Camera capture: attach the stream via a ref callback so the preview is
no longer a black frame (the ref could be read before the video mounted).
- Notes: fetch the full note body on attach. The list endpoint truncates
content to 1000 chars, so attached notes were cut off; getNote returns
the complete markdown.
Replace the single attach popover with an Open WebUI-style multi-level
menu covering every attachment source:
- Upload Files: images stay inline (vision); other files are uploaded and
grounded via retrieval (`file-{id}` collection).
- Capture: take a photo from the webcam as an inline image.
- Attach Webpage: ingest a URL (existing).
- Attach Files: pick files already stored in Open WebUI.
- Attach Notes: inject the contents of a note as context.
- Attach Knowledge: nested picker with checkboxes, expandable to a base's
files (replaces the flat checkbox list).
- Reference Chats: bring a previous conversation in as context.
Knowledge bases, web pages and files are grounded by retrieval; notes and
referenced chats are injected verbatim as context blocks. Adds the
supporting integration methods (list files/notes, knowledge files, upload
file), request handlers and tRPC procedures.
Extend the AI Chat widget with three ways to ground a conversation:
- Images: attach images to a message, sent inline as OpenAI multimodal
content parts for vision-capable models. A hint warns when the selected
model likely has no vision support.
- Knowledge bases: pick RAG collections exposed by /api/v1/knowledge/.
- Web pages: ingest a URL via /api/v1/retrieval/process/web.
Open WebUI does not run RAG from the chat-completions `files` parameter,
so attached collections (web pages and knowledge bases) are grounded by
retrieving the most relevant chunks at send time
(/api/v1/retrieval/query/collection) and injecting them as system context.
Also add a Retry action to re-run the last turn after a model failure, and
handle the paginated `{ items, total }` shape of the knowledge endpoint on
newer Open WebUI versions.
Adds a new "openWebUi" integration kind (under a new "ai" category) and an
AI Chat widget for chatting with models served by Open WebUI.
- Integration: Bearer API key auth, model list, native chat history CRUD, and
token-by-token streaming chat completions (SSE).
- Widget: in-input model picker, streaming responses, and an in-widget history
panel. Reconstructs conversations from Open WebUI's message tree with a flat
fallback.
- Streaming is delivered over the WebSocket subscription link.
- Adds integration and widget documentation.
Thomas Camlong
Lok3sh-kumar
Lok3sh-Kumar