diff --git a/apps/docs/docs/management/integrations/index.mdx b/apps/docs/docs/management/integrations/index.mdx index 1a2e2c06f..f934ba268 100644 --- a/apps/docs/docs/management/integrations/index.mdx +++ b/apps/docs/docs/management/integrations/index.mdx @@ -67,29 +67,6 @@ or groups. Lists are filtered on the server and only include connections the cur ## Connection problems -### Docker service URLs - -Use the service name and **container port** when Homarr and the service share a user-defined Docker network. -For example, Sonarr with `ports: ["6767:8989"]` is reachable from Homarr at `http://sonarr:8989`. -Use `http://sonarr:6767` only if Sonarr actually listens on `6767` inside its container. -The browser-facing app URL can use the published host port or reverse proxy. -See [Docker Compose networking](https://docs.docker.com/compose/how-tos/networking/). - -Homarr uses native name resolution by default. Leave `ENABLE_DNS_CACHING` unset or set it to `false`: -the optional cache can retain an old IP after a container is recreated and can override `extra_hosts` entries. -If an outbound proxy is configured, exclude local service names with `NO_PROXY`, for example `sonarr,radarr,jellyfin`. - -To check Docker DNS independently of Homarr, replace `homarr` with your container name: - -```sh -docker exec homarr node -e 'require("node:dns").lookup("sonarr", { all: true }, (error, addresses) => console.log(error?.code ?? addresses))' -``` - -`ENOTFOUND` here indicates a container resolver or network problem. Check both services' network membership and -Docker's embedded DNS; changing a Next.js setting cannot repair that resolver. - -### Other connection failures - If a connection test fails: 1. Check the integration page for the required base URL and credentials. diff --git a/docs/research/docker-network-hostnames.md b/docs/research/docker-network-hostnames.md deleted file mode 100644 index 96b83dfe7..000000000 --- a/docs/research/docker-network-hostnames.md +++ /dev/null @@ -1,86 +0,0 @@ -# Docker service hostname investigation - -Investigated against `dev` on 2026-10-03. The fix makes Homarr DNS caching opt-in and leaves native -name resolution active by default. - -## Confirmed application failure - -Homarr's HTTP agent imports the DNS initializer, which previously enabled `dns-caching@0.2.9` -globally with a minimum five-minute TTL. The dependency replaces callback and promise `dns.lookup` -with direct A/AAAA queries and caches the resulting address. It consults the OS resolver only after -DNS fails. This also changes `/etc/hosts` precedence and can bypass Compose `extra_hosts` entries. - -Sources: [published dependency source](https://registry.npmjs.org/dns-caching/-/dns-caching-0.2.9.tgz), -[Node DNS implementation considerations](https://nodejs.org/docs/latest-v24.x/api/dns.html#implementation-considerations), -[Compose extra_hosts](https://docs.docker.com/reference/compose-file/services/#extra_hosts). - -A temporary user-defined Docker network reproduced stale-address reuse using Node 24.18.0, -Undici 7.29.1, and the actual bundled Homarr `UndiciHttpAgent` and DNS initializer: - -1. Start an HTTP fixture listening on container port `6767` with network alias `sonarr`. -2. Request `http://sonarr:6767` through Homarr's agent; receive HTTP 200. -3. Recreate only the fixture at a different IP, retaining the `sonarr` alias. -4. An original, native `dns.lookup` reference resolves the new IP immediately. -5. A new Homarr HTTP agent still connects using the cached IP and times out. - -The request agents were recreated between requests, ruling out pooled-socket reuse. Environment -validation was enabled so the initializer applied its real default. Proxies were explicitly bypassed -to isolate DNS. This initial reproduction used a bundled agent rather than the production image. - -Docker explicitly requires clients to look up a recreated service's new IP: -[Compose networking: update containers on the network](https://docs.docker.com/compose/how-tos/networking/#update-containers-on-the-network). - -## Separate deployment causes - -- Same-network URLs use the container listening port. A published mapping `6767:8989` requires - `http://sonarr:8989` internally. [Docker Compose networking](https://docs.docker.com/compose/how-tos/networking/). -- Native lookup in the existing local Homarr container returned `ENOTFOUND` for its neighboring - services, although direct-IP HTTP reached Sonarr on `8989`. Fresh temporary containers resolved - the same alias successfully. That is a separate Docker resolver failure; this PR cannot repair - existing daemon/network registrations. No existing services or networks were modified. -- Homarr's `EnvHttpProxyAgent` respects outbound proxy variables. Docker aliases need `NO_PROXY` - entries when the proxy cannot reach them. Preserve the operator's explicit proxy configuration. - [Undici 7.29.1 EnvHttpProxyAgent](https://github.com/nodejs/undici/blob/v7.29.1/docs/docs/api/EnvHttpProxyAgent.md). -- Next.js `images.dangerouslyAllowLocalIP` applies to image optimization. Integration requests use - Undici directly, so that setting does not solve integration connectivity. - [Next.js Image configuration](https://nextjs.org/docs/app/api-reference/components/image#dangerouslyallowlocalip). - -## Scope and tradeoff - -Default requests now use the OS resolver and see Docker address changes on new connections. -`ENABLE_DNS_CACHING=true` remains available for operators who deliberately accept cached addresses -and DNS-first resolution. This may increase DNS queries compared with the old default; HTTP connection -reuse remains unchanged. Existing explicit opt-ins must be removed or set to `false` to get native lookup. - -The same manual recreation probe passed after the default changed: both native lookup and Homarr's -HTTP agent followed the new service IP and received HTTP 200 without restarting the client process. - -## Production Compose acceptance - -The production Dockerfile was built locally and run with real Sonarr 4.0.19.2979 and Radarr 6.3.0.10514 -on a fresh user-defined Compose network. The smoke runner creates a disposable administrator/API key, -calls the real Next.js integration connection tests, and requests each service's `/ping` and authenticated -`/api/v3/system/status` through `/api/integrations/request`. All returned HTTP 200. - -The same Next.js process reached a health fixture on port `6767` after its IP changed. A conflicting -Docker DNS alias and `extra_hosts` entry confirmed native hosts-file precedence. Re-enabling -`ENABLE_DNS_CACHING=true` in the same production image reproduced both DNS-first hosts-file override -and a failed request after another fixture IP change, while a fresh native request still reached it. - -Run from the checkout: - -```sh -docker build -t homarr:dns-network-6997 . -node scripts/smoke/docker-network.mts -CI=true pnpm exec vitest run packages/core/src/test/infrastructure/dns/init.spec.ts packages/core/src/test/infrastructure/http/http-agent.spec.ts --project dom -``` - -The smoke runner uses a fresh `homarr-dns-` project, verifies its namespace is empty before -startup, and removes only that invocation's containers, network and disposable volumes afterward. -Homarr binds only to loopback; Sonarr/Radarr images are pinned. The fixed `10.203.199.0/24` subnet -prevents concurrent runs; a collision fails without cleaning another project. - -Focused DNS/HTTP tests passed (20 tests). Temporarily restoring the old default made the new -address-family contract fail, confirming that it detects a regression. The read-only review found no -actionable defects; explicit cache opt-ins and pre-existing Docker resolver failures remain outside -the corrected default behavior. diff --git a/scripts/smoke/docker-network.compose.yml b/scripts/smoke/docker-network.compose.yml deleted file mode 100644 index e330f5a59..000000000 --- a/scripts/smoke/docker-network.compose.yml +++ /dev/null @@ -1,75 +0,0 @@ -services: - homarr: - image: ${HOMARR_DNS_IMAGE:-homarr:dns-network-6997} - environment: - SECRET_ENCRYPTION_KEY: "0000000000000000000000000000000000000000000000000000000000000000" - NO_EXTERNAL_CONNECTION: "true" - ports: - - "127.0.0.1::7575" - extra_hosts: - - "overridden-service:10.203.199.11" - volumes: - - homarr-data:/appdata - healthcheck: - test: - [ - "CMD", - "node", - "-e", - "fetch('http://127.0.0.1:7575/api/health/ready').then(r=>process.exit(r.status===200?0:1)).catch(()=>process.exit(1))", - ] - interval: 2s - timeout: 5s - retries: 90 - sonarr: - image: ghcr.io/hotio/sonarr@sha256:71d939c562a273767d97f51516f349e8459542cf868058e4b49891513210bdd5 - volumes: - - sonarr-data:/config - healthcheck: - test: ["CMD", "curl", "--fail", "--silent", "http://127.0.0.1:8989/ping"] - interval: 2s - timeout: 5s - retries: 90 - radarr: - image: ghcr.io/hotio/radarr@sha256:c054e15a3bb7a0fd9e450b12ceed1298a8230a0b44382234d0cec5e0767044a2 - volumes: - - radarr-data:/config - healthcheck: - test: ["CMD", "curl", "--fail", "--silent", "http://127.0.0.1:7878/ping"] - interval: 2s - timeout: 5s - retries: 90 - fixture: - image: node:24.18.0-alpine - environment: - PROOF_INSTANCE: ${PROOF_INSTANCE:-original} - command: - [ - "node", - "-e", - "require('node:http').createServer((req,res)=>{res.setHeader('content-type','application/json');res.end(JSON.stringify({healthy:true,instance:process.env.PROOF_INSTANCE}))}).listen(6767,'0.0.0.0')", - ] - networks: - default: - ipv4_address: ${PROOF_SERVICE_IP:-10.203.199.10} - aliases: ["overridden-service"] - hosts-override: - image: node:24.18.0-alpine - command: - [ - "node", - "-e", - "require('node:http').createServer((req,res)=>{res.setHeader('content-type','application/json');res.end(JSON.stringify({healthy:true,instance:'hosts-override'}))}).listen(6767,'0.0.0.0')", - ] - networks: - default: - ipv4_address: 10.203.199.11 -volumes: - homarr-data: - sonarr-data: - radarr-data: -networks: - default: - ipam: - config: - - subnet: 10.203.199.0/24 diff --git a/scripts/smoke/docker-network.mts b/scripts/smoke/docker-network.mts deleted file mode 100644 index 04727cc57..000000000 --- a/scripts/smoke/docker-network.mts +++ /dev/null @@ -1,232 +0,0 @@ -import assert from "node:assert/strict"; -import { execFile } from "node:child_process"; -import { randomUUID } from "node:crypto"; -import { mkdtemp, rm, writeFile } from "node:fs/promises"; -import { tmpdir } from "node:os"; -import { resolve } from "node:path"; -import { promisify } from "node:util"; - -const execAsync = promisify(execFile); -const composeFile = resolve(import.meta.dirname, "docker-network.compose.yml"); -const project = `homarr-dns-${randomUUID()}`; -const temporaryDirectory = await mkdtemp(`${tmpdir()}/homarr-dns-proof-`); -const overrideFile = `${temporaryDirectory}/cache.json`; -let override = false; -let baseUrl = ""; -let apiKey = ""; -let ownsProject = false; - -async function assertUnusedProject() { - const label = `label=com.docker.compose.project=${project}`; - for (const resource of ["container", "network", "volume"]) { - const args = [resource, "ls", "--quiet", "--filter", label]; - if (resource === "container") args.push("--all"); - const result = await execAsync("docker", args); - assert.equal(result.stdout.trim(), "", "Disposable proof project already has resources; no mutation allowed"); - } -} - -async function compose(args: string[], environment: Record = {}) { - const files = ["-f", composeFile]; - if (override) files.push("-f", overrideFile); - const result = await execAsync("docker", ["compose", "-p", project, ...files, ...args], { - env: { ...process.env, ...environment }, - maxBuffer: 4 * 1024 * 1024, - }); - return result.stdout.trim(); -} - -async function refreshUrl() { - const address = await compose(["port", "homarr", "7575"]); - baseUrl = `http://${address}`; -} - -async function nextProcessIdentity() { - const identity = await compose([ - "exec", - "-T", - "homarr", - "node", - "-e", - ` - const fs = require('node:fs'); - for (const pid of fs.readdirSync('/proc').filter(value => /^\\d+$/.test(value))) { - try { - if (fs.readFileSync('/proc/'+pid+'/cmdline','utf8').startsWith('next-server')) { - const stat = fs.readFileSync('/proc/'+pid+'/stat','utf8'); - console.log(pid, stat.slice(stat.lastIndexOf(')')+2).split(' ')[19]); - } - } catch {} - } - `, - ]); - assert.ok(identity, "Next.js process was not found"); - return identity; -} - -async function api(path: string, input: unknown) { - const response = await fetch(`${baseUrl}${path}`, { - method: "POST", - headers: { "Content-Type": "application/json", ApiKey: apiKey }, - body: JSON.stringify(input), - signal: AbortSignal.timeout(25000), - }); - const result = await response.json(); - assert.equal(response.status, 200, `Homarr ${path} returned HTTP ${response.status}`); - return result; -} - -async function createIntegration(name: string, url: string, kind: string, token: string) { - const result = await api("/api/trpc/integration.create", { - json: { name, url, kind, secrets: [{ kind: "apiKey", value: token }], attemptSearchEngineCreation: false }, - }); - const data = result.result?.data?.json ?? result.result?.data; - const integration = data?.integration; - assert.ok( - integration?.id, - `Connection test/create failed for ${name}: ${JSON.stringify(data?.error?.type ?? result.error?.message ?? Object.keys(result))}`, - ); - console.log(`PASS Next.js connection test: ${name} at ${url}`); - return integration.id as string; -} - -async function request(integrationId: string, path: string) { - const result = await api("/api/integrations/request", { integrationId, path, method: "GET" }); - assert.equal(result.ok, true); - assert.equal(result.status, 200); - return result.data; -} - -async function upstreamKey(service: string) { - const config = await compose(["exec", "-T", service, "cat", "/config/config.xml"]); - const token = config.match(/([^<]+)<\/ApiKey>/)?.[1]; - assert.ok(token, `Missing API key for disposable ${service}`); - return token; -} - -async function recreateFixture(address: string, instance: string) { - await compose(["up", "-d", "--no-deps", "--force-recreate", "fixture"], { - PROOF_SERVICE_IP: address, - PROOF_INSTANCE: instance, - }); - const deadline = Date.now() + 10000; - while (true) { - try { - await compose([ - "exec", - "-T", - "fixture", - "node", - "-e", - "fetch('http://127.0.0.1:6767/health').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))", - ]); - return; - } catch { - assert.ok(Date.now() < deadline, "Recreated fixture never became ready"); - await new Promise((done) => setTimeout(done, 100)); - } - } -} - -try { - await assertUnusedProject(); - // The unpredictable, empty namespace belongs only to this invocation, including partial startup. - ownsProject = true; - await compose(["up", "-d", "--wait", "--wait-timeout", "180"]); - await refreshUrl(); - for (const path of ["/api/health/ready", "/api/health/live"]) { - const response = await fetch(`${baseUrl}${path}`); - assert.equal(response.status, 200); - console.log(`PASS production Homarr ${path}: HTTP 200`); - } - - // Provision only the disposable database. Keep the generated API key out of logs. - apiKey = await compose([ - "exec", - "-T", - "homarr", - "node", - "-e", - ` - const Database = require('better-sqlite3'); - const db = new Database(process.env.DB_URL, {nativeBinding:'/app/build/better_sqlite3.node'}); - const token = require('node:crypto').randomBytes(24).toString('hex'); - const hash = require('bcrypt').hashSync(token, 10); - db.transaction(() => { - db.prepare('INSERT OR IGNORE INTO user(id,name) VALUES (?,?)').run('dns-proof-user','DNS proof'); - db.prepare('INSERT OR IGNORE INTO "group"(id,name,position) VALUES (?,?,?)').run('dns-proof-group','DNS proof',999); - db.prepare('INSERT OR IGNORE INTO groupMember(group_id,user_id) VALUES (?,?)').run('dns-proof-group','dns-proof-user'); - db.prepare('DELETE FROM groupPermission WHERE group_id=?').run('dns-proof-group'); - db.prepare('INSERT INTO groupPermission(group_id,permission) VALUES (?,?)').run('dns-proof-group','admin'); - db.prepare('INSERT OR REPLACE INTO apiKey(id,api_key,user_id) VALUES (?,?,?)').run('dns-proof-key',hash,'dns-proof-user'); - })(); - db.close(); - process.stdout.write('dns-proof-key.'+token); - `, - ]); - - for (const [service, port] of [ - ["sonarr", 8989], - ["radarr", 7878], - ] as const) { - const id = await createIntegration(service, `http://${service}:${port}`, service, await upstreamKey(service)); - const health = await request(id, "/ping"); - assert.equal(health.status, "OK"); - const status = await request(id, "/api/v3/system/status"); - assert.ok(status.version); - console.log( - `PASS Next.js -> ${service}:${port}/ping and authenticated system/status: HTTP 200, version ${status.version}`, - ); - } - - const fixture = await createIntegration("DNS fixture", "http://fixture:6767", "sonarr", "fixture-token"); - const hosts = await createIntegration("Hosts override", "http://overridden-service:6767", "sonarr", "fixture-token"); - assert.equal((await request(hosts, "/health")).instance, "hosts-override"); - console.log("PASS extra_hosts takes precedence over the conflicting Docker DNS alias"); - assert.equal((await request(fixture, "/health")).instance, "original"); - const originalContainer = await compose(["ps", "-q", "homarr"]); - const originalNextProcess = await nextProcessIdentity(); - await recreateFixture("10.203.199.12", "replacement"); - assert.equal((await request(fixture, "/health")).instance, "replacement"); - assert.equal(await compose(["ps", "-q", "homarr"]), originalContainer); - assert.equal(await nextProcessIdentity(), originalNextProcess); - console.log("PASS fixture IP 10.203.199.10 -> 10.203.199.12: HTTP 200 without restarting Homarr"); - - // Re-enable the old default as a control, using the same production image and API routes. - await writeFile( - overrideFile, - JSON.stringify({ services: { homarr: { environment: { ENABLE_DNS_CACHING: "true" } } } }), - ); - override = true; - await compose(["up", "-d", "--no-deps", "--force-recreate", "--wait", "--wait-timeout", "90", "homarr"]); - await refreshUrl(); - assert.equal((await request(fixture, "/health")).instance, "replacement"); - assert.equal((await request(hosts, "/health")).instance, "replacement"); - console.log("PASS control ENABLE_DNS_CACHING=true reproduces DNS overriding extra_hosts"); - await recreateFixture("10.203.199.13", "control-replacement"); - const native = JSON.parse( - await compose([ - "exec", - "-T", - "homarr", - "node", - "-e", - "fetch('http://fixture:6767/health').then(async r=>console.log(JSON.stringify(await r.json()))).catch(()=>process.exit(1))", - ]), - ); - assert.equal(native.instance, "control-replacement"); - await assert.rejects( - request(fixture, "/health"), - (error: unknown) => error instanceof assert.AssertionError && [502, 504].includes(error.actual as number), - ); - console.log("PASS control ENABLE_DNS_CACHING=true reproduces failure after fixture IP change"); -} finally { - try { - if (ownsProject) { - await compose(["down", "--volumes"]); - console.log("Removed this invocation's disposable Compose stack and volumes"); - } - } finally { - await rm(temporaryDirectory, { recursive: true, force: true }); - } -}