mirror of
https://github.com/outline/outline.git
synced 2026-11-21 13:34:02 +03:00
* feat: Support OAuth Client ID Metadata Documents for MCP clients MCP clients can send an https URL as their client_id. The server fetches the metadata document at that URL and caches one team-less client row per URL, so MCP works on the root domain without dynamic registration. Closes #14010 * fix: Address review findings for client ID metadata documents - Only count unexpired tokens when cleaning up metadata document clients - Reject backslashes, whitespace and trailing host dots in client_id URLs - Strip trailing dots before installation host comparisons - Type clientSecret as nullable * fix: Keep OAuth clients with pending authorization codes during cleanup * refactor: Simplify client ID metadata document handling - Gate metadata document fetches on the MCP preference inside the helper - Add isMCPClient getter and a shared client ID prefix constant - Enforce the team/metadata document CHECK in both directions - Index oauthClientId on OAuth token and code tables - Reuse parseDomain, clamp and time helpers * revert: Remove pending authorization code check from OAuth client cleanup * fix: Block all special-use addresses when fetching client metadata documents ALLOWED_PRIVATE_IP_ADDRESSES no longer applies to client ID metadata document requests (draft -02 section 8.6). The fetch wrapper takes an allowIPAddressList option to override the environment list. * fix: Reject empty userinfo in client metadata document URLs https://@host and https://:@host have a userinfo component that the URL parser drops, so check the raw authority (draft -02 section 3). * fix: Accept a root path in client metadata document URLs Draft -02 section 3 marks a root path as NOT RECOMMENDED, not invalid, so https://example.com/ is a valid client_id. A URL with no path is still rejected. * fix: Reject client metadata documents with private key material A jwks key with private or symmetric members (d, p, q, dp, dq, qi, oth, k) makes the document invalid, so it is not cached (draft -02 section 4.1). Public keys are still ignored. * fix: Respect no-cache, no-store and Age for client metadata documents no-store and no-cache now expire the cached document at once, so it is fetched again before the next authorization. Freshness from max-age or Expires (measured from Date) is reduced by Age before clamping (draft -02 section 5.2). * feat: Copy DCR and CIMD client logos to file storage in the background logo_uri was rendered directly on the consent screen, which lets the logo host track users and change the image at any time (draft -02 sections 8.8 and 9.2). A background task now fetches the logo with the strict fetch rules, accepts only PNG, JPEG, GIF and WebP by their leading bytes, and stores it content addressed in the public bucket. Requests never wait for the upload. * fix: Match loopback redirect URIs on any port Native apps such as Claude Code register http://localhost/callback and then listen on a free port, so an exact match rejected them. Loopback redirect URIs now match on any port (RFC 8252 section 7.3); the scheme, host and path must still match, and other URIs still match exactly. * fix: Only break the redirect URL mid-word on the consent screen break-all on the whole notice split words such as "continue" in the loopback warning. It now applies only to the redirect URL. * chore: Shorten the loopback warning on the consent screen * fix: Show the punycode host of client metadata documents on the consent screen A lookalike Unicode host was shown as written. Hostnames now come from a shared getHostname helper that returns the punycode form. * refactor: Move image type detection to ImageHelper and use it for avatars Avatars and integration logos copied from a URL are now stored only if they are PNG, JPEG, GIF or WebP, with the content type detected from the file data rather than the remote host. * fix: Address review findings for OAuth client logos and metadata caching - Count the time since the Date header as age when caching metadata documents, so a document is not cached past its Expires time - Log logo scheduling failures instead of failing registration requests - Run dynamic client registration in a transaction - Delete copied logos from storage when no client uses them * fix: Store one logo per OAuth client and return a distinct metadata document error - Logos are stored per client, so a file is only deleted when its own client stops using it. This removes the reference count and its race. - A metadata document that cannot be fetched or is invalid now returns invalid_client_metadata with the reason, not a 404. Network errors are only logged. - Move scheduleOptional to BaseTask so that any task can use it.
343 lines
11 KiB
TypeScript
343 lines
11 KiB
TypeScript
/* oxlint-disable no-restricted-imports, react/rules-of-hooks */
|
|
import { promises as dns } from "node:dns";
|
|
import type http from "node:http";
|
|
import type https from "node:https";
|
|
import * as net from "node:net";
|
|
import nodeFetch, {
|
|
Headers,
|
|
type RequestInit,
|
|
type Response,
|
|
} from "node-fetch";
|
|
import { getProxyForUrl } from "proxy-from-env";
|
|
import tunnelAgent, { type TunnelAgent } from "tunnel-agent";
|
|
import { errToString } from "@shared/utils/error";
|
|
import env from "@server/env";
|
|
import { InvalidRequestError } from "@server/errors";
|
|
import Logger from "@server/logging/Logger";
|
|
import { capitalize } from "es-toolkit/compat";
|
|
import {
|
|
type RequestFilteringAgentOptions,
|
|
useAgent as useFilteringAgent,
|
|
validateIPAddress,
|
|
} from "./requestFilteringAgent";
|
|
|
|
interface UrlWithTunnel extends URL {
|
|
tunnelMethod?: string;
|
|
}
|
|
|
|
const DefaultOptions = {
|
|
keepAlive: true,
|
|
timeout: 1000,
|
|
keepAliveMsecs: 500,
|
|
maxSockets: 200,
|
|
maxFreeSockets: 5,
|
|
maxCachedSessions: 500,
|
|
};
|
|
|
|
export type { RequestInit, Response } from "node-fetch";
|
|
export { Headers };
|
|
|
|
/**
|
|
* Default user agent string for outgoing requests.
|
|
*/
|
|
export const outlineUserAgent = `Outline-${
|
|
env.VERSION ? `/${env.VERSION.slice(0, 7)}` : ""
|
|
}`;
|
|
|
|
/**
|
|
* Fake Chrome user agent string for use in fetch requests to
|
|
* improve reliability.
|
|
*/
|
|
export const chromeUserAgent =
|
|
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36";
|
|
|
|
/**
|
|
* Resolves the URL's hostname and validates every returned address against the
|
|
* filtering rules. Used as a pre-flight check when a proxy is configured,
|
|
* since both proxy code paths in buildAgent() bypass the per-connection DNS
|
|
* hook in the filtering agent.
|
|
*
|
|
* @param url The target URL to validate.
|
|
* @param options Allow/deny rules to apply.
|
|
* @param signal Optional abort signal — if it fires (e.g. fetch timeout),
|
|
* the validation rejects with an AbortError so the configured timeout
|
|
* applies to slow DNS resolution as well.
|
|
* @throws An error if any resolved address is disallowed.
|
|
*/
|
|
const validateTargetURL = async (
|
|
url: URL,
|
|
options: RequestFilteringAgentOptions,
|
|
signal?: AbortSignal | null
|
|
): Promise<void> => {
|
|
const host = url.hostname;
|
|
const lookup =
|
|
net.isIP(host) !== 0
|
|
? Promise.resolve([{ address: host, family: net.isIP(host) }])
|
|
: dns.lookup(host, { all: true });
|
|
|
|
const addresses = signal
|
|
? await Promise.race([
|
|
lookup,
|
|
new Promise<never>((_, reject) => {
|
|
if (signal.aborted) {
|
|
reject(Object.assign(new Error("aborted"), { name: "AbortError" }));
|
|
return;
|
|
}
|
|
signal.addEventListener(
|
|
"abort",
|
|
() =>
|
|
reject(
|
|
Object.assign(new Error("aborted"), { name: "AbortError" })
|
|
),
|
|
{ once: true }
|
|
);
|
|
}),
|
|
])
|
|
: await lookup;
|
|
|
|
for (const { address, family } of addresses) {
|
|
const err = validateIPAddress({ address, family, host }, options);
|
|
if (err) {
|
|
throw err;
|
|
}
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Wrapper around fetch that uses the request-filtering-agent in cloud hosted
|
|
* environments to filter malicious requests, and the fetch-with-proxy library
|
|
* in self-hosted environments to allow for request from behind a proxy.
|
|
*
|
|
* @param url The url to fetch
|
|
* @param init The fetch init object
|
|
* @param init.allowIPAddressList Private IP addresses or CIDR ranges that may
|
|
* be requested. Defaults to the ALLOWED_PRIVATE_IP_ADDRESSES environment
|
|
* variable; pass an empty list to block every private address.
|
|
* @returns The response
|
|
*/
|
|
export default async function fetch(
|
|
url: string,
|
|
init?: RequestInit & {
|
|
allowPrivateIPAddress?: boolean;
|
|
allowIPAddressList?: string[];
|
|
timeout?: number;
|
|
}
|
|
): Promise<Response> {
|
|
Logger.silly("http", `Network request to ${url}`, init);
|
|
|
|
const {
|
|
allowPrivateIPAddress,
|
|
allowIPAddressList = env.ALLOWED_PRIVATE_IP_ADDRESSES ?? [],
|
|
timeout,
|
|
...rest
|
|
} = init || {};
|
|
|
|
// Create AbortController for timeout if specified
|
|
let abortController: AbortController | undefined;
|
|
let timeoutId: NodeJS.Timeout | undefined;
|
|
|
|
if (timeout && !rest.signal) {
|
|
abortController = new AbortController();
|
|
timeoutId = setTimeout(() => {
|
|
abortController?.abort();
|
|
}, timeout);
|
|
}
|
|
|
|
const signal = abortController?.signal || rest.signal;
|
|
|
|
try {
|
|
// When a proxy is configured, the request-filtering-agent's per-connection
|
|
// DNS hook does not see the target host (the tunnel-agent path bypasses it
|
|
// entirely, and the http-over-http path only filters the proxy's IP).
|
|
// Pre-resolve and validate the target ourselves so SSRF protection still
|
|
// applies to user-supplied URLs.
|
|
//
|
|
// We resolve DNS locally, but the proxy performs its own DNS resolution
|
|
// when forwarding the request. A determined attacker controlling DNS
|
|
// could return a public IP here and a private IP to the proxy. Closing
|
|
// that gap would require the proxy itself to enforce egress rules.
|
|
const parsedURL = new URL(url);
|
|
if (getProxyForUrl(parsedURL.href)) {
|
|
await validateTargetURL(
|
|
parsedURL,
|
|
{ allowPrivateIPAddress, allowIPAddressList },
|
|
signal
|
|
);
|
|
}
|
|
|
|
const headers = new Headers({ "User-Agent": outlineUserAgent });
|
|
new Headers(rest?.headers).forEach((value, key) => {
|
|
headers.set(key, value);
|
|
});
|
|
|
|
const response = await nodeFetch(url, {
|
|
...rest,
|
|
headers,
|
|
signal,
|
|
agent: buildAgent(url, {
|
|
signal,
|
|
allowPrivateIPAddress,
|
|
allowIPAddressList,
|
|
}),
|
|
});
|
|
|
|
if (!response.ok) {
|
|
Logger.silly("http", `Network request failed`, {
|
|
url,
|
|
status: response.status,
|
|
statusText: response.statusText,
|
|
headers: response.headers.raw(),
|
|
});
|
|
}
|
|
|
|
return response;
|
|
} catch (err) {
|
|
if (err instanceof Error && "name" in err && err.name === "AbortError") {
|
|
throw new Error(`Request timeout after ${timeout}ms`);
|
|
}
|
|
const message = errToString(err);
|
|
if (message.startsWith("DNS lookup")) {
|
|
throw InvalidRequestError(
|
|
env.isCloudHosted
|
|
? message
|
|
: `${message}\n\nTo allow this request, add the IP address or CIDR range to the ALLOWED_PRIVATE_IP_ADDRESSES environment variable.`
|
|
);
|
|
}
|
|
throw err;
|
|
} finally {
|
|
if (timeoutId) {
|
|
clearTimeout(timeoutId);
|
|
}
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Parses the proxy URL and returns an object with the properties
|
|
*
|
|
* @param url The URL to be fetched
|
|
* @param proxyURL The proxy URL to be used
|
|
* @returns An object containing the parsed proxy URL and tunnel method
|
|
*/
|
|
const parseProxy = (url: URL, proxyURL: string) => {
|
|
const proxyObject = new URL(proxyURL) as UrlWithTunnel;
|
|
const proxyProtocol = proxyObject.protocol?.replace(":", "");
|
|
const proxyPort =
|
|
proxyObject.port || (proxyProtocol === "https" ? "443" : "80");
|
|
proxyObject.port = proxyPort;
|
|
proxyObject.tunnelMethod = url.protocol
|
|
?.replace(":", "")
|
|
.concat("Over")
|
|
.concat(capitalize(proxyProtocol));
|
|
return proxyObject;
|
|
};
|
|
|
|
/**
|
|
* Builds a tunnel agent for the given proxy URL and options. Note that tunnel
|
|
* agents do not perform request filtering.
|
|
*
|
|
* @param proxy The parsed proxy URL
|
|
* @param options The request options
|
|
* @returns A tunnel agent for the proxy
|
|
*/
|
|
const buildTunnel = (proxy: UrlWithTunnel, options: RequestInit) => {
|
|
if (!proxy.tunnelMethod) {
|
|
Logger.warn("Proxy tunnel method not defined");
|
|
return;
|
|
}
|
|
if (!(proxy.tunnelMethod in tunnelAgent)) {
|
|
Logger.warn(`Proxy tunnel method not supported: ${proxy.tunnelMethod}`);
|
|
return;
|
|
}
|
|
|
|
const proxyAuth =
|
|
proxy.username || proxy.password
|
|
? `${proxy.username}:${proxy.password}`
|
|
: undefined;
|
|
|
|
return tunnelAgent[proxy.tunnelMethod as keyof TunnelAgent]({
|
|
...options,
|
|
proxy: {
|
|
port: proxy.port,
|
|
host: proxy.hostname,
|
|
proxyAuth,
|
|
},
|
|
});
|
|
};
|
|
|
|
/**
|
|
* Creates a http or https agent for the given URL, applying request filtering
|
|
* if necessary. If a proxy is detected in the environment, it will use that
|
|
* proxy agent to tunnel the request.
|
|
*
|
|
* @param url The URL to fetch.
|
|
* @param options Options controlling agent behavior.
|
|
* @param options.signal An abort signal used to destroy the agent on cancellation or timeout.
|
|
* @param options.allowPrivateIPAddress Whether to allow requests to private IP addresses.
|
|
* @param options.allowIPAddressList Private IP addresses or CIDR ranges that may be requested.
|
|
* @returns An http or https agent configured for the URL.
|
|
*/
|
|
function buildAgent(
|
|
url: string,
|
|
options: {
|
|
signal?: AbortSignal | null;
|
|
allowPrivateIPAddress?: boolean;
|
|
allowIPAddressList?: string[];
|
|
} = {}
|
|
) {
|
|
const agentOptions = { ...DefaultOptions };
|
|
const parsedURL = new URL(url);
|
|
const proxyURL = getProxyForUrl(parsedURL.href);
|
|
let agent: https.Agent | http.Agent | undefined;
|
|
|
|
const filteringOptions = {
|
|
...agentOptions,
|
|
allowPrivateIPAddress: options.allowPrivateIPAddress,
|
|
allowIPAddressList:
|
|
options.allowIPAddressList ?? env.ALLOWED_PRIVATE_IP_ADDRESSES ?? [],
|
|
};
|
|
|
|
if (proxyURL) {
|
|
const parsedProxyURL = parseProxy(parsedURL, proxyURL);
|
|
|
|
Logger.silly("http", `Using proxy for request`, {
|
|
url: parsedURL.toString(),
|
|
proxy: parsedProxyURL.href,
|
|
tunnelMethod: parsedProxyURL.tunnelMethod,
|
|
username: parsedProxyURL.username || undefined,
|
|
password: parsedProxyURL.password ? "******" : undefined,
|
|
});
|
|
|
|
if (parsedProxyURL.tunnelMethod?.startsWith("httpOver")) {
|
|
const proxyURL = new URL(parsedProxyURL.href);
|
|
proxyURL.pathname = (parsedURL.protocol ?? "")
|
|
.concat("//")
|
|
.concat(parsedURL.host ?? "")
|
|
.concat(parsedURL.pathname + parsedURL.search);
|
|
if (parsedProxyURL.username || parsedProxyURL.password) {
|
|
proxyURL.username = parsedProxyURL.username;
|
|
proxyURL.password = parsedProxyURL.password;
|
|
}
|
|
agent = useFilteringAgent(proxyURL.toString(), filteringOptions);
|
|
} else {
|
|
// tunnel-agent bypasses the filtering agent's per-connection DNS hook,
|
|
// so SSRF protection for this branch comes from the validateTargetURL
|
|
// pre-flight in fetch() above.
|
|
agent =
|
|
buildTunnel(parsedProxyURL, agentOptions) ||
|
|
useFilteringAgent(parsedURL.toString(), filteringOptions);
|
|
}
|
|
} else {
|
|
agent = useFilteringAgent(parsedURL.toString(), filteringOptions);
|
|
}
|
|
|
|
if (options.signal) {
|
|
options.signal.addEventListener("abort", () => {
|
|
if (agent && "destroy" in agent) {
|
|
agent.destroy();
|
|
}
|
|
});
|
|
}
|
|
|
|
return agent;
|
|
}
|