Commit Graph
182 Commits
Author SHA1 Message Date
Tom Moor 2f57a36d50 Validate awareness identity in collaboration service (#13804)
* Validate awareness identity against connection context

* Validate awareness messages on the sending connection

* Encode real removal states and isolate ownership check in tests
2026-09-19 13:24:35 -04:00
Tom Moor 1a1a3398e0 fix: Do no report expired wss auth to error tracker (#13488) 2026-08-19 08:06:58 -04:00
Tom MoorandClaude Opus 5 ac07b97a85 fix: Align CSRF cookie resolution between attach and verify paths (#13426)
* fix: Align CSRF cookie resolution between attach and verify paths

The middleware that attaches the token picked one cookie name based on
whether the request was secure, while the verifier accepted either name.
Resolve the name through a single helper so both paths always agree, and
warn at startup when proxy headers are untrusted on an https install.

* Remove excessive tests

* fix: Describe the request scheme on the MCP-derived API context

Helpers shared between the MCP tools and the HTTP routes read the scheme
from the context, which the MCP stub did not provide.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: Do not warn about proxy headers when the app terminates TLS itself

The socket is already encrypted in that case, so the request scheme is
known without X-Forwarded-Proto. Extract the certificate check that main
was doing inline so both call sites stay in step.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* simplify

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 17:56:39 -04:00
Tom MoorandClaude Opus 5 15daf2e8a7 fix: Unhandled socket error during websocket upgrade crashes server (#13409)
* fix: Unhandled socket error during websocket upgrade crashes server

Node removes its own socket error handling before emitting the "upgrade"
event, so any socket detached from the HTTP server without a listener of
its own will terminate the process when the client goes away.

Handle errors for every upgraded socket in one place, and destroy sockets
we reject rather than leaving them half-open to be reset at any point in
the future.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: Flush rejection response before destroying the socket

Destroying immediately after the write can truncate the response if the
socket's write buffer has not drained, leaving the client with a reset in
place of the rejection.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* noisy comment

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 18:59:06 -04:00
Tom Moor 11ef69e381 fix: Unhandled rejection when a Redlock lock release fails (#13316)
* fix: Unhandled rejection when a Redlock release fails

Releasing a lock is best-effort, a lock that has already expired or been
taken over by another process cannot be released and that is not an error
for the caller. Previously the ExecutionError thrown from release replaced
the successful return value of CacheHelper.populate.

Also catch failures when joining rooms after websocket authentication,
which was a floating promise and so crashed the process.

* fix: Coerce caught values to Error before logging
2026-08-06 00:52:26 +00:00
Tom MoorandClaude Fable 5 0c2374dec1 perf: Index-friendly popularity score scan, spread runs over an hour (#13306)
* perf: Index-friendly candidate scan in popularity score task

The chunked scan that populates the working table combined "has a non-zero
score" and "has recent insights" into a single OR, which forced a scan of
every document in the partition with a correlated subquery per row. In
production this reached a p95 of 16.9s per chunk on the read replica, which
also serves user reads.

Query each source separately so both can use an index, then union. Limiting
each branch and then the union still yields the globally lowest ids, so keyset
pagination is unchanged. Validity is now a join rather than a predicate, with
the cursor tracked over candidates so filtered rows can't end the walk early.

Adds a partial index on documents(id) where the score is non-zero; the full
index on popularityScore was dropped as unused in 20251217161051 and would not
help this access pattern.

* perf: Drop per-batch COUNT(*) from popularity score task

Every batch issued a COUNT(*) over the working table purely to decide whether
to continue, doubling the round-trips for a loop that already runs thousands of
iterations. processBatch already reports how many rows it handled, so a short
batch is sufficient to detect a drained table.

setupWorkingTable now returns the row count it inserted, which removes the
remaining call site.

* perf: Spread popularity scoring across teams over six hours

The task partitioned by document id over a 30 minute window, which the cron
route expands to one partition per minute. Thirty partitions each took minutes
to run, so they piled up and saturated the database for the whole window.

Partition by team over six hours instead: 360 slices, one starting each minute,
each covering a fraction of the team space. Peak concurrency is bounded by how
long a slice takes rather than by the number of partitions.

Spreading past the hour breaks the interval gate, which read the wall clock at
execution time and so would skip every partition that ran in a later hour --
and worse, would let the next hour's cron schedule a second full run. Cron now
records when a run was scheduled and the gate reads that instead.

The partial index added for the score walk leads with teamId to match the new
access pattern; it is not deployed yet, so it is amended in place.

* perf: Re-key popularity partitions to document id, reduce spread to one hour

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: Compute scheduledAt once per cron run

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 07:12:11 -04:00
Tom MoorandClaude 17452bda5b Move socket management from client -> server (#13120)
* Move socket management from client -> server

* feedback

* fix: Reconcile collection rooms on visibility change

Handle collections.update transitions between private and team-visible by
joining or rebuilding the collection channel, route collection events through
the members channel so guests are excluded, and use removeData for the
collection ids cache invalidation.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-26 17:47:34 -04:00
Tom Moor 9884e0379f perf: Memory usage fixes (#13101)
* fix: Module level cache in headingToSlug

* fix: Remove retained full User and Team model on every ws connection

* feedback
2026-07-23 08:14:35 -04:00
Tom Moor a60373118c fix: Handle transient errors in HealthMonitor (#12913) 2026-07-06 16:13:32 -04:00
Tom Moor c3669a1f1a fix: Guard against orphaned jobs in queue (#12819) 2026-06-24 17:54:08 -04:00
Tom Moor 66eee9007f chore: Upgrade to Typescript 6 (#12737)
* chore: Upgrade TypeScript to 6.0
2026-06-18 17:34:05 -04:00
Tom MoorandClaude Opus 4.8 9113501906 Add PROXY_HEADERS_TRUSTED env (#12676)
* Add PROXY_HEADERS_TRUSTED env

* Don't trust X-Forwarded-Proto for HTTPS redirect when proxy headers untrusted

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 19:58:16 -04:00
Tom MoorandClaude Opus 4.8 0c0facc2a1 perf: Avoid empty webhook processor work via cached subscription lookup (#12593)
* Avoid empty webhook processor work via cached subscription lookup

WebhookProcessor ran for every event but most teams have no matching
webhook subscription, costing an empty processor job and a database query
per event.

Cache a team's enabled subscriptions ({ id, events }) in Redis, invalidated
by model lifecycle hooks, and add an optional BaseProcessor.shouldQueue hook
consulted by the global event queue so the webhook processor only enqueues a
job when a matching subscription exists.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feedback

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 17:53:40 -04:00
Tom MoorandClaude Opus 4.8 b189c308e5 perf: Avoid loading unused services (#12537)
* fix: Run single process when only the worker service is enabled

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* perf: Improve memory consumption through lazy service loading

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 16:48:31 -04:00
Tom MoorandClaude Opus 4.7 fca10221b9 chore: promote no-explicit-any from warn to error (#12244)
* chore: promote no-explicit-any from warn to error and resolve violations

Upgrades the oxlint rule severity and removes all 40 existing
`no-explicit-any` warnings across the codebase. Most call sites gained
proper types (SharedEditor refs, JSONNode/JSONMark for ProseMirror JSON
walking, DocumentsStore, dd-trace `Span` parameter inference, prosemirror
Fragment public API in place of internal `(fragment as any).content`).
A few load-bearing `any` uses were preserved with scoped disable
comments where changing the type would cascade widely (Sequelize JSONB
columns on `Event`, the `withTracing` higher-order function generic,
`Extension.options` consumed by many subclasses, dd-trace's `req`
patching).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-02 12:14:23 -04:00
Copilotandtommoor d54a861894 Avoid reporting max payload size exceeded errors to Sentry in collaboration server (#12002)
Agent-Logs-Url: https://github.com/outline/outline/sessions/ec7f1b13-6d7e-49d7-a8a4-f1223ba07a93

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>
2026-04-11 19:29:14 -04:00
Ashley SommerandTom Moor 8048b7e530 feat: Add support for configurable proxy IP header in environment settings (#11595)
* feat: Add support for configurable proxy IP header in environment settings

* Update server/env.ts

Remove mention of Koa from docs

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Update .env.sample

Remove mention of Koa from env sample.

Co-authored-by: Tom Moor <tom.moor@gmail.com>

---------

Co-authored-by: Tom Moor <tom.moor@gmail.com>
2026-02-27 19:27:24 -05:00
1937043aed feat: MCP Server (#11464)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 20:14:18 -05:00
Tom Moor 00fb4d1af7 chore: Update node style imports (#11277)
- crypto → node:crypto
  - fs → node:fs
  - fs/promises → node:fs/promises
  - path → node:path
  - http → node:http
  - https → node:https
  - stream → node:stream
  - buffer → node:buffer
  - url → node:url
  - os → node:os
  - net → node:net
  - dns → node:dns
  - events → node:events
  - readline → node:readline
  - querystring → node:querystring
  - util → node:util
2026-01-26 20:51:50 -05:00
Tom Moor 838a1e7428 fix: Suppress ECONNRESET (#11276) 2026-01-27 01:31:52 +00:00
Tom Moor 1cc4d879dc fix: Sync API changes to clients in realtime (#11186)
* fix: Sync API changes to clients in realtime

* Add editMode parameter

* rename

* Add error handling

* refactor
2026-01-19 13:28:44 -05:00
Tom Moor d9aec40313 chore: Store service in JWT (#11136)
* chore: Store service in JWT

* docs

* fix: Remove early return
2026-01-10 12:28:00 -05:00
Tom Moor 2e1d3f0e65 chore: Add missing error logging on wss connection (#11015) 2025-12-28 10:40:13 -05:00
Tom Moor bf45e97641 chore: Enforce type import consistency (#10968)
* Update types

* fix circular dep

* type imports

* lint type imports and --fix
2025-12-19 23:07:02 -05:00
Tom Moor a54e66e19a chore: Test improvements (#10945)
* Lazy queues, correctly closing Redis and server

* feedback

* fix: Tests not correctly split across matrix
2025-12-17 23:15:55 -05:00
Tom Moor 42959d66db chore: Add cron task partitioning (#10736)
* wip

* Implementation complete

* tidying

* test

* Address feedback

* Remove duplicative retry logic from UpdateDocumentsPopularityScoreTask.
Now that we're split across many runs this is not neccessary

* Refactor to subclass, config to instance

* Refactor BaseTask to named export

* fix: Missing partition

* tsc

* Feedback
2025-11-27 16:57:52 +01:00
Tom Moor 76691e8aaa fix: Add yet another guard against crawlers consuming magic links (#10457) 2025-10-23 08:24:10 -04:00
Tom Moor f6315875b4 fix: CSRF validation issues on Firefox (#10317) 2025-10-06 19:10:25 -04:00
Tom Moor 0a9bd39aac Add CSRF middleware (#10051)
ref OUT-Q325-03
2025-08-31 06:35:35 -04:00
Hemachandar 04c3d81b1f chore: Setup missing oxlint configs (#9862)
* shared

* server

* app

* remove vestigial eslintrc files

* update comment directives
2025-08-06 19:54:22 -04:00
Tom Moor a83adc4ecf feat: Allow horizontal scaling of collaboration service (#9625)
* stash

* Horizontal scaling of collaboration service
2025-07-17 08:53:52 -04:00
f284a27941 feat: Add OIDC well-known endpoint discovery support (#9308)
* feat: Add OIDC well-known endpoint discovery support

Co-authored-by: codegen-sh[bot] <131295404+codegen-sh[bot]@users.noreply.github.com>
Co-authored-by: Tom Moor <tom@getoutline.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-27 07:56:02 -04:00
codegen-sh[bot]andTom Moor d2aba1de96 feat: Add POST method option to redirectOnClient (#9228)
* feat: Add POST method option to redirectOnClient helper

* Applied automatic fixes

* fix: Add missing closing HTML tag in redirectOnClient GET method

* fix: Use lodash escape for form field values to prevent XSS

* Applied automatic fixes

* fix: Add missing lodash/escape import

* Applied automatic fixes

* fix: Escape all URLs in redirectOnClient function

* Update index.ts

* fix: CSP

* Refactor CSP middleware

* docs, only use for email signin

---------

Co-authored-by: codegen-sh[bot] <131295404+codegen-sh[bot]@users.noreply.github.com>
Co-authored-by: Tom Moor <tom@getoutline.com>
2025-05-17 09:06:32 -04:00
Tom Moor 08b7c11461 Normalize Collection.findByPk (#9193) 2025-05-14 00:05:45 -04:00
Tom Moor a06671e8ce OAuth provider (#8884)
This PR contains the necessary work to make Outline an OAuth provider including:

- OAuth app registration
- OAuth app management
- Private / public apps (Public in cloud only)
- Full OAuth 2.0 spec compatible authentication flow
- Granular scopes
- User token management screen in settings
- Associated API endpoints for programatic access
2025-05-03 19:40:18 -04:00
codegen-sh[bot]andTom Moor a226889143 Update task scheduling to use instance method (#9092)
* Update task scheduling to use instance method

* Delete update_task_schedule.sh

* Applied automatic fixes

* tsc

---------

Co-authored-by: codegen-sh[bot] <131295404+codegen-sh[bot]@users.noreply.github.com>
Co-authored-by: Tom Moor <tom@getoutline.com>
2025-04-29 06:47:51 -04:00
Tom Moor f5c659f902 fix: Prevent cross-domain websocket connections to on-premise instances (#9064) 2025-04-28 17:27:40 -04:00
Hemachandar 6e98568e5b API importer for Notion (#8710) 2025-03-23 12:19:13 -07:00
Tom Moor 0ec6440506 Prevent outdated clients from connecting to collaboration server (#8751)
* Move editor version check to collaboration server connection

* connected -> onConnect

* docs

* Remove hardcoded event codes
2025-03-22 14:35:45 -07:00
Tom Moor 6fde025ce4 Revert "Send editor version down websocket and force reload (#8582)" (#8750)
This reverts commit 13f45e1a1c.
2025-03-22 20:06:54 +00:00
Tom Moor 13f45e1a1c Send editor version down websocket and force reload (#8582) 2025-02-26 19:25:13 -08:00
Tom Moor e90a86737f Add task to cleanup old events, change strategies to hourly (#8446) 2025-02-17 13:34:30 -08:00
Tom Moor 8fec6758b8 fix: Move compression middleware to cover all /api and /auth routes 2025-01-14 19:01:51 -05:00
Tom Moor c5278a71de Increase ping/pong rate to increase Heroku compatibility 2024-10-20 09:56:59 -04:00
Tom Moor 9680e57849 chore: Remove suppressImplicitAnyIndexErrors TS rule (#7760) 2024-10-11 12:46:46 -07:00
Tom Moor d65037d4e7 perf: Increase default concurrency on worker 2024-09-02 09:31:18 -04:00
Tom Moor 0642396264 perf: Policies refactor (#7460) 2024-08-28 06:10:38 -07:00
Apoorv Mishra 5d85a3a093 Specify time conversion unit (#7458)
* fix: specificity in time units

* fix: milliseconds -> ms
2024-08-25 18:57:45 +05:30
Tom Moor c02f7c9c85 Remove gist.github.com, gitlab.com from default CSP (#7008) 2024-06-08 07:54:55 -07:00
Tom Moor 2ec4ff3cd0 chore: Add tags to queue processor traces 2024-02-25 11:07:28 -05:00