Commit Graph
524 Commits
Author SHA1 Message Date
Tom MoorandClaude Opus 5.5 5647621224 chore: Prune redundant comments and type suppressions (#14124)
* chore: Prune redundant comments and normalize lint directives

Removes comments that restate the following line and stale TODOs, drops
the passive event listener feature check now that all supported browsers
support it, and renames eslint directives to oxlint.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BehTjuZwKbN5csMCp3H4QH

* fix: Evaluate restricted team domains at validation time

The restricted email provider list was captured when the model loaded, so
it could not follow the hosting environment and the tests that covered it
never reached the model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BehTjuZwKbN5csMCp3H4QH

* chore: Replace type suppressions with accurate types

Adds a Concrete helper type for registries of abstract classes, declares
missing Sequelize and refractor typings, uses public ProseMirror APIs, a
typed mock context helper in middleware tests, and a StaticLink component
for sidebar rows being edited.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BehTjuZwKbN5csMCp3H4QH

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-17 17:07:39 -04:00
Tom Moor e51b9f3151 chore: Export provisionFirstCollection, skip migration check when cloud hosted (#14116) 2026-10-17 11:01:55 -04:00
Tom Moor eceba73a14 chore: Upgrade koa-helmet to v9 and koa-body to v8 (#14046)
* chore: Upgrade koa-helmet to v9 and koa-body to v8

* fix: Keep decimal parsing for pagination limit and offset
2026-10-16 02:11:11 +00:00
Tom MoorandClaude Opus 5.5 9e2565184e chore: Prune stale comments, TODOs, and type suppressions (#14092)
* chore: Remove unused lint disable directives

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJegM4QsoYSKrBKb6eB4CH

* chore: Replace TODO type suppressions with typed code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJegM4QsoYSKrBKb6eB4CH

* chore: Remove obsolete workaround comments in tests and MediaDimension

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJegM4QsoYSKrBKb6eB4CH

* chore: Remove Tippy compatibility props from Tooltip

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJegM4QsoYSKrBKb6eB4CH

* chore: Decouple DropdownMenu props from Tooltip and clarify sharing styles

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJegM4QsoYSKrBKb6eB4CH

* chore: Remove stale retention TODOs from cleanup tasks

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJegM4QsoYSKrBKb6eB4CH

* chore: Register relations synchronously and clarify workaround comments

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJegM4QsoYSKrBKb6eB4CH

* chore: Replace Notion import TODOs with descriptive comments

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJegM4QsoYSKrBKb6eB4CH

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-15 08:54:37 -04:00
Tom Moor a58962101d fix: Tighten email sign-in link handling (#14089) 2026-10-13 22:49:05 -04:00
Tom Moor f8496ea2f7 feat: Personal collections (#14062)
* feat: Personal collections backend

Adds an owner to collections. A personal collection is private, one per user
per team, created lazily through collections.personal, and only its owner can
see or manage it. Admin bypasses in policies, lists, exports, events, webhooks
and file operations exclude the personal collections of other users. Turning
off the PersonalDocs preference makes the collection read-only.

* test: Access matrix for personal collections

Checks that collection and document endpoints, list and search results, MCP
tools, collaboration and websocket rooms agree with the read policy for the
owner, collaborators, members, admins, guests and other teams.

* feat: Personal collection in the sidebar

Adds a Personal sidebar section that renders the personal collection with the
collection document tree, creates the collection on the first new document,
and keeps it out of the Collections section. Adds the workspace setting, the
personal sidebar context, and hides collection sharing and membership UI for
personal collections.

* fix: Drag and drop in shared with me trees

Anchors the reorder drop cursor inside the row, and prefetches documents on
hover so that nested rows that were never opened can be dragged.

* feat: Personal collection owner for collaborators, no collection page

Presents the owner of the personal collection on documents, so that
collaborators see the owner in the share popover. Root rows in the Personal
section show the default document icon. A personal collection has no page:
the scene shows not found, and the breadcrumb, command bar and delete redirect
no longer link to it.

* fix: Personal sidebar indent and breadcrumb items without a link

Children of rows at depth 0 skip a level so that they start beyond the row
icon. Breadcrumb items without a destination, such as the personal and deleted
collection, are shown as plain text.

* test: Reduce personal collection tests to list paths

Endpoints that authorize a single model are already covered by the policy
tests. Keeps the lists and searches, which filter in queries, and the
presenter.

* test: Move personal collection list tests into collections tests

* feat: Create documents in the personal collection by alias

documents.create, documents.import and the MCP create_document tool accept
the collectionId "personal", which refers to the personal collection of the
user and creates it when it does not exist yet.

* refactor: Remove collections.personal route

The sidebar creates its first personal document with the personal
collectionId alias, so documents.create is the only path that creates the
personal collection.

* feat: Personal collection name, starring, templates and default

Personal collections cannot be starred, renamed or hold templates, and the
PersonalDocs preference is on by default. The app shows collections through a
displayName getter, which is the translated "Personal" for a personal
collection.

* fix: Use the same policy for starring and unstarring collections

* fix: No pins or viewer insights for personal documents

* revert: Show personal collection events in the audit log

Admins with audit access should see events in personal collections, even when
they cannot open the documents.

* fix: Load collection owner in share scopes

* fix: Keep personal collections out of workspace exports and webhooks

Workspace exports skip all personal collections, while an explicit export of
a personal collection stays available to its owner. Events in personal
collections, including attachments and documents moved after queueing, are
never delivered to workspace webhooks. Personal collections cannot be
renamed or made public through any path, cannot hold pins, and are not
offered as mentions.

* Change icon to avatar

* fix: Personal documents cannot be unpublished

* feat: Personal collection context for MCP

Collections are presented with an isPersonal flag, the server instructions
explain the personal collection, and move_document and update_document accept
the personal collectionId alias.

* fix: Keep moved personal exports private and remove pins on move

Document exports record the owner of the personal collection they came from,
so they stay private after the document moves. Moving documents into a
personal collection removes their pins, including pins to home.

* fix: Only create the personal collection in MCP when it is used
2026-10-13 07:53:06 -04:00
Tom Moor b4601a61ff feat: Support OAuth Client ID Metadata Documents for MCP clients (#14015)
* feat: Support OAuth Client ID Metadata Documents for MCP clients

MCP clients can send an https URL as their client_id. The server fetches
the metadata document at that URL and caches one team-less client row per
URL, so MCP works on the root domain without dynamic registration.

Closes #14010

* fix: Address review findings for client ID metadata documents

- Only count unexpired tokens when cleaning up metadata document clients
- Reject backslashes, whitespace and trailing host dots in client_id URLs
- Strip trailing dots before installation host comparisons
- Type clientSecret as nullable

* fix: Keep OAuth clients with pending authorization codes during cleanup

* refactor: Simplify client ID metadata document handling

- Gate metadata document fetches on the MCP preference inside the helper
- Add isMCPClient getter and a shared client ID prefix constant
- Enforce the team/metadata document CHECK in both directions
- Index oauthClientId on OAuth token and code tables
- Reuse parseDomain, clamp and time helpers

* revert: Remove pending authorization code check from OAuth client cleanup

* fix: Block all special-use addresses when fetching client metadata documents

ALLOWED_PRIVATE_IP_ADDRESSES no longer applies to client ID metadata
document requests (draft -02 section 8.6). The fetch wrapper takes an
allowIPAddressList option to override the environment list.

* fix: Reject empty userinfo in client metadata document URLs

https://@host and https://:@host have a userinfo component that the URL
parser drops, so check the raw authority (draft -02 section 3).

* fix: Accept a root path in client metadata document URLs

Draft -02 section 3 marks a root path as NOT RECOMMENDED, not invalid, so
https://example.com/ is a valid client_id. A URL with no path is still
rejected.

* fix: Reject client metadata documents with private key material

A jwks key with private or symmetric members (d, p, q, dp, dq, qi, oth,
k) makes the document invalid, so it is not cached (draft -02 section 4.1).
Public keys are still ignored.

* fix: Respect no-cache, no-store and Age for client metadata documents

no-store and no-cache now expire the cached document at once, so it is
fetched again before the next authorization. Freshness from max-age or
Expires (measured from Date) is reduced by Age before clamping (draft -02
section 5.2).

* feat: Copy DCR and CIMD client logos to file storage in the background

logo_uri was rendered directly on the consent screen, which lets the logo
host track users and change the image at any time (draft -02 sections 8.8
and 9.2). A background task now fetches the logo with the strict fetch
rules, accepts only PNG, JPEG, GIF and WebP by their leading bytes, and
stores it content addressed in the public bucket. Requests never wait for
the upload.

* fix: Match loopback redirect URIs on any port

Native apps such as Claude Code register http://localhost/callback and
then listen on a free port, so an exact match rejected them. Loopback
redirect URIs now match on any port (RFC 8252 section 7.3); the scheme,
host and path must still match, and other URIs still match exactly.

* fix: Only break the redirect URL mid-word on the consent screen

break-all on the whole notice split words such as "continue" in the
loopback warning. It now applies only to the redirect URL.

* chore: Shorten the loopback warning on the consent screen

* fix: Show the punycode host of client metadata documents on the consent screen

A lookalike Unicode host was shown as written. Hostnames now come from a
shared getHostname helper that returns the punycode form.

* refactor: Move image type detection to ImageHelper and use it for avatars

Avatars and integration logos copied from a URL are now stored only if
they are PNG, JPEG, GIF or WebP, with the content type detected from the
file data rather than the remote host.

* fix: Address review findings for OAuth client logos and metadata caching

- Count the time since the Date header as age when caching metadata
  documents, so a document is not cached past its Expires time
- Log logo scheduling failures instead of failing registration requests
- Run dynamic client registration in a transaction
- Delete copied logos from storage when no client uses them

* fix: Store one logo per OAuth client and return a distinct metadata document error

- Logos are stored per client, so a file is only deleted when its own
  client stops using it. This removes the reference count and its race.
- A metadata document that cannot be fetched or is invalid now returns
  invalid_client_metadata with the reason, not a 404. Network errors
  are only logged.
- Move scheduleOptional to BaseTask so that any task can use it.
2026-10-12 11:01:55 -04:00
Tom Moor 086bd9e9e4 chore: Replace deprecated koa-router with @koa/router v15 (#14042)
* chore: Replace deprecated koa-router with @koa/router v15

* fix: Run optional authentication on all auth provider routes
2026-10-11 16:47:26 -04:00
Tom Moor e6ae742f0e fix: Preserve percent signs in exported file names (#14019) 2026-10-11 09:11:45 -04:00
Tom Moor e571208871 fix: Spelling, grammar, punctuation, and translation string errors (#14017)
* fix: Spelling, grammar, punctuation, and translation string errors

* fix: Remove duplicate period and comma splice from review feedback
2026-10-11 09:07:07 -04:00
Tahsin Ahmed RefatandClaude Opus 5.5 62ef10d629 fix: Allow private-use URI schemes for OAuth redirect URIs (#13945)
* fix: Allow private-use URI schemes for OAuth redirect URIs

Native apps receive OAuth redirects on a private-use URI scheme
(RFC 8252 section 7.1), e.g. com.example.app:/oauth2redirect or
myapp://callback. These were rejected when registering a client
because IsUrlList only accepted http, https and ftp, and the
recommended com.example.app:/path form was also rejected at authorize
time because it has no host.

Add isPrivateUseSchemeUrl, which accepts private-use schemes while
excluding web schemes and schemes that can execute or embed content,
and use it in both places.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: Require a protocol for OAuth redirect URIs

isURL treats a bare word such as "callback" as a hostname, so a client
could be saved with a redirect URI that validateRedirectUri would always
reject at authorize time. Require a protocol when validating the list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: Accept one-character private-use URI destinations

The destination check compared the pathname length, so a valid opaque URI
such as myapp:a was rejected along with the empty myapp: and myapp:/.
Reject only those empty forms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 17:59:34 -04:00
Tom Moor 36951f0032 feat: Align MCP server and OAuth provider with latest MCP spec guidance (#13961)
* feat: Align MCP server and OAuth provider with newer MCP spec recommendations

- Upgrade @modelcontextprotocol/sdk to 1.31.0
- Include iss in authorization responses and advertise support (RFC 9207)
- Include required scopes in the WWW-Authenticate challenge on /mcp
- Expose title, website and icons in the MCP server info
- Centralise issuer, scopes, grant types and client type mapping in OAuthHelper

* Remove OAuth authorization issuer changes
2026-10-05 19:54:22 -04:00
Tom Moor 6630ffcc5a fix: Improve collaboration connection lifecycle (#13949) 2026-10-05 08:52:14 -04:00
Tom Moor 7f3bc8e3d7 feat: Redirect root domain app links to the signed-in workspace (#13923)
* feat: Redirect root domain app links to the signed-in workspace

* fix: Treat non-object sessions cookie values as empty
2026-09-30 22:24:27 -04:00
Tom MoorandClaude Fable 5.1 abc47b442a feat: Sync group descriptions and re-sync groups on SSO validation (#13889)
* feat: Sync group descriptions and re-sync groups on SSO validation

- Sync group descriptions from group sync providers alongside names
- Lock name and description edits for externally synced groups at the
  API layer and in the edit dialog
- Move group sync into a SyncUserGroupsTask, scheduled from login and
  from ValidateSSOAccessTask after a fresh provider validation
- Add authenticationProviders.startGroupSync for providers that require
  an admin consent flow before group sync can be enabled

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix: Make group sync setup generic and deduplicate sync jobs

- Expose groupSyncRequiresSetup from the provider list and use it in the
  settings scene instead of checking for the azure provider name
- Rename the setup redirect query param to groupSync
- Give SyncUserGroupsTask a job id per user authentication so concurrent
  syncs for the same authentication are not queued twice

* Use existing hook

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-25 19:49:26 -04:00
Tom Moor 6ac407de84 chore: Run server tests with a shared module registry (#13799)
Adds a server-shared Vitest project with isolate disabled and moves every
server test into it by default. Files that mock or reset modules opt out
with a header marker and stay in the isolated project.

Restores env values that leaked between files: the test URL is reset after
each file, and tests that mutate env restore the previous value. The test
server no longer closes the shared sequelize pool.
2026-09-18 17:45:39 -04:00
Tom Moor 324b2b94ef fix: Validate share id on public share endpoints (#13766)
* fix: Validate share id on shares.sitemap, shares.info, and shares.subscribe

Fixes OUTLINE-CLOUD-CWY

* test: Add regression tests for invalid share ids on shares.info and shares.subscribe

* test: Use an id that fails share slug validation and assert the message
2026-09-17 02:43:57 +00:00
Tom Moor 85deb8934b Fix GitLab webhook integration updates (#13764)
* fix GitLab webhook integration updates

* support shared GitLab webhook credentials
2026-09-16 22:11:24 -04:00
Tom Moor ca9317f4cc fix: Clarify missing OAuth state errors (#13734) 2026-09-13 15:53:34 -04:00
Tom Moor af2608536a fix: robots.txt disallowed crawlers on self-hosted installs (#13687)
Closes #13685
2026-09-06 22:44:49 -04:00
Tom MoorandClaude Fable 5.1 b2a1e10e48 Batch large table deletes in teamPermanentDeleter (#13610)
* fix: batch large deletes in teamPermanentDeleter to shorten lock time

Delete events, collections, and documents in batches of ids outside the
final transaction so row locks are held briefly instead of for the whole
team deletion, which blocked the hourly old-events cleanup for hours.

* fix: type batched destroy helper structurally

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix: guard whole team deletion with a non-blocking mutex

Move the concurrency guard from the final transaction to the whole
routine so a second worker skips before running the batched deletes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 06:12:12 -04:00
Tom MoorandClaude Opus 5 f49b047648 feat: Convert @ prefixed Markdown links to mentions on server writes (#13403)
* checkpoint

* feat: Convert @ prefixed links to mentions on server Markdown writes

A Markdown link preceded by "@" now becomes a mention when brought in
through the documents API, MCP, or an import, so an exported mention
survives a round trip instead of degrading to a plain link.

The editor rule creates a generic mention carrying the href, and the
type of resource it points at is resolved by the plugin that recognizes
the URL through a new MentionProvider hook. Mentions of external
resources also serialize to their real URL rather than a mention:// URI,
so the reference resolves outside of Outline.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: Escape values interpolated into rendered mention HTML

markdown-it leaves markup in a link label intact when html is disabled,
so the label and attributes are escaped before being written into the
tag.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 17:42:28 -04:00
Tom Moor aab83e8378 fix: Reliability of count-based limit checks (#13542)
* fix: Make count-based limit checks reliable under concurrency

Limits enforced by counting rows before an insert could be exceeded by
requests that ran at the same time, as each one read the same count
before any of them had written. Add a LockHelper that takes a Postgres
advisory lock for the remainder of the transaction, and use it to
serialize the check for webhook subscriptions, team domains, share
subscriptions and pins. The counts now also run on the transaction so
they include rows written earlier in the same request.

* Remove excessive tests

* Also guard simultaneous row removal
2026-08-24 19:14:55 -04:00
Tom MoorandClaude Opus 5 7652d26d07 fix: Consistent rate limiter keys across path variants (#13504)
* fix: Normalize path when keying rate limiters

The router matches paths case-insensitively and with an optional trailing
slash, so those variants each received their own quota. Collapse them onto
a single key at both registration and consumption.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feedback

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:59:21 +00:00
Tom Moor fc4fe65155 feat: Filter DSL (#12176)
* feat: Normalized filter DSL for documents.list and search

Add a structured filter expression language (field/operator/value leaves
composed with AND/OR groups) shared by documents.list, documents.search and
documents.searchTitles. Values are validated against each field's column type
and per-field operator allowlists at the input layer, and translated to
Sequelize where-clauses server-side.

Supersedes the ad-hoc statusFilter/dateFilter/collectionId/userId parameters,
adds ISO 8601 duration matching for date fields, documentId subtree expansion
for search, and preserves draft visibility and collection authorization.

* chore: Fix flaky Encrypted "different key" test

AES-256-CBC is unauthenticated, so decrypting with the wrong key only
raises "bad decrypt" when the garbage plaintext has invalid PKCS#7 padding
(~255/256 of the time). Roughly 1 in 245 runs it decrypts to a non-matching
value instead of throwing, failing the strict toThrow assertion. Accept
either outcome, mirroring the adjacent tampered-data test.

* Hardening
2026-08-15 13:25:10 -04:00
Tom MoorandClaude Opus 5 ac07b97a85 fix: Align CSRF cookie resolution between attach and verify paths (#13426)
* fix: Align CSRF cookie resolution between attach and verify paths

The middleware that attaches the token picked one cookie name based on
whether the request was secure, while the verifier accepted either name.
Resolve the name through a single helper so both paths always agree, and
warn at startup when proxy headers are untrusted on an https install.

* Remove excessive tests

* fix: Describe the request scheme on the MCP-derived API context

Helpers shared between the MCP tools and the HTTP routes read the scheme
from the context, which the MCP stub did not provide.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: Do not warn about proxy headers when the app terminates TLS itself

The socket is already encrypted in that case, so the request scheme is
known without X-Forwarded-Proto. Extract the certificate check that main
was doing inline so both call sites stay in step.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* simplify

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 17:56:39 -04:00
Tom Moor 187a5f8926 fix: Tighten one-time credential handling in OAuth token exchange (#13417)
* fix: Consume OAuth authorization codes and refresh tokens atomically

The exchange of an authorization code, and the rotation of a refresh
token, read the credential and deleted it in two steps. Concurrent
requests with the same credential could both pass the read and each
receive a token.

Claim the credential with a single conditional statement instead, so
that only one of a set of concurrent requests continues to token
issuance.

* fix: Move OAuthAuthentications delete to predicate-scoped destroy.
2026-08-13 22:32:22 -04:00
Tom Moor 9bfb1b9930 fix: Preserve custom port in client-side redirects (#13401)
Proxies commonly forward a Host header without the port the installation is
publicly reachable on, so redirects derived from the incoming request pointed at
the wrong address – most visibly breaking the email magic sign-in link.
2026-08-12 21:49:49 -04:00
Tom MoorandClaude Opus 5 399cd4041d perf: Reduce database and Redis round-trips on the MCP endpoint (#13333)
* perf: Reduce database and Redis round-trips on the MCP endpoint

Profiling production traces for the /mcp endpoint showed the time is
spent in repeated permission and breadcrumb lookups rather than in the
MCP layer itself.

- Cache Document.membershipDocumentIds in Redis, mirroring the TTL and
  key pattern of User.collectionIds. The two are resolved together for
  every search and this half was uncached, measuring ~200ms.
- Resolve each collection's cached document structure once, concurrently,
  when building breadcrumbs for a batch of documents. It was awaited per
  document inside a loop, so a page of search results meant a serial
  Redis round-trip for every result.
- Walk each collection structure once into a document ID to breadcrumb
  map instead of searching the whole tree per document.
- Reuse the collection association already loaded by Document.findByPk
  when resolving a single document's breadcrumb, rather than querying it
  again under the same membership scope.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: Invalidate cached document membership IDs on permission changes

The cache added alongside membershipDocumentIds relied on its TTL alone,
so a revoked grant stayed searchable until it expired. Clear the key on
the paths that change which documents a user is a member of, keeping the
TTL as a backstop.

- UserMembership create and destroy where documentId is set.
- GroupMembership create and destroy where documentId is set, for every
  user in the group.
- GroupUser create and destroy, as joining or leaving a group changes
  which group grants reach the user.

Sourced memberships are written with hooks disabled and destroyed without
individual hooks, so the parent membership invalidates once for the group
rather than once per descendant.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 19:01:02 -04:00
Tom Moor 355d15ad4f chore: Requests bearing API+OAuth tokens should not fall through to default limiter (#13328) 2026-08-06 22:10:40 -04:00
Tom Moor 11ef69e381 fix: Unhandled rejection when a Redlock lock release fails (#13316)
* fix: Unhandled rejection when a Redlock release fails

Releasing a lock is best-effort, a lock that has already expired or been
taken over by another process cannot be released and that is not an error
for the caller. Previously the ExecutionError thrown from release replaced
the successful return value of CacheHelper.populate.

Also catch failures when joining rooms after websocket authentication,
which was a floating promise and so crashed the process.

* fix: Coerce caught values to Error before logging
2026-08-06 00:52:26 +00:00
Tom MoorandClaude Opus 5 000f2f98fb feat: Add TextPack (.textpack) support (#13235)
* feat: Add TextPack (.textpack) import support

Adds a single-document importer for TextPack, the zipped variant of
TextBundle used by Bear, Ulysses, iA Writer and others. Bare .textbundle
directories are not supported, as a directory cannot be delivered through
a browser file input.

The bundle's text entry may use any extension, per the spec, with
info.json's type deciding whether it can be read as markdown. Assets are
inlined as data URIs for the existing attachment pipeline to pick up,
bounded by the attachment size limit and a memory ceiling, and only for
media types markdown-it accepts as a link destination.

Also fixes an existing bug where a file embedded in an HTML or email
import as a data URI was stored in the document as base64 rather than
being uploaded as an attachment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Refactor to individual converters

* refactor

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 20:42:05 -04:00
Tom MoorandClaude Opus 5 13b9b2c62d feat: Add support for web page and eml import (#13229)
* feat: Add support for web page and email import

* fix: Type mailparser result explicitly

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Combine HTML and Web Page

* fix: Validate mime type before inlining MIME archive parts

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 18:30:06 -04:00
Tom MoorandClaude Opus 5 8b35f7bd17 feat: Import file dialog (#13228)
* feat: Replace import document file picker with a modal

Choosing "Import document" now opens a dialog that lists the supported
file formats with an icon for each, and accepts files either dropped onto
it or chosen with the system file picker.

Also closes the gaps between the formats the dialog advertises and those
the importer actually accepts:

- `.tsv` was offered by the file picker but rejected by the server, it is
  now converted to a table like CSV
- `.csv` and `.htm` are now recognised by extension, so they still import
  when the browser reports an unhelpful mime type
- `.markdown` is now offered by the file picker, it was already supported

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dj1UnjnRX336CQcdzw3ott

* Add intermediate dialog for file import

* tweaks, polish

* Update failure message

* test

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-01 17:46:47 -04:00
Tom Moor 6e8b20e35f fix: OIDC forced secure cookie (#13217)
closes #13180
2026-08-01 07:24:31 -04:00
Tom Moor 9a50a980f8 Update CSRF handling to set secure host-bound where possible (#13203) 2026-07-30 17:27:11 -04:00
Tom Moor 7b751ab04b perf: Attach output stream before adding zip entries (#13167)
* perf: Attach output stream before adding zip entries

* fix: Handle failure
2026-07-27 19:13:18 -04:00
Tom Moor 5befd94fb8 perf: Fix streaming file outputs passthrough (#13153)
* perf: Fix streaming file outputs passthrough correctly

* test
2026-07-27 18:23:51 -04:00
Tom Moor 8267f22c31 chore: Remove client boolean guard in redirectOnClient (#13136) 2026-07-25 12:16:28 -04:00
Tom Moor 64fad39230 Remove dead ImportHelper (#13134) 2026-07-25 11:50:54 -04:00
Tom MoorandClaude 2d608d95e7 fix: Temporary directory left behind after S3 storage file handle (#13127)
* fix: Temporary directory left behind after S3 storage file handle

* Remove unused ZipHelper.extract

Use pipeline everywhere instead of manually wiring streams

* fix: Cleanup failure should not mask original error

Removing the temporary directory is best-effort, log the failure and
propagate the underlying error instead.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-25 08:17:04 -04:00
Tom Moor d63372694b chore: More lint warnings -> errors (#13122) 2026-07-25 07:56:13 -04:00
Tom Moor 64132a28d5 fix: Ensure JSDOM’s window is torn down (#13112)
* fix: Ensure JSDOM’s window (and its retained timers/resources) is torn down

* feedback

* catch
2026-07-23 20:36:04 -04:00
Tom MoorandClaude Opus 4.8 c8d4e8c0e7 perf: Bound MutexLock retries and de-dupe same-key cache misses (#13110)
* perf: Bound MutexLock retries and de-dupe same-key cache misses

MutexLock configured redlock with retryCount: 120, retryDelay: 1000 — a
2-minute retry window for locks with a 4s default TTL. Under cache-miss
storms (e.g. websocket reconnects all missing user.collectionIds) N
contenders piled onto one lock key, retaining ~K²/2 error objects plus
pending timers/promises. Bound retries to ~the lock TTL (retryCount: 15,
retryDelay: 250) and add a per-process in-flight de-dupe in
CacheHelper.getDataOrSet so N same-key misses share one lock acquisition
and callback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: Return falsy cached values instead of treating them as misses

getDataOrSet checked `if (cache)`, so a cached null/false/0/"" was treated
as a miss and re-populated on every read (e.g. Collection.getCachedDocument
Structure caches NavigationNode[] | null via the AfterSave hook). Check for
undefined — the only value getData returns on a genuine miss — so falsy
cached values are returned without an extra lock and DB query.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* deslop

* fix: Scope bounded lock retries to cache locks only

The tightened retry window (15×250ms) was set on the shared Redlock
instance, so it also affected long-lived callers — notably the 10-minute
"migrations" lock in checkMigrations, which fatal-exits on acquire failure
and would crash concurrent boots. Restore the generous default on the
shared instance and pass a bounded retry override (MutexLock.cacheRetry
Settings) only from the cache path via a new per-acquire retry option.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: Cache falsy callback results in getDataOrSet

populate only cached truthy callback results, so a valid null (e.g. from
Collection.getCachedDocumentStructure) was never stored and every miss
re-ran the lock + DB query. Skip caching only when the result is undefined,
and guard the CacheResult detection against null (typeof null === "object").

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 20:00:54 -04:00
Tom Moor 5386d689da perf: Refactor to run boot migration check as separate process (#13049)
* perf: Refactor to run boot migration check as separate process

* Apply suggestions from code review
2026-07-19 11:36:14 -04:00
Tom MoorandClaude Fable 5 24f95cadc3 perf: Reduce server memory usage and add container memory best practices (#13042)
* Reduce server memory usage and add container memory best practices

Profiling a full-service boot showed ~300MB RSS per service process, much
of it client-only dependencies pulled into the server module graph, and no
heap limits applied in containers:

- Import date-fns locales individually rather than via the locale index,
  which loaded all ~90 locales into every process
- Load @sentry/react dynamically in insertFiles so the browser Sentry SDK
  stays out of the server-side editor graph
- Replace class-validator isHexColor with the existing validateColorHex
  in the Highlight mark
- Extract icon names into a standalone IconNames module so server-side
  schema validation no longer imports Font Awesome icon packs, with a
  test to keep it in sync with IconLibrary
- Require S3Storage lazily so the AWS SDK and native CRT binding are only
  loaded when S3 file storage is configured
- Default each forked service process to a V8 heap limit derived from the
  cgroup memory constraint, preventing several processes from together
  committing more memory than the container allows
- Cap glibc malloc arenas in the Docker image to reduce resident memory
  fragmentation in multi-threaded processes

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016YMX5ReGHyxrR4ZE5ZbaXa

* Require both file storage backends lazily for consistency

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016YMX5ReGHyxrR4ZE5ZbaXa

* Load AWS SDK via dynamic imports compatible with test environment

Requiring the storage TypeScript modules lazily broke under Vitest, which
cannot resolve require() of source files and does not apply vi.mock to
bare requires. Move the laziness into S3Storage instead: the module is
imported statically, and the AWS SDK packages are loaded with dynamic
imports on first use, including deferred creation of the S3 client.

Also address review feedback: handle rejection of the dynamic Sentry
import, and warn when the minimum per-process heap limit exceeds the
memory budget of the container.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016YMX5ReGHyxrR4ZE5ZbaXa

* Consolidate S3 SDK loading into a single lazy accessor

The commands are loaded together with the client in getS3() rather than
as separate dynamic imports per method. The global S3Client test mock is
now a class, as a mock function is not constructable when reached through
a dynamic import.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016YMX5ReGHyxrR4ZE5ZbaXa

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-18 18:34:32 -04:00
Tom MoorandClaude Opus 4.8 533efdfcfe fix: OIDC 502 on sign-in from oversized id_token cookie (#12986) (#12995)
* fix: OIDC 502 on sign-in from oversized id_token cookie (#12986)

Storing the full OIDC id_token in the `oidcIdToken` cookie inflated the
sign-in response headers enough to exceed reverse proxy buffers, causing a
502 Bad Gateway. Store the id_token server-side in Redis keyed by a short
session identifier and keep only that identifier in the cookie, preserving
spec-compliant RP-initiated logout via `id_token_hint`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: Make OIDC logout token persistence best-effort

A Redis failure when storing or reading the logout id_token_hint should not
block sign-in or logout; fall back to omitting the hint instead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs

* refactor: Extract portable LogoutTokenStore for provider logout hints

Move the cookie-referenced, Redis-backed logout token persistence out of the
OIDC router into a provider-agnostic `LogoutTokenStore`. Any auth provider that
supports provider-initiated logout can now persist and consume a logout hint via
`new LogoutTokenStore(providerId)` without duplicating the cookie/Redis/error
handling. The OIDC-specific end-session URL construction stays in the router.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 10:24:12 -04:00
Tom MoorandClaude Opus 4.8 a054809b51 Use full keyspace for email verification codes (#13031)
* Use full keyspace for email verification codes

Email verification codes were generated with randomInt(100000, 1000000),
which can never produce a value below 100000. That excluded every code
with a leading zero — shrinking the keyspace from 1,000,000 to 900,000
and letting an attacker assume the first digit is never zero. It also
made the existing padStart(6, "0") dead code.

Draw uniformly from the full 0–999999 range and rely on padStart to
render leading zeros, restoring the full six-digit keyspace. Add tests
covering the six-digit invariant across many samples and the presence of
leading-zero codes.

* Simplify verification code jsdoc and drop leading-zero test

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EWfLg5uQdYFuzj1UBEo1og

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-18 09:35:42 -04:00
Tom MoorandClaude Opus 4.8 3ffd088624 fix: Consistent usage of configured proxy with auth providers (#13018)
* fix: Consistent usage of configured proxy with auth providers

* lint

* fix: Harden proxy agent helper against missing OAuth2 client

Degrade to a logged warning instead of crashing auth at boot if a
strategy's internal _oauth2 client is ever absent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 19:58:32 -04:00
6a14ac8ae6 feat: Slab importer (#12859)
* Add Slab importer backed by the Markdown import pipeline

Slab exports a zip of Markdown documents that is structurally identical to
the Outline Markdown export, with one difference: images are referenced as
remote signed URLs rather than files inside the archive.

Rather than build a parallel importer, this reuses the Markdown task and
processor and teaches the shared import pipeline to download remote images:

- Enable per-page attachment upload for the Markdown task. The base
  APIImportTask already downloads remote image/video/attachment URLs and
  rewrites them to internal redirect URLs; the Markdown task previously
  opted out because its attachments live in the zip.
- Make the base upload step skip URLs that are already internal, so local
  zip attachments (resolved to redirect URLs during rewriteMarkdown and
  uploaded from the archive in onAllTasksCompleted) pass through untouched
  while remote URLs are fetched and re-hosted. Also guards the URL rewrite
  against nodes not present in the download map.
- Introduce IntegrationService.Slab as a distinct, importable service that
  routes through the Markdown task/processor, so imports are tracked and
  labelled as "Slab" with no duplicated import logic.
- Wire the create API (schema + route), the import settings UI (Slab card
  and dialog), and a placeholder logo asset.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AC5Ho2SHBjNpxEAMr5uL6o

* Add shell SlabAPIImportTask as a Slab-specific seam

Slab imports previously ran through MarkdownAPIImportTask directly. Introduce
a thin SlabAPIImportTask subclass so Slab gets its own task name (for
scheduling, tracing, and retries) and a dedicated place to override
behavior as Slab exports diverge from the generic Markdown shape.

The subclass inherits all conversion/attachment/persistence logic and only
overrides scheduleNextTask to keep the whole import chain on the Slab class.
MarkdownImportsProcessor now selects the task implementation by the import's
service when scheduling the first task.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AC5Ho2SHBjNpxEAMr5uL6o

* Unwrap single root "slab" directory in Slab imports

Slab exports wrap the entire workspace in a single top-level directory named
"slab". Without handling this, the import produces one "slab" collection
containing everything, instead of mapping each workspace area to its own
collection.

Add a `resolveCollectionRootNodes` seam on the Markdown bootstrap phase
(default: pass entries through unchanged) and override it in SlabAPIImportTask
to descend into a lone, case-insensitive "slab" root directory so its child
directories become collections. Paths are left intact, so the attachment
manifest, completion re-walk, and internal-link resolution stay consistent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AC5Ho2SHBjNpxEAMr5uL6o

* Use the filename, not the first heading, as the title for Slab imports

In Slab a document's leading heading is real content, not its title — the
filename is authoritative. Add an `extractTitle` option to
DocumentConverter.convert (default true, preserving existing callers) that,
when false, skips lifting a leading H1 into the title and leaves it in the
body. Expose it through a `shouldExtractTitleFromHeading` seam on the
Markdown task and override it in SlabAPIImportTask. With no extracted title,
the page falls back to the filename-derived title.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AC5Ho2SHBjNpxEAMr5uL6o

* Update logo

* Narrow MarkdownAPIImportTask generic back to Markdown

Slab imports now run through SlabAPIImportTask, so the base Markdown task no
longer handles the Slab service directly and its generic can return to
Markdown-only. SlabAPIImportTask keeps a Markdown | Slab union on its
scheduleNextTask override, which is required so the override stays a valid
(contravariant) override of the base's Markdown-typed parameter.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AC5Ho2SHBjNpxEAMr5uL6o

* Optimised images with calibre/image-actions

* reorder

* Move Slab importer into its own plugin

Restructure the Slab importer to live in plugins/slab, mirroring the Notion
plugin, instead of being threaded through the core Markdown importer.

- plugins/slab/server registers SlabImportsProcessor (Hook.Processor) and
  SlabAPIImportTask (Hook.Task). The processor extends MarkdownImportsProcessor
  to claim the Slab service and schedule the Slab task; the task keeps its
  Slab-specific overrides (remote image download, root "slab" dir unwrap,
  filename-as-title).
- plugins/slab/client registers the Slab import card (Hook.Imports) and owns
  the import dialog.
- Revert MarkdownImportsProcessor to Markdown-only and drop the hardcoded Slab
  card from the Import settings screen.

The Slab service enum, import schema, and create route remain in core, as the
Notion plugin's equivalents do.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AC5Ho2SHBjNpxEAMr5uL6o

* Generalize DropToImport to accept an importable service

DropToImport previously branched on `format` to map MarkdownZip/JSON/Slab
to specific imports.create calls, with a dead collections.import fallback.
Replace this with a single `service` prop and one generic imports.create
call. Core no longer hardcodes any service name (notably the Slab plugin
service), and the Markdown, JSON, and Slab dialogs each pass their service.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AC5Ho2SHBjNpxEAMr5uL6o

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-07-10 19:05:08 -04:00