Commit Graph
488 Commits
Author SHA1 Message Date
Tom Moor 7b751ab04b perf: Attach output stream before adding zip entries (#13167)
* perf: Attach output stream before adding zip entries

* fix: Handle failure
2026-07-27 19:13:18 -04:00
Tom Moor 5befd94fb8 perf: Fix streaming file outputs passthrough (#13153)
* perf: Fix streaming file outputs passthrough correctly

* test
2026-07-27 18:23:51 -04:00
Tom Moor 8267f22c31 chore: Remove client boolean guard in redirectOnClient (#13136) 2026-07-25 12:16:28 -04:00
Tom Moor 64fad39230 Remove dead ImportHelper (#13134) 2026-07-25 11:50:54 -04:00
Tom MoorandClaude 2d608d95e7 fix: Temporary directory left behind after S3 storage file handle (#13127)
* fix: Temporary directory left behind after S3 storage file handle

* Remove unused ZipHelper.extract

Use pipeline everywhere instead of manually wiring streams

* fix: Cleanup failure should not mask original error

Removing the temporary directory is best-effort, log the failure and
propagate the underlying error instead.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-25 08:17:04 -04:00
Tom Moor d63372694b chore: More lint warnings -> errors (#13122) 2026-07-25 07:56:13 -04:00
Tom Moor 64132a28d5 fix: Ensure JSDOM’s window is torn down (#13112)
* fix: Ensure JSDOM’s window (and its retained timers/resources) is torn down

* feedback

* catch
2026-07-23 20:36:04 -04:00
Tom MoorandClaude Opus 4.8 c8d4e8c0e7 perf: Bound MutexLock retries and de-dupe same-key cache misses (#13110)
* perf: Bound MutexLock retries and de-dupe same-key cache misses

MutexLock configured redlock with retryCount: 120, retryDelay: 1000 — a
2-minute retry window for locks with a 4s default TTL. Under cache-miss
storms (e.g. websocket reconnects all missing user.collectionIds) N
contenders piled onto one lock key, retaining ~K²/2 error objects plus
pending timers/promises. Bound retries to ~the lock TTL (retryCount: 15,
retryDelay: 250) and add a per-process in-flight de-dupe in
CacheHelper.getDataOrSet so N same-key misses share one lock acquisition
and callback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: Return falsy cached values instead of treating them as misses

getDataOrSet checked `if (cache)`, so a cached null/false/0/"" was treated
as a miss and re-populated on every read (e.g. Collection.getCachedDocument
Structure caches NavigationNode[] | null via the AfterSave hook). Check for
undefined — the only value getData returns on a genuine miss — so falsy
cached values are returned without an extra lock and DB query.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* deslop

* fix: Scope bounded lock retries to cache locks only

The tightened retry window (15×250ms) was set on the shared Redlock
instance, so it also affected long-lived callers — notably the 10-minute
"migrations" lock in checkMigrations, which fatal-exits on acquire failure
and would crash concurrent boots. Restore the generous default on the
shared instance and pass a bounded retry override (MutexLock.cacheRetry
Settings) only from the cache path via a new per-acquire retry option.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: Cache falsy callback results in getDataOrSet

populate only cached truthy callback results, so a valid null (e.g. from
Collection.getCachedDocumentStructure) was never stored and every miss
re-ran the lock + DB query. Skip caching only when the result is undefined,
and guard the CacheResult detection against null (typeof null === "object").

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 20:00:54 -04:00
Tom Moor 5386d689da perf: Refactor to run boot migration check as separate process (#13049)
* perf: Refactor to run boot migration check as separate process

* Apply suggestions from code review
2026-07-19 11:36:14 -04:00
Tom MoorandClaude Fable 5 24f95cadc3 perf: Reduce server memory usage and add container memory best practices (#13042)
* Reduce server memory usage and add container memory best practices

Profiling a full-service boot showed ~300MB RSS per service process, much
of it client-only dependencies pulled into the server module graph, and no
heap limits applied in containers:

- Import date-fns locales individually rather than via the locale index,
  which loaded all ~90 locales into every process
- Load @sentry/react dynamically in insertFiles so the browser Sentry SDK
  stays out of the server-side editor graph
- Replace class-validator isHexColor with the existing validateColorHex
  in the Highlight mark
- Extract icon names into a standalone IconNames module so server-side
  schema validation no longer imports Font Awesome icon packs, with a
  test to keep it in sync with IconLibrary
- Require S3Storage lazily so the AWS SDK and native CRT binding are only
  loaded when S3 file storage is configured
- Default each forked service process to a V8 heap limit derived from the
  cgroup memory constraint, preventing several processes from together
  committing more memory than the container allows
- Cap glibc malloc arenas in the Docker image to reduce resident memory
  fragmentation in multi-threaded processes

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016YMX5ReGHyxrR4ZE5ZbaXa

* Require both file storage backends lazily for consistency

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016YMX5ReGHyxrR4ZE5ZbaXa

* Load AWS SDK via dynamic imports compatible with test environment

Requiring the storage TypeScript modules lazily broke under Vitest, which
cannot resolve require() of source files and does not apply vi.mock to
bare requires. Move the laziness into S3Storage instead: the module is
imported statically, and the AWS SDK packages are loaded with dynamic
imports on first use, including deferred creation of the S3 client.

Also address review feedback: handle rejection of the dynamic Sentry
import, and warn when the minimum per-process heap limit exceeds the
memory budget of the container.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016YMX5ReGHyxrR4ZE5ZbaXa

* Consolidate S3 SDK loading into a single lazy accessor

The commands are loaded together with the client in getS3() rather than
as separate dynamic imports per method. The global S3Client test mock is
now a class, as a mock function is not constructable when reached through
a dynamic import.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016YMX5ReGHyxrR4ZE5ZbaXa

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-18 18:34:32 -04:00
Tom MoorandClaude Opus 4.8 533efdfcfe fix: OIDC 502 on sign-in from oversized id_token cookie (#12986) (#12995)
* fix: OIDC 502 on sign-in from oversized id_token cookie (#12986)

Storing the full OIDC id_token in the `oidcIdToken` cookie inflated the
sign-in response headers enough to exceed reverse proxy buffers, causing a
502 Bad Gateway. Store the id_token server-side in Redis keyed by a short
session identifier and keep only that identifier in the cookie, preserving
spec-compliant RP-initiated logout via `id_token_hint`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: Make OIDC logout token persistence best-effort

A Redis failure when storing or reading the logout id_token_hint should not
block sign-in or logout; fall back to omitting the hint instead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs

* refactor: Extract portable LogoutTokenStore for provider logout hints

Move the cookie-referenced, Redis-backed logout token persistence out of the
OIDC router into a provider-agnostic `LogoutTokenStore`. Any auth provider that
supports provider-initiated logout can now persist and consume a logout hint via
`new LogoutTokenStore(providerId)` without duplicating the cookie/Redis/error
handling. The OIDC-specific end-session URL construction stays in the router.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 10:24:12 -04:00
Tom MoorandClaude Opus 4.8 a054809b51 Use full keyspace for email verification codes (#13031)
* Use full keyspace for email verification codes

Email verification codes were generated with randomInt(100000, 1000000),
which can never produce a value below 100000. That excluded every code
with a leading zero — shrinking the keyspace from 1,000,000 to 900,000
and letting an attacker assume the first digit is never zero. It also
made the existing padStart(6, "0") dead code.

Draw uniformly from the full 0–999999 range and rely on padStart to
render leading zeros, restoring the full six-digit keyspace. Add tests
covering the six-digit invariant across many samples and the presence of
leading-zero codes.

* Simplify verification code jsdoc and drop leading-zero test

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EWfLg5uQdYFuzj1UBEo1og

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-18 09:35:42 -04:00
Tom MoorandClaude Opus 4.8 3ffd088624 fix: Consistent usage of configured proxy with auth providers (#13018)
* fix: Consistent usage of configured proxy with auth providers

* lint

* fix: Harden proxy agent helper against missing OAuth2 client

Degrade to a logged warning instead of crashing auth at boot if a
strategy's internal _oauth2 client is ever absent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 19:58:32 -04:00
6a14ac8ae6 feat: Slab importer (#12859)
* Add Slab importer backed by the Markdown import pipeline

Slab exports a zip of Markdown documents that is structurally identical to
the Outline Markdown export, with one difference: images are referenced as
remote signed URLs rather than files inside the archive.

Rather than build a parallel importer, this reuses the Markdown task and
processor and teaches the shared import pipeline to download remote images:

- Enable per-page attachment upload for the Markdown task. The base
  APIImportTask already downloads remote image/video/attachment URLs and
  rewrites them to internal redirect URLs; the Markdown task previously
  opted out because its attachments live in the zip.
- Make the base upload step skip URLs that are already internal, so local
  zip attachments (resolved to redirect URLs during rewriteMarkdown and
  uploaded from the archive in onAllTasksCompleted) pass through untouched
  while remote URLs are fetched and re-hosted. Also guards the URL rewrite
  against nodes not present in the download map.
- Introduce IntegrationService.Slab as a distinct, importable service that
  routes through the Markdown task/processor, so imports are tracked and
  labelled as "Slab" with no duplicated import logic.
- Wire the create API (schema + route), the import settings UI (Slab card
  and dialog), and a placeholder logo asset.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AC5Ho2SHBjNpxEAMr5uL6o

* Add shell SlabAPIImportTask as a Slab-specific seam

Slab imports previously ran through MarkdownAPIImportTask directly. Introduce
a thin SlabAPIImportTask subclass so Slab gets its own task name (for
scheduling, tracing, and retries) and a dedicated place to override
behavior as Slab exports diverge from the generic Markdown shape.

The subclass inherits all conversion/attachment/persistence logic and only
overrides scheduleNextTask to keep the whole import chain on the Slab class.
MarkdownImportsProcessor now selects the task implementation by the import's
service when scheduling the first task.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AC5Ho2SHBjNpxEAMr5uL6o

* Unwrap single root "slab" directory in Slab imports

Slab exports wrap the entire workspace in a single top-level directory named
"slab". Without handling this, the import produces one "slab" collection
containing everything, instead of mapping each workspace area to its own
collection.

Add a `resolveCollectionRootNodes` seam on the Markdown bootstrap phase
(default: pass entries through unchanged) and override it in SlabAPIImportTask
to descend into a lone, case-insensitive "slab" root directory so its child
directories become collections. Paths are left intact, so the attachment
manifest, completion re-walk, and internal-link resolution stay consistent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AC5Ho2SHBjNpxEAMr5uL6o

* Use the filename, not the first heading, as the title for Slab imports

In Slab a document's leading heading is real content, not its title — the
filename is authoritative. Add an `extractTitle` option to
DocumentConverter.convert (default true, preserving existing callers) that,
when false, skips lifting a leading H1 into the title and leaves it in the
body. Expose it through a `shouldExtractTitleFromHeading` seam on the
Markdown task and override it in SlabAPIImportTask. With no extracted title,
the page falls back to the filename-derived title.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AC5Ho2SHBjNpxEAMr5uL6o

* Update logo

* Narrow MarkdownAPIImportTask generic back to Markdown

Slab imports now run through SlabAPIImportTask, so the base Markdown task no
longer handles the Slab service directly and its generic can return to
Markdown-only. SlabAPIImportTask keeps a Markdown | Slab union on its
scheduleNextTask override, which is required so the override stays a valid
(contravariant) override of the base's Markdown-typed parameter.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AC5Ho2SHBjNpxEAMr5uL6o

* Optimised images with calibre/image-actions

* reorder

* Move Slab importer into its own plugin

Restructure the Slab importer to live in plugins/slab, mirroring the Notion
plugin, instead of being threaded through the core Markdown importer.

- plugins/slab/server registers SlabImportsProcessor (Hook.Processor) and
  SlabAPIImportTask (Hook.Task). The processor extends MarkdownImportsProcessor
  to claim the Slab service and schedule the Slab task; the task keeps its
  Slab-specific overrides (remote image download, root "slab" dir unwrap,
  filename-as-title).
- plugins/slab/client registers the Slab import card (Hook.Imports) and owns
  the import dialog.
- Revert MarkdownImportsProcessor to Markdown-only and drop the hardcoded Slab
  card from the Import settings screen.

The Slab service enum, import schema, and create route remain in core, as the
Notion plugin's equivalents do.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AC5Ho2SHBjNpxEAMr5uL6o

* Generalize DropToImport to accept an importable service

DropToImport previously branched on `format` to map MarkdownZip/JSON/Slab
to specific imports.create calls, with a dead collections.import fallback.
Replace this with a single `service` prop and one generic imports.create
call. Core no longer hardcodes any service name (notably the Slab plugin
service), and the Markdown, JSON, and Slab dialogs each pass their service.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AC5Ho2SHBjNpxEAMr5uL6o

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-07-10 19:05:08 -04:00
Tom MoorandClaude Opus 4.8 96372c97b6 fix: Hand desktop sign-in back to the app on self-hosted deployments (#12952)
* fix: Hand desktop sign-in back to the app on self-hosted deployments

The desktop `/desktop-redirect` handoff — which bounces the transfer token
back into the app via the `outline://` protocol — was nested inside the
`env.isCloudHosted && team.subdomain` branch of `signIn`. On a self-hosted
deployment that branch is never taken, so a desktop OIDC (or any) sign-in fell
through to the single-domain path, which set an `accessToken` cookie in the
browser and redirected to the app's home. The desktop app never received the
token and appeared to hang on the SSO screen.

Move the desktop handoff out of the cloud-hosted guard so it runs for every
deployment. The cloud sessions cookie is still written for cloud-hosted teams
(web and desktop alike), and the web / single-domain redirects are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wv8LuN54DQoxofSoWmTPVK

* refactor: Encode transfer token in redirect URLs and clarify comment

Address PR review: URL-encode the transfer token before interpolating it into
the desktop-redirect and auth/redirect query strings so the value can't
truncate the query if the token format ever changes, and reword the comment
that inaccurately described a cookie being set in this branch.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wv8LuN54DQoxofSoWmTPVK

* docs: Trim desktop redirect comment

Remove the trailing explanation added earlier; the comment reads cleaner
without it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wv8LuN54DQoxofSoWmTPVK

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-10 17:42:33 -04:00
Tom MoorandClaude Fable 5 032379da93 feat: Add support for OIDC provider returning JWT from userinfo endpoint (#12879)
* feat: Add support for OIDC provider returning JWT from userinfo endpoint

closes #12878

* PR feedback

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 16:28:13 +02:00
Tom MoorandClaude Fable 5 a1d0520c9f Add missing JSDoc to exported members in app and server utils (#12940)
* Add missing JSDoc to exported members in app and server utils

Documents all exported functions, classes, enums, and constants in
app/utils and server/utils that were missing JSDoc, per the project
documentation guidelines.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AsEQYA8zN813thxu7GjjzU

* Use @returns tag to match codebase convention

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AsEQYA8zN813thxu7GjjzU

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-08 08:30:11 -04:00
Tom MoorandClaude Fable 5 3fcef8d68e fix: validateUrlNotPrivate hardening (#12938)
* fix: Validate all DNS records in validateUrlNotPrivate SSRF check

A hostname with multiple A/AAAA records would pass validation as long as
the first record returned was public, even if another record pointed at a
private or link-local address the connection could actually use. Resolve
with { all: true } and reject if any record is private and not allowlisted.

Also wrap DNS resolution failures in InvalidRequestError so an unresolvable
hostname returns a clean 400 instead of an unhandled ENOTFOUND.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UotVRi1xphwNwAchiZZNi5

* chore: Add explicit type for resolved DNS addresses

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UotVRi1xphwNwAchiZZNi5

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-08 08:28:39 -04:00
Tom MoorandClaude Fable 5 db1d93f9b5 perf: Use SCAN and batched UNLINK instead of KEYS in CacheHelper.clearData (#12934)
* perf: Use SCAN and batched UNLINK instead of KEYS in CacheHelper.clearData

The KEYS command blocks the Redis event loop while it walks the entire
keyspace, which stalls all other clients on large installations —
notably on startup where the whole unfurl cache prefix is cleared.
Replace it with incremental SCAN iteration and delete the matched keys
in batches with UNLINK, which reclaims memory asynchronously, so
neither discovery nor deletion blocks the server. Adds test coverage
for CacheHelper including multi-page prefix clears.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015K9AAwaGTLVQZnp7uv5Ebj

* fix: Address review comments on CacheHelper.clearData

Unlink each SCAN batch as the scan progresses instead of buffering all
matching keys in memory, repeating until a full pass matches nothing so
implementations with unstable scan cursors still clear every key. Wrap
the operation in try/catch with logging for consistency with the other
CacheHelper methods, and mark constants as readonly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015K9AAwaGTLVQZnp7uv5Ebj

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-08 04:42:02 -04:00
Tom Moor ec3b26241f chore: Reduce file scanning in PluginManager on startup (#12911) 2026-07-06 22:57:39 +02:00
Tom MoorandClaude Fable 5 2b6ee79778 fix: all _FILE suffix secrets in env are cemented at startup (#12889)
* fix: Resolve _FILE env secrets lazily to avoid clobbering third-party variables

The Docker-style secrets support added in #11906 eagerly copied every
*_FILE environment variable into its base variable at boot. This broke
AWS SDK credential refresh on EKS Pod Identity: the rotating token in
AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE was frozen into the static
AWS_CONTAINER_AUTHORIZATION_TOKEN variable, which the SDK prefers and
never re-reads, so S3 access failed once the boot-time token expired.
It similarly resolved the standard OpenSSL SSL_CERT_FILE CA bundle into
Outline's SSL_CERT setting, failing validation at startup.

File secrets are now resolved lazily through a proxy when a variable is
read off the environment export, which limits resolution to variables
declared on the Environment classes, and SSL_CERT_FILE is explicitly
reserved for OpenSSL.

Fixes #12885

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MqP3Gb29iG5bJmhYuEAAjF

* refactor: Generalize reserved file variable documentation

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MqP3Gb29iG5bJmhYuEAAjF

* refactor: Clear SSL_CERT_FILE in test environment instead of reserving it

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MqP3Gb29iG5bJmhYuEAAjF

* docs: Trim withFileSecrets JSDoc

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MqP3Gb29iG5bJmhYuEAAjF

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-06 16:14:03 -04:00
Tom Moor ec3200c7d2 chore: babel -> swc (#12821)
* chore: babel -> swc

* Add ls guard

* PR feedback
2026-06-29 17:11:02 -04:00
Tom Moor 97c6cc3ab3 fix: Guard missing key in manifest (#12806) 2026-06-23 21:16:20 -04:00
Tom Moor 4c36bd83de fix: Prevent concurrent OAuth refresh attempts from revoking grant (#12752)
* Prevent concurrent refresh attempts from revoking grant

* test
2026-06-19 15:58:51 -04:00
Tom Moor 66eee9007f chore: Upgrade to Typescript 6 (#12737)
* chore: Upgrade TypeScript to 6.0
2026-06-18 17:34:05 -04:00
Tom Moor a2f9962958 fix: Update validateUrlNotPrivate to match implementation in SSRF (#12636) 2026-06-08 19:35:46 -04:00
Tom MoorandClaude Opus 4.8 0c0facc2a1 perf: Avoid empty webhook processor work via cached subscription lookup (#12593)
* Avoid empty webhook processor work via cached subscription lookup

WebhookProcessor ran for every event but most teams have no matching
webhook subscription, costing an empty processor job and a database query
per event.

Cache a team's enabled subscriptions ({ id, events }) in Redis, invalidated
by model lifecycle hooks, and add an optional BaseProcessor.shouldQueue hook
consulted by the global event queue so the webhook processor only enqueues a
job when a matching subscription exists.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feedback

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 17:53:40 -04:00
Tom Moor e864684d56 fix: Private IP lookup should be invalid request rather than internal error (#12592) 2026-06-05 09:19:05 -04:00
Tom Moor d0ede882c6 perf: More memory improvements (#12539)
* perf: Lazy import mailparser, @fast-csv, and franc deps

Moves heavy dependencies off the startup path into the narrow async code
paths that actually use them, mirroring the mammoth lazy-import change:

- mailparser: only needed for Confluence Word imports (confluenceToHtml)
- @fast-csv/parse: only needed for CSV imports (csvToMarkdown)
- franc / iso-639-3: only needed by the DocumentUpdateText worker task

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* perf: Lazy import jsdom dep

jsdom is one of the heaviest server dependencies but is only needed for
HTML export (ProsemirrorHelper.toHTML) and HTML import
(DocumentConverter.htmlToProsemirror). Move it to a lazy `await import`
inside those methods so its dependency tree stays off the startup path.

Both methods become async; all callers were already in async contexts.
The type-only usage in patchGlobalEnv is now an `import type`.
2026-05-30 17:31:04 -04:00
Tom Moor cecc9ef576 perf: Lazy import Mammoth dep (#12538) 2026-05-30 16:48:19 -04:00
Tom MoorandClaude Opus 4.7 76a3ba4e83 fix: Normalize IP addresses to avoid validation errors (#12500)
* fix: Normalize IP addresses to avoid validation errors on audit columns

Koa's `ctx.request.ip` can yield values that fail Sequelize's `isIP`
validation (X-Forwarded-For chains, IPv6 zone identifiers, "unknown"
from misconfigured proxies). This drops the IP metadata silently
instead of raising a 500 on Event/User writes.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test: Cover IP normalization on User setters

Reviewer feedback. Also switches the column-options `set` to TypeScript
get/set accessors — the original approach was shadowed by the class
field declaration and never actually fired, which the new tests would
have caught.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-27 22:52:05 -04:00
Tom Moor a4a67f2cdd fix: Upgrade yauzl, improve stream close handling 2026-05-27 20:33:33 -04:00
Tom Moor 0f2513346a Hardening of scope validation (#12490) 2026-05-27 18:27:34 -04:00
Tom Moor 8d44a0fd92 chore: Migrate from JSZip to Yazl (#12408)
* chore: Migrate from JSZip to Yazl

* Add koa stream helper, PR feedback
2026-05-21 23:27:23 -04:00
841ab022a6 Sanitize Windows-invalid characters in exported filenames (#12407)
* Sanitize Windows-invalid ZIP filename characters

Agent-Logs-Url: https://github.com/outline/outline/sessions/539082bc-597f-463d-b77c-6eb1bcf9bffa

Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>

* Refine Windows filename sanitization regex handling

Agent-Logs-Url: https://github.com/outline/outline/sessions/539082bc-597f-463d-b77c-6eb1bcf9bffa

Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>

* PR feedback

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>
Co-authored-by: Tom Moor <tom@getoutline.com>
2026-05-20 17:49:30 -04:00
Tom Moor ee5164290d perf: Move Markdown importer to zip stream (#12372)
* perf: Move Markdown importer to zip stream

* refactor

* refactor: Extract zip walk + tree builder into ZipHelper

Adds `ZipHelper.walk` and `ZipHelper.toFileTree` so other importers can
stream zip contents without extracting to disk. Tree construction uses
an O(1) path → node map; `./`-prefixed entries are normalized, while
dotfiles, `__MACOSX`, and `..` segments are filtered.

* PR feedback
2026-05-18 18:32:58 -04:00
Tom Moor 77cee2806c chore: getJWTToken -> getSessionToken (#12371)
* getJWTToken -> getSessionToken

Ensure expiry is included in payload

* Refactor test harness to avoid direct usage of getSessionToken
2026-05-17 16:58:52 -04:00
Tom Moor 879d2b8198 fix: Allow connecting additional auth providers on custom domain (#12364)
* fix: Unable to link secondary auth provider on custom domain

* doc

* chore: Custom -> Apex transfer token

* Refactor, address security concerns

* Ensure OAuth intent is single-use

* Secure OAuth state actor binding

* Use scrypt for OAuth actor session binding
2026-05-16 19:56:21 -04:00
Tom Moor 935e0bb7b9 chore: Fix all no-misused-spread lint warnings (#12327) 2026-05-12 17:30:08 -04:00
fc01deeefd chore(deps-dev): bump oxlint-tsgolint from 0.14.2 to 0.22.1 (#12320)
* chore(deps-dev): bump oxlint-tsgolint from 0.14.2 to 0.22.1

Bumps [oxlint-tsgolint](https://github.com/oxc-project/tsgolint) from 0.14.2 to 0.22.1.
- [Release notes](https://github.com/oxc-project/tsgolint/releases)
- [Commits](https://github.com/oxc-project/tsgolint/compare/v0.14.2...v0.22.1)

---
updated-dependencies:
- dependency-name: oxlint-tsgolint
  dependency-version: 0.22.1
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Switch tsconfig to bundler resolution for tsgolint 0.22.1

oxlint-tsgolint 0.22.1 removed support for moduleResolution=node10
(the alias for "node"). Switch to "bundler" with resolvePackageJsonExports
disabled so packages whose exports field omits a types condition still
resolve. Update markdown-it type imports to sub-paths since the package's
.d.mts entry only re-exports a subset of named types.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix: Resolve type-aware lint errors caught by tsgolint 0.22.1

oxlint-tsgolint 0.22.1 catches several await-thenable, no-floating-promises,
and no-meaningless-void-operator cases the prior 0.14.2 missed:

- Drop redundant inner `await` from Promise.all([await x, await y]) call sites
  so the array entries are real Promises rather than already-resolved values.
- Replace Promise.all wrappers around synchronous presenters (presentEvent,
  presentTemplate, presentPublicTeam) with plain map / direct calls.
- Wrap non-promise branches of ternaries inside Promise.all with
  Promise.resolve so the array remains thenable across both arms.
- Add `void` to the unawaited provider.connect() in the auth-failed retry
  chain, and remove `void` from the disconnect() call which returns void.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tom Moor <tom@getoutline.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 07:59:13 -04:00
Tom Moor 7ff1c84530 chore: Short-circuit common scanner/crawler routes (#12306)
* Shortcircuit common scanner/crawler routes

* PR feedback, remove query strings
2026-05-09 11:32:17 -04:00
Mark Steward fa990a33c0 Only preconnect to S3 if it's being used (#12298) 2026-05-08 14:41:32 -04:00
Tom MoorandClaude Opus 4.7 091346dfe8 chore: Migrate to vitest (#12272)
* wip

* Remove obsolete snapshots

* simplify

* chore(test): Convert mocks to TypeScript and tighten fetch mock types

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Remove unneccessary patches

* Migrate to msw instead of custom fetch mock

* Address PR review comments

- Split chained vi.useFakeTimers().setSystemTime() into separate calls.
- Switch test setup to dynamic imports so EventEmitter.defaultMaxListeners
  assignment runs before module init (static imports were hoisted above it).
- Drop redundant NODE_ENV guard in monkeyPatchSequelizeErrorsForJest; its
  sole caller already gates on env.isTest.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-06 21:10:51 -04:00
Tom Moor 0139b91b5d chore: Replace lodash with es-toolkit (#12281)
* chore: Replace lodash with es-toolkit

Migrate all direct lodash imports to es-toolkit/compat for a smaller,
faster, lodash-compatible utility library. Transitive lodash usage from
other packages remains unchanged.

* fix: Restore isPlainObject semantics in CanCan policy

The lodash migration aliased `isObject` to `lodash/isPlainObject` and
the codemod incorrectly mapped the local name to es-toolkit's `isObject`,
which also returns true for arrays and functions. This caused condition
objects in policy definitions to be skipped, breaking authorization
checks across the codebase.

* fix: Restore unicode-aware length counting in validators

es-toolkit/compat's size() returns string.length, while lodash's _.size()
counts unicode code points. Switch to [...value].length to preserve the
previous behavior so multi-byte characters like emoji count as one.
2026-05-06 21:03:47 -04:00
Tom MoorandClaude Opus 4.7 ad7e6c98ab chore: Vendor request-filtering-agent (#12266)
* chore: Vendor request-filtering-agent

* fix: honor fetch timeout and undefined allow list in proxy pre-flight

Default allowIPAddressList to [] so an unset ALLOWED_PRIVATE_IP_ADDRESSES
env var doesn't overwrite the agent's default and crash on .length, and
race the pre-flight DNS lookup against the request's abort signal so the
configured fetch timeout applies to slow DNS resolution.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-06 20:26:52 -04:00
Tom Moor f50bb00b29 Refactor of OAuth account linking flows (#12246)
* Refactor of OAuth account linking flows

* PR feedback
2026-05-02 18:54:38 -04:00
Tom MoorandClaude Opus 4.7 fca10221b9 chore: promote no-explicit-any from warn to error (#12244)
* chore: promote no-explicit-any from warn to error and resolve violations

Upgrades the oxlint rule severity and removes all 40 existing
`no-explicit-any` warnings across the codebase. Most call sites gained
proper types (SharedEditor refs, JSONNode/JSONMark for ProseMirror JSON
walking, DocumentsStore, dd-trace `Span` parameter inference, prosemirror
Fragment public API in place of internal `(fragment as any).content`).
A few load-bearing `any` uses were preserved with scoped disable
comments where changing the type would cascade widely (Sequelize JSONB
columns on `Event`, the `withTracing` higher-order function generic,
`Extension.options` consumed by many subclasses, dd-trace's `req`
patching).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-02 12:14:23 -04:00
Tom MoorandClaude Opus 4.7 269e7d048f chore: resolve lint warnings in oauth routes and BaseStorage (#12243)
Replaces `as any` casts when constructing OAuth2Server Request/Response
with explicit objects containing the fields the library actually
consumes, and switches BaseStorage's manual header spread to a
node-fetch `Headers` instance to avoid the no-misused-spread warning.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-02 09:53:57 -04:00
Tom MoorandClaude Opus 4.7 1f097b0fdd chore: resolve no-explicit-any lint warnings in plugins (#12237)
* chore: resolve no-explicit-any lint warnings in plugins

Replaces uses of `any` in the plugins directory with concrete types,
`unknown`, or structured type assertions, addressing the remaining
typescript-eslint(no-explicit-any) warnings flagged by oxlint.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore: address review feedback in GitLabIssueProvider

Drop trailing semicolon from log string and add early return in
`destroyNamespace` when neither `user_id` nor `full_path` is present
to avoid an unnecessary full-scan transaction.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-01 08:29:58 -04:00
Tom Moor 4c8a1c89b2 chore: resolve no-explicit-any and no-base-to-string lint warnings (#12217) 2026-04-29 17:45:02 -04:00