20 Commits
Author SHA1 Message Date
Tom Moor eff1a43d9b fix: Keep serving requests when Redis is unavailable (#14457)
* fix: Fail fast on Redis for cache and rate limits, lock migrations in Postgres

* fix: Use the fail-fast Redis client for locks

* fix: Require a database pool of at least two connections

* chore: Rename cache Redis client to ephemeralClient
2026-11-03 08:45:56 -05:00
Tom MoorandClaude Opus 4.8 c8d4e8c0e7 perf: Bound MutexLock retries and de-dupe same-key cache misses (#13110)
* perf: Bound MutexLock retries and de-dupe same-key cache misses

MutexLock configured redlock with retryCount: 120, retryDelay: 1000 — a
2-minute retry window for locks with a 4s default TTL. Under cache-miss
storms (e.g. websocket reconnects all missing user.collectionIds) N
contenders piled onto one lock key, retaining ~K²/2 error objects plus
pending timers/promises. Bound retries to ~the lock TTL (retryCount: 15,
retryDelay: 250) and add a per-process in-flight de-dupe in
CacheHelper.getDataOrSet so N same-key misses share one lock acquisition
and callback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: Return falsy cached values instead of treating them as misses

getDataOrSet checked `if (cache)`, so a cached null/false/0/"" was treated
as a miss and re-populated on every read (e.g. Collection.getCachedDocument
Structure caches NavigationNode[] | null via the AfterSave hook). Check for
undefined — the only value getData returns on a genuine miss — so falsy
cached values are returned without an extra lock and DB query.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* deslop

* fix: Scope bounded lock retries to cache locks only

The tightened retry window (15×250ms) was set on the shared Redlock
instance, so it also affected long-lived callers — notably the 10-minute
"migrations" lock in checkMigrations, which fatal-exits on acquire failure
and would crash concurrent boots. Restore the generous default on the
shared instance and pass a bounded retry override (MutexLock.cacheRetry
Settings) only from the cache path via a new per-acquire retry option.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: Cache falsy callback results in getDataOrSet

populate only cached truthy callback results, so a valid null (e.g. from
Collection.getCachedDocumentStructure) was never stored and every miss
re-ran the lock + DB query. Skip caching only when the result is undefined,
and guard the CacheResult detection against null (typeof null === "object").

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 20:00:54 -04:00
Tom MoorandClaude Fable 5 db1d93f9b5 perf: Use SCAN and batched UNLINK instead of KEYS in CacheHelper.clearData (#12934)
* perf: Use SCAN and batched UNLINK instead of KEYS in CacheHelper.clearData

The KEYS command blocks the Redis event loop while it walks the entire
keyspace, which stalls all other clients on large installations —
notably on startup where the whole unfurl cache prefix is cleared.
Replace it with incremental SCAN iteration and delete the matched keys
in batches with UNLINK, which reclaims memory asynchronously, so
neither discovery nor deletion blocks the server. Adds test coverage
for CacheHelper including multi-page prefix clears.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015K9AAwaGTLVQZnp7uv5Ebj

* fix: Address review comments on CacheHelper.clearData

Unlink each SCAN batch as the scan progresses instead of buffering all
matching keys in memory, repeating until a full pass matches nothing so
implementations with unstable scan cursors still clear every key. Wrap
the operation in try/catch with logging for consistency with the other
CacheHelper methods, and mark constants as readonly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015K9AAwaGTLVQZnp7uv5Ebj

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-08 04:42:02 -04:00
Tom Moor 66eee9007f chore: Upgrade to Typescript 6 (#12737)
* chore: Upgrade TypeScript to 6.0
2026-06-18 17:34:05 -04:00
Tom MoorandClaude Opus 4.7 281b778b2d fix: Suspended users should not be included in cached member count (#12197)
* fix: Suspended users should not be included in cached member count for groups

* fix: Defer CounterCache hook registration until model is initialized

The previous test-only no-op hid a timing bug where setImmediate could
fire before the Sequelize instance had registered the related model,
causing "Model not initialized" failures. Poll until the model is
ready, and unref the pending immediate so it does not keep the event
loop alive in environments where the database is never initialized.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* perf: Reduce overhead of group member count invalidation

Select only the groupId column with raw queries and de-duplicate before
issuing Redis deletes, avoiding loading full GroupUser rows into memory
when a user belongs to many groups.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore: unref Redis healthcheck interval

Don't keep the Node event loop alive solely for the periodic ping; the
event loop should drain on its own when the application is shutting
down or a Jest worker is finishing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor: Centralize counter cache key in RedisPrefixHelper

Avoid duplicating the "count:<Model>:<relation>:<id>" string between
the CounterCache decorator and the User suspension hook by routing
both through a single getCounterCacheKey helper.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix: Walk to parent transaction when scheduling cache invalidation

Nested savepoints commit independently of their outer transaction, so
afterCommit callbacks attached to the inner transaction may run after
the outer rolls back, or never run at all. Match the pattern used in
Collection, Event, and base/Model and walk to the parent transaction
so the cache invalidation fires after the real outer commit.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 11:24:44 -04:00
Tom Moor a411e08f1f chore: Address code quality findings (#11960)
* chore: Address code quality findings

* Round 2, quality findings

* fix: Add fallback for MediaQueryList.addEventListener in test env

The jsdom test environment doesn't implement addEventListener on
MediaQueryList. Prefer addEventListener but fall back to the
deprecated addListener when unavailable.
2026-04-04 16:11:10 -04:00
40bbfc78cd Refactor: Extract Redis cache key generation to RedisPrefixHelper (#11376)
* Initial plan

* Refactor Redis cache keys: delegate CacheHelper to RedisPrefixHelper and update callers

Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>

* Add JSDoc documentation to getCollectionDocumentsKey method

Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>

* Remove unused indirection

* Remove mock

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>
Co-authored-by: Tom Moor <tom@getoutline.com>
2026-02-09 14:03:02 -05:00
Tom Moor 446a0e1071 fix: Do not show embed option for unembeddable links (#11323)
* fix: Do not show embed option for unembeddable links

* test
2026-01-31 13:57:25 -05:00
Tom Moor bffd11b593 fix: urls.unfurl thundering herd (#11194)
* Add tracing around unfurl calls

* fix: Prevent thundering-herd unfurls

* tracer -> traceFunction
2026-01-15 19:52:26 -05:00
Tom Moor 89425ccdab chore: Add a mutex lock around migrations to ensure in multi-instance deployments multiple machines don't attempt to run migrations at once (#10560) 2025-11-04 17:01:44 -05:00
Tom Moor 201fbb56eb perf: Add cache for document structure (#9196)
* Normalize Collection.findByPk

* Add caching of documentStructure

* fix: Do not set cache before transaction is flushed

* Mock Redis
2025-05-18 18:45:00 -04:00
Tom Moor 9ce7133837 fix: Increase lock timeout for calculating document diff (#8902) 2025-04-08 17:59:54 -07:00
Hemachandar 5c4eb32c26 fix: Release redis lock only when it hasn't expired (#8765)
* fix: Suppress redlock release errors

* release only when lock hasn't expired
2025-03-24 14:37:36 -07:00
Tom Moor fd5391cbb6 Cache diff generation for email notifications (#7987)
* Cache diff generation, closes #7982

* Handle cannot acquire lock

* Refactor to guard
2024-11-20 14:45:12 -08:00
Apoorv Mishra 5d85a3a093 Specify time conversion unit (#7458)
* fix: specificity in time units

* fix: milliseconds -> ms
2024-08-25 18:57:45 +05:30
Hemachandar e3cd9af6df fix: Set redis expiry in seconds (#7456)
* fix: Set redis expiry in seconds

* change expiry for github and iframely
2024-08-24 05:36:12 -07:00
Tom Moor 0ab8b52582 feat: Cache count of group members (#7377) 2024-08-17 14:34:12 -07:00
Apoorv Mishra 6a4628afef Refactor unfurling related types (#6730)
* wip

* fix: refactor unfurl types
2024-04-03 07:28:30 +05:30
Apoorv MishraandTom Moor 450d0d9355 Github integration (#6414)
Co-authored-by: Tom Moor <tom.moor@gmail.com>
2024-03-23 07:09:28 -07:00
Apoorv MishraandTom Moor bd2b32f9d5 Helper for cache related utilities (#6696)
Co-authored-by: Tom Moor <tom@getoutline.com>
2024-03-21 16:49:38 -07:00