Commit Graph
378 Commits
Author SHA1 Message Date
Tom Moor b8cb10248e lint 2026-03-05 20:28:10 -05:00
Tom Moor 762a0c78c7 perf: Introduce max task timeout on queue 2026-03-05 20:23:53 -05:00
38a3e651a7 fix: Rebuild of sourced permissions on document move (#11229)
* only recalculate permssions for moved document

* stop duplicating permissions

* add tests

* fix comment

* filter duplicates

* filter duplicates

* minor fixes

* fix child document permissions not being recalculated

* expand tests

* Update DocumentMovedProcessor.test.ts

* Update server/queues/processors/DocumentMovedProcessor.test.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* requested changes

* remove all sourced permissions before calculating new ones

* remove all sourced permissions before recalculating

* Add additional tests

---------

Co-authored-by: Tom Moor <tom@getoutline.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-05 08:22:03 -05:00
Copilotandtommoor 89cfc76d8f Remove artificial statement timeout from popularity score calculation (#11652)
* Initial plan

* fix: remove transaction wrapper and artificial statement timeout from calculateScoresForDocuments

The method was wrapping a read-only CTE query in a transaction solely to
SET LOCAL statement_timeout = '30000ms'. On instances with large tables this
30-second cap caused SequelizeDatabaseError: canceling statement due to
statement timeout. Removed the unnecessary transaction and the STATEMENT_TIMEOUT_MS
constant so the query runs unimpeded under the database server's own timeout policy.

Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>
2026-03-05 08:11:48 -05:00
Tom Moor 1f251d4829 perf: Include collection updatedAt property in websocket payloads (#11628) 2026-03-03 18:06:41 -05:00
Tom Moor 749bf49ebe fix: Avatar upload (#11624)
* fix: Avatar upload

* Restore internal public ACL
2026-03-03 08:06:17 -05:00
Tom Moor 371665b35c fix: Cannot access attribute text on document omitted from attributes 2026-03-02 22:09:55 -05:00
5a14944d0c Remove starred document when actor archives it (#11611)
* Initial plan

* Add DocumentArchivedProcessor to remove archived documents from actor's stars

Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>

* Fix unused variable in test

Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>

* Remove unnecessary transaction wrapper for single operation

Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>

* Add JSDoc documentation to perform method

Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>

* Address code review: add @throws annotation and fix spacing

Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>

* Fix TypeScript errors in test file - add non-null assertions for collectionId

Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>

* Use ctx method for events

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>
Co-authored-by: Tom Moor <tom@getoutline.com>
2026-03-01 17:47:17 -05:00
Tom Moor 0f23b63e35 perf: Remove unnecessary complex joins (#11581)
* perf: Remove unneccessary complex joins

* Further reduce joins and data loading
2026-02-26 09:11:37 -05:00
Tom Moor 112731a52b fix: Incorrect spread in DocumentImportTask
closes #11538
2026-02-23 23:50:45 -05:00
Tom Moor 957ce69d2e perf: Move image download out of transaction (#11528)
* perf: Move image download out of transaction

* loop
2026-02-23 22:14:03 -05:00
Tom Moor 0f45778d79 perf: Protect against ValidateSSOAccessTask thundering herd (#11532)
* perf: Protect against many tabs reloading at once

* PR feedback
2026-02-23 21:05:01 -05:00
Tom Moor 7be893f9a3 Refactor templates (#11027)
closes #8674
2026-02-20 18:53:00 -05:00
957648a588 feat: OAuth dynamic client registration (#11462)
* feat: DCR first pass

* Add cleanup task, management endpoints

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* wip

* Combine migrations

* Self review

* fix: Guard OAuth policies

* fix: Application access list not updating on deletion

* feat: Add OAUTH_DISABLE_DCR env var to disable dynamic client registration

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: Validate max length of redirect URIs in DCR schemas

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Self review

* Use withCtx methods for correct event creation

* Remove incorrect scopes_supported

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 17:30:19 -05:00
Tom Moor fe9daa0a75 fix: Collections with the same name overwrite in export (#11443) 2026-02-14 09:19:30 -05:00
Tom Moor 797c28a12e fix: Edits that only include a mention below edit distance do not trigger mention (#11434) 2026-02-13 18:02:47 -05:00
Tom Moor 38880f8335 fix: Missing check for disabled group mentions (#11435) 2026-02-13 08:06:24 -05:00
Tom Moor 0722b42613 fix: Potential task queue saturation in Notion importer (#11428)
* fix: Potential task queue saturation in Notion import

* Reduces concurrent Notion API pressure from 3× the recursive call depth down to 1
2026-02-12 21:56:00 -05:00
Tom Moor a06174b627 Revert "perf: Reduce database contention in ImportTask (#11361)" (#11411)
This reverts commit 8209f56e56.
2026-02-10 22:59:46 -05:00
40bbfc78cd Refactor: Extract Redis cache key generation to RedisPrefixHelper (#11376)
* Initial plan

* Refactor Redis cache keys: delegate CacheHelper to RedisPrefixHelper and update callers

Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>

* Add JSDoc documentation to getCollectionDocumentsKey method

Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>

* Remove unused indirection

* Remove mock

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>
Co-authored-by: Tom Moor <tom@getoutline.com>
2026-02-09 14:03:02 -05:00
Tom Moor 8209f56e56 perf: Reduce database contention in ImportTask (#11361)
* perf: Reduce database contention in ImportTask

* fix: Reuse transaction when available
2026-02-07 17:02:35 -05:00
Tom Moor 0354548f22 perf: Improve slow query in CleanupDeletedDocumentsTask (#11317) 2026-01-30 06:50:30 -05:00
Tom Moor e1e82ef4ac perf: Remove remaining usage of problematic findOrCreate (#11306)
* perf: Remove remaining usage of problematic findOrCreate

* fix spread order
2026-01-29 22:54:09 -05:00
Tom MoorandCopilot c853917502 perf: More popularity scoring performance improvements (#11305)
* perf: More popularity scoring performance improvements

* Update server/queues/tasks/UpdateDocumentsPopularityScoreTask.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-29 22:25:18 -05:00
Tom Moor dbfe9eb0e4 perf: More tweaks to popularity scoring job (#11293) 2026-01-27 20:06:34 -05:00
Tom Moor fcd40a93a4 feat: Make markdown importer generic (#11278) 2026-01-26 22:11:04 -05:00
Tom Moor 00fb4d1af7 chore: Update node style imports (#11277)
- crypto → node:crypto
  - fs → node:fs
  - fs/promises → node:fs/promises
  - path → node:path
  - http → node:http
  - https → node:https
  - stream → node:stream
  - buffer → node:buffer
  - url → node:url
  - os → node:os
  - net → node:net
  - dns → node:dns
  - events → node:events
  - readline → node:readline
  - querystring → node:querystring
  - util → node:util
2026-01-26 20:51:50 -05:00
Tom Moor 86811a5556 fix: Hanging read stream handle (#11237)
* fix: Hanging read stream handle

closes #11236

* comment
2026-01-21 22:53:49 -05:00
Tom Moor ed03b9d548 fix: ImportJSONTask file resolution (#11231) 2026-01-21 04:16:41 +00:00
yy 0230e1c5a5 chore: fix typos in comments and messages (#11203) 2026-01-17 10:50:06 -05:00
HemachandarandTom Moor 5584089441 feat: Figma integration (#11044)
* OAuth

* store logo

* unfurl support

* refresh token

* support for list

* embed list

* mention menu for all embeds in a list

* multi-level list

* logo

* account level connection

* tsc

* Update Icon.tsx

* coderabbit feedback

* RFC 6749 suggestion

---------

Co-authored-by: Tom Moor <tom@getoutline.com>
2026-01-15 20:27:00 -05:00
Tom Moor dfc2102450 fix: Export no longer includes private collections (#11153) 2026-01-11 22:06:14 -05:00
NewtonandTom Moor 2d1092a2ca feat: Use signed urls in "copy as markdown" (#10821)
* feat: Use signed urls (ttl configurable) for attachment urls in "copy markdown"

* make signed urls a parameter, not a instance configuration

* feat(fix): copy as markdown: refactor toMarkdown (use signed urls)

* attempt to fix tests failing

* adjust markdown exporting in revisions too

* formatting

---------

Co-authored-by: Tom Moor <tom@getoutline.com>
2026-01-10 21:42:44 -05:00
HemachandarandTom Moor 50759d40e8 feat: Option to export nested documents (#9679)
* add migration file

* documents.export API

* download dialog

* file ops list item

* export task

* download modal styling

* cleanup

* lint

* Restore individual download actions

---------

Co-authored-by: Tom Moor <tom@getoutline.com>
2026-01-10 21:19:33 -05:00
Tom Moor 57b6e9aca4 feat: Passkey support (#11065)
closes #6930
2026-01-05 19:58:46 -05:00
7e2d2542b0 Split group mention notifications into separate background task (#11040)
* Initial plan

* Split group notifications into separate background task

- Created GroupMentionedInCommentNotificationsTask for handling group mention notifications
- Updated CommentCreatedNotificationsTask to delegate group notifications
- Updated CommentUpdatedNotificationsTask to delegate group notifications
- Added comprehensive tests for the new task
- Improved scalability with batch processing (10 members per batch)
- Enhanced resilience by isolating group notification failures

Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>

* Optimize database queries in GroupMentionedInCommentNotificationsTask

- Batch fetch users instead of individual queries for better performance
- Add defensive comment explaining duplicate mentions check
- Create user map for O(1) lookups within batch

Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>

* Add comment explaining concurrency control in batch processing

Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>

* Remove unnecessary document loading - use event properties directly

- Removed Document import and document loading query
- Use event.documentId and event.teamId directly
- Reduces database queries and improves performance
- canUserAccessDocument loads document internally when needed

Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>

* Fix TypeScript errors and test failures

- Fixed test: buildComment requires documentId parameter
- Fixed type error: use nullish coalescing for group.actorId ?? event.actorId
- Fixed type definition: use Omit<CommentEvent, "data"> to avoid type conflicts
- All TypeScript errors resolved

Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>

* test

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: tommoor <380914+tommoor@users.noreply.github.com>
Co-authored-by: Tom Moor <tom@getoutline.com>
2026-01-03 10:14:56 -05:00
Tom Moor 222d86d084 fix: Flaky ErrorTimedOutFileOperationsTask test (#11011) 2025-12-27 14:34:38 -05:00
Tom Moor a3b2615edf chore: Remove future public bucket usage (#10977)
* No longer upload avatars to public bucket

* Public redirect

* tests

* test

* test
2025-12-24 20:24:58 -05:00
Tom Moor 041c4217b2 perf: Update documents popularity score (#10986) 2025-12-22 08:10:27 -05:00
Tom Moor bf45e97641 chore: Enforce type import consistency (#10968)
* Update types

* fix circular dep

* type imports

* lint type imports and --fix
2025-12-19 23:07:02 -05:00
Tom Moor a54e66e19a chore: Test improvements (#10945)
* Lazy queues, correctly closing Redis and server

* feedback

* fix: Tests not correctly split across matrix
2025-12-17 23:15:55 -05:00
Tom Moor 5d5e56251c fix: HealthMonitor check can fail if long running job (#10883) 2025-12-14 19:15:43 -05:00
Tom Moor 8cb07889ce perf: Further break up popularity batch querying (#10862) 2025-12-11 18:58:18 -05:00
Tom Moor 42959d66db chore: Add cron task partitioning (#10736)
* wip

* Implementation complete

* tidying

* test

* Address feedback

* Remove duplicative retry logic from UpdateDocumentsPopularityScoreTask.
Now that we're split across many runs this is not neccessary

* Refactor to subclass, config to instance

* Refactor BaseTask to named export

* fix: Missing partition

* tsc

* Feedback
2025-11-27 16:57:52 +01:00
Tom Moor b6a8986235 chore: Update UpdateDocumentsPopularityScoreTask to run 4 times per day (#10729)
* chore: Update UpdateDocumentsPopularityScoreTask to run 4 times per day

* chore: Add index to popularityScore column
2025-11-26 01:39:16 +01:00
Tom Moor ac820e4e2a fix: Speed up popularity score calculation further (#10728)
* fix: Speed up popularity score calculation further

* Add READ_ONLY database connection

* UNNEST performs better

* Move config to env
2025-11-26 01:06:24 +01:00
Tom Moor 9dfdf9a1ec fix: Refactor to decrease lock contention (#10727) 2025-11-25 23:32:55 +01:00
Tom Moor 8b2214fa5e fix: Popularity score should be calculated hourly, not daily (#10723) 2025-11-25 12:36:59 +00:00
Tom Moor 747fde1105 feat: Sync avatars automatically from iDP (#10718)
* feat: Sync avatars automatically from iDP unless user has manually uploaded

* Update test for new logic
2025-11-25 13:36:15 +01:00
Tom MoorandCopilot b51692cdc5 feat: Add popularity scoring (#10721)
* Simple first pass

* Use findAllInBatches

* Add comments,views,revisions

* Add 'popular' tab to Home

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Add 'Popular' tab to collections

* Boost search results based on popularityScore

* Move to unlogged temp table

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-25 13:11:05 +01:00