Commit Graph
12 Commits
Author SHA1 Message Date
Tom MoorandCopilot c853917502 perf: More popularity scoring performance improvements (#11305)
* perf: More popularity scoring performance improvements

* Update server/queues/tasks/UpdateDocumentsPopularityScoreTask.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-29 22:25:18 -05:00
Tom Moor dbfe9eb0e4 perf: More tweaks to popularity scoring job (#11293) 2026-01-27 20:06:34 -05:00
Tom Moor 00fb4d1af7 chore: Update node style imports (#11277)
- crypto → node:crypto
  - fs → node:fs
  - fs/promises → node:fs/promises
  - path → node:path
  - http → node:http
  - https → node:https
  - stream → node:stream
  - buffer → node:buffer
  - url → node:url
  - os → node:os
  - net → node:net
  - dns → node:dns
  - events → node:events
  - readline → node:readline
  - querystring → node:querystring
  - util → node:util
2026-01-26 20:51:50 -05:00
Tom Moor 041c4217b2 perf: Update documents popularity score (#10986) 2025-12-22 08:10:27 -05:00
Tom Moor bf45e97641 chore: Enforce type import consistency (#10968)
* Update types

* fix circular dep

* type imports

* lint type imports and --fix
2025-12-19 23:07:02 -05:00
Tom Moor 8cb07889ce perf: Further break up popularity batch querying (#10862) 2025-12-11 18:58:18 -05:00
Tom Moor 42959d66db chore: Add cron task partitioning (#10736)
* wip

* Implementation complete

* tidying

* test

* Address feedback

* Remove duplicative retry logic from UpdateDocumentsPopularityScoreTask.
Now that we're split across many runs this is not neccessary

* Refactor to subclass, config to instance

* Refactor BaseTask to named export

* fix: Missing partition

* tsc

* Feedback
2025-11-27 16:57:52 +01:00
Tom Moor b6a8986235 chore: Update UpdateDocumentsPopularityScoreTask to run 4 times per day (#10729)
* chore: Update UpdateDocumentsPopularityScoreTask to run 4 times per day

* chore: Add index to popularityScore column
2025-11-26 01:39:16 +01:00
Tom Moor ac820e4e2a fix: Speed up popularity score calculation further (#10728)
* fix: Speed up popularity score calculation further

* Add READ_ONLY database connection

* UNNEST performs better

* Move config to env
2025-11-26 01:06:24 +01:00
Tom Moor 9dfdf9a1ec fix: Refactor to decrease lock contention (#10727) 2025-11-25 23:32:55 +01:00
Tom Moor 8b2214fa5e fix: Popularity score should be calculated hourly, not daily (#10723) 2025-11-25 12:36:59 +00:00
Tom MoorandCopilot b51692cdc5 feat: Add popularity scoring (#10721)
* Simple first pass

* Use findAllInBatches

* Add comments,views,revisions

* Add 'popular' tab to Home

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Add 'Popular' tab to collections

* Boost search results based on popularityScore

* Move to unlogged temp table

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-25 13:11:05 +01:00