25 Commits
Author SHA1 Message Date
Tom Moor f43614713e perf: Replace keyspace SCAN with single-key cache deletes after commit (#14279)
* perf: Use single-key cache deletes after commit and add afterCommit helper

* fix: Await notification event scheduling without a transaction
2026-10-24 09:07:19 -04:00
Tom Moor aab83e8378 fix: Reliability of count-based limit checks (#13542)
* fix: Make count-based limit checks reliable under concurrency

Limits enforced by counting rows before an insert could be exceeded by
requests that ran at the same time, as each one read the same count
before any of them had written. Add a LockHelper that takes a Postgres
advisory lock for the remainder of the transaction, and use it to
serialize the check for webhook subscriptions, team domains, share
subscriptions and pins. The counts now also run on the transaction so
they include rows written earlier in the same request.

* Remove excessive tests

* Also guard simultaneous row removal
2026-08-24 19:14:55 -04:00
Tom Moor 77b01f11fc chore: Remove Fix decorator (#13100)
* chore: Remove Fix decorator

* test
2026-07-23 09:10:17 -04:00
Tom Moor ec3200c7d2 chore: babel -> swc (#12821)
* chore: babel -> swc

* Add ls guard

* PR feedback
2026-06-29 17:11:02 -04:00
Tom MoorandClaude Opus 4.8 8183c8e032 fix: Allow webhook urls without a top-level domain when self-hosted (#12728)
* fix: Allow webhook urls without a top-level domain when self-hosted

Webhook URL validation required a top-level domain, rejecting valid
internal hostnames such as http://webhook on self-hosted installations.
The TLD requirement is now only enforced when cloud hosted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* refactor: Extract webhook url validation into reusable IsUrl decorator

Move the conditional top-level domain validation into a reusable IsUrl
validator alongside the other model validators, and restore mocks
between create tests to keep hosting mode isolated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: Move webhook url validation tests onto IsUrl validator

The top-level domain validation is owned by the IsUrl decorator, so test
it directly via a collocated unit test rather than through the
webhookSubscriptions API.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* PR feedback

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 20:29:03 -04:00
Tom MoorandClaude Opus 4.8 0c0facc2a1 perf: Avoid empty webhook processor work via cached subscription lookup (#12593)
* Avoid empty webhook processor work via cached subscription lookup

WebhookProcessor ran for every event but most teams have no matching
webhook subscription, costing an empty processor job and a database query
per event.

Cache a team's enabled subscriptions ({ id, events }) in Redis, invalidated
by model lifecycle hooks, and add an optional BaseProcessor.shouldQueue hook
consulted by the global event queue so the webhook processor only enqueues a
job when a matching subscription exists.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feedback

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 17:53:40 -04:00
Tom Moor 0139b91b5d chore: Replace lodash with es-toolkit (#12281)
* chore: Replace lodash with es-toolkit

Migrate all direct lodash imports to es-toolkit/compat for a smaller,
faster, lodash-compatible utility library. Transitive lodash usage from
other packages remains unchanged.

* fix: Restore isPlainObject semantics in CanCan policy

The lodash migration aliased `isObject` to `lodash/isPlainObject` and
the codemod incorrectly mapped the local name to es-toolkit's `isObject`,
which also returns true for arrays and functions. This caused condition
objects in policy definitions to be skipped, breaking authorization
checks across the codebase.

* fix: Restore unicode-aware length counting in validators

es-toolkit/compat's size() returns string.length, while lodash's _.size()
counts unicode code points. Switch to [...value].length to preserve the
previous behavior so multi-byte characters like emoji count as one.
2026-05-06 21:03:47 -04:00
Tom Moor 6874b02cc7 Add max character count to inputs (#12006) 2026-04-12 11:44:28 -04:00
Tom Moor 00fb4d1af7 chore: Update node style imports (#11277)
- crypto → node:crypto
  - fs → node:fs
  - fs/promises → node:fs/promises
  - path → node:path
  - http → node:http
  - https → node:https
  - stream → node:stream
  - buffer → node:buffer
  - url → node:url
  - os → node:os
  - net → node:net
  - dns → node:dns
  - events → node:events
  - readline → node:readline
  - querystring → node:querystring
  - util → node:util
2026-01-26 20:51:50 -05:00
Tom Moor bf45e97641 chore: Enforce type import consistency (#10968)
* Update types

* fix circular dep

* type imports

* lint type imports and --fix
2025-12-19 23:07:02 -05:00
Tom Moorandgraphite-app[bot] 92db179230 chore: Add script to reset encrypted data (#10238)
* chore: Add script to reset encrypted data

* Update server/models/WebhookSubscription.ts

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* Update server/scripts/reset-encrypted-data.ts

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* Add confirmation

---------

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-09-23 08:34:46 -04:00
Hemachandar 5dd993adf5 Convert WebhookSubscription mutations (#8161)
* Convert 'WebhookSubscription' mutations

* add tests

* remove unnecessary assignment
2024-12-30 16:11:32 -08:00
Tom Moor dd1df68e74 chore: Refactor @Encrypted decorator (#7381)
* chore: Simplify encrypted decorator

* fix: Correctly handle and type nullable encrypted fields

* docs
2024-08-14 03:54:37 -07:00
Apoorv MishraandTom Moor 7e61a519f1 Type server models (#6326)
* fix: type server models

* fix: make ParanoidModel generic

* fix: ApiKey

* fix: Attachment

* fix: AuthenticationProvider

* fix: Backlink

* fix: Collection

* fix: Comment

* fix: Document

* fix: FileOperation

* fix: Group

* fix: GroupPermission

* fix: GroupUser

* fix: Integration

* fix: IntegrationAuthentication

* fix: Notification

* fix: Pin

* fix: Revision

* fix: SearchQuery

* fix: Share

* fix: Star

* fix: Subscription

* fix: TypeError

* fix: Imports

* fix: Team

* fix: TeamDomain

* fix: User

* fix: UserAuthentication

* fix: UserPermission

* fix: View

* fix: WebhookDelivery

* fix: WebhookSubscription

* Remove type duplication

---------

Co-authored-by: Tom Moor <tom.moor@gmail.com>
2024-01-12 22:33:05 +05:30
Tom Moor b7bfc4bb1a chore: Remove optimize imports to allow vite upgrade (#5691) 2023-08-14 11:44:58 -07:00
Newton 738fa55e12 fix: use real boolean instead of aws's bool (#5313) 2023-05-07 14:13:16 -07:00
dependabot[bot]andTom Moor 9d9df81dec chore(deps): bump sequelize from 6.20.1 to 6.29.0 (#4921)
* chore(deps): bump sequelize from 6.20.1 to 6.29.0

Bumps [sequelize](https://github.com/sequelize/sequelize) from 6.20.1 to 6.29.0.
- [Release notes](https://github.com/sequelize/sequelize/releases)
- [Commits](https://github.com/sequelize/sequelize/compare/v6.20.1...v6.29.0)

---
updated-dependencies:
- dependency-name: sequelize
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* lint

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tom Moor <tom.moor@gmail.com>
2023-02-23 19:19:12 -08:00
Apoorv MishraandTom Moor 7a590550c9 Sign webhook requests (#4156)
Co-authored-by: Tom Moor <tom.moor@gmail.com>
2022-09-24 14:19:26 -07:00
Tom Moor b45e6c504f fix: Prevent webhook delivery for deleted teams 2022-08-08 11:15:04 +02:00
Tom Moor b37a848914 Add limit of 10 webhooks/team 2022-08-08 10:58:47 +02:00
Tom Moor 47e73cee4e feat: Cleanup api keys and webhooks for suspended users (#3756) 2022-07-13 00:59:31 -07:00
Tom Moor 4a46d19846 fix: Improved model validation for Collection (#3749)
* fix: Added improved model validation for Collection attributes

* sp

* fix: Enforce title length in UI
2022-07-08 11:10:22 -07:00
Tom Moor 8ebe4b27b1 fix: Add additional model validation (#3725) 2022-07-02 14:29:01 -07:00
Tom Moor f744d488f6 chore: Soft delete webhooks (#3722) 2022-07-02 10:41:28 -07:00
Tom MoorandCorey Alexander 10f86ed218 feat: Webhooks (#3691)
* Webhooks (#3607)

* Get the migration and the model setup. Also make the sample env file a bit easier to use. Now just requires setting a SECRET_KEY and besides that will boot up from the sample

* WIP: Start getting a Webhook page created. Just the skeleton state right now

* WIP: Getting a form created to create webhooks, need to bring in react-hook-forms now

* WIP: Get library installed and make TS happy

* Get a few checkboxes ready to go

* Get creating and destroying working with a decent start to a frontend

* Didn't mean to enable this

* Remove eslint and fix other random typescript issue

* Rename some events to be more realistic

* Revert these changes

* PR review comments around policies. Also make sure this inherits from IdModel so it actually gets an id

* Allow any admin on the team to edit webhooks

* Start sending some webhooks for some User events

* Make sure the URL is valid

* Start recording webhook deliveries

* Make sure to verify if the subscription is for the type of event we are looking at

* Refactor sending Webhooks and follow better webhook schema

This creates a presenter to unify the format of webhooks. We also
extract the sending of webhooks and recording their deliveries to a
method than can be used by each of the different event type methods

We also add a status to WebhookDelivery since we need to save the record
before we make the HTTP request to get its id. Then once we make the
request and get a response we can update the delivery with the HTTP info

* Turn off a subscription that has failed for the last 25 deliveries

* Get a first spec passing. Found a bug in my returning of promises so good to patch that up now

* This looks nicer

* Get some tests added for the processor

* Add cron task to delete older webhooks

* Add Document Events to the Processor

* Revisions, FileOperations and Collections

* Get all the server side events added to the processor and make Typescript make sure they are all accounted for

* Get all the events added to the Frontend and work on styling them a bit, still needs some love though

* Get UI styled up a bit

* Get events wired up for webhook subscriptions

* Get delete events working and test at least one variant of them

* Get deletes working and actually make sure to send the model id in the webhook

* Remove webhook secrets from this slice

* Add disabled label for subscriptions that are disabled

* Make sure to cascade the delete

* Reorg this file a bit

* Fix association

* I removed secret for the moment

* Apply Copy changes from PR Review

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Actually apply the copy changes

TIL that if you Resolve a conversation it _also_ removes the 'staged suggestion' from your list on Github

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Update app/scenes/Settings/Webhooks.tsx

Missed this copy change before

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* Add disabled as yellow badge

* Resolve frontend comments

* Fixup Schema a bit and remove the dependency on the subscription

* Add test to make sure we don't disable until there are enough failures, and fix code to actually do that. Also some test fixes from the json response shape changes

* Fix WebhookDeliveries to store the responses as Text instead of blobs

* Switch to text better for response bodies, this is using the helpers better and makes the code read better

* Move the logic to a task but run in through the processor cause the tests expect that right now, moving the tests over next

* Split up the tests and actually enqueue the events from the WebhookProcessor instead of doing them inline

* Allow any team admin to see any webhook subscription for the team

* Add the indexes based on our lookup patterns

* Run eslint --fix to fix auto correct issues from when I tried to use Github to merge copy changes

* Allow subscriptions to be edited after creation

* Types caught that I didn't add the new event to the webhook processor, also added it to the frontend here

* I think this will get these into the translations file

* Catch a few more translations, use styled components better and remove usage of webhook subscription in the copy

Co-authored-by: Tom Moor <tom.moor@gmail.com>

* fix: tsc
fix: Document model payload empty

* fix: Revision webhook payload
Add custom UA for hooks

* Add webhooks icon, move under Integrations settings
Some spacing fixes

* Add actorId to webhook payloads

* Add View and ApiKey event types

* Spacing tweaks, fix team payload

* fix: Webhook not disabled after 25 failures

* fix: Enable webhook when editing if previously disabled

* fix: Correctly store response headers

* fix: Error in json/parsing/presentation results in hanging 'pending' webhook delivery

* fix: Awkward payload for users.invite webhook

* Add BaseEvent, ShareEvent

* fix: Add share events to form

* fix: Move webhook delivery cleanup to single DB call
Remove some unused abstraction

* Add user, collection, group context to membership webhook events
Some associated refactoring

Co-authored-by: Corey Alexander <coreyja@gmail.com>
2022-06-28 22:44:50 -07:00