Skip to content

Conversation

@chris-bes
Copy link
Contributor

@chris-bes chris-bes commented Jun 30, 2025

Issue:

RN-1545: Epic datatrak sync


Note

Adds a new sync-server and shared sync utilities to enable offline-first Datatrak syncing via streaming APIs and a local PGlite DB, updates clients/types/infrastructure, and bumps key deps.

  • Sync Architecture:
    • New @tupaia/sync utilities (snapshotting, lookup, stream framing) and sync-server microservice with streaming sync endpoints and scheduled tasks.
  • Datatrak Web (offline-first):
    • Local DB via PGlite + DatatrakDatabase, ModelRegistry in-browser, ClientSyncManager, and new Sync/Database context providers.
    • New client sync routes and stream helpers; queries/mutations adapted to use local DB (offlineFirst) with server sync.
  • Database/Models:
    • Add sync tables/models (sync_session, sync_queued_device, sync_device_tick, sync_lookup, tombstone) and lookup-table population.
    • Expose toggle tombstone triggers in tests; move model typings to @tupaia/tsmodels.
  • Shared Libraries:
    • Server-boilerplate supports stream/pipe routes; server-utils adds StreamMessage, file helpers, ScheduledTask refactor.
    • Types/constants expanded (sync enums/requests, updated_at_sync_tick across models); new task filter utilities.
  • API Servers:
    • datatrak-web-server adds sync endpoints; entity-server filters via @tupaia/tsmodels helpers.
  • Infrastructure:
    • Nginx config and PM2 updated to host sync-api; build/deploy scripts include sync-server.
  • Dependencies:
    • Upgrade express and date-fns; add polyfills/plugins for browser DB/Node APIs.

Written by Cursor Bugbot for commit 5cf5909. This will update automatically on new commits. Configure here.

@jaskfla
Copy link
Contributor

jaskfla commented Jul 28, 2025

Looks like @tupaia/datatrak-web is missing these dependencies:

  • case (used in ./src/sync/insertSnapshotRecords.ts), and
  • knex (dev dependency used in ./src/database/DatatrakDatabase.ts)

@jaskfla jaskfla force-pushed the rn-1545-epic-datatrak-offline branch from c40ca28 to 5ef38f4 Compare September 11, 2025 06:11
cursor[bot]

This comment was marked as outdated.

@jaskfla jaskfla force-pushed the rn-1545-epic-datatrak-offline branch from be3a920 to d13f7e3 Compare September 26, 2025 01:33
cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

jaskfla and others added 30 commits October 20, 2025 12:59
…-project

feat(sync): RN-1545: Allow switching projects in datatrak sync
feat(datatrak): RN-1490: get permission group users offline
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants