Skip to content

Sync streams #112

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 27 commits into
base: main
Choose a base branch
from
Open

Sync streams #112

wants to merge 27 commits into from

Conversation

simolus3
Copy link
Contributor

@simolus3 simolus3 commented Jul 22, 2025

This adds support for sync streams to the core extension.

Local sync stream subscriptions are managed by invoking powersync_control('subscriptions', ...), where the second argument can be used to

  • subscribe to a stream, supplying its name, parameters to use for the subscription and optionally a TTL + priority.
  • unsubscribe from a stream by its name and parameters.

Stream subscriptions are stored in a new ps_stream_subscriptions table (with the stream's name and subscription parameters being the primary key). Rows are created

  • when a stream is subscribed to explicitly.
  • when we receive a stream from the sync service by default

It's also possible to explicitly subscribe to a default stream by not supplying any parameters - this is useful to e.g. override the priority of such a stream. Another reason for storing default streams is that they should be visible when a user requests a list of all streams. We also store whether a particular stream has been synced (and when it has last been synced).

When the SDK wants to connect, it supplies a list of all stream subscriptions that are currently (SDKs don't have to take TTL/expiry into consideration) active in the start command. The Rust client will automaticall increase the expiry date of active streams when receiving keepalive messages.
Because we expect the set of currently active stream subscription to potentially update very frequently (whenever subscribe or unsubscribe is called anywhere in the app, potentially on every navigation), the set of currently active subscriptions can be updated within a stream iteration. Only when the client detects that a TTL has expired or an new subscription not previously cached has been added would it request a new iteration.

The sync status interface is expanded to include information about all streams part of the current sync information. We also include all bucket names for each stream here, which is useful for client SDKs to compute sync progress for each stream.

Finally, there's a new powersync_offline_sync_status() function that returns the sync status with the initial hasSynced and lastSyncedAt fields for every stream and priority. Client SDKs can use this to replace the manual SQL query they have for that today.

@simolus3 simolus3 marked this pull request as ready for review August 13, 2025 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant