Mobile Bytes #5: Understanding Partition-based Sync and Flexible Sync

Community Bytes Mobile Bytes
1

Hello Everybody,

Last week, we talked about Realm Relationships and types and how they are implemented client-side (on mobile).

This week I will focus on Realm Sync and discuss the differences between Partition-based Sync and Flexible Sync. This will help you choose the best approach for syncing your mobile data to MongoDB Atlas.

Create your Cluster and Realm App

Please feel free to follow previous realm-bytes on understanding cluster configuration . After creating your cluster, you create a Realm App and link to cluster

Enabling Sync

There are two ways to enable Sync in your mobile application:

Generate Schema if you have data in Atlas

When you have data in Atlas already or can load it easily, Realm will generate your client data models for you. There is also a Sync guide available on your application Dashboard page. The guide explains how to configure your collections for Realm Sync

1600×812 196 KB

Turn Development Mode On

If you aren’t starting with data, Development Mode can be very useful for getting started quickly – it allows you to build your mobile app from scratch and sync your data to Atlas. Once you have finished creating your client data models, development mode can be turned off and it will lock in a backend schema based on the models that you’ve created.

Sync Types

There are two Sync Types: Flexible Sync and Partition Based.

Partition Based

Partition-based Sync allows you to choose a single field called a Partition Key across all collections to divide Atlas data into partitions based on the field’s value.

Partition Key

This plays an important role in partition-based Sync. If you have opted for “Generate Schema” in the previous step, you can choose one of the fields from your Schema to split the data across your MongoDB collections into partitions/Realms based on the value. If you opted for Development mode and you are creating your application from scratch, make sure to choose a field that either exists in your client application schema or you create a new one.

For complex use-cases, please refer Partitioning Strategies documentation

Sync Permissions

Sync permissions is another important concept. This can vary depending on your use-case. For example if I have to use the Book and Author model explained previously, I would want users to read all information but write in their own private realm. For different use-cases, check Sync Permissions and Rules documentation.

Please Note - When you have Sync enabled on your cluster. Sync Permissions will serve as the permissions for all requests in the application.

Below is a snapshot of a random key and permissions for Book Author Model

1376×1562 141 KB

Flexible Sync

Flexible Sync is in preview but offers far more flexibility in data synchronization across devices and MongoDB Atlas.

Please Note: Flexible Sync requires MongoDB 5.0 and above

Flexible Sync uses subscriptions and permissions to determine which data to sync with your Realm App.

Some basic terminology used in Flexible Sync is explained below:

Queryable Fields

This refers to the fields in your client schema that your application can query. These queries will define the data that is synced down to your device and replace the concept of partitions. When you configure Flexible Sync on the backend, the field names are specified there. You can choose upto 10 queryable fields.

If you choose development mode, you can create fields that are part of your client schema. For example, for Book and Author model, I created the following queryable fields

1374×622 70.3 KB

Subscriptions

The query and its metadata are represented by a subscription. For flexible sync, it sends an RQL (Realm Query Language) query that the client app is trying to sync on in comparison to the partition key sent in the Partition-based Sync. Flexible Sync does not support all the operators available in RQL. See Flexible Sync RQL limitations for details.

Flexible Sync allows you to define a query in the client, and sync only the objects that match the query. When the client-side makes a query, Realm searches the server-side data set for documents matching the query.

The respective Realm SDKs provide a Subscription API to modify the queries. For example, if you are syncing on [author == “Rowling”] and [isRead ==true], you can remove the second query, add another one or update one of these, and the server will re-sync with the new data.

Rules and Permissions

Flexible Sync has a more powerful permission system and can be applied on a per-document level in comparison to partition-based permission systems that do not offer granular filtering.

When you set up permissions on the backend, you can choose from a provided template or design your own permissions from scratch.

For the Book and Author model, I chose “Users can read all data but only write their own data” from the provided options below:

1402×898 94.1 KB

The JSON expression for the selected permission is as below and this will be applied to all collections in the database

{
"rules": {},
 "defaultRoles": [
  {
    "name": "owner-write",
    "applyWhen": {},
     "read": true,
     "write": {
     "owner_id": "%%user.id"
       }
    }
  ]
}

If there is a requirement to apply more granular rules, those can be applied in the following way

{
defaultRoles: [],
    rules: {
       Author: [
           { name: "role1", applyWhen: { userId: "abc" }, read: true, write: { field1.user == %%user.id } },
          { name: "role2", applyWhen: true, read: true, write: { field1.user == %%user.id } },
        ]
    }
 }

To Note

  • The order in which the roles are listed is important. The role that returns true (in apply when condition) is selected at connection time and persists for the entire session. Once the role is selected, the read/write permissions are used to evaluate permissions per document.
  • If a collection has write permissions, then read is automatically true, even if “read” evaluates to false for a document.
  • The queries can be thought of as logical “AND” of the client’s subscriptions and the read permissions for the role assigned to the client

Please refer to Flexible Sync Role Strategies for more information.

I hope the information provided is helpful. Please feel free to share your experience, thoughts of using Realm Sync Types and Modes.

Until next week


Cheers :performing_arts:

4 Likes
2

Is there any guidance on when you should opt for one or the other sync mode? What use-cases would be suitable for each? If I start off with partition-based sync, does that lock me in or will I be able to combine the approaches in the future?

If I am not yet syncing data, would it be of benefit to wait until flexible sync is generally available? When is this expected to be available so I can used it in production? A lot of questions, but I find it a bit hard to understand what approaches to take without more information.

3

Hello @Simon_Persson,

Thank you for raising your concerns and these are really great questions :smiley:

Is there any guidance on when you should opt for one or the other sync mode? What use-cases would be suitable for each?

This will depend on your application requirement and needs. Flexible-Sync will work for almost all use-cases. For example, I have a news-reader app, I would want all users to read all news, but perhaps save topics to their own private realms. This use-case can be implemented with either of the Sync Type. If you want to extend the functionality, for example, users should access only certain titles and articles then Flexible Sync would be a better choice.

If I start off with partition-based sync, does that lock me in or will I be able to combine the approaches in the future?

You can convert from one type to another. Please refer Alter Sync Configuration in the MongoDB docs. Currently, it’s not possible to have both Flexible and Partition Sync in your application but this may change in the future.

If I am not yet syncing data, would it be of benefit to wait until flexible sync is generally available? When is this expected to be available so I can use it in production? A lot of questions, but I find it a bit hard to understand what approaches to take without more information.

Unfortunately, we don’t have a timeline available at this moment but will update here once more information comes to light. I would suggest testing your use case with both the options and making your decision from there.

The feedback from our users is always appreciated, so please feel free to raise any questions that you may have while working with either of the options.

I look forward to your response.

Cheers :performing_arts:

1 Like
4

I just read the Alter Sync Configuration link and it doesn’t say anything about converting from one type to another?

Are there any differences in performance between the two? I know that the legacy query based sync was removed because of performance. Is this resolved with partition based?

In my case. The main use case is that each user has their own data. This maps well to partitions. But there might be future use cases where users might want to share data with each other. What I don’t want to run into is that I start with one, and then discover that I have the need for another sync type and can’t change it if that makes sense.

On one hand, flexible sync sounds like it has pretty high complexity to me and I prefer to keep things simple. On the other hand, Client Reset is only mentioned in the partition based sync. This was a major headache for me when trying to use the legacy realm sync and I found it pretty much impossible to test during development. Is client reset not a thing with flexible sync?

5

Initially, there are performance differences between the two sync types, however during this preview we aim to bring flexible sync in line with the performance profile of partition-based sync, and if successful, flexible sync will become the default going forward. We believe this should be readily achievable because you can basically think of partition-based sync as an incredibly simple version of query-based sync ( give me all documents, across all collections, where this field matches this value). Of course, the performance of sync queries greatly depends on the type of query you run, essentially the big-O notation, if the query is slow to run on MongoDB then it will also be slow to sync.

Client reset will still be a thing with flexible sync, however, we realize that it is an undertaking for a developer to implement it themselves which is why we are shipping a new feature that performs the client reset for the developer automatically. The next iteration of the feature will automatically recover the data as well. See discardLocal enhancement here -

1 Like
6

Thanks Ian. If flexible sync is something that will become the default going forward, then that is a strong argument to try it out. I strongly suspect that my use case can be accommodated using both models, but it is good to know that this is something you are aiming for.

And great info with the client resets and good that you are working on them. Makes me super happy to hear that.

1 Like
7

There are some interesting question and answers on Flexible Sync that hopefully will be helpful to you all.

Thank you @Reveel for asking these great questions :smiley: Linking them here for others as well :smiling_face:

Cheers, :performing_arts:

1 Like