Flexible sync, subscription based on other subscriptions results

MongoDB Atlas App Services & Realm Atlas Device Sync
, , ,
1

With flexible sync linked objects do not get downloaded to the client when an object gets downloaded. That, and another use case of our app lead to the following question:

How would one best get objects based on the results of a subscription?
So let’s say I have a collection “Assignment” where each assignment contains an “ownerId” and a “foreignUserId” (or a linked User object)
These Assignments are the source of truth and the only place where the client can find what other users to sync. So the client has a subscription on these Assignments where he gets all the assignments with his id as “ownerId”. But now the clients needs the actual other users he has an assigment for, so he would need a different subscription on Users, based on all the "foreignUserId"s retrieved from the first subscription. How could that be done with the Swift SDK?

Of course this could be solved easily by having a field “owner” inside a User and syncing directly the Users. But in our app, a User can have multiple "owner"s and each “Assignment” contains also additional information, different for each pair of “owner” and “foreignUser”. Additionally, we would like to separate the source of truth for the relations of these users from the actual users, since not every user should see the assigments of other users.

A different example where this could be useful is when you have employer and notes. lets say you query for the employers based on some of their properties. Now you want to sync all notes that are “createdBy” these employers. So you would also need a subscription on “Notes” based on the results of a “Employer” subscription.

This seems like a very usual use-case, yet I could not find anything in the swift SDK docs.

Thank you for your help!

P.S: if that was not understandable, or more information is needed I can always elaborate…

cc @Ian_Ward @Tyler_Kaye

2

I have thought about using “realm.objects(Example.self).observe” but quite unsure if there is no more straight forward way to do this.

“getting objects based on the results of a subscription” is necessary whenever linked objects do not contain themselves the same properties that were used to query the “parent” objects, which in my opinion is the case most of the time… I feel like I am missing something.

I suppose one could also use onComplete, or Async/await as described here

but then again I am not sure if it makes sense to add a new subscription for each “parent” object as there might be hundreds

3

As I thought, onComplete is only run when subscriptions are newly set or removed, but not if the actual data being synced changes. So I still do not have an idea how to setup subscriptions that rely on results of other subscriptions.

4

Hi, sorry for the delayed response. Do you mind sharing a little JSON snippet of your data for the two collections and what you are trying to do? Having a bit of a difficult time understanding what you are describing.

5

Thanks for your response @Tyler_Kaye. Sure I can elaborate. Let’s take the second example I stated above:
There are employees and notes. Then let’s say a manager has a few employees assigned to him, so he has a query for employees, based on properties of the employee (e.g. region).
He also needs all the notes of the employees that he gets as a result of his query, but the note object itself does not contain the necessary fields (region) that he used to query the employees. So the subscription on notes objects would have to be based on the results of the employee subscription. For example a subscription that is effectively “give me every Note that has creatorId == (id of one of my employees)”
Here would be example swift models:

class Employee: Object, ObjectKeyIdentifiable {
    @Persisted(primaryKey: true) var _id = UUID().uuidString
    @Persisted var region: String
}

class Note: Object, ObjectKeyIdentifiable {
    @Persisted(primaryKey: true) var _id = UUID().uuidString
    @Persisted var text: String?
    @Persisted var creatorId: String? // this is needed, because flexible sync does not allow queries on embedded objects (?)
    @Persisted var creator: Employee?
    // putting region in here as well is not an option
}
6

Just in case my answer went unnoticed, I’ll bump this question, as it is still open.
@Tyler_Kaye

7

Hi! Apologies on the delay again, will try to not let it become a pattern. I think that right now you are correct that this is a slightly awkward experience for you. We have two projects that have already started work that may be of help to you:

  1. Extending the IN operator in RQL to take a list. So you could do something like: creator IN {"list", "of", "employee" "ids"}
  2. Automatically including in linking objects when requested. This is a little bit further off but the work for it has started already. This will allow you to specify which links you would like to traverse, so you can query for “EMPLOYEES.region == “whatever”” and specify that you want to pull in the “notes” linking objects (would have to be a field in Employee".

I suspect that 1 will make your experience more pleasant in making this query, but am curious for your thoughts on the two projects / solutions.

Thanks,
Tyler

8

What’s the roadmap for the linked objects? It would literally remove the consideration of ever using any other database for mobile

9

Hi, automatically including linked objects in the result of a subscription is unlikely to happen in the near term. The reasons for this are:

  1. Performance: Doing so would be a significant performance degradation for the average user and would encourage data modeling that is not ideal for Atlas Device Sync at scale
  2. Usability: Most people do not want this and therefore we would need to extend the query language to opt-in for this. Additionally, there became many difficult theoretical problems with this approach in determining link depth, cycle detection, and reference counting as the same object could be linked to from several places and it was a further performance degradation to hold on to all of this information and ensure that only when the last object linking to an object is removed the linked object should also be removed.
  3. Permissioning: It becomes a little less clear how permissions are impacted across different collections in this model of synchronization
  4. Alternatives: The beauty of MongoDB is in the flexible and embedded data model. Instead of linking objects across collections, you can embed the objects directly for maximal performance. Additionally, you can always structure your data such that you can query on a partner collection based on the values of the first collection that are sent to the device.

I hope this explains our choice. I would be happy to hear if you have any additional feedback about this.

Best,
Tyler

10

Hello Tyler,

Thank you for your response.

Well, firstly the performance loss sounds awful so I understand the hesitation. But it might definitely be useful even in a very simple capacity. For example, I use flexible sync to sync all recent Posts but the comments are not embedded so I have to add a new subscription every time somebody opens a post to make sure the right comments are syncing. That process would be way simpler if I could just add a subscription for all comments in the initial config by just adding a predicate that all the linked comments should be synced along with the posts with the same permissions as the parent object.

Side note:
2) Is there a way to do this without using linking objects? I there a way I can somehow always have access to a dynamic list of all the POSTIDS so I can maintain a subscription for the comments using those as keys? I know realm swift SDK is updated very frequently but the docs are somewhat lagging or confusing sometimes. So if there’s a way to do that. That would really help me out. Thank you!

1 Like