MongoDB aggregation equivalent in Realm/ App services

MongoDB Atlas App Services & Realm
,
1

I have some complex aggregation pipelines I need to execute, but my clients are using app services. Aggregation needs to be done by connecting to the data source as I understand, which executes the query in atlas (cloud). And it takes a good 2-3 seconds in my machine, to do one aggregation and get the result back.

I know that there are some aggregation operators in RQL which are not exactly as powerful as to the actual aggregation.

I want to do some grouping, counting, and count by date fields properties (data/ dayOfMonth, dayOfYear
and so on) 
etc. Is there a way to do at least some of those through realm objects?

e.g: How am I supposed to do a simple counting by a field, like the following pipeline result, using realm sdk?

[
  {
    $group: {
      _id: "$state",
      count: { $sum: 1 }
    }
  }, ...
]
2

The question is a little unclear.

Are you stating you’re using Realm locally only (as App Services is depreciated) and have a local database you want to run aggregate functions against?

Can you clarify your coding platform and also provide sample data (as mentioned, Realm objects that have dayOfMonth etc properties) and then some code you’ve attempted so we can get more clarity on what’s being asked?

This is also a cross post to here - in case someone comes up with a solution

3
  1. We are using app services with sync (but clients can go offline sometimes, for a short while), so I want to know if it is possible to run the aggregation on realm objects which would be faster than aggregation on the actual collection
  2. Here’s a partial document from the “tasks” collection I have.
    {
  _id: ObjectId('66fbbfb826a136f8f67b422a'),
  on: ISODate('2024-10-01T09:24:08.719Z'),
  robot: 'peter',
  status: 'Completed',
}
    I want to get some data for analytics from this collection. Say I want to count tasks by day of week, by status. I can run this aggregation on the collection, but it is very slow with the atlas cloud DB.
    [
  {
    $group: {
      _id: {
        status: "$status",
        dayOfWeek: { $dayOfWeek: "$on" }
      },
      count: { $sum: 1 }
    }
  }
]
    With realm objects, I can just run a “for realm.objects(Task)” loop over all the documents and calculate this, but I want to know if there is another, more realm/mongoDB way to do it.

I am using the nodejs sdk with typescript, if that helps.

4

Something that will make this way faster is to denormalize your data - a common task with NoSQL databases.

Include the dayOfWeek as a separate property

{
  _id: ObjectId('66fbbfb826a136f8f67b422a'),
  on: ISODate('2024-10-01T09:24:08.719Z'),
  robot: 'peter',
  status: 'Completed',
  theDay:  "Tuesday"
}

Then your query is simply on all tasks on “Tuesday” that are “Completed”.

Keeping in mind that in less than a year, all Atlas services will be gone (depreciated) so there will be no sync’ing or access to Atlas from a client.

It’s a good time to move to another platform.

1 Like
5

It’s a good time to move to another platform.

Yeah we will have to unfortunately, but it takes time to migrate. And right now, we want to use what we have.

De-normalizing this way would be expensive. We might want to analyze our data in 1000 different ways, and we do not want 10-20 additional fields for that in each collection. (the example is a partial document, and it has more fields, including embedded objects)

So your answer is there is no way to do something similar to normal aggregation, from realm objects?

6

The date is a single field and denormalizing would be adding one additional field which will greatly speed up any analysis and make aggregate functions ‘simpler’.

Disk space is cheap so the cost increase would be negligible.

Based on your example,

With locally stored, denormalized data and 1M records, the response is a couple of seconds, so without understanding the scope of the actual task, it’s hard to say what a better solution would be.

1 Like