Update the aggregation output to two different collections

Working with Data
,
1

I have a 10 stage pipeline. After stage 5 I need to “$merge” it to collection "A: and then continue with the pipeline and after stage 10 I need to “$merge” it to collection “B”. Is this possible and what is the best way to achieve it

2

By the documentation about $merge you cannot as $merge as to be the last stage of a pipeline.

But nothing stops you from running one pipeline per $merge that you have to do.

1 Like
4

Does the aggregation starts on A or B or a third one?

5

Yes, that can be done but the objective is to avoid querying the whole bunch of documents and processing them with the same logic again

6

You mean the second aggregation - it starts from the beginning. I want to avoid querying the documents and processing them up to stage 5 unnecessarily because I have already done it

7

That was clear from the beginning.

No. I mean the first aggregation. You did not really shared the pipeline so I will try to illustrate my question.

Is FirstCollection the same as A or B or is it a third one that is not A nor B? If it is a third one, then an idea that might work is that you create an on-demand materialized view from the first 5 stages. The you could $merge the materialized into A, and then start an aggregation on the materialized view with step-6 that you would merge into B.

This way step-1 to step-5 are executed only once when the materialized view is created.

1 Like
8

@Santosh_Gonsalves, I have spent time working on your issue. I think I have provided valuable information. I would appreciate a followup and perhaps closure with one of my post marked as the solution.

9

@steevej This is resolved. Appreciate the solution!!!

1 Like
10

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.