Hi @Heiaha,
The large sizes are for debug builds. Stripped binaries (as shipped in the install packages) are much smaller. The strip utility can be used to remove debug info, but you will need some working space to compile. I would choose the most minimal SCons target you need, for example install-mongod.
If you have access to a better resourced desktop machine you can cross-compile for RPi4. Instructions should be similar to cross-compiling MongoDB v4.4 but I don’t have an Rpi4 handy to test the build process or outcomes.
Assuming your requirement isn’t to try out some of the new 5.0 features, the most straightforward path would be to install the pre-built MongoDB 4.4 packages 
Regards,
Stennie