Recommend Data Schema

David_Thompson · June 18, 2022, 8:14am

Hello all… I am new to mongoDB and I have a couple questions. I plan on asking them one at at time so here goes for my first question.

I am currently coding an application in Python… That being said, the switch for me from .net to Python isn’t without its pains. This brings be to my first question.

In taking the Developer Courses I have come to one big question… Embed Objects verses using Lookup.

I currently am running one database titled KLD9_Bazaar with 7 collections. They are

Cart
Country
Invoice
Product
Session
Store
User

The User collection is where my question begins. I currently am trying to embed objects within the user object. Below is my current schema for the user collection.

Specifically Address and Card (Credit Card) are embedded objects.

First… Is this the best way to do this, or should I use a lookup query to embed the other objects within the user display when returned?

Example: John Doe logs in to purchase some items… When he logs in, the lookup is used for both the card (or cards) he uses and his identified addresses (billing and shipping)?

It seems that I am trying to over complicate this and I wanted to get some recommendations. I will use the recommendations to further develop the application… The store collection will have product in it… Populate the products in the store by using the lookup for the product collection to add the products when the store is loaded.

for example:

ABC and XYZ sell shirts. Then the customer looks at ABC store, the products that ABC sells will be loaded by using the lookup query.

Is it better to use lookup or embed… and if embedding is better, should it be embedded as an object?

Thanks for taking the time to read this and give me some pointers… Once this is answered… I will move on to my next question… Indexing… LOL.

Respectfully,
Dave Thompson
Co-Founder
Global Kloud Solutions

NeNaD · June 18, 2022, 12:30pm

Hi,

Here are some main pointers:

If you use embedded objects, that is faster because it is only one database query. When you use $lookup, that is additional database query.
You don’t want to use embedded approach when you will have a lot of embedded objects, because that would not scale. MongoDB has a limit of 16MB for each document in a collection.

For your use case I would suggests this:

For User collection, go ahead and add address and card as embedded objects. You can add them as objects or as an array of objects, but either way, every user will probably not have more that 5-10 addresses and cards.
For Stores and Products collections, use $lookup. This would not be a good place to use embedded approach, because each store can potentially have thousands of products, and if you would put thousands of products inside store document, you can potentially reach limit of 16MB for the document.

David_Thompson · June 21, 2022, 1:18am

Sorry @NeNaD I have been busy looking at what you said and applying it to my current project… I have now gotten the hang of nesting objects. This brings me to my next question.

I have been looking at how to embed a document into the Users database (the address and card field like you recommended) and I believe those are the objects right? (I also forgot to add the cart since each user should have their own cart with products in it)

But upon looking up multiple questions I came across the mongoengine…

My current classes are written like this:

from dataclasses import dataclass
from model.cart import Cart


@dataclass
class User(object):
    
   
    # __useritems = None
    
    # @staticmethod
    # def get_user_items():
    #     if User.__useritems == None:
    #         User.__useritems = []
    #     return User.__useritems
    
    def __init__(self, first_name, last_name, phone, email, uid, cart):
        self.first_name = first_name
        self.last_name = last_name
        self.phone = phone
        self.email = email
        self.uid = uid
        self.cart = cart
       

    def __getattr__(self, key):
        return getattr(self, key)

    def __repr__(self):
        return "first_name: {}, last_name: {}, uid: {}, cart: {}, card: {}".format(self.first_name, self.last_name, self.uid, self.cart, )


    first_name: str
    last_name: str
    phone: str
    email: str
    uid: str
    cart: Cart

I find the documentation on mongoengine to define classes quite differently. They define classes like this:

class User(Document):
    email = StringField(required=True)
    first_name = StringField(max_length=50)
    last_name = StringField(max_length=50)

I cannot find any documentation that tells me what format I need to use… Can you point me in the right direction for clarification? I am dying to read about this and whether I need to use the schema outlined in the mongoengine or if I can use the schema like all the tutorials explain in Python. I.E. can I use the standard class definition or do I need to use the mongoengine verison.

From what I can gather, the mongoengine version enforces validation. If I don’t use this format, I’m assuming I need to program in the validation. (the required portion and also max length, etc)

Am I on the right track?

Respectfully,

Dave Thompson

NeNaD · June 21, 2022, 1:41am

Hi @David_Thompson,

I am not familiar with Python implementation.

You should probably wait for someone that is familiar with Python implementation to answer this.

David_Thompson · June 21, 2022, 1:47am

Thanks… Should I move it to an entirely new topic?

NeNaD · June 21, 2022, 2:07am

I would say just create a new topic with that question.

It’s kinda new question anyway, and it’s more language specific than your original question.