MongoDB Launches MongoDB Ventures Investment Fund

Fifteen years ago, MongoDB was founded to give developers an intuitive way to work with ever-increasing volumes of highly varied data, moving in or near real-time. Since then, developers, tired of working with data within the narrow rows and columns of relational databases, have enthusiastically embraced MongoDB's flexible document data model to scale further and build faster. With more than 31,000 customers — and approaching $1 billion in annualized revenue — MongoDB is leading the disruption in the data infrastructure market.

Since the beginning, we have continued to invest aggressively in our product roadmap to keep enabling developers to build great software. But we are not just building a product, we are building an ecosystem. We want to help foster other companies that will join us in creating a developer-friendly, modern data experience designed for the next generation of applications.

Today we announce the launch of MongoDB Ventures, a strategic investment fund to empower innovators who help solve the growing challenges and complexities related to data and developer experience. Alongside investing, MongoDB Ventures also will pair entrepreneurs with executive mentors, support product and go-to-market initiatives, and provide access to partners and customers.

How will MongoDB Ventures work in practice? Let’s look at our inaugural investments.

Apollo GraphQL is a leader in open source and commercial GraphQL technologies that help engineering teams accelerate application development. Developers trust Apollo to implement their graphs, with more than 17 million monthly open source downloads and more than 10 billion daily queries run on the Apollo graph.

"We are excited to welcome MongoDB Ventures as an investor because MongoDB is legendary for its obsession with empowering developers — an obsession we share,” said Geoff Schmidt, Chief Executive Officer of Apollo GraphQL. “The investment will deepen our partnership and drive product innovation, with an initial focus on making it even more seamless for developers to build applications with the Apollo Graph Platform and MongoDB Atlas.”

BigID is a leading data intelligence platform that enables organizations to know their enterprise data and take action for privacy, security, and governance. With BigID, customers can make better decisions with their data, achieve compliance, scale with the evolving data privacy and protection landscape, improve data trust, and ultimately reimagine how they manage their data.

"Bringing MongoDB Ventures in as a strategic technical investor makes all the sense in the world given MongoDB's success in building an independent, best-in-class cloud offering," said Dimitri Sirota, Chief Executive Officer of BigID. "We're committed to innovating across privacy, security, and governance, and are thrilled to partner with MongoDB to reimagine data management in the cloud."

With 15 years spent honing the MongoDB developer experience, we're in a unique position to help discover and empower innovators who share our focus on data and developers. If this sounds like you, please contact us at ventures@mongodb.com.

← Previous

MongoDB Connector for Apache Kafka 1.7 Available Now

Today, MongoDB has released version 1.7 of the MongoDB Connector for Apache Kafka! This article highlights some of the key features of this new release! MongoDB errors to the Dead Letter Queue Apache Kafka version 2.6 added support for handling errant records. The MongoDB Kafka Connector for Apache Kafka automatically sends messages that it cannot process to the dead letter queue. This includes messages that fail during conversion but up until this release did not include errors that were generated within MongoDB. For example, consider the scenario where we have a topic, “Sales.OrderStaging” This topic includes messages that contain an ‘order-id’ field. The application needs to insert a new document into MongoDB and use that order-id as the primary key or ‘_id’ of the document. If there happens to be a duplicate order-id entered on the kafka topic, the kafka message should be routed to a dead letter queue topic and the mongodb connector should continue to process other orders. the following sink configuration highlights the configuration parameters that support this scenario: "errors.tolerance":"all", "mongo.errors.tolerance":"all", "mongo.errors.log.enable":"true", "errors.log.include.messages":"true", "errors.deadletterqueue.topic.name":"orders.deadletterqueue", "errors.deadletterqueue.context.headers.enable":"true", "writemodel.strategy":"com.mongodb.kafka.connect.sink.writemodel.strategy.InsertOneDefaultStrategy", "document.id.strategy":"com.mongodb.kafka.connect.sink.processor.id.strategy.PartialValueStrategy", "document.id.strategy.overwrite.existing":"true", "document.id.strategy.partial.value.projection.type": "AllowList", "document.id.strategy.partial.value.projection.list": "order-id" For example consider a kafka message with an order-id=5 and another message with the same order-id of 5. The sink connector will try to insert that second message with the same _id and there will be a MongoDB Error generated as expected. The kafka topic message which caused the error will be written to the orders.deadletterqueue topic. Once on the dead letter queue, you can inspect the errant records, update them, and resubmit them for processing. Setting the errors.deadletterqueue.context.headers.enable to true will add metadata to the DLQ message. This extra information may help with any automatic processing of errors in the queue. In addition to the DLQ, you can set errors.log.enable and error.log.include.messages configuration to write errors in the kafka connect log. Here is an example error from our scenario above: com.mongodb.kafka.connect.sink.dlq.WriteException: v=1, code=11000, message=E11000 duplicate key error collection: Sales.Orders index: id dup key: { _id: { order-id: 5 } }, details={} Bulk write improvements Today the connector sink process works with a bulk insert in an ordered fashion. For example, consider these 10 documents in a bulk operation: [1,2,3,4,5,6,7,8,9,10] If document number 5 failed, perhaps due to a duplicate _id error, the MongoDB driver would return this error back to the connector and the rest of the documents would not be written to MongoDB e.g. only [1,2,3,4] is written in MongoDB in the above example. While this might be acceptable for some use cases, for other scenarios with large batch sizes this can make reprocessing messages cumbersome. In Kafka 1.7, we introduced a new parameter bulk.write.ordered that by default is set to true which is the behavior as it exists today with the Kafka connector. Setting to false and running the above scenario will result in an end state of [1,2,3,4,6,7,8,9,10] written to MongoDB with 5 being written to the topic defined in the dead letter queue. Note that the actual order of the documents may be different since we specified false to bulk.write.ordered. For more information on error handling including information on the format of the DLQ headers check out the MongoDB Kafka Connector documentation . To setup a dead letter queue, check out the Creating a dead letter queue section within the Confluent Kafka Connect documentation. Changed retry logic Currently the Kafka Connector manages retries of the writes to MongoDB using the max.num.retries and retries.defer.timeout configuration properties . This feature was originally intended to address challenges such as network connection issues. Since that time the MongoDB drivers have implemented native capabilities that handle retry logic: The Kafka Connector uses the MongoDB Java driver and has retries enabled by default so there are no changes or extra configuration you need to do to enable retry in the Kafka Connector. Note: If you set retryWrites to false in the connection.uri configuration property, then retries are disabled for the sink connector. If you would like to leverage the drivers native retry capability, simply remove the “retryWrites”' parameter from the connection.uri. Allow disk use when copying The copy.existing.allow.disk.use configuration copies existing data from the source. It uses an aggregation pipeline that filters change stream events coming from the MongoDB source. In certain situations this pipeline can use up large amounts of memory. This flag enabled by default allows the copy existing aggregation to use temporary disk storage if required by the query. Note the default is true, but set to false if the process running MongoDB doesn't have the permissions for disk access. For more information see the ‘allowDiskuse’ option in the aggregate() documentation .

February 17, 2022
Next →

Staff Engineering at MongoDB: Your Path to Making Broad Impact

Andrew Whitaker is a Senior Staff Engineer at MongoDB. His previous experience spans tiny startups to enormous organizations like AWS, where he held several different roles focusing on databases. Before joining MongoDB, he worked at a startup building optimized machine learning models in the cloud. Read on to learn more about why Andrew decided to join MongoDB in a senior-level engineering role and how his work is driving improvement within our engineering organization. Why MongoDB I have long been a fan of MongoDB’s products and services. MongoDB the database has always been a pleasure to work with – the system “brings joy” to quote a phrase. As a Python developer, I appreciate how the Python driver feels “Pythonic” in a completely natural way. The programmer interacts with the database using Python constructs: dictionaries, lists, and primitive types. By contrast, SQL databases force me to change my mental model, and the query language feels like an add-on that does not blend with the core language. As an engineer, I am always looking to expand my knowledge and grow my skills. The scope of challenges engineers face at MongoDB is what triggered my interest in the company. We obviously have people working on core databases and distributed systems. But, we also have teams dedicated to machine learning, streaming data, analytics, networking, developer tooling, drivers, and many more areas. It is very hard to get bored working at MongoDB. Finally, I would be remiss if I did not mention the people. Overall, MongoDB’s engineering culture prioritizes intelligence, low ego, and an ability to get stuff done. CL/CI (Continuous Learning, Continuous Improvement) Working at MongoDB has provided me with opportunities for continued learning and growth. Though I do not program as much as I did earlier in my career, I have recently been exploring the Rust language. I’m excited by Rust because it avoids the tradeoffs between predictable performance and safety. My work in the search space has given me exposure to the fast moving world of AI: vector embeddings, RAG, etc. For various reasons, I think MongoDB is uniquely positioned to do well in this area. On top of this, I’m working on some initiatives that are not fully public. I can say that one focus area is improving the sharding experience for our customers. We believe MongoDB sharding is best-in-breed. Still, the process requires more manual configuration than we think is ideal: customers select the shard key, cluster type, shard count, etc. We give guidance here, but I think we can raise the bar in terms of offering a seamless experience with less “futz”. I’m also working with the search team. We believe there is a natural affinity between MongoDB’s document model and AI/ML workloads. We have some features in the works that extend this integration in new and interesting ways. I also spend a fair bit of time driving quality improvements across our suite of products. Our CTO Jim Scharf frequently refers to our “ big 4 ” goals: security, durability, availability, and performance. These goals are more important than any feature we build. I’ve been working across the company to help teams define their availability SLO/SLAs. It turns out that measuring availability is a subtle topic. For example, a naive approach of counting the percentage of failed requests can underestimate downtime because customers make fewer requests when a service is unavailable. So, the first step is to clarify the definition of availability. Finally, as a lapsed academic (in a distant life, I was a graduate student at the University of Washington Department of Computer Science and Engineering), I’m always interested in finding ways to bridge theory and practice. I’ve been collaborating with some folks in our research team to drive improvements to our replication protocols. There are theoretical results that suggest it is impossible to simultaneously achieve low latency and strong consistency (“linearizability” in the technical jargon). However, we believe there are intermediate points in the consistency/latency spectrum that have not been fully explored. This work hasn't been made into a product yet, but stay tuned. Flexible working MongoDB is a hybrid company. Like many of our engineers, I work outside the company headquarters in New York City (I live in Seattle). I appreciate MongoDB’s approach to hybrid working and that company leadership, starting with Dev , cares about the well-being of their employees. It seems there are companies that don’t seem to trust their employees to make decisions, such as which days to come into the office, so I’m thankful for the autonomy I receive at MongoDB to work in a way that’s best for me. Remote work has its challenges, but I would say that the benefit for my work/life balance has been transformative. Final thoughts I have found MongoDB engineers demonstrate a strong mix of technical depth, pragmatism, and empathy. I have yet to find the “smart jerk” prototype that seems to exist throughout the tech industry. Overall, I have found MongoDB is open to change and growth at both the team level and the individual level. There is a willingness to evolve and improve that aligns with the company’s values and leadership principles and enables the success of our technology and people. Find out more about MongoDB culture and career opportunities by joining our talent community .

November 20, 2024