Keller Williams transforms into a technology innovator with MongoDB

INTRODUCTION

A global innovator harnessing the power of technology to build relationships

Keller Williams is the world‘s leading real estate company.

Or, at least, that is how it used to be known.

Co-Founder and now Executive Chairman Gary Keller said in 2018 that the company wanted to be recognized as a technology company. That might seem counterintuitive for a company founded in a single office in 1983 on the core belief that real estate is all about relationships. In truth, however, Keller Williams is masterfully synergizing the two, using technology to deliver a seamless user experience so agents are free to focus on what the founders knew matters most: fostering meaningful relationships.

Today, Keller Williams is a technology innovator that in 2020 closed on 1.2 million homes for $407.4 billion in sales volume and earned $10.5 billion. The company has more than 190,000 associates around the globe. The Keller Cloud platform consists of two main components: Command and Consumer. A smart CRM-plus solution, Command is a unified platform that supports internal teams with everything from lead generation to database management. Command also contains a consumer-facing component that streamlines home buying and selling processes, enabling agents and customers to interact and manage listings, contacts, marketing profiles, campaigns, and more.

Jim McClarty, Software Architect, Keller Williams, is on the Consumer team, serving up the real-time data that is the driving force supporting all KW.com searches; 182,000+ personalized sites (one for every agent); and much more.

The technology focus at Keller Williams (KW) is now part of the company’s DNA, said McClarty. “We build technology. That means we hire technologists.”

Technology is also key to what McClarty said is another core precept at KW: helping everyone take their performance and careers to the next level. This core belief is reflected all the way to the executive suite. In February 2021 KW named an executive who was previously its Co-Director of Growth as its new leader: Marc King, President, Keller Williams.

THE CHALLENGE

Making the right data instantly available to agents and customers

KW manages more than 500GB of listing data totaling approximately 70 million records across two databases. One database, Fast Facets, supports faceted search so consumers and agents can quickly obtain information about many properties. The other database, Master Dataset, or MDS, contains all of the property details that come into play once someone is interested in a particular property and wants to drill down to learn more.

KW ingests data from real estate multiple listing services (MLS) across the US. It was running massive and complex queries that ran slower than what was optimal to support a real-time user experience for either agents or customers. That was due in part to the reality that even relatively simple queries, such as map results showing all properties within a given area, had to access MDS.

KW needed to make changes in how its Command and Consumer applications were accessing and leveraging data. The company needed to evolve its data infrastructure such that connection strings would start using the smaller, faster database (Fast Facets) and stop using the larger, slower database (MDS).

THE SOLUTION

A microservices architecture built on MongoDB

When KW started on the path from real estate agency to technology titan, it had already been running its own self-managed instance of MongoDB. “Part of the reason I was excited to join Keller Williams was my years of experience using MongoDB with a startup in Austin [TX, USA], and I wanted to get back to it,” said McClarty.

The team decided rather than continuing to manage its own MongoDB instance, it wanted to focus time, attention, and resources on adding value across the organization.

So KW mapped out a new data strategy on MongoDB Atlas. It launched its Consumer application on MongoDB Atlas in late 2018/early 2019 and migrated Command from an alternative solution to MongoDB Atlas soon thereafter. “We executed a replatform to MongoDB Atlas,” explained McClarty. “We are now running four microservices on Atlas, and everything is working so much better now.”

“We made the decision to give our developers the best way to work with data — freeing developers to develop — by outsourcing the management of the data platform,” said McClarty. “Our strategy was to democratize access to the data infrastructure while freeing ourselves from the burden of data management by letting the experts at MongoDB manage the data clusters and machines for us.”

KW uses Atlas App Services Triggers to maintain active listings in a MongoDB Collection. “When a record comes in, we’re triggered on the listing and it flows into the smaller database (Fast Facets),” explained McClarty. An array of key value pairs for all facets enables quick lookups.

Using the Trigger, the team sets up a MongoDB Materialized View to proactively shape its database as it wants instead of simply reacting by using MDS data as it comes in. This enables the team to filter the data down quickly and easily, subscribing to the services it wants and aggregating the data. “With MongoDB Materialized Views, I am no longer beholden to the decisions of developers who went before me. If I need data in a different shape, I can get it done quickly and easily.”

McClarty said Triggers replace the functionality that would normally be covered in a cron job and a lot more because they are also event-based, and because you can have Triggers processing functions and even writing out data as a result of an aggregation you’ve already worked with. “Aggregation gets the data into just the shape your application needs. Now you can build out your data into a view that’s materialized, and have indexes built off the data. That has been tremendous.”

“I don’t know how we would have done this without Triggers,” he added. “We get listings from all connected sites and systems every 15 minutes and now, instead of experiencing data loss, we are equipping agents and consumers with the most complete and accurate information about properties in real time.”

McClarty characterized MongoDB Charts, the native visualization tool for MongoDB, as another standout feature. “We use MongoDB Charts a lot,” he said. “MongoDB Charts is helping us manage everything from core functions — for example, whether we’re getting data ingestion consistently — to more user-facing considerations, such as how many consumers currently using the system are associated with a given agent.”

“MongoDB Charts helps me give our product managers the information they are always asking for, and best of all, I can create a chart once and they can self-serve going forward,” said McClarty. “Thanks to Charts I don’t have to run queries for my product managers to report back to upper management. I can configure the charts once and get on with my other work.”

“MongoDB Charts helps our teams visualize what is happening in the data,” he added. “So, for example, when the pandemic hit, we could assess: What steps did you take and what were the results? Did it increase our reach or surpass where we were before?”

The Aggregation Pipeline Builder in the native GUI for MongoDB, MongoDB Compass, enables McClarty and his team to quickly build aggregation pipelines to process data. Documents in a MongoDB Collection or View pass through stages where Compass processes them into a set of aggregated results, and the team can change the stages and results to suit its needs. “When I was first using MongoDB years ago I wrote a lot of FIND queries. Then as I got more advanced, I started to use the Aggregation Pipeline Builder in MongoDB Compass. Now I point our people directly to the Aggregation Pipeline Builder and don’t even bother with FIND queries,” said McClarty. “The Aggregation Pipeline Builder is a game-changer.”

Other capabilities built into Atlas and across the platform are covering a range of technological bases for KW so the team can stay focused on adding value. In McClarty’s words, “The world's best chefs may know how to build a fire, but they don’t usually do it when they are preparing a 3-Star meal.”

McClarty said, “The autoscaling, alerts, and backups are all things I might be able to put on an Ops hat and do, but having someone else do that means I can focus on the needs of the customers.” He added that the geolocation and integrated search capabilities built into Atlas address various needs, including the ability to share with someone how many open houses are happening right now, or how many new users there are in a given state.

The robust features and flexibility of Atlas App Services are pointing to new ways to use the product, said McClarty. “We recently used Atlas Functions to avoid a full rollback waiting for a new rule to be ready. We simply put in the business logic to make it happen — it took literally two minutes to prepare — and our live data kept performing flawlessly straight through.”

Other features that are saving time and enhancing workflows include MongoDB’s alerting when an index might help a certain type of query, and the GUI for user permissions control.

In KW’s view, the benefits transcend products and features.

“The documentation with MongoDB is great; I can always find what I need. For many other products I will typically go to other resources first, but for MongoDB I come to your documentation first.”

KW also finds high value in MongoDB educational offerings. “I definitely recommend MongoDB University,” McClarty explained. “We have all of our developers take MongoDB Basics 001 and more advanced courses. Even our QA team and other colleagues including our Scrum Master have taken MongoDB Basics 001.”

“For us, it all adds up to the fact that MongoDB is a true modern, multi-cloud data platform (Atlas),” said McClarty.

THE RESULTS

Incredible transformation with MongoDB resulting in market-leading growth

KW has worked with MongoDB to achieve a remarkable transformation from real estate company to technology innovator. Empowering agents, buyers, and sellers with the right mix of real-time data that is easy to navigate helps all parties be agile, better-informed, and able to move faster, creating millions of happy homeowners each year.

This transformation has also helped fuel substantial growth outpacing the real estate market as a whole. In 1Q2021 KW transactions were up 21.3% YoY versus the industry average of 13.9%, and sales volume increased by 39.7% as compared with the industry average of 27.6%. KW contracts written volume was up 35.1% YoY. That growth is likely to accelerate as KW expands its worldwide presence beyond North America. Consumers and agents will soon be able to search for properties in other world regions on KW.com, supported by the underlying modern database platform (Atlas) with integrated Atlas Search built on top of Apache Lucene.

One of the more notable potential future developments KW is currently considering is using Atlas App Services to serve up that new worldwide listing service, which would reduce complexity by eliminating an entire microservice. This is made possible because Atlas App Services offers the ability to manage GraphQL on the MongoDB side. Atlas App Services may also provide the backend in KW’s MEAN Stack as a replacement for Express.js.

The challenge in making these moves is the cost/benefit analysis with regard to translating business rules that must occur to make it all work. The plus side, per McClarty: “Adding some of these things could have us living happily in MongoWorld someday without having to deal with much of anything else.”

As mentioned earlier in this story, part of the attraction for McClarty to join Keller Williams was his years of experience using MongoDB at another company. When we spoke with him for this story, McClarty extended a similar thought to potential job-seekers: “If anyone wants to learn more about MongoDB and other cool technologies, Keller Williams is a great place to do that.”