SHOPLINE powers 350,000 merchants across Asia on MongoDB Atlas and AWS

INTRODUCTION

Helping merchants sell to over 680 million customers

SHOPLINE is Asia’s biggest smart commerce platform powered by a complex data management ecosystem. Founded in 2013, SHOPLINE has helped over 350,000 merchants reach more than 680 million customers via ecommerce, social commerce and offline point of sale (POS) transactions.

With key features including inventory and sales management tools, regional logistics, payment gateways, data analytics, storefront theme designs and local customer support, merchants have everything they need to build a successful online store.

In addition to different commerce needs, SHOPLINE also provides a full range of services such as Advertising Agency Service, Smart Ads System, SHOPLINE Payments, SHOPLINE Logistics and more to give businesses everything they need to succeed in retail, and support businesses to succeed everywhere.

SHOPLINE employs over 1500 people across regions including Hong Kong, Taipei, Ho Chi Minh City, Shenzhen, Guangzhou, Kuala Lumpur, Jakarta, Singapore and Bangkok.

THE CHALLENGE

From a three person team to hundreds of millions of customers

Scaling from a three-person operation, SHOPLINE’s first task was to build a platform that was dynamic, could host a wide variety of data and would adapt to changes as the company grew. Early on, they didn’t know exactly how the service would be used or how merchants' requirements would change.

The next challenge they knew they would face was scaling. For retailers there are certain days which are incredibly important, such as Black Friday, 11/11 and other key events. The SHOPLINE team had to be able to add extra capacity and ensure the retailer's customers had a smooth shopping experience even when millions of shoppers were hitting the servers at the same time.

To address those two key components from the beginning, the team used MongoDB and its flexible document data model to build the core service. Engineering Director at SHOPLINE, Ian Chan, has been with the business from its humble beginnings and has long been an advocate of MongoDB.

“Our biggest challenge when starting out in e-commerce was keeping up with merchants requiring the flexibility to change orders, add new products or product variations, which represented a lot of different data points, and so we immediately chose MongoDB because of the dynamism and scale it gave us,” said Ian.

As the company continued to grow, it also grew in complexity, size and geographic coverage. This led to even more complex compliance and security. At the same time Ian’s small dev ops team realised they couldn't meet all these demands without extra support.

Then, in 2019, SHOPLINE’s e-commerce success had paved the way to include social commerce and POS verticals into their offering which added even more complex data sources to manage and integrate.

MongoDB community, deployed on AWS, was scaling and supporting their business, but managing the infrastructure themselves meant Ian's limited devops team was at breaking point. Self-management also added extra friction when developers wanted to deploy new services or build new features.

According to Ian, above and beyond rising to the challenge of SHOPLINE’s mind boggling complex data infrastructure, the number one golden rule of SHOPLINE is “no downtime permitted”.

THE SOLUTION

Developer freedom and focus

To help address the company's scale, the first step they had taken was to shard their MongoDB deployment, which is a method for distributing data across multiple servers. This helps ensure seamless service, even with very large data sets and high throughput.

“Our ability to embrace sharding with MongoDB was a breakthrough for us as the system became increasingly complicated with more and more layers of data being added. We needed to feel confident our database could handle increased load to a nearly unlimited degree by providing increased read/write throughput, storage, capacity and high availability,” Ian said.

Sharding only solved one part of the puzzle: scale. To address the other piece, Ian's team wanted to retain the power and flexibility of MongoDB's database but run it as a managed service. Something that would provide out of the box monitoring, one click updates, seamless deployments and maintenance. The team also wanted even more granular control of their data.

So in 2019 SHOPLINE moved to MongoDB Atlas, the fully managed cloud database service. While making changes to the database layer of such a large and complex infrastructure is never easy, the team were able to make the migration with zero downtime.

This shift enabled them to retain a lean and dedicated team of five developers, while the business invested in its broader engineering workforce comprising around 500 people.

“We asked ourselves, do we want our engineers to be developing features or maintaining databases? So it made perfect sense to move to Atlas. It was also becoming hard to hire talent with the technical ability to work with our ever increasingly complex data infrastructure, and we have confidence in MongoDB’s team providing technical support working around the clock for us,” Ian explained.

MongoDB Atlas on AWS is SHOPLINE’s primary operational data. However, the team lives in a multi-cloud world. They also have a close partnership with Google and use Google's data analytics tool Big Query for data analytics.

THE RESULTS

Merchants selling anywhere, everywhere and always on

SHOPLINE now has more than 20 clusters spread across multiple shards on Atlas. The platform is delivering a number of key benefits for the company:

• Real time – operational monitoring of the entire ecosystem which allows developers to easily update indexes, respond to issues and kill slow queries.
• Best in class security – easy to use security and permissions which support Google login and enable developers to encrypt data as well as strict controls and auditing on who can access or view what data.
• Automatic scale – scaleable disk management that easily expands cluster size to accommodate usage and automatically archives cold data. Especially crucial around the large retail dates like 11/11.
• 24/7 support – MongoDB’s professional technical support team is on hand 24/7 to help with everything from index suggestions to performance analysis.
• Resilience – providing backups and point in time restoration down to the specific minute ensuring confidence there is never any down time.
• Launch fast – creating a new database for a developer to build or trial something new previously took more than half a day, now it's done in three minutes.

“Having the Atlas service saves us a lot of time, reduces cost to the business, provides optimum performance and gives me peace of mind the system is always on and performing. From the viewpoint of merchants, every microsecond of latency can cost them a lot of money losing customers and sales,” Ian said.

That agility and scale became even more crucial as COVID-19 descended, driving people into their homes and working remotely. SHOPLINE's services and infrastructure were perfectly placed to build the new era of social commerce and was able accommodate hundreds of thousands merchants selling live, via video and social. The fully featured platform continues to grow and expand into new regions and add new capabilities, all on a mission to empower merchants to succeed everywhere, including but not limited to omnichannel retailing, social commerce and cross-border sales.

“We credit MongoDB as one of our trusted partners behind the scenes for helping SHOPLINE rise up to become Asia’s largest online and offline smart commerce company, adhering to the highest standards of security and compliance as we continue to grow and take advantage of new opportunities,” Ian concluded.