Dominic Wellington

Director, Field Initiatives & Readiness, EMEA

> Some features mentioned below will be deprecated on Sep. 30, 2025. [Learn more](https://www.mongodb.com/products/updates/product-support-deprecation?tck=notice_product_deprecation_2024).
>
>
>Please note: This article discusses Stitch. Stitch is now MongoDB Realm. All the same features and functionality, now with a new name. Learn more [here](https://www.mongodb.com/realm). We will be updating this article in due course.
>
>

"This'll be simple," I thought. "How hard can it be?" I said to myself, unwisely.

*record scratch*

*freeze frame*

Yup, that's me. You're probably wondering how I ended up in this
situation.

Once upon a time, there was a small company, and that small company had an internal newsletter to let people know what was going on. Because the company was small and everyone was busy, the absolute simplest and most minimal approach was chosen, i.e. a Google Doc that anyone in the Marketing team could update when there was relevant news. This system worked well.

As the company grew, one Google Doc became many Google Docs, and an automated email was added that went out once a week to remind people to look at the docs. Now, things were not so simple. Maybe the docs got updated, and maybe they didn't, because it was not always clear who owned what. The people receiving the email just saw links to the docs, with no indication of whether there was anything new or good in there, and after a while, they stopped clicking through, or only did so occasionally. The person who had been sending the emails got a new job and asked for someone to take over the running of the newsletter.

This is where I come in. Yes, I failed to hide when the boss came asking for volunteers.

I took one look at the existing system, and knew it could not continue as it was — so of course, I also started looking for suckers er I mean volunteers. Unfortunately, I could not find anyone who wanted to take over whitewashing this particular fence, so I set about trying to figure out how hard it could be to roll my own automated fence-whitewashing system to run the newsletter back end.

Pretty quickly I had my minimum viable product, thanks to MongoDB Atlas and Stitch. And the best part? The whole thing fits into the free tier of both. You can get your own free-forever instance [here](https://www.mongodb.com/cloud/atlas/register), just by supplying your email address. And if you ask me nicely, I might even throw some free credits your way to try out some of the paid features too.

>
>
>If you haven't yet set up your free cluster on [MongoDB Atlas](http://bit.ly/mongodbatlas), now is a great time to do so. You have all the instructions in this [blog post](https://www.mongodb.com/developer/quickstart/free-atlas-cluster/).
>
>

<img src="https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/article/good-intentions-rewarded-usual-way/atf-sample.png" class="align-center" alt="Above The Fold screenshot" />

## Modelling Data: The Document

The first hurdle of this project was unlearning bad relational habits. In the relational database world, a newsletter like this would probably use several JOINs:

-   A table of issues
-   Containing references to a table of news items
-   Containing references to further tables of topics, authors

In the document-oriented world, we don't do it that way. Instead, I defined a simple document format:

``` javascript
{
   _id: 5e715b2099e27fa8539274ea,
   section: "events",
   itemTitle: "[Webinar] Building FHIR Applications with MongoDB, April 14th",
   itemText: "MongoDB and FHIR both natively support the JSON format, the standard e...",
   itemLink: "https://www.mongodb.com/webinar/building-fhir-applications-with-mongod...",
   tags: ["fhir", "healthcare", "webinar"],
   createdDate: 2020-03-17T23:01:20.038+00:00
   submitter: "marketing.genius@mongodb.com",
   updates: [],
   published: "true",
   publishedDate: 2020-03-30T07:10:06.955+00:00
   email: "true"
}
```

This structure should be fairly self-explanatory. Each news item has:

-   A title
-   Some descriptive text
-   A link to more information
-   One or more topic tags
-   Plus some utility fields to do things like tracking edits

Each item is part of a section and can be published simply to the web, or also to email. I don't want to spam readers with everything, so the email is curated; only items with `email: true` go to email, while everything else just shows up on the website but not in readers' inboxes.

One item to point out is the updates array, which is empty in this particular example. This field was a later addition to the format, as I realised when I built the edit functionality that it would be good to track who made edits and when. The flexibility of the document model meant that I could simply add that field without causing any cascading changes elsewhere in the code, or even to documents that had already been created in the database.

So much for the database end. Now we need something to read the documents and do something useful with them.

I went with [Stitch](https://www.mongodb.com/cloud/stitch), which together with the Atlas database is another part of the MongoDB Cloud platform. In keeping with the general direction of the project, Stitch makes my life super-easy by taking care of things like authentication, access rules, MongoDB queries, services, and functions. It's a lot more than just a convenient place to store files; using Stitch let me write the code in JavaScript, gave me somewhere easy to host the application logic, and connects to the MongoDB Atlas database with a single line of code:

``` javascript
client = stitch.Stitch.initializeDefaultAppClient(APP_ID);
```

`APP_ID` is, of course, my private application ID, which I'm not going
to include here! All of the code for the app can be found in my [personal Github repository](https://github.com/riotnrrd-eu/Above-the-Fold); almost all the functionality (and all of the code from the examples below) is in a [single Javascript file](https://github.com/riotnrrd-eu/Above-the-Fold/blob/master/atf.js).

## Reading Documents

The newsletter goes out in HTML email, and it has a companion website, so my Stitch app assembles DOM sections in Javascript to display the
newsletter. I won't go through the whole thing, but each step looks
something like this:

``` javascript
let itemTitleContainer = document.createElement("div");
itemTitleContainer.setAttribute("class", "news-item-title");
itemContainer.append(itemTitleContainer);

let itemTitle = document.createElement("p");
itemTitle.textContent = currentNewsItem.itemTitle;
itemTitleContainer.append(itemTitle);
```

This logic showcases the benefit of the document object model in MongoDB. `currentNewsItem` is an object in JavaScript which maps exactly to the document in MongoDB, and I can access the fields of the document simply by name, as in `currentNewsItem.itemTitle`. I don't have to create a whole separate object representation in my code and laboriously populate that with relational queries among many different tables of a database; I have the exact same object representation in the code as in the database.

In the same way, inputting a new item is simple because I can build up a JSON object from fields in a web form:

``` javascript
workingJSON[e.name] = e.value;
```

And then I can write that directly into the database:

``` javascript
submitJSON.createdDate = today;
if ( submitJSON.section == null ) { submitJSON.section = "news"; }
submitJSON.submitter = userEmail;
db.collection('atf').insertOne(submitJSON)
   .then(returnResponse => {
      console.log("Return Response: ", returnResponse);
      window.alert("Submission recorded, thank you!");
   })
.catch(errorFromInsert => {
   console.log("Error from insert: ", errorFromInsert);
   window.alert("Submission failed, sorry!");
});
```

There's a little bit more verbose feedback and error handling on this one than in some other parts of the code since people other than me use this part of the application!

## Aggregating An Issue

So much for inserting news items into the database. What about when someone wants to, y'know, read an issue of the newsletter? The first thing I need to do is to talk to the MongoDB Atlas database and figure out what is the most recent issue, where an issue is defined as the set of all the news items with the same published date. MongoDB has a feature called the [aggregation pipeline](https://docs.mongodb.com/manual/core/aggregation-pipeline/), which works a bit like piping data from one command to another in a UNIX shell. An aggregation pipeline has multiple stages, each one of which makes a transformation to the input data and passes it on to the next stage. It's a great way of doing more complex queries like grouping documents, manipulating arrays, reshaping documents into different models, and so on, while keeping each individual step easy to reason about and debug.

In my case, I used a very simple aggregation pipeline to retrieve the most recent publication dates in the database, with three stages. In the first stage, using [$group](https://docs.mongodb.com/manual/reference/operator/aggregation/group/#pipe._S_group), I get all the publication dates. In the second stage, I use [$match](https://docs.mongodb.com/manual/reference/operator/aggregation/match/) to remove any null dates, which correspond to items without a publication date — that is, unpublished items. Finally, I sort the dates, using — you guessed it — [$sort](https://docs.mongodb.com/manual/reference/operator/aggregation/sort/) to get the most recent ones.

``` javascript
let latestIssueDate = db.collection('atf').aggregate( [
   { $match : { _id: {$ne: null }}},
   { $group : { _id : "$publishedDate" } },
   { $sort: { _id: -1 }}
]).asArray().then(latestIssueDate => {
   thisIssueDate = latestIssueDate[0]._id;
   prevIssueDate = latestIssueDate[1]._id;
   ATFmakeIssueNav(thisIssueDate, prevIssueDate);
theIssue = { published: "true", publishedDate: thisIssueDate };
db.collection('atf').find(theIssue).asArray().then(dbItems => {
   orderSections(dbItems); })
   .catch(err => { console.error(err) });
}).catch(err => { console.error(err) });
```

As long as I have a list of all the publication dates, I can use the next most recent date for the navigation controls that let readers look at previous issues of the newsletter. The most important usage, though, is to retrieve the current issue, namely the list of all items with that most recent publication date. That's what the `find()` command does, and it takes as its argument a simple document:

``` javascript
{ published: "true", publishedDate: thisIssueDate }
```

In other words, I want all the documents which are published (not the drafts that are sitting in the queue waiting to be published), and where the published date is the most recent date that I found with the aggregation pipeline above.

That reference to `orderSections` is a utility function that makes sure that the sections of the newsletter come out in the right order. I can also catch any errors that occur, either in the aggregation pipeline or in the find operation itself.

## Putting It All Together

At this point publishing a newsletter is a question of selecting which items go into the issue and updating the published date for all those items:

``` javascript
const toPublish = { _id: { '$in': itemsToPublish } };
let today = new Date();
const update = { '$set': { publishedDate: today, published: "true" } };
const options = {};
db.collection('atf').updateMany(toPublish, update, options)
   .then(returnResponse => {console.log("Return Response: ", returnResponse);})
   .catch(errorFromUpdate => {console.log("Error from update: ", errorFromUpdate);});
```

The [updateMany() command](https://docs.mongodb.com/manual/reference/method/db.collection.updateMany/index.html) has three documents as its arguments.

-   The first, the filter, specifies which documents to update, which here means all the ones with an ID in the `itemsToPublish` array.
-   The second is the actual update we are going to make, which is to set the `publishedDate` to today's date and mark them as published.
-   The third, optional argument, is actually empty in my case because I don't need to specify any options.

## Moving The Mail

Now I could send emails myself from Stitch, but we already use an external specialist service that has a nice REST API. I used a Stitch Function to assemble the HTTP calls and talk to that external service. Stitch Functions are a super-easy way to run simple JavaScript functions in the Stitch serverless platform, making it easy to implement application logic, securely integrate with cloud services and microservices, and build APIs — exactly my use case!

I set up a simple HTTP service, which I can then access easily like this:

``` javascript
const http = context.services.get("mcPublish");
```

As is common, the REST API I want to use requires an API key. I generated the key on their website, but I don't want to leave that lying around. Luckily, Stitch also lets me define a secret, so I don't need that API key in plaintext:

``` javascript
let mcAPIkey = context.values.get("MCsecret");
```

And that (apart from 1200 more lines of special cases, admin functions, workarounds, and miscellanea) is that. But I wanted a bit more visibility on which topics were popular, who was using the service and so on. How to do that?

## Charting Made Super Easy

Fortunately, there's an obvious answer to my prayers in the shape of Charts, yet another part of the MongoDB Cloud platform, which let me very quickly build a visualisation of activity on the back-end.

<img src="https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/article/good-intentions-rewarded-usual-way/atf-charts.png" class="align-center" alt="MongoDB Charts" />

Here's how simple that is: I have my database, imaginatively named "newsletter", and the collection, named "atf" for Above the Fold, the name of the newsletter I inherited. I can see all of the fields from my document, so I can take the `_id` field for my X-axis, and then the `createdDate` for the Y-axis, binning by month, to create a real-time chart of the number of news items submitted each month.

<img src="https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/old_images/article/good-intentions-rewarded-usual-way/atf-chart-settings.png" class="align-center" alt="Settings for MongoDB Charts" />

It really is that easy to create visualizations in Charts, including much more complicated ones than this, using all MongoDB's rich data types. [Take a look at some of the more advanced options](https://www.mongodb.com/products/charts) and give it a go with your own data, or with the sample data in a [free instance of MongoDB Atlas](https://www.mongodb.com/developer/quickstart/free-atlas-cluster/).

It was a great learning experience to build this thing, and the whole exercise gave me a renewed appreciation for the power of MongoDB, the document model, and the extended MongoDB Cloud platform - both the Atlas database and the correlated services like Stitch and Charts. There's also room for expansion; one of the next features I want to build is search, using [MongoDB Atlas' Text Search](https://www.mongodb.com/atlas/search) feature.

## Over To You

As I mentioned at the beginning, one of the nice things about this project is that the whole thing fits in the free tier of MongoDB Atlas, Stitch, and Charts. You can [sign up](https://www.mongodb.com/cloud/atlas/signup) for your own free-forever instance and start building today, no credit card required, and no expiry date either. There's a helpful onboarding wizard that will walk you through loading some sample data and performing some basic tasks, and when you're ready to go further, the [MongoDB docs](https://docs.mongodb.com/) are top-notch, with plenty of worked examples. Once you get into it and want to learn more, the best place to turn is [MongoDB University](https://university.mongodb.com/), which gives you the opportunity to learn MongoDB at your own pace. You can also get certified on MongoDB, which will get you listed on our public list of [certified MongoDB professionals](https://university.mongodb.com/certified_professional_finder/).

How I ended up building a whole CMS for a newsletter — when it wasn't even my job

Dominic Wellington

Content by Dominic Wellington

Build a Newsletter Website With the MongoDB Data Platform