MongoDB aggregation: create your data pipelines


MongoDB is one of the leading NoSQL databases, and its aggregation framework enables powerful queries, as well as data operations. We will see how to quickly create data pipelines and create code you can use in your projects.

The power of a Compass

You use MongoDB as data repository because it’s so simple and powerful, with its flexible schema? I can relate.

You bang your head on the screen, because your background is in Python and the JavaScript-like syntax of MongoDB drives you nuts? I can relate VERY much.

Luckily, there is a piece of software that can ease the process: Compass. MongoDB made this fantastic software free, and it can really help you to get the most out of your NoSQL database.

The Aggregation tab is where I spend most of my time. You can test your data pipeline fr extraction and transformation, and finally get code for your favorite programming language to perform the same pipeline in your script/application/whatever.

mongodb aggregation data pipelines
Compass Aggregation tab. Sweet!

The piece of magic is the small icon near the the green “SAVE” button. Once you are done with the creation of your data pipeline, adding different stages and directly visualizing some results, you want to integrate this with the rest of your code. And yes, Python 3 is the option I select all the time!

Compass exporting your data pipeline to Python3 code. Sweeter 🙂

You can grab the code, optionally include also the Driver Syntax (that will also include your MongoDB connection string, with username and password). And you are good to go! You can paste that code in your Jupyter notebook or Python file, and move on with your project.

I hope you find this useful! I plan to write more MongoDB-related posts over time, you can check them here!

  • MongoDB Compass link
  • MongoDB related posts link

Do you like our content? Check more of our posts in our blog!