Skip to content
Studio 3T - The professional GUI, IDE and client for MongoDB
  • Tools
    • Aggregation Editor
    • IntelliShell
    • Visual Query Builder
    • Export Wizard
    • Import Wizard
    • Query Code
    • SQL Query
    • Connect
    • Schema Explorer
    • Compare
    • SQL ⇔ MongoDB Migration
    • Data Masking
    • Task Scheduler
    • Reschema
    • More Tools and Features
  • Solutions
  • Resources
    • Knowledge Base
    • MongoDB Tutorials & Courses
    • Tool/Feature Documentation
    • Blog
    • Community
    • Testimonials
    • Whitepapers
    • Reports
  • Contact us
    • Contact
    • Sales Support
    • Feedback and Support
    • Careers
    • About Us
  • Store
    • Buy Now
    • Preferred Resellers
    • Team Pricing
  • Download
  • My 3T
search

Studio 3T Knowledge Base

  • Documentation
  • Tutorials
  • Workshops
Take the fastest route to learning MongoDB. Cover the basics in two hours with MongoDB 101, no registration required.
Start the free course

How to Run MongoDB Map-Reduce Jobs

Posted on: 13/07/2018 (last updated: 04/08/2021) by Thomas Zahn

In this post, we’ll show you the simplest way of writing, debugging, and running MongoDB map-reduce jobs using Studio 3T’s Map-Reduce screen.

Don’t have Studio 3T on your machine? Download it here, available for Windows, Mac, and Linux.

MongoDB Map-Reduce vs Aggregation Pipeline

MongoDB’s Map-Reduce is the flexible cousin of the Aggregation Pipeline.

In general, it works by taking the data through two stages:

  • a map stage that processes each document and emits one or more objects for each input document
  • a reduce stage that combines emitted objects from the output of the map operation

The main advantage over the Aggregation Pipeline is that Map-Reduce may use arbitrary JavaScript for each stage enabling otherwise impossible operations though at the expense of lower performance (potentially higher execution times). You can read more about it in MongoDB’s reference documentation.

MongoDB recommends the Aggregation Pipeline for most aggregation options. As an alternative to Map-Reduce, please also check out Aggregation Editor, Studio 3T’s MongoDB aggregation query builder.

{
     "_id" : 592341,
     "tags" : [
         "cats",
         "kittens",
         "travel"
     ]
 }

A map-reduce example

In this example, our objective is to group images by tag except for those which include the “work” tag.

To achieve this, we will need to write a Map-Reduce job that will:

  1. Exclude all images which include the “work” tag.
  2. Have the map() function emit the image id for each of the tags as key.
  3. Have the reduce() function combine the image ids for each tag.

Let us start by opening Studio 3T’s new Map-Reduce screen by selecting the Open Map-Reduce option from the context menu:

Open Map-Reduce in MongoChef
Opening the Map-Reduce in Studio 3T

Filtering the input data

Clicking on the “Input data” tab and then the “Preview Input” toolbar button shows us a preview of the collection data. It is here that we can shape the data fed into the Map-Reduce job and omit any image tagged “work”. This is achieved by the following query

{ "tags": { $ne: "work" } }

We can inspect the data that will be fed into the map function by clicking the “Preview Output” toolbar button.

Input data sample
Input data sample

map() function

For the second step, we move to the “map()” tab.

In this tab we want to specify the function responsible for emitting one or more key-pairs for each document. The following function gets the job done:

function () {
     for (var index in this.tags) {
         emit ( this.tags[index] , this._id );
     }
 }

We can sample the map() function’s output by clicking the preview button, verifying that this function was successful. The preview feature is extremely useful, in particular before submitting jobs that could take hours to run. The “map() sample output” tab gives us a detailed breakdown of how our map() function operates, showing the emitted key/value pairs as well as their original document _id.

Map output preview
Map output preview

reduce() function

Studio 3T’s default implementation of the reduce() function takes care of the rest:

function (key, values) {    
     var reducedValue = "" + values;
     return reducedValue;
 }

Again, the Preview Output toolbar button will let us verify that our function is successful. If we were writing a more complex reduce() function or trying to debug what was being fed in, we could sample the input by clicking on the Preview Input button. This gives us a few of the key-value pairs that are emitted and then reduced.

Reduce sample input
Reduce sample input

finalize() function

MongoDB allows for a final stage to a Map-Reduce job for doing some final processing with use of a finalize() function. Let’s use this just so the output is easier to read:

function (key, reducedValue) {
     var finalValue = "tag '" + key + "' was found in images: " + reducedValue;
     return finalValue;
 }

A quick inspection of finalize()’s sample output and we are ready to submit a job that will process all of the data.

Finalize sample output
Finalize sample output

Running the map-reduce job

Now that we have set all the parameters of the job, and are sure that all our functions run as intended, we can submit the Map-Reduce job to run through the whole collection dataset by clicking the “Execute” button on the toolbar.

This action will open a new tab which will contain the results of the job when it is finished:

Finished job
Finished Job!

Clicking on Show details will bring up a dialog showing execution statistics as well as a configuration summary for this job.

Job statistics
Finished Map-Reduce job statistics

Epilogue

Now that the Map-Reduce job is finished, we can save all this work as a script. The format is 100% JavaScript code, which allows the saved file to be run in IntelliShell or even the basic mongo shell and will produce identical results.

// *** 3T Software Labs, MongoChef: MapReduce Job ****
 // Variable for db
 var __3t_mongochef_db = "exam";
 // Variable for map
 var __3t_mongochef_map = function () {
     for (var index in this.tags) {
             emit ( this.tags[index] , this._id );
     }
 }
 ;
 // Variable for reduce
 var __3t_mongochef_reduce = function (key, values) {    
     var reducedValue = "" + values;
     return reducedValue;
 };
 // Variable for finalize
 var __3t_mongochef_finalize = function (key, reducedValue) {
     var finalValue = "tag '" + key + "' was found in images: " + reducedValue;
     return finalValue;
 }
 ;
 db.runCommand({ 
     mapReduce: "images",
     map: __3t_mongochef_map,
     reduce: __3t_mongochef_reduce,
     finalize: __3t_mongochef_finalize,
     out: { "inline" : 1},
     query: { "tags": { $ne: "work" } },
     sort: { },
     inputDB: "exam",
  });

Do you have an existing script that you’ve been working with already? No problem, Studio 3T will load it into the Map-Reduce screen, just click on the “Open Map-Reduce File” toolbar button, select the file and there you have it!

Once you’re done running MongoDB map-reduce jobs, why not keep the momentum by learning how to build MongoDB aggregation queries, and discover our MongoDB shell integration, IntelliShell.


How helpful was this article?
This article was hideous
This article was bad
This article was ok
This article was good
This article was great
Thank you for your feedback!

About The Author

Thomas Zahn

Having grown up with a living room that was essentially the office of his mother’s software start-up in the 80s, Thomas is a dyed-in-the-wool software engineer. In the past, he has worked for large outfits such as Microsoft Research and Nokia as well as for specialised engineering shops and start-ups. He lives in Berlin with his wife and two kids, and loves tennis and hiking (though, bizarrely, he constantly seems to find no time to do much of either those two). Thomas holds a Ph.D. in Computer Science from the Freie Universität Berlin.

Article navigation

Related articles

  • Why do I see an error when I re-run an IntelliShell script? #Studio3T_AMA
  • Lesson 2, Exercise 1: Using IntelliShell to build and run find statements
  • Lesson 2, Exercise 2: Using Visual Query Builder to build and run find statements
  • Lesson 2, Exercise 3: Using Query Code and IntelliShell to modify and run a find statement
  • Lesson 6, Exercise 1: Using the SQL Query tool to run SQL statements

Studio 3T

MongoDB Enterprise Certified Technology PartnerSince 2014, 3T has been helping thousands of MongoDB developers and administrators with their everyday jobs by providing the finest MongoDB tools on the market. We guarantee the best compatibility with current and legacy releases of MongoDB, continue to deliver new features with every new software release, and provide high quality support.

Find us on FacebookFind us on TwitterFind us on YouTubeFind us on LinkedIn

Education

  • Free MongoDB Tutorials
  • Connect to MongoDB
  • Connect to MongoDB Atlas
  • Import Data to MongoDB
  • Export MongoDB Data
  • Build Aggregation Queries
  • Query MongoDB with SQL
  • Migrate from SQL to MongoDB

Resources

  • Feedback and Support
  • Sales Support
  • Knowledge Base
  • FAQ
  • Reports
  • White Papers
  • Testimonials
  • Discounts

Company

  • About Us
  • Blog
  • Careers
  • Legal
  • Press
  • Privacy Policy
  • EULA

© 2023 3T Software Labs Ltd. All rights reserved.

  • Privacy Policy
  • Cookie settings
  • Impressum

We value your privacy

With your consent, we and third-party providers use cookies and similar technologies on our website to analyse your use of our site for market research or advertising purposes ("analytics and marketing") and to provide you with additional functions (“functional”). This may result in the creation of pseudonymous usage profiles and the transfer of personal data to third countries, including the USA, which may have no adequate level of protection for the processing of personal data.

By clicking “Accept all”, you consent to the storage of cookies and the processing of personal data for these purposes, including any transfers to third countries. By clicking on “Decline all”, you do not give your consent and we will only store cookies that are necessary for our website. You can customize the cookies we store on your device or change your selection at any time - thus also revoking your consent with effect for the future - under “Manage Cookies”, or “Cookie Settings” at the bottom of the page. You can find further information in our Privacy Policy.
Accept all
Decline all
Manage cookies
✕

Privacy Preference Center

With your consent, we and third-party providers use cookies and similar technologies on our website to analyse your use of our site for market research or advertising purposes ("analytics and marketing") and to provide you with additional functions (“functional”). This may result in the creation of pseudonymous usage profiles and the transfer of personal data to third countries, including the USA, which may have no adequate level of protection for the processing of personal data. Please choose for which purposes you wish to give us your consent and store your preferences by clicking on “Accept selected”. You can find further information in our Privacy Policy.

Accept all cookies

Manage consent preferences

Essential cookies are strictly necessary to provide an online service such as our website or a service on our website which you have requested. The website or service will not work without them.

Performance cookies allow us to collect information such as number of visits and sources of traffic. This information is used in aggregate form to help us understand how our websites are being used, allowing us to improve both our website’s performance and your experience.

Google Analytics

Google Ads

Bing Ads

Facebook

LinkedIn

Quora

Hotjar

Functional cookies collect information about your preferences and choices and make using the website a lot easier and more relevant. Without these cookies, some of the site functionality may not work as intended.

HubSpot

Social media cookies are cookies used to share user behaviour information with a third-party social media platform. They may consequently effect how social media sites present you with information in the future.

Accept selected