Mongosh: How to Run More Processes

In the previous two parts of this series, we’ve looked at how to read and write files. But what’s the point of writing files if you can’t do anything with them after you’ve finished writing them? Well, you could always run another process to consume your file and do something with it. Until the arrival of Mongosh however, you would have had to do this manually. Now with Mongosh, we can leverage its access to the rich Node.js library to run a process.

The key to running processes is the child_process module of Node, specifically the execSync command.

child_process.execSync(command[, options])

If you look at the manual page, you’ll see there are many options for this command but that’s the nature of running processes; there are a lot of moving parts.

Getting practical with the mongo shell

Let’s look at a practical example, using what we created in the previous article, when we turned a MongoDB collection into a HTML page. In that case we wrote a HTML table of all the matching records to a file and did nothing to publish it to the world. Let’s fix that now.

Our “Publish” command

We don’t want to get into the weeds over what or how we are publishing to the web. For this example, publishing will consist of calling a shell script called publish.sh with a file name. And all that script will do is copy the file.

cp $1 /tmp/$1

That’s it: it’s enough to show the principles at work. All this does is copy the named file (from the first argument – $1) over to /tmp/ with the same name. Let’s save that as publish.sh in our home directory, where we write our page.html file.

Calling the Publish command

At the end of our script, we can now add this:

var a = child_process.execSync("sh publish.sh page.html", 
       { cwd:"/Users/dj" }, 
       (error, stdout, stderr) => {
           if (error) {
           print(`exec error: ${error}`);
           return; }
        }
);

The first parameter to our execSync call is the command:

sh publish.sh page.html

This calls the default shell (sh) and asks it to run publish.sh in the current directory. And we give that script the argument page.html. Nothing too out of the ordinary there. This is where you could drop in another executable or command to be run.

Where are we currently working?

The thing you should be asking is “How does the script know where all these HTML files are?” The answer is, it doesn’t. And that’s why we have one parameter being set in the options:

{ cwd:"/Users/dj" }, 

This sets the current working directory (cwd) to my home directory, where I previously wrote out the page.html file and where I wrote and saved the publish.sh script. With this set, when sh runs it will look in the current working directory for publish.sh (and find it) and when publish.sh runs it will look for page.html in the current working directory, and find it just as we wrote it.

The last part of the execSync parameters is a function which defines what happens when the command being run finishes running. This function is called after it exits.

(error, stdout, stderr) => {

It defines an anonymous function that will be handed the error (if any), the command’s standard output and the command’s standard error output. It’s up to the function what is done with these parameters.

     if (error) {
           print(`exec error: ${error}`);
           return; }
        });

And in this case, we only respond if an error has been reported, by printing out the error. In this case we don’t do anything else. And that’s the end of the call.

Running the script

When you are ready, click the Run button and the script will run, creating page.html in your home directory, then running a shell script to copy that file to /tmp. It’s as simple as that. This is, of course, just an example. What you do with this capability depends on your own workflows.

One important point, with Studio 3T, the ability to schedule scripts like these opens the way to even more powerful automation. For example, any project which needs to produce public reports, for compliance or transparency, would benefit from scheduling a fully automated web publication process to remove the possibility of the report not being produced. 

Here’s the whole example script in its entirety:

db = db.getSiblingDB("sample_training");
var cursor=db.getCollection("companies").find(
    {
        "$and" : [
            {
                "homepage_url" : {
                    "$exists" : true
                }
            },
            {
                "homepage_url" : {
                    "$ne" : null
                }
            },
            {
                "homepage_url" : {
                    "$ne" : ""
                }
            }
        ]
    },
    {
        "name" : 1.0,
        "homepage_url" : 1.0
    }
).sort(
    {
        "name" : 1.0
    }
);

process.chdir("/Users/dj/");
var out = fs.openSync('page.html', 'w')
fs.writeSync(out,"<html><body><table>\n")
while (cursor.hasNext()) {
    var r = cursor.next();
    fs.writeSync(out,`<tr><td><a href='${r.homepage_url}'>${r.name}</a></td></tr>\n`);
}
cursor.close();
fs.writeSync(out,"</table></body></html>\n")
fs.closeSync(out);

var a = child_process.execSync("sh publish.sh page.html",
      { cwd:"/Users/dj" },
      (error, stdout, stderr) => {
          if (error) {
          print(`exec error: ${error}`);
          return; }
        }
);

Final notes on our MongoDB shell example

First, we’ve only touched on what you can do with execSync. As I mentioned, there are many options available with it, beyond cwd, which let you configure the environment, path and behavior of the executable when run. 

Secondly, reading and writing files or running processes are just a few of the things that are possible to do using Mongosh, now that it has a full JavaScript/NodeJS engine underpinning it. For example, network connections to get data from APIs or to trigger web automation are possible, although admittedly currently a little fiddly in terms of the gymnastics needed to make those calls simply and reliably; short version, there’s no synchronous versions of the web/network calls. But, the state of the art in what underlies Mongosh has moved on, so look out in future for those improvements to appear in Mongosh too, making easy networking possible.

Finally, have fun with Mongosh, it’s the most powerful thing! In creating this series, our experiments have led to creating self publishing map websites which are automatically deployed to the web, complete with fresh data every day and emailed reports landing in inboxes with attached spreadsheets. What will you create? Let us know!