Time Series Collections in Studio 3T

MongoDB’s Time Series collections are a way of more efficiently indexing and storing the data which makes up a time series, without the overhead of each data point containing multiple repetitions of the same fields. 

There are three components to a time series record. Firstly, it includes the acquisition time for the data. Then identification for the source, called the metadata field. Finally, the measurements taken for the data point. 

A time series collection may appear to be a normal collection. Behind the scenes though, MongoDB is managing a system collection for the data which compresses each source’s measurements into one bucket document. When a bucket document is full, MongoDB creates a new bucket document. This reduces the storage requirement and enables queries to be more efficient. 

This does mean that you have to create the collection explicitly as a time series collection. 

Creating a Time Series collection in Studio 3T

When you add a new collection, the option to Create a Time Series Collection is available. Select this option and then configure the Time Series collection settings, Time Field, Meta Field, Granularity and Expire After Seconds.

Time Field: This will be the name of the field that contains the time information for a datapoint.

Meta Field: This is  the name of a field which will contain information about the source – this may be an embedded document where there are multiple fields identifying a source, but there will only be one Meta field containing that data. This field is optional, but note that you can only use the Meta Field field to identify data for updates and deletion.

Granularity: This field defaults to seconds. It hints to MongoDB how granular your time series is, allowing it to optimize around minute or hourly updates. 

Expire After Seconds: This is another optional field. When set, the automatic removal of data points that are older than the seconds value in this field takes place. 

You will need to enter a collection name, a Time Field name and a Meta Field name. Then you can click Create to create your Time Series collection.

Querying a Time Series collection in Studio 3T

You can use the Studio 3T Collections Tab, Aggregation Editor or IntelliShell to query your Time Series collections like every other MongoDB collection. There are some things to note though.

• Update and Delete operations are restricted by MongoDB Time Series collection limitations. Specifically:
  • Update and Delete queries can only match on the meta field values, so document deletion by reference to the _id field is not possible.
    • This means that the queries generated by default in the update and remove dialogs will not be valid without rewriting.
  • Updates may only change the metaField values.
  • You cannot use limit to restrict the number of documents to be updated. All updates must have multi: true set or use UpdateMany().
  • Upserts are not available.
• Time Series collections manifest as writable non-materialized views. As such, they are also subject to View Limitations.

Duplicate _ids

While normal MongoDB collections do not allow entries with duplicate _id values, Time Series collections do. This is due to the way the data is stored in the “compressed” format in another table. There, the original id is saved alongside the readings.

Because of this, there are limitations on Studio 3T as to how they can handle duplicated data. These limitations affect import, copy and migration tools which offer “insertion options” upon detecting duplicate _id. These tools will not be able to detect the duplicate _id values and will insert duplicates into the time series collection. 

Importing from JSON

It is currently not possible to import JSON data as a time series collection. The import process is not aware of the data being time series data. It will therefore import it as a normal collection. Importing into an already existing Time Series collection will generate an error. This applies currently to all importable data formats. One option is to import as a normal collection and then follow the MongoDB guide Migrate Data into a Time Series Collection which involves creating a transient collection, backing that up and restoring it into a time series collection.

Mongodump and Time Series Collections

You can dump Time Series collections by exporting using Mongodump BSON from Studio 3T’s Export tool. When you restore this data using Studio 3T’s import Mongodump BSON, the collection will import as a time series collection. When importing Time Series collections like this, set –-noIndexRestore to ensure no warnings about a lack of indexes.

You should only restore Time Series collections like this; if you are restoring other normal collections, you may wish to restore them separately so you can restore their indexes.