When the term NoSQL appeared in the technology mainstream, it was specifically about eschewing SQL. In its place would be lighter, more agile databases that didn’t bring with them the old constraints of crusty relational databases. Drop SQL and you can talk directly to your data storage and query engine. That was the theory at least.
The arrival of NoSQL
What we saw was an explosion of NoSQL databases. Each of which specialized in a particular way of storing data and falling into a range of NoSQL categories. All of these databases came with their own, usually unique APIs, and specialized query languages. Like how they stored data, the APIs and languages addressed the needs of the particular problems they optimized for.
MongoDB rose to the challenge with its JSON document-centric approach. Wide-column databases like Cassandra and ScyllaDB broke up the document and spread it out over multiple stores. Redis’s in-memory data structures offered rapid key/value lookups. Graph databases like Neo4J emphasized the connected structure of data in their APIs and query languages.
Time series databases like InfluxDB focused on managing data that was linked, stored, and queried by time. Search-driven databases like Elasticsearch created their own market, providing fuzzy searching on hard data. All of these databases took different approaches to storing and querying data. This means that they also offer different capabilities for their users, from simpler data access for applications, to global distribution of data or affording high-performance for particular roles.
So what is it really?
So the answer to the question “What is NoSQL” is really, that it’s a different approach to storing and querying data. Most NoSQL databases are relatively focused on a particular use. It shouldn’t be a surprise that MongoDB is one of the most successful NoSQL databases and that it is the one that set out from the start to be a general purpose database.
And yet SQL is still here. Arguably it’s more popular than ever with six of the top 10 databases in the DB-Engines rankings being SQL. That’s partly through the momentum of the old big players. But also partly because databases like PostgreSQL have come along with their own solutions to storing and searching JSON documents. SQL standards have developed too, to encompass new challenges.
Much like the CISC/RISC processor debates of the nineties, we have also seen SQL databases like CockroachDB emerge, with a key/value NoSQL foundation and an SQL query engine on top.
The future of data is SQL and NoSQL
In our annual surveys of Studio 3T users, we found that the average user was exposed to three different databases with an even mix of SQL and NoSQL technology throughout. And that average was rising. This is the future where general purpose databases do the operational work to support the enterprise, and NoSQL databases fill specialist niches where their tactical advantages stand out. The software developer has become a polyglot developer for the languages they work in. That same polyglot ability serves them well when working with databases of all kinds. Different SQL databases have their own SQL accents.
SQL isn’t going away but neither is the NoSQL design philosophy or the urge to try out new ways of storing and interrogating data. That’s one of the reasons why Studio 3T fills an ever increasing need. It provides tooling to help SQL and NoSQL work optimally together. It lets the user run SQL queries directly against your MongoDB database, makes migrating SQL databases across to MongoDB Atlas a breeze, and saves days of error prone work by translating SQL scripts into driver code in multiple languages.
People like to say data is the new oil. Shipping, refining and delivering data in the 21st century is going to require all the complexity and specialization that the oil industry has taken over a hundred years to grow.
Data is the energy that drives industry forward in the 21st century. And, as with energy, the challenge is not generating it, but capturing, transporting, filtering, managing and delivering it. It’s the same task energy providers have constantly grappled with, developing new infrastructure of both complexity and specialization over hundreds of years. Studio 3T is there to plug you into that new grid.