If you were to create a list of the buzz words that are filtering through the world of IT right now, “noSQL” would have to be at the top of your list when you are talking about the importance of information technology. CIOs everywhere have decided that they’ve had enough of traditional databases and the high vendor fees that come with them. Open source noSQL databases seem to appear to be the solution to all of their data processing needs. However, it turns out that this just isn’t the case…
CIO’s ACID Problem
When we get presented with a new technology, it can be all too easy to start to think that it is a solution that we can apply to every problem that we are currently facing. noSQL is one such technology. However, the key is to realize that not all data that the IT department has been asked to process is created the same.
The data that the IT department has traditionally been asked to process generally all looks similar. This is the data that we feed to the company’s mission critical systems. The acronym ACID has been created to describe this data:
Atomic: each transaction is executed completely and can be rolled back if something goes wrong.
Consistent: no transaction will be permitted to leave the database if for some reason it creates an inconsistency with the stored data.
Isolated: each transaction does not affect another transaction
Durable: before a transaction can be considered to be complete, it must first be recorded permanently in the database.
What CIOs Need To Know About BASE
In the new world of “big data” in which we find ourselves, clearly not all data is going to meet the ACID criteria. This is where the door of opportunity opens for noSQL databases.
When we start to consider web and social media applications, we start to have to deal with data that is orders of magnitude larger than most standard corporate databases. This means that developers need to become more flexible when dealing with this much information.
The data properties of this kind of new data workload have been captured in the acronym BASE. This stands for:
Basically Available: just what it sounds like – the database is no longer required to be consistently real-time atomic.
Soft-State: Database states are now permitted to change and expire instead of always having to have to be durable.
Eventual Consistency: This flexibility is in contrast to a traditional ACID database’s requirement to provide stringent transactional consistency.
Which Database Is The Right One To Use?
I’m sorry about this; however, although it’s important to understand what kind of data set you are dealing with (ACID or BASE) , that’s not going to be enough to tell you which type of SQL / noSQL database you need to use for your next IT project. The CIO job is just a bit more complicated than that.
noSQl will be the right database to use when you have BASE workloads that are clearly not ACID, when you have a great deal of data, and when you want to be able to run your database using commodity hardware and software.
There is one other point for you to consider when you are trying to decide between an SQL or noSQL solution. If you try to use an SQL database for an application that must deal with high-volume workloads that are delivered via the web, then you’re going to see your database collapse because of the overhead. Instead, in these situations go with a noSQL solution.
What All Of This Means For You
It’s never been easy to be in the CIO position and lately it seems as though even those things that we thought that we had under control, like databases, are undergoing significant changes. One of these changes is the arrival of no SQL databases – when should we use them?
It turns out that not all data sets are created equally. Data that can be classified as being ACID are well suited to being processed by a standard database. However, data that can be classified as being BASE would be better handled by a noSQL database. Additional issues such as the quantity of data needs to be taken into consideration also.
What this means for you as a CIO is that what might have once been a fairly standard decision (“throw it into the database”), has now become yet another issue that you need to take a careful look at before making up your mind. Take the time to learn how to do this correctly and you’ll find that you are making the right decisions for your company.
– Dr. Jim Anderson
Blue Elephant Consulting –
Your Source For Real World IT Department Leadership Skills™
Question For You: Do you think that you should create separate DB teams to handle noSQL or should this be part of your existing DB team?
Click here to get automatic updates when The Accidental Successful CIO Blog is updated.
P.S.: Free subscriptions to The Accidental Successful CIO Newsletter are now available. Learn what you need to know to do the job. Subscribe now: Click Here!
What We’ll Be Talking About Next Time
Us CIOs are a very optimistic lot. If you take a survey of CIOs more often than not what you’re going to find out is that we truly do believe in the importance of information technology and we believe that each and every IT project that the IT department is scheduled to work on this year is going to result in money being saved for the company. How very interesting considering how much money past IT projects have cost the company…