What Do CIOs Need To Know About Hadoop?

Hadoop is an open source database that has a lot of CIOs excited
Hadoop is an open source database that has a lot of CIOs excited
Image Credit:

If there is one IT buzzword that gets everyone excited right now, then it’s “Big Data”. The idea behind Big Data is that with the right set of tools, companies can finally take a look at all of the data that they collect from multiple streams and pull it together in order to answer important business questions. Finally, the importance of information technology can be realized in everyone’s company. In order to perform Big Data operations, you need the right type of database. Hadoop is an open source database for dealing with big data that CIOs are getting excited over. However, should they be?

What Can You Use Hadoop To Do?

Hopefully, you like everyone else with the CIO job, look at any new IT buzzword with a bit of disdain. What, something new that promises to solve all of my problems? Well, in this case Hadoop is probably something that you should at least be taking a closer look at. Every IT department is currently drowning in too much information that is flowing in through the front door. Hadoop is an open source database that offers the promise of allowing you to get on top of this flood of data and to be able to make sense out of it.

What makes Haddop so special, and different from traditional databases, is that it can spread it’s uncategorized data over a network of thousands upon thousands of cheap computers. The end result of this is that the processing of your company’s wealth of data just got a whole lot cheaper to do.

If you had a Hadoop solution installed in your IT shop, then you could process the data that you are receiving in real-time. You could use social media, cell phone information, and customer purchase histories together to create custom product offerings. Yes, this is powerful stuff; however, there are some bumps in the road as CIOs are finding out.

What Are The Problems That CIOs Are Having With Hadoop?

One of the biggest problems that CIOs are encountering with their Hadoop projects that that all too often these types of projects just flat out fail. The folks over at research firm Gartner are reporting that through 2017 they are predicting that 60% of all big data will fail to go beyond the piloting and experimentation phase and will eventually end up being abandoned.

Another issue that CIOs are discovering is that the world that Hadoop lives in is brand new. However, the majority of data that is currently within the company lives in older enterprise databases. Joining these two worlds is turning out to be much harder than anyone thought that it would be. Creating a single unified data store for the firm that handles both old and new data is a very large and complex project.

Finally, although Hadoop is a powerful database system, it turns out that it may not have the horsepower that is needed to drive enterprise wide applications. What CIOs are learning is that although Hadoop may be faster than some of the traditional databases that they are currently using, it may not be fast enough to support information that is coming in in real time or be able to support a very large number of simultaneous queries. This could end up having serious limitations on where and when a Hadoop solution could be used within an enterprise.

What Does All Of This Mean For You?

CEOs and those in the CIO position have become convinced that the future of IT involves Big Data. Everyone wants to be able to take a look at all of the data that the company is collecting and use it to answer critical business questions. However, in order to do this special databases, such as Hadoop, have to be used and this is causing problems.

CIOs need to create Hadoop databases in order to store large quantities of unstructured company data. Their goal is to then be able to search this data in real-time in order to find answers to critical questions. However, what they are discovering is that it can be very hard to get a Hadoop database to work with the company’s existing databases. Additionally, Hadoop is not set up to support real-time operations. These challenges are leading many IT big data project to never be able to move out of the lab.

Hadoop is a powerful new tool for CIOs to use in order to understand the data that they have. However, Hadoop is brand new – all of the bugs have not yet been worked out. This is a situation where CIOs need to be very careful to not over promise and under deliver on what their IT department can do with Hadoop. Use Hadoop; in a lab setting and start to experiment with it in order to find out what it can do for your company. It will be able to do a lot more for you tomorrow and you are going to want to be ready when that day arrives!

– Dr. Jim Anderson
Blue Elephant Consulting –
Your Source For Real World IT Department Leadership Skills™

Question For You: Do you think that only IT staff should be allowed to access your company’s copy of Hadoop?

Click here to get automatic updates when The Accidental Successful CIO Blog is updated.

P.S.: Free subscriptions to The Accidental Successful CIO Newsletter are now available. Learn what you need to know to do the job. Subscribe now: Click Here!

What We’ll Be Talking About Next Time

Sure, as the person with the CIO job, there are a lot of different things that you could be doing. However, have you ever spent any time thinking about the things that you should be doing? It turns out that given the importance of information technology it just might be what you and your IT team is not doing that may end up causing both you and your company the most trouble. In the world that we live in today, at any time your company may be sued. When this happens there just may be a request for electronic records – are you going to be ready to locate them?