The IT department over at the Nasdaq stock exchange had a rough year in 2013. Their exchange, which is used by thousands of traders every day, suffered two debilitating outages. I can understand one outage, things happen. But two? Where was the CIO?
Two Outages Too Many
The Nasdaq is a stock market where traders can buy and sell stocks that are traded there. One of the ways that the Nasdaq gets the information that it needs is to exchange information with other stock exchanges. However, in order for those exchanges to occur, the Nasdaq has to send to those other exchanges timely stock pricing and trading volume information. Clearly, the Nasdaq understands the importance of information technology.
On January 3, 2013, a data feed that the Nasdaq uses to send price quotes to other exchanges experienced a brief outage. When this happened, those other exchanges halted trading of Nasdaq listed stocks. Needless to say, if you are one of the companies that decided to list your company on the Nasdaq instead of the NYSE, this is a big deal.
Bradley Peterson got the Nasdaq CIO job by being appointed the Nasdaq’s global chief information officer in February of 2013. You would think that this outage would have been one of the main items on the incoming CIO’s to do list – what caused it and how could future events be prevented?
However, clearly nothing was done. On August 22, 2013, the Nasdaq experienced yet another outage that was very similar to the one that they had experienced in January. The effect of this outage was much more severe. Trading in Nasdaq listed stocks was halted for over three hours – this included shares in Apple and Google.
Lots Of Talk And No Action
So what happened here? Wasn’t one major outage enough to get the Nasdaq IT department to wake up and realize that they had a problem on their hands? Apparently not.
After the first outage, the committee that oversees the feed that failed met. This committee consists of industry representatives that are affected by the feed. No changes can be made to the feed or its performance without a unanimous vote from the committee.
In the end, the committee did not take any action based on the issues that they discussed. In fact, they let the issue drop and they spent their time discussing other issues. That was until August rolled around and they experienced a much more severe outage. Now the U.S. Securities and Exchange Commission is getting involved and things have become much more complicate.
What didn’t happen here is Bradly Peterson didn’t make fixing this problem a top priority of the Nasdaq’s CIO. Yes, dealing with the committee that controls the data feed would be a big issue – you might need their permission to make any changes to the behavior of the data feed. However, fixing the infrastructure that supports the data feed is the CIOs responsibility and that could have been done. CIOs can’t sit around and wait to be told what to do. We need to take action to prevent bad things from happening.
What All Of This Means For You
As CIOs, we are in charge of our company’s IT systems. Unfortunately, this means that we are also responsible for any outages that the IT systems experience. The IT functions at the Nasdaq stock exchange are a critical part of how the company operates and in 2013 they experienced two major outages.
The first outage occurred in January before the current CIO came on board. However, the second, and more damaging, outage occurred in August. What went wrong? It appears as though there was a great deal of discussion about the root causes of the outage; however, in the end nobody did anything about it.
CIOs own the proper performance of their company’s IT systems. In the case of the Nasdaq, the person in the CIO position had responsibility for the second outage. The most troubling factor here is that he already knew that he had a problem because of the first outage. Don’t get yourself in the same spot – prevent outages from happening and if they do, take the time to both uncover and fix their root causes!
– Dr. Jim Anderson
Blue Elephant Consulting –
Your Source For Real World IT Department Leadership Skills™
Question For You: What do you think the Nasdaq CIO should have done after joining the company shortly after a major outage?
Click here to get automatic updates when The Accidental Successful CIO Blog is updated.
P.S.: Free subscriptions to The Accidental Successful CIO Newsletter are now available. Learn what you need to know to do the job. Subscribe now: Click Here!
What We’ll Be Talking About Next Time
In a perfect world, IT systems would always work and they would always work correctly. However, we don’t live in that perfect world. What this means for CIOs is that glitches happen. It’s our job to minimize them, but we’ll never be able to eliminate them completely. We’re always going to have to come up with a plan for dealing with them and their aftermath when they do occur. U.S. stock exchanges, who know about the importance of information technology, have had their fair share of glitches in 2013. I wonder how their CIOs are planning dealing with the aftermath of that?