Until recently, large scale data processing, analysis and computational statistics meant working with companies like SAS, Informatica, Teradata, SPSS, and Oracle. Today, those companies face an “innovator’s dilemma.” Do they keep their prices high to maintain revenue and risk losing customers to new Big Data and Analytics vendors, or embrace the new, lower-cost computing options and reduce their large legacy revenue stream? Tomorrow may be too late for them to establish themselves in this evolving market, or protect their customer base.
The Innovator’s Dilemma: When New Technologies Cause Great Firms to Fail, published by Harvard Business School professor Clayton Christensen in 1997, outlines the dilemma now faced by SAS and the others. Responses to this dilemma can be seen as Microsoft moves from their highly profitable shrink-wrapped Office desktop software to push their cloud version to compete with Google Docs and other online options. Siebel didn’t move to the cloud when Salesforce.com entered their market, and they wound up selling to Oracle in 2005 at a substantial discount.
Mentioned in an earlier post, Oracle has missed their revenue projections in three of the last eight quarters. Teradata’s stock is currently 31 percent below their 52 week high. New Open Source technologies dramatically lower the cost of Big Data statistical analysis.
SAS and the rest of the group’s innovator’s dilemma is being forced by the success of open source technologies like R, Hadoop, and NoSQL. R is an open source statistical programming language now used by over 2 million analysts. With open source projects like RHadoop, it can be scaled to run on clusters of computers using Hadoop, the open source standard for processing large data sets across clusters of computers. This combination offers an order of magnitude more performance at an order of magnitude lower price.
The ability for companies to run big clusters that analyze data more affordably than ever before is already here. The issue is not technical – from a business standpoint, if SAS, SPSS, Oracle, and the others price their products to win Big Data contracts, their established customers could save a lot of money by porting to the new architecture, and the companies lose a lot of revenue.
As happens in technology on a regular basis, new startups like us, Revolution Analytics, and others offer better performance at lower prices. We don’t do everything SAS and Oracle and the others do, but what we do, we do well. The ability to harness massive resources at disruptively low cost is enabling a revolution whereby the enterprise can tap all its data and move from intuition to data-driven decisions.