Hadoop World: Hadoop Ecosystem is Exploding, Says Cloudera’s Olson in Keynote

READ MORE

Mike Olson kicked off Hadoop World 2011 with a history lesson. Just two or three years ago, the majority of Hadoop contributions were made to the core distribution. Today. Nearly 70% of contributions are made to Hadoop sub-projects and other complimentary technologies.

(Continuing live coverage of Hadoop World 2011 via #TheCUBE at siliconangle.tv)

In short, the Hadoop ecosystem is large and growing, said Olson, Cloudera CEO, and in the next 12 months there will be an “explosion” of business-focused application built on top of the open source Big Data framework.

“Big Data is Big Deal. It’s not just a Web thing,” Olson said. “It’s not enough to have a platform (Hadoop) that sophisticated Java developers know how to operate. Business users, others need to have access to this.”

Olson then highlighted several examples of vendors building applications and platforms on top of Hadoop:

- WibiData has built an application that leverages Hadoop to analyze mobile phone data. It’s a “phone doctor” of sorts, allowing mobile providers to “help customers debug their phones.”

- Cetas is developing a self-service business intelligence platform that sits on top of Hadoop. It gives business users, not power Hadoop users, access to real-time, interactive analytics. It eliminates the complexity of working with Hadoop so business analysts need not be programmers to work with Hadoop data, said Olson.

- Revolutions Analytics has optimized R, the open source advanced analytics language, to run on Hadoop. Wall Street analysts can now work with a tool they know well, R, on an infrastructure that is totally novel, said Olson.

- Digital Reasoning’s Synthesis platform is used by the US Army and numerous government intelligence organizations to collect financial intelligence, risk data, patent data and other publicly available data to run complex analytics on unstructured and semi-structured data and deliver results in a “beautiful user interface.”

- Tidemark has built an enterprise performance management platform on top of Hadoop that allows executives to conduct planning, budgeting and forecasting at scale on top of Hadoop. All this, without executives needing to understand the underlying infrastructure.

Vendors aren’t the only one’s taking notice. During the keynote, Ping Li from Accel Partners joined Olson on stage to announce the $100 million Big Data Fund. The fund is meant to finance Big Data start-ups that are developing the next generation of applications on top of Hadoop.

“Hadoop is happening,” Olson said as he wrapped up his keynote at Hadoop World. And he urged attendees to “build applications that solve real problems.”

Said Li: “Big Data is going mainstream,” Ping said.

ServicesAngle

There will no doubt be more Hadoop-based solutions delivered via the cloud as a service making their debut in the coming months and years. This is yet another opportunity to bring Big Data to “traditional” enterprises, especially those that lack the internal expertise to leverage Hadoop on their own.

About Jeffrey Kelly

Jeffrey F. Kelly is a Principal Research Contributor at The Wikibon Project, an open source research and advisory firm based in Boston. His research focus is the business impact of Big Data and the emerging Data Economy. Mr. Kelly's research has been quoted and referenced by the Wall Street Journal, the Financial Times, Forbes, CIO.com, IDG News, TechTarget and more. Reach him by email at jeff.kelly@wikibon.org or Twitter at @jeffreyfkelly.