LucidEra a casualty of poor design

The rumors could be more than rumors for the SaaS BI company that focused on analytics for Salesforce.com. LucidEra used a variety of open source BI software to address a market need. People wanted SaaS BI and were paying for it. So what was the problem?

If you haven't heard, the rumor is that LucidEra is no longer in business.

I want to say rumor until someone from the company verifies it. However I've heard from a couple sources now, which is good enough for me to publish this post. Plus their blog is down. Some Salesforce.com customers are looking for BI vendors again. And senior executives have moved onto other companies.

What went wrong?

Rumor #1: LucidEra had more servers than NetSuite!

One advantage that SaaS should bring to the table is ease of scalability. Without that you're just on-premise software in someone else's data center adding servers for new customers. SaaS architecture needs to have automated scalability.

Rumor #2: Not a multi-tenant architecture.

A few things could have taken them away from a pure multi-tenant architecture. The open source BI software, like Mondrian, may not have lended itself well to multi-tenancy. (Maybe someone technically savvy could verify or refute this?) Another issue can be the data model design. Replicating data models for each new customer isn't multi-tenancy. I'm not sure why or where LucidEra moved away from a pure SaaS model. But that's what I'm hearing.

Rumor #3: Customers loved them and their product.

If you want to please your customers, find out how LucidEra did their sales and customer support. They are an example of how a SaaS business needs excellent sales and customer support AND savvy engineers and architects.

At least this casualty wasn't because of the economy or so the rumors go.

10 Questions with Bruce Armstrong, CEO Kickfire

When you talk with a person who has worked for successful companies and spent time as a venture capitalist sitting on multiple boards, you tend to actively listen to the stories and glean what insight you can. When I spoke with Bruce Armstrong, CEO of Kickfire, it was such a conversation.

Bruce is looking to take Kickfire and disrupt the data warehouse market. Their product, as Bruce put it, helps bridge MySQL (which doesn't understand data warehousing) with data warehousing (which doesn't understand open source). He is now in the midst of a David and Goliath battle with Teradata -- his former employer.

Question: Hi Bruce. Care to describe this battle with Teradata?

Answer: Actually, Tom, we believe the battle is less with Teradata’s existing high-end data warehouse business and more for the vast, under-served data warehouse “mass” market out there. As such, we don’t plan on taking the battle directly to Teradata or anyone else, but rather create a new market by providing affordable, easy-to-deploy data warehouse and data mart appliances for the mass market. The mass market for data warehousing includes rapidly growing small & medium businesses as well as departments in larger organizations who simply can’t afford either their own Teradata machine or in a lot of cases even the charge-backs from central IT to time share on a Teradata machine. So, we hope to be complementary to Teradata, but realize there may be some skirmishes for budget dollars in the market today!

Question: You started as employee #16 at Teradata who boasts Walmart as a customer. Was that your start that brought you to running Kickfire today?

Answer: Yes, I was at Teradata for 15 years, Tom. I worked my way up from programmer to President after we went public and were acquired by NCR/AT&T. I spent almost 10 years in the field working with customers – including Walmart – so I developed a pretty good idea of what they are looking for at the high end. After Teradata, I was GM of the Server Group at Sybase where one of the products I launched was Sybase IQ, the first column-store database on the market. Later, I was EVP of Sales & Marketing at Broadbase – the second column-store database on the market and a pioneer of data marts. Since then, I have been involved in doing due diligence for a lot of the data warehouse startups, including Netezza, Greenplum, and Vertica.

Question: What did you learn from Teradata back then that you are applying to Kickfire today?

Answer: While I learned many things at Teradata that are applicable to Kickfire, the key concept for me was just how different the data warehousing high end is from the mass market. At the high end, once you’re able to break into the market with a new approach, it quickly becomes less about technology and more about services and support. Large customers just simply demand more hand holding. In the mass market, it’s just the opposite. Customers in the mass market expect the product to be highly packaged – easy to buy, install, and manage. They simply can’t afford lots of services and support, so the product needs to deliver high performance in a plug-and-play way.

Question: You have years of history in the data warehouse industry. Care to explain what makes you think the data warehouse industry needs changing?

Answer: While I believe every sector of the data warehouse market is looking for more performance, faster deployments, and lower cost, our goal at Kickfire is not necessarily to change the data warehouse industry but rather to enable more people in the market to take advantage of the benefits that come from data warehousing. Again, we’re going after an under-served market with rapidly growing small and medium sized businesses and departments in enterprises that simply can’t afford Teradata or Netezza. The change that’s required to enable the mass market is to get high-end performance out of a mass-market machine. Kickfire achieves this through our patented parallel-processing SQL chip. In a single chip, we are able to get the performance of dozens of general purpose CPUs. So, in the same way that NVIDIA has radically changed the dynamics of the graphics industry by encoding the graphics processing language in silicon, Kickfire has the same opportunity in the data warehouse industry.

Question: There have been recent acquisitions. Sun purchased MySQL. Then Oracle purchased Sun. Should the MySQL community be concerned?

Answer: We do not think so. We believe that with Solaris, Java, and MySQL, Oracle now has the kind of assets they will need to truly make Microsoft worry. With MySQL, Oracle will finally be able to undercut Microsoft as well as to dominate the high end in databases. As such, we think Oracle is going to continue to invest in MySQL. In fact, Oracle has already proven to be a good steward for InnoDB, the most popular MySQL storage engine they bought several years ago.

Question: About Kickfire, how does this chip technology bridge MySQL with data warehousing?

Answer: MySQL has become the de facto standard for online businesses by virtue of its open source business model and the fact that it has increasingly become production-ready for transaction processing applications. For data warehousing applications, though, MySQL remains very primitive. As such, customers struggle with database volumes as small as 50GB when it comes to reporting and analytics. The primary technical issue with MySQL regarding data warehousing is that MySQL does not have any query parallelization capabilities. This is where Kickfire comes in: using MySQL’s pluggable storage engine API, Kickfire takes over a query from MySQL and provides parallel-processing with the SQL chip allowing queries to run 10x – 1000x faster.

Question: Are you getting any response from the MySQL or data warehouse communities? Are they seeing potential benefit?

Answer: Yes! We have a large and rapidly growing pipeline of opportunities in the MySQL community. One of our early customers, Mamasource, has been using MySQL for years to drive their online community website. However, when it came to analyzing the clickstream data about their community in order to improve the user experience and increase ad revenue, MySQL hit the wall at less than 30GB of data. With Kickfire, Mamasource is now able to run their queries on average 20x faster and up to 600x faster for complex queries. More importantly, Mamasource can now scale their data warehouse to over 300GB, which allows them to increase from just one month of data to a full year.

Question: It's hard to ignore Microsoft. How does Microsoft with SQL Server fit in the Kickfire world view?

Answer: Microsoft and Oracle are the predominant databases in the mass market today. As MySQL continues to penetrate the market (at a rate of 70,000 downloads a day, by the way), Kickfire will be brought into more and more Microsoft and Oracle shops. As such, one could imagine us building more specific features to co-exist with those databases in addition to MySQL.

Question: You spent time as a Venture Capitalist. We won't hold that against you but what are your take-aways from being a VC?

Answer: My time as a venture capitalist was really more about board work than investing. As an operator, the VC firm I was with was interested in my ability to help identify interesting spaces, provide input on business plans and teams, and then work with portfolio companies from a board perspective in order to help generate value for shareholders. A lot of what I did was to dive deep into each function (sales, marketing, development, etc) of the portfolio companies and provide input to the boards (including the CEO’s) on where things could improve. Having served on 20+ boards, I got a pretty good idea of what works and what doesn’t work in start-ups and rapidly growing companies. The lessons I learned were really quite simple:

1. make sure you have a big market opportunity;
2. understand very clearly who your customer is and why they should buy from you;
3. build a very high quality product;
4. regularly assess your market position and look for ways to change the game in your favor;
5. don’t be afraid to tell the truth about the business – there’s always more than one way to create a market leader.

Question: It’s been a pleasure talking with you Bruce. Do you have any additional links or information about Kickfire you want to share?

Answer: Thank you – the pleasure’s all mine. Please come visit us at www.kickfire.com!

Don't live in the past, predict the future

If Jethro Tull is "living in the past".  And hindsight is 20/20.  Then you're reading this blog in the moment to learn how to predict the future.  From a business intelligence perspective, that is.

Thanks to Accutate for providing this short article on Predictive Analytics.

"Predictive analysis can be an extremely useful tool for many different types of businesses. In fact, where there is any type of data warehousing there should be implementation of a business intelligence program that includes predictive analysis. However, in order to learn how your business can profit from this facet of business intelligence, you are going to have to understand exactly how and why predictive analysis works.  

The main idea of predictive analysis is to use current and past data to predict future events. The goal of the statistical techniques used in predictive analysis is to determine market patterns, identify risks, and predict potential opportunities for growth. In addition, data relationships can be reordered to determine the most plausible outcome of possible solutions and patterns can be recognized that might have the power to alter the outcome of a probable event.  

One of the most important aspects to reliable predictive analysis is data quality. The information provided by predictive analysis can only be as effective as the abundance and accuracy of data available. Data quality is absolutely necessary to the process of predictive analysis. In order to attain accurate business intelligence, companies must maintain quality data.  Predictive analysis requires both past and current data about many different things including customers, businesses, products, and the economy. All of this information is used to draw relationships and patterns between sets of data. If the data is accurate and well maintained, then the business intelligence produced will be high quality as well.   

In the past, predictive analysis was mainly used for newly emerging technologies. However, in recent years these practices have quickly started to become common for mainstream businesses. There are a few differences between the ways that these techniques are currently used and how they were used in the past. One of the main reasons for these differences is why companies use predictive analysis. In the past, these techniques were used for long-term analysis of market and consumer trends. However, in recent years, the mainstream implementation of predictive analysis techniques has tended to focus more on immediate, tactical uses. Because of the “real-time” nature of this business intelligence, more and more companies are using predictive analysis as standard in making predictions about particular industry markets and consumer trends.  

Some of the industries that have started utilizing these business intelligence techniques include telecom, insurance, pharmaceutical, and financial industries. All of the companies in these different business sectors have been able to use predictive analysis to make the right decisions to move their businesses in a positive direction. These processes can help with economic predictions as well as predicting the behavior of businesses and consumers. This type of information, made available in an efficient manner by business intelligence, is understandably invaluable. It can turn a simple prediction into intelligence that is more precise than even the most educated guesses. Predictive analysis with appropriate attention paid to data quality has made it easier than ever for businesses to make accurate market and consumer predictions and thus smarter decisions for the growth of their company."

Microsoft bundles BI software - what they are doing unveiled

The inevitable nature of big companies is to lock customers into their suite of products. For instance, Oracle started with databases, now bundles financial and integration software with their database. Customers are told it's easier to use their add-on, value-add products than use another vendor and get into compatibility issues. Add-on, value-add products are sometimes free, enticing you even more.

So it comes to no surprise that Microsoft has announced they have tightly bundled their BI software, PerformancePoint Server, with their document management software, Sharepoint. Joined at the hip, you cannot buy one without getting the other.

Comparatively to other BI vendors, Microsoft has the only bundled BI, document management, workflow, collaboration software on the market (I think). IBM Cognos, SAP Business Objects, and Oracle Hyperion are behind with this endeavour - they haven't tightly integrated BI with their other software.

Each of these competitors to Microsoft have similar software breadth but Microsoft's tight integration is a step ahead. It may take years for IBM, SAP, and Oracle to properly integrate BI, document management, workflow and collaboration.

But what about the customers? On the surface, the tight bundling can be viewed as positive. You could be using Sharepoint and now want some BI reporting. Or maybe you have PerformancePoint reports and need to manage documents.

Tackling both BI and document management pieces is challenging.  Without considering planning, governance, and skillsets, you will most certainly encounter serious setbacks.  Or worse your vision is restricted to an unimpressive rollout.

Here are some problems you may encounter with tight bundling of BI and document management.

1. Standardizing on Microsoft, Oracle, IBM, SAP technologies requires highly paid IT resources and/or consultants (not great for this economic climate)
   
2. Very different skillsets are required to build Sharepoint sites vs PerformancePoint reports (need to hire more staff and/or pay for more IT training)
3. Goverenance and best practices are very different for managing BI vs documents (different knowledge needed, so twice the effort)
4. Locked into sole-source agreements (difficult switching BI or document mgmt products for a better and more cost effective product in the future)

Putting those concerns aside, there are positives in Microsoft's story.  Microsoft's price point is unbeatable when compared to other on-premise vendors.  This can offset additional costs (see points above) and make your entrance into Document Management and Business Intelligence a little easier.

The question is whether the competitors - Oracle, SAP, IBM - will compete directly by integrating their products with BI.  Here's my thoughts on where these companies are going.

• In the case of SAP Business Objects, it's natural direction would be to simply use Business Objects as the reporting engine for the SAP application (do away with SAP's current BW).  Perhaps keep Business Objects as a stand-alone BI product set... perhaps.
  
• Hyperion is a natural fit for Oracle Financials.  Hyperion was deeply entrenched within the financials of large companies.  But Oracle has many other products that could work along side Hyperion's BI.  For instance, SOA/messaging integration replacing traditional ETL (or parts of it).
• As for IBM... guessing where they are going is just a gamble.  I don't have any insiders that could give me hints.  IBM and Cognos were working together prior to the acquisition and you look at the breadth of products, consulting division and R&D coming out of IBM... well your guess is probably as good or better than mine.

Is Business Objects losing ground?

"Urgent: Time is running out". The title to an email today from Business Objects, an SAP company. I'm on their mailing list so I get emails once in a while.

But what was so urgent that I couldn't miss out on today? Perhaps they were informing me of a Christmas charity that needed donations to save the Congo children. Maybe a holiday offer of free software licenses? They say it is urgent so it must be important.

Where's the value

So what did I get that required me to jump out of my seat and click on the link in the email before time ran out? I have now "opt-in to receive emails from BO". Wow, time truly was of the essence (I'm trying to contain my sarcasm). I will receive "announcements, events, product offers, technical tips, and other vital information for SAP customers." [I thought I was already signed up, which is how I received the email in the first place??]

Lame campaign

Anyway, this was a lame marketing campaign from BO's marketing department (maybe this is from SAP's marketing department...). Anyway the technique of inducing urgency and fear is decades old and rubs people the wrong way. Our time is valuable, so if it's "urgent", it had better be.

Fear within Business Objects

This could also point to a fear coming from within BO/SAP, although I cannot substantiate this claim. I base this on history where fear marketing comes from companies that are losing ground to competitors. Or they simply go delusional (like the automakers who continued marketing the same gas guzzling vehicles when the economy and environment were screaming for change). They are detached from their customers.

I suspect Business Objects has a strong revenue base from existing customers and services so I'm not trying to induce fear that BO will be asking for Federal bailout money. But maybe they are seeing declines in new customers and new license revenue.

For instance, I think it is safe to say they failed with their Insight Portal (sorry I can't find the link), failing with their Information On Demand, and I don't see them getting much uptake from their BI On Demand web portal.

Until something more substantial comes out, I'll wait for future correspondence from BO like a 7 year old child waiting for Santa on Christmas Eve.

10 Questions with Mark Windrim

The "data warehouse for anyone who needs it" has gone open source.  I previously talked with Miriam Tuerk, Infobright’s CEO, when the company launched.  Recently I spoke with Mark Windrim, VP of Community for Infobright, who is at the helm promoting their open source data warehouse, Infobright Community Edition (ICE).  Mark, as I discovered, has a history of spearheading initiatives from their infancy.

In the 90s, Mark started a community called MAGIC while working at Apple Computer. MAGIC grew quickly into one of the largest ISPs serving the Toronto, Canada region. At that time MAGIC was the largest Macintosh user group in the world. He left to join Newstar Technologies in 1996 (where he met Miriam). Newstar evolved and became BCE Emergis.

Now his focus is set squarely on building an open source community for Infobright. It won't be easy. Starting a community in an established company with an existing product - a paid product at that – can be even more challenging.

Question: So Mark, let's get right to it. Why did Infobright decide to go open source?

Answer: As you may know, Infobright built its technology underneath MySQL so we have always had open source roots. Open source has always been part of our plan. In terms of timing, we wanted to prove out our technology with early customers before releasing an open source version. We wanted to make sure it was highly scalable, easy to download and implement, and robust - not an immature project. We see a large untapped opportunity in the market to bring an ultra low-cost, easy-to-use data warehouse to many companies who don’t have the big budgets and resources needed to implement traditional solutions. Open source is the best way to enable them to try it.

Question: What does your community ecosystem look like today?

Answer: We’ve got our main infobright.org site that contains information about ICE. It supports a Wiki and forums where users can mingle and get support directly from Infobright and other community members. We have a great relationship with MySQL/Sun and continue to work closely with them in supporting the open source data warehousing community with ICE. Additionally, we are building partnerships with many organizations. Today, those partners include Pentaho, Jaspersoft and Talend, and we work with their communities in addition to our own.

Infobright also started to build out our integration communities through relationships with OpenBI in Chicago and Lincube in Stockholm, Sweden. Many more are to come. Both OpenBI and Lincube have extensive experience in implementing BI Solutions with open source offerings.

Question: What is the planned direction for your new open source community over the next few years?

Answer: We will be adding a great deal of educational content about data warehousing. The Infobright software itself will be enhanced and updated on a monthly basis. Since the ICE launch, we have already added a 32-bit version of the product and a new release has been posted to the community.

The goal is to create an environment where people can go to be introduced to data warehousing, as well as receive help both from Infobright and the community itself. We’ve begun offering free webinars as an introduction to data warehousing, and the response (and attendance, I might add) has been very good. Many of our community members are turning to us because their current platform is no longer able to meet their requirements, and they’re starting to build data warehouses as a result. Normally that is a very expensive proposition but open source, and ICE, has made it available to the masses. We want to be that go-to location when individuals and businesses need to know how to build these systems.

Infobright will also be adding content about implementing an open source data warehouse through support of our partners. We’re already moving down that path and recently announced an open source bundle with Jaspersoft. Shortly you will see more of this as we try and make the on ramp to open source data warehousing as easy as possible.

Question: What challenges have you encountered starting an open source model within a company selling a paid product?

Answer: We had to engage the entire Infobright team in the vision of a different kind of company. Everyone’s job was going to change in some way and the venture would only be successful with 100 percent employee participation. To be honest, that education process was fairly complex. We implemented a comprehensive plan for the entire company over six months to ensure that everyone was on the same page, and that we had everything absolutely ready. We also researched other open source projects – both successful and unsuccessful ones to understand the best practices of open source. Those learnings have impacted everyone in some manner and through them we have developed guidelines for all to follow.

Question: Infobright's open source database was released in September 2008. So far, what has been the response?

Answer: Response to Infobright’s open source announcement has been incredible! Immediately after launch it became apparent that the community wanted a 32-bit version of ICE, even though we had previously thought that the software needed more memory in the form of a 64-bit version. We responded immediately by offering a 32-bit version, and it quickly became the hottest download on the site. Infobright is also receiving reports from community members that they are using ICE in ways that we never would have thought was a great fit for us, but the sheer volume of people using the platform is giving us some fantastic new insights into where we need to focus our future efforts. I don’t think there is a day that goes by where there isn’t an “oh wow” comment coming from a community member that loves ICE. Now the community is asking us for Windows support, so you’ll be seeing that coming down the pipe shortly.

Question: How has the open source initiative affected Infobright as a company?

Answer: As I mentioned earlier, the way we do virtually everything has changed. Infobright.org community members commenting on code, or recommending new features, has a direct impact on our product roadmap. With many more people using Infobright’s software, we are discovering new ways people can benefit from its use not previously considered. For example, we are now looking into use cases that didn’t make sense to us six months ago - such exploration is a direct result of community members having an impact on the organization. From a business perspective, the last six weeks has been a hockey stick in terms of number of new customers and revenue. By the end of Q1, we expect to have surpassed most, if not all, of our competitors in terms of number of customers using our product. And more users result in a better product – we aspire to deliver the best product in the market!

Question: Has the direction of Infobright changed as a result of being an active part of the open source community?

Answer: Yes, absolutely. The expectations of the open source user are different than that of the commercial one. The open source user demands from a technical perspective are very high – you can’t hide issues and weaknesses; your bug list is entirely public. They’re doing a phenomenal amount of testing and performance stress testing. All these insights benefit every part of Infobright as we add new features to ICE. Such feedback is even changing the types of webinars we present, which user groups on which we focus, and what features we add to the product. We also closely monitor a variety of metrics in order to determine the best ways to support the Infobright.org community.

Question: There is probably the value proposition of lower cost, less implementation time, and less maintenance effort. However, can an organization really run an enterprise-wide data warehouse on open source?

Answer: Every aspect of a data warehouse infrastructure can utilize open source technologies. A company can utilize Linux as the operating system and integrate with open source ETL tools like Talend or Kettle (Pentaho). Many open source platforms will use their own home grown ETL processes built upon Python, Perl, or Ruby (or C/C++, for that matter). The data warehouse can be built upon ICE, and users can use BI tools like Jaspersoft or Pentaho. Support is readily available for purchase or for free through community forums. Every one of these platforms is extremely robust and stable. There is no reason that an organization could not run their data warehouse using open source technologies.

And businesses are recognizing this. Open source BI is a fact of life in enterprises across industry. A recent Gartner survey highlighted that by the end of this year 69 percent of enterprises surveyed either plan to have implemented and be using open source databases. And in the area of BI tools, the number is 34 percent (a 100 percent growth over the previous year). These statistics were gathered before the recent economic downturn. Now, companies are even more pressed to find the fastest and most cost effective way to deliver BI and data warehousing solutiuons to their businesses – and that way is open source.

Question: What is the benefit to your partners and developers of being a part of Infobright's open source community?

Answer: In addition to the obvious and huge benefit to end users, the data warehouse is a key enabler of many enterprise applications. As such, our partners and developers will be able to achieve great benefits to their businesses and products thru integration with Infobright. Now organizations can accelerate their business by reducing the cost and time required for the data warehouse component and focusing their value on the application. Additionally, our partners and developers not only support our community, but they cross pollinate each other’s communities as well. With open source, there is much more willingness among partners to support each other.

Question: It’s been a pleasure talking with you Mark. Do you have any additional links or information about Infobright's open source community you want to share?

Answer: We really encourage people to get involved in our forums (www.infobright.org/Forums). There is a vast amount of information about ICE available there and we really appreciate reports on how people are using ICE. We’d also encourage people to look at Pentaho, Jaspersoft and Talend.

BI in the year 2020

A theme heard throughout the Microsoft conference was "thinking bigger about BI". I'm not sure the people from companies who are struggling to get BI started were interested in thinking bigger but I can appreciate their visionary approach.

A panel discussion was setup with a group of senior Microsoft partners to discuss and debate where BI will be in the future, in the year 2020. Moderated by Bob Lokken, Senior Director of Business Intelligence for Microsoft, the panel included:

• Greg Todd, Senior Executive, Accenture's Information Management Services practice
  
• Andrew Horgett, Sr. Manager, Microsoft Alliance Team at Dell
  
• Mike Fahey, Director, HP's Product Development
  
• Todd Price, VP, Hitachi's National BI Practice
  
• Terje Rugland, CTO, Profitbase
  

There were a couple comments about collaboration, social networking, and instant messaging but I would say that is a year or two away from mainstream. Not exactly 2020. They talked about data mining, optimization of the DW, and being proactive. Sigh. These are current topics being worked out by innovative companies today.

However here are the top 10 predictions/recommendations I gathered from the discussion. I'll let you comment on whether you feel these are in line with what you see.

1. 90% of BI will be delivered on mobile devices.
   
2. Much larger volumes of data, beyond terabytes, will need to be handled changing the hardware and software paradigm.
   
3. Information will be managed by business professionals and not technology professionals.
   
4. Do not listen to BI best practices that say "18 months to deliver" and "high costs for licenses/services". We need to think out of the box.
   
5. Envision a new role within an organization called the Information Architect (based from a Data Architect). This person understands the entire life cycle of information from where it comes from to where it ends up.
   
6. Better adoption of BI will happen when BI is integrated within productivity suites, like Office. BI will be an extension of business productivity.
   
7. BI should not be packaged with ERPs because the role for BI should be to integrate disparate systems. However Performance Management should be packaged with ERPs.
   
8. BI should be considered a service for business people.  Business is used to leveraging services already.  However IT is not yet matured to the point of understanding this role.
9. SaaS BI's success will be based on two serious roadblocks: Data Quality and Data Security.
10. Organizations will have 2 application footprints:  ERP systems for capturing information and IRP (information resource planning) systems for massaging and delivering information.

Did this help with your future BI plans and 5-year implementation roadmap?

[Other MS BI Conference posts:
ETL World Record and 150 TB Fact Tables and
Ben Stein talks about BI]

Ben Stein talks about BI

How many funny, engaging stories can someone tell in their first 10 minutes of getting on stage? Well Ben Stein surely approached the limit. Ben Stein has done many things on tv and off, been a lawyer, is a well established economist, writer, and columnist.

However for the MS BI Conference crowd of several hundred, his clarity on the financial crisis drew feelings of frustration over the financial community and government mismanagement. He referred to how the financial community could have used a little Business Intelligence -- and could definitely use some now. His clear chronological overview of how the financial community got into this mess (and brought us along with them) was responded to with claps and a few hoots of support.

Briefly Ben lays out the crisis with this:

1. "It started with the liberals wanting equal housing opportunities for every American."
   
2. "Greenspan pushed interest rates down and then banks couldn't make much money from normal mortgages, so they had to be creative."
   
3. "Banks came up with Side Bets. Basically banks bet on whether or not mortgage bonds would be paid or failed to be paid." -- Yes banks bet on whether you will pay off your mortgage.
   
4. "When house prices fell, costs of price swaps (the side bets) went up. This created such a huge loss that it dwarfed the financial impact from defaulted loans."
   
5. "Side bets are the Weapons of Mass Financial Destruction allowed by the Bush administration."
   

He summarized with "Wall Street has taken trillions out of the economy from everyday people over the years". Needless to say he doesn't agree with the people taking on the risk by giving billions in a bailout package.

Ben Stein also had good things to say but it was clear he was there for his entertainment and inspirational value... a BI tech talk he was not about to do. If you haven't seen him, then he is just like he is on his tv shows, like "Win Ben Stein's Money" and "America's Most Smartest Model".

[Other MS BI Conference posts: ETL World Record and 150 TB Fact Tables and
BI in the year 2020]

ETL World Record and 150TB fact tables

At the Microsoft BI conference in Seattle today and wanted to give you highlights as I jump from session to session. One point will be about the world record.

MS Message: Think Bigger about BI

MS goal: accelerate decision making using structured and unstructured information.

Trends: Despite BI being on top of CIO spending list, there are less than 20% of people using BI? Why? Still not easy for people to use BI to get their job done.

MS Prediction: BI won't be on top of CIO spending list in the coming 3 years.

Okay about the world record. Microsoft recently set an ETL world record. I didn't even realize there were Olympics for BI but I guess there are. So by using SQL Server, they loaded 1 TB of data within 30 minutes. To put this in context, Oracle's last record was 1 TB within 45 minutes. Of course they are using SQL Server 2008 to accomplish this.

And on that note, Project Madison! If you're interested in large, large volumes of data, this could be the next big leap. An R&D partnership with DATAllegro who produces DW hardware, is building a combination of SQL Server and DATAllegro using massively parallel processing (MPP). They have loaded 150 TB of data with 1 trillion fact table rows using 14 servers. The live demonstration showed response times from Analysis Services reporting within seconds. Impressive.

More as I come across anything that is excellent.

[Other MS BI Conference posts: Ben Stein talks about BI and
BI in the year 2020]

10 Questions with Glen Rabie

Being in the finance industry may seem boring to many but in Glen Rabie's case it gave him a unique perspective on the challenges a large financial company was having with their information.

Then a life changing event inspired him to become an innovative entrepreneur. Glen founded a company called Yellowfin where they are all about making it easy for people to get access to information.

Question 1: Hi Glen. So what motivated you to do something about improving the BI industry?

Answer: The light went on whilst working on a large data warehouse project for a global financial institution. The project, budgeted at $30M, only rolled out 300 licenses of the BI tool to an organization of 50,000 employees. This is where we saw the opportunity. Reduce the complexity and cost of deploying BI so that every person that needs access to data and analysis, to do their job effectively, can do so.

Question 2: Let's talk about the market. You had an interesting take on the recent acquisitions of Cognos, BO, and Hyperion. What should we expect to see from them with innovations?

Answer: The acquisitions and the high multipliers paid for them put the new owners in a delicate situation. Their primary concern will be to generate a return on their investment, which to do so will mean that additional investment in those products is likely to be stymied. What we have seen since the acquisitions is a greater focus on rationalization of product set rather than news of product improvements. I think this is what we can continue to expect in the short to medium term.

Question 3: Has Cognos, BO, and Hyperion already won the BI tools market? Is there room for new products?

Answer: The top 5 players only own 70% of the market. This compared with most IT markets, such as the database market, can be considered to be highly disaggregated. As for the remaining 30% this is the growing segment. The BI landscape is changing from the traditional high cost but smaller power user deployments to mass deployments. I do not believe that BO, Cognos or Hyperion have the business model to support this change, both in terms of pricing and deployment complexity. It is this which is creating the new opportunity for vendors of easy to deploy, price scalable and browser based solutions.

Question 4: Interesting that BI doesn't have a gorilla in the market. How does that affect our customers?

Answer: In reality BI tools are a critical piece of IT infrastructure. Making a strategic choice is crucial since not all BI tools are a fit for all business scenarios. Without a Gorilla in the market I think it forces customers to evaluate solutions and make business / technical decisions based on their needs. If a gorilla exists then there is a tendency towards ‘me to’ buying behavior. So the current state of play is actually good for customers.

Question 5: You think that BI is complicated but is making the tools easier to use the solution?

Answer: It is part of the solution. There are aspects that will continue to be challenging – not matter the size of deployment you still have to invest in the design of and build the back end processes to support front end analysis. So in terms of ROI the more users of your BI solution the better. Having easy to use tools that can be rolled out with little to no training to many, who then in turn utilize the BI infrastructure to drive business benefit is key. This is where Yellowfin focuses its efforts – it is not just about being easier to use per se, but also easier to deploy (via a browser), easier to administer and manage security etc.

Question 6: The BI industry promotes methodology for an enterprise view and consolidating across the enterprise (e.g. customers across marketing, sales, finance). Does this make sense for the management team?

Answer: Yes and No. The reality is that even in Global organizations Senior Management and the executive team are primarily responsible for and remunerated on their performance within their area of the business. Consolidating data, such as a single view of the customer may have little to no actionable impact for these Managers (Great for analytical Marketing and Customer relationship Managers).

However, for management having a single consolidated view of business performance is critical. The data may exist in data silos but the delivery and presentation may well be consolidated to provide a global view of the organization. This is a large distinction – consolidation of data versus consolidation of presentation.

Question 7: Where are you positioning Yellowfin in the market?

Answer: Yellowfin is positioned as a viable alternative to the traditional vendors – Cognos, Business Objects, Hyperion etc. Our focus though is purely on the presentation layer. We are not building a BI stack. There are plenty of great specialized BI components for ETL, Budgeting and Forecasting which is not our area of specialization. So for we are driven by developing very easy to use presentation layer which can be deployed for 100s and 1000’s of users. It is in this space that we excel.

Question 8: Should learning to use a BI system be as quick and simple as, say, CRM or even Google?

Answer: Yes – for too long there has been this aura surrounding BI that it is a hugely complex undertaking, but let’s be honest with ourselves – when it boils down to it BI is just charts and tables. Well Ok maybe a bit more complex but the mind set has to change, from being a hugely difficult task to making it easy. End users are becoming more analytical and demanding greater access to data - going forward embedded BI applications are going to be seamless to the end user. They will use these tools almost without being aware of them. For companies are not going to have the time or the capability to train the vast majority of day-to-day users in how to use their BI tools – they will have to be easy and intuitive to use.

Question 9: Where do you envision the BI market going over the next 5 or even 10 years?

Answer: he biggest change in the BI market will be the emergence and dominance of Embedded BI. BI as a standalone application is going to be a very small segment of the market.

Embedded BI is needed to support the 1000’s of organizational information stakeholders. Process oriented workers and customers want access to BI at the point when they need it to complete their tasks within the application that they are using for transactional purposes – they do not want to access an alternate application for their reports and data. It is this area of embedded BI that is really going to make BI pervasive, bite size, tailored to the business process and rich in collaborative functionality.

Question 10: Excellent talking with you Glen. Do you have any additional links or information about Yellowfin you want to share?

Answer: Thanks Tom, it’s been a pleasure. If you want to find out more then go to www.yellowfin.bi.

Really Simple Integration

Companies are challenged these days. So are individuals. The vast amount of information available and being created every minute is growing so fast how can one leverage that into something meaningful?

Along comes aggregation. It's seen for blog/news feeds, like AllTop, for the thousands of "top bloggers".

But what about organizations and integrating their information needs? Along comes SnapLogic with an open source RESTful architecture integration app. I spoke with Chris Marino, CEO, and John Bennett, Director of Marketing. They explained how they empower organizations through self-service... to "loosely couple a federation of systems" for enterprise mashups of data.

Here's a partial quote from their press release today:

"Really Simple Integration is a new approach to data integration" that "enables enterprises to quickly and easily make core IT data from data warehouses, Master Data Management data marts, SaaS apps, SOA Web Services and other sources."

Be sure to check them out if you're attending O'Reilly's Web 2.0 Expo in San Francisco this week. In Web 2.0/open source fashion, they have started a publicly available collection of free components, including a screencast showing a mashup of LinkedIn and SalesForce.com.

So where does this put Business Intelligence and more specifically ETL tools and the static nature of data warehouses. I think this is another step towards the end of the ETL era as we know it. Products like SnapLogic provide transformation functionality but have access to more than internal data sources. Ever heard of an ETL tool able to scrap data off a public website to merge with your sales data?

Business Intelligence has been typically limited to using an organization's internal databases, such as finance, CRM, marketing, and sales. But we're in the age of the Internet now (actually we've been here for quite some time) and to be competitive you need access to the vast amounts of information from external sources of information, such as SaaS applications and information websites.

The small to mid-sized market is where SnapLogic is positioned today. Tiny companies can leverage the open source community, while IT staff of mid-sized companies would deploy SnapLogic for efficiencies. So once the data warehouse is built, users/departments will start asking for the data in different ways. (The static issue with a DW). Or merrying DW data with data not in the DW (and they shouldn't wait 18 months for additional DW implementation to address this need). IT departments can shine by having an easy-to-use tool like SnapLogic.

Sometimes there is no time to wait for the perfect enterprise dimensional model to be designed. Organizations are organic and need to stay competitive and ever changing to keep ahead. Access to information is key.

10 Questions for Miriam Tuerk

It has been said by many, including Gartner and Forrester, that the next big innovation for BI & DW will most likely come from the data warehouse side. Sure Visualization is a hot topic lately but the "pain points" for many clients are on the back-end.

After speaking with Miriam for a few minutes, she mentioned their tool produces results from "three billion rows of data and resolved queries in seconds". So she caught my attention. And when Miriam Tuerk, CEO of Infobright, mentions a client roster of the likes of the Royal Bank of Canada, Xerox, and TradeDoubler, you know they are onto something.

Our conversation continued as we discussed Infobright's innovative solution, which I'm sharing with you.

Question 1: Hi Miriam, let’s start with your statement “Research shows that the volume of the world's data approximately doubles every three years... 92% of new information is stored on hard-disks." Do you think Infobright can help organizations analyze data faster in a more flexible way?

Answer: We have proved it at our customer sites. Our customers have wrestled with the problem of how to extract valuable information from the huge volume of data they collect. They know, as we do, that being able to quickly access key information about their business or their customers can be the difference between business success and business failure. That is why recent studies confirm that Business Intelligence is the #1 investment area for CIO’s today. Infobright designed an analytic data warehouse solution from the ground up specifically designed to provide fast answers to ad hoc, complex analytic queries without burdening IT with lengthy, resource-intensive projects.

Question 2: Okay, let's get right to the heart of Infobright's business. Why do I need your DW solution when I know I can already build dimensional models, cubes, reports, etc?

Answer: Three reasons – time, money, and the unknown. Given enough time and money, IT can develop a system perfectly designed to answer any question quickly – as long as they know the question. In today’s changing business world, however, business people don’t know in advance all of the questions they will need answers to in the future. They want the answers today, but most systems require a lot of manual work on the part of IT and database administrators to set up and maintain environments each time the business users want to perform new and different analytics on their business. Today, providing fast access to massive amounts of data requires a lot of IT resources and time. Infobright’s solution eliminates all of that work by IT, and doesn’t require buying lots of servers and storage as other products do. Instead, we developed a simple but very powerful solution that provides business users access to all of the data they need to get fast answers to unpredictable questions.

Question 3: How did you and Infobright get started? Was it a grass-roots entrepreneurial effort?

Answer: Infobright was born out of pioneering work done by a group of internationally recognized mathematicians in the emerging science of Rough Set Mathematics. They realized that they could use information about the data itself to quickly provide answers to complex queries, rather than require IT to do extensive work up front or rely on brute force from massive amounts of hardware. Seeing the benefits of this approach, RBC Capital and Flybridge Capital Partners (formerly IDG Ventures) funded the company and brought in an experienced management team to turn raw technology into industry-leading products and services. Over the past year we have expanded the capabilities of our software while growing our customer base and establishing Infobright as an emerging player in the market. For example, Infobright is the first analytic data warehouse provider to be named a MySQL Certified Storage Engine Partner. The combination of Infobright’s solution and MySQL provides organizations an analytic data warehouse that delivers unprecedented scalability, performance and ease-of-use.

Question 4: Where do you envision the DW market going, especially with recent consolidation of BI vendors?

Answer: In the “old” days – the 1990s, which is really not so long ago – smaller volumes of data, smaller and less diverse sets of users, fewer subject areas and simpler queries, allowed vendors to recommend one data warehouse solution to meet all of the business needs of the users. Just like hardware, where once we only had only the mainframe, the market is evolving and maturing such that there is no longer a “one-stop shopping” solution to meet the BI needs of businesses today. There are really two different types of workload in a data warehouse:

1. One requirement is where you have a lot of users running the same query over and over. An example would be if you had a customer data warehouse being used to support a call center for a cell phone company. Every time a customer calls in, the customer profile is pulled from the data warehouse. This is a repetitive OLTP-like query and for this a highly designed and engineered system, optimized and tuned for the specific and repetitive queries are the best solution.


2. A second requirement for data warehousing is analytics. Here, marketing, finance, sales, compliance, risk management, operations groups in companies are performing ad hoc, changing and unknown queries such as:

• “How did our 2007 Christmas sales campaign perform as compared to our 2006 campaign? Was the customer retention higher – did more of those customers buy the value-add services?” or


• “Let’s do a trend analysis understanding why there are more mortgage defaults in this area than previously – lets run a trend analysis of the last 12 months versus the last five years. Can we identify any indicators that would allow us to re-estimate/extrapolate what the defaults will be thru the end of 2008?”

These parts of the business use the data warehouse to design marketing and sales campaigns, to understand what the risk, compliance and security issues may be – and use that to operate and manage the business. Today, IT needs to assign resources and do manual work in support of all of these queries. And every day the business has new or different queries, IT must do more work. Business users need to have a “Google-like” experience for this type of data warehouse workload. They need to be able to just run the query against the data warehouse without manual intervention of IT.

This is the really big, hidden story in BI, that the growing analytic requirement is causing IT to drown under the workload it requires. They need a way to make things simpler and really change how things are run. Some technology companies are delivering value by consolidating platforms and creating integrated solutions. We have chosen to focus on the analytic use case and deliver the only product in the market that effectively solves that problem.

Question 5: What are the benefits of Brighthouse for an organization or manager looking for information?

Answer: Brighthouse delivers fast response to ad hoc, complex analytical queries across a large volume of data. It does so without requiring IT to spend time and effort to create new schemas, create indices or partition data. It is also lowers total cost of ownership through industry-leading compression that significantly reduces the amount of storage needed to support all this data. Business users get the answers they need quickly, and IT can meet high service levels with minimal effort or cost.

Question 6: You mentioned the phrase, "use the intelligence of the data." Can you share what you mean by that?

Answer: When data is loaded into the Brighthouse system, it is tightly compressed and stored in “data packs.” The Knowledge Grid automatically creates a highly compact set of metadata, which stores information about the relationship between packs and statistical information about the contents.

When a query is initiated, Brighthouse searches the grid to intelligently decide which data packs, if any, are required to resolve the query. The Knowledge Grid is created on-the-fly, dramatically increasing data load times, and eliminating the need for specialized data partitioning and indexing.

Question 7: So, what about competitors, like BI appliances, database vendors, and such? Are you taking data warehousing one step further?

Answer: The fact that there have been new entrants into the market in recent years is a clear indication that current technologies do not meet the needs of the business today. They are also an indication of high demand across a very broad spectrum of new requirements. Traditional solutions are very expensive, take a lot of time to build, and in fact, are not well suited to support the analytic queries of the business. That is why IT struggles to keep up with the demand of the business users. Many of the newer products on the market are very good at what they are designed to do – provide very fast query performance to predictable queries – but are not designed for ad hoc, unpredictable complex queries. What’s more, they all require substantial work on the part of database administrators and IT to implement and maintain.

Infobright’s solution is markedly different – it is incredibly simple to implement and maintain. Rather than extract-transform-load data, our solution is Load and Go. No new schemas, no index creation, no data partitioning. Brighthouse is simple, powerful and extremely cost effective – the best solution if you need fast answers to evolving business questions.

Question 8: How can your Brighthouse product fit within an existing BI/DW solution? Or is it better to use your product at the beginning of designing a new BI system?

Answer: When we built our product strategy, this was a very important question for us. Organizations have invested millions into their existing data warehouses and BI infrastructure. Offering a solution that leverages those investments and works within that environment was key to us and is a big part of our go-forward product road map. Brighthouse is very well suited to be added to an existing data warehouse environment. Because of our “just load it and go” capability, you can re-use all of the data modeling, ETL, and BI reports that you have already built. Many of our customers have large data warehouses already, but they aren’t able to support business users requests for ad hoc queries due to its performance impact on other users or high cost. In that case, they’ll implement Brighthouse as a complementary warehouse to provide the services their business users need.

In other cases, Brighthouse is implemented as the sole data warehouse for the company.

Question 9: During our call, you mention several amazing results seen by clients. Care to share some of those with readers?

Answer: I’d be glad to. A good example is the use of our technology in support of online advertising. Companies that advertise online want to track how well marketing campaigns are attracting their target audiences as well as detailed ROI of these campaigns. The marketing analytics providers depend on being able to rapidly run complex, ad hoc queries against huge amounts of click stream data and provide this to their customers.

Using Brighthouse, one digital marketer found that it could load 3.2 billion rows of data at an average rate of more than 300,000 rows per second! Brighthouse also compressed all-important fact tables at a ratio of 40:1 – meaning that 40 GB of raw data resulted in only 1 GB of storage, leading to huge savings in storage costs as well as improved performance.

Another user—a company that manages major online customer loyalty and incentive programs—found that Brighthouse returned query results 15 times faster than an existing solution. Brighthouse also surpassed this solution’s ability to compress data, reducing the footprint of fact tables some 35 to 43:1.

Question 10: Excellent talking with you Miriam. Do you have any additional links or information about Infobright you want to share?

Answer: For your readers who are looking for more information about what is new in data warehousing, we have an excellent white paper on our web site written by Claudia Imhoff, a well known expert in the field. Those people interested in finding out more about Brighthouse can also find additional information on our web site at http://www.infobright.com/, or contact us at any time via email at info@infobright.com.

Open source gets VC funding

$12M in a round of Series C funding by Benchmark Capital who has backed other open source companies like Red Hat, Zimbra, and MySQL. This makes a total of $26M of VC funding given to this US-based BI open source company.

Yes, the Pentaho group picks up another round of funding according to this Performance Guys post. The article suggests that open source is one of three disruptive technologies with Software as a Service and pre-loaded appliances being the others.

The question still remains. Can Pentaho, and really open source BI in general, compete with other on-premise vendors, such as Cognos, Business Objects, Microsoft, and Hyperion? According to the article, Pentaho reports "brisk" uptake. My take is they still need to continue converting the 3 million downloads into open source licenses (i.e. sales).

Pentaho has built a location intelligence dashboard mashup with Google Maps. And is working with Greenplum's Bizgres-based data warehousing appliance platform. So they are getting out there and integrating their products for a variety of revenue streams.

But what differentiates them enough from Cognos, BO, Microsoft, and Hyperion?

I guess the licensing costs could be significantly less. The flip side to low upfront costs is the backend support and how that support is perceived. Is there enough of a developer community to support customer implementations? Are there enough help desk and knowledge areas to give IT departments support when questions and issues arise?

Perhaps the open source BI play is in the small to mid-sized company range for now. I'm all for Pentaho and other open source BI companies as they bring competitiveness to the industry. And as they take on more market share, the proprietary vendors will be forced to adjust and improve. And that ultimately provides more choice and flexibility for customers.

Microsoft leader in execution

I'm sure you've heard the Gartner quadrants are out again. The BI vendors are graded and ranked according to Gartner criteria. And as you can expect, vendors want to be high and to the right.

This year Microsoft is a leader in platform BI and ahead of the pack for "ability to execute" on the BI vision. This means Microsoft has the competitiveness, success from it's BI products, investment in BI, and can execute on it's pricing model.

Tidbit: Did you know Microsoft spends $6B (with a 'B') company-wide on research and development annually?

And it's pricing model... which I have to say is aligned with the BI industry's goal: deliver BI for the masses. No longer will an organization have to pay hundreds of thousands of dollars on licenses -- this is a cost prior to building anything for end users! Microsoft's model flips the traditionally expensive licensing model upside down.

Plus Gartner thinks Microsoft's PerformancePoint Server is coming to a desktop near you. Aimed at the mid-market and above with a CPM focus, Microsoft's BI stack integrates well with it's integration engine (BizTalk), Office products, and portal software (Sharepoint). What else could most organizations need?

And if you're interested, here are the Top 10 benefits of PerformancePoint Server from Microsoft's marketing department.

Okay, there are a few short comings but which vendor doesn't have them. One being a not so competitive data mining story. I view this as a small portion of most organizations, so I shall not dwell. The second is how they intend to deliver their BI products to customers -- through a partner community.

Yes, Microsoft isn't interested in implementing their own software, unlike Cognos and Oracle. They want to educate and train partners to do this. Today the downside is the limited choice of BI Systems Implementation partners.

The huge upside is when more partners come on board, watch out. These partners will have the collective brain-power to Outsmart, Outwit, and Outlast (yes I watch the Survivor series on TV) any in-house services group from the likes of Cognos, Oracle, etc.

So I like the Microsoft model and their products and expect to see them competing along side the other leaders that are high and to the right.

The SEC side-steps BI

Could the SEC be side-stepping business intelligence forcing companies to report financial data in an XML standard? Or could this turn into the investor's version of "financial intelligence"?

The SEC launched an XBRL-based online tool that allows investors to extract, compare and analyze executive compensation for large U.S. companies. This tool is so important to the SEC that they are pressing to mandate it as a requirement for all companies to post their financials for investors to view.

Investors can do analysis and reporting on companies to determine investment worthiness. And the best part, no Data Warehouse required.

Business Need: Ensure financial accuracy - In recent years, regulatory reporting requirements prescribed by new legislation, such as the Sarbanes-Oxley Act in the United States, have raised the importance of reporting accuracy and transparency.

Companies are forced to produce a public BI system, of sorts, hosted by the SEC.

Microsoft: As we know, MS Office Word and Excel have long been used to help compile, report, and consume financial information. Together, XBRL and Microsoft Office hide the complexities and shoot for wide-scale adoption in the financial community. Microsoft has a considerable lead in the market as financial people are very familiar with Excel.

With SEC's XBRL, contextual information is stored, while the language and accounting standards are irrelevant.

What is it? Extensible Business Reporting Language (XBRL) is a worldwide industry standard for the publishing, exchange, and analysis of financial reports and data based on the XML language. The XBRL technical standard is being developed by XBRL International, a not-for-profit consortium with 200+ members worldwide.

So where does this leave you if you're managing BI and your CFO needs to produce these XBRL financial filings? Most likely the financial department will want to export directly from their financial system. In my mind, this would break BI's one shared truth concept for an organization. Or does it?

This boils down to what BI is today and what BI can do for an organization in the future. Today, SEC filings may not be your BI system's mandate. In the future, BI needs to expand it's definition. Should it always include ETL and a data warehouse? Or should BI focus on delivering content throughout the organization and to external partners or customers? I think the later.

BI has potential but is rot with problems -- failed projects, high costs, low returns. Some organizations have made BI very successful. The US Veterans Affairs is one of the largest Microsoft Analysis Services deployments in the world with a profound cost savings of over a billion dollars! Now that is successful BI!

So you may want to consider how XBRL should be apart of your BI system using an overall mandate of providing content to the masses. As for XBRL, couldn't they at least come up with a friendlier acronym!?

Acquisitions bad for customers

In the short term at least. Acquisitions can kill R&D funding and unfocus company direction during the internal integration process. Then there is all the time and effort spent on a new company message announcing the "new" product and service offerings - marketing, architectures, training.

Thank you Fayu for sending in this IT Week article about acquisitions in BI will stem innovation.

As well, it could be that during the Oracle, SAP, and IBM acquisition/integration efforts, it may create a vacuum while these big three BI vendors (geez I guess they are now) focus on internal integration. This downtime vacuum may open up a space in the market for the entrepreneurially minded. I hope it does!

Sure business for the acquired (BO, Cognos, Hyperion) will continue to sell licenses. However I agree with the article that behind the scenes people are going to be pulled into committees, working groups and potentially let go, okay, re-shuffled.

This all equates to not pushing the envelope nor being laser-focused on customer needs and the competition. Unfortunate for them... Opportunity for others!

Who's Next

Who's next in acquisition fever?

Yes, there is already debate on who's to be acquired next. I guess it could be the natural progression of things but more likely the herd mentality is cropping up. I know, I know, BI is hot right now.

But like speculators on the stock exchange who drive prices up, make their money by selling at the top, and watch the prices fall on those unaware investors... aka us normal folk who are just trying to make a buck...

I've noticed attention being focused on the remaining tier 1 independent BI vendors - but I don't want to jump on the propaganda band-wagon for companies that didn't really standout before. Not meant as an insult but there are excellent reasons why Cognos, Business Objects, and Hyperion were acquired first (I would also include Microsoft with these best of breed companies).

Then I read the typical "watch for the up & comers" called tier 2 vendors. I believe both these tier 1 & 2 vendors should all be considered either "up & comers" or "been-there-and-done-that'ers".

The question is, if we sober up from our high on acquisitions, down deep in places we don't mention in polite company, "do you really feel the need for the industry to continue consolidating?"

I think not! Any acquisition now would come across as a follower in a sea of leaders. I may exclude acquiring SAS from that list, although there are post-acquisition, merger problems with a privately owned company such as SAS.

Alas where people, investors, acquisition-hungry companies should focus their attention on are innovative, thinking-out-of-the-box companies, technologies or people that will shift BI away from lengthy, costly implementations; allowing BI to permeate throughout a company delivering on the "BI Promise"!

Otherwise BI will be downgraded to simply a component or attachment to an ERP initiative, perceived as the second-cousin, something we will do later after the "must-haves" are complete. You know, similar to what operational reporting is today. There are winners to this downgrade... IBM, SAP, and Oracle's of the world win by having a complete solution to offer, where customers can go shopping in one place for everything they need.

Another word for this... WALMART!

And how do you compete with Walmart? Carefully, not head-on and not on price. Luckily IBM, SAP, and Oracle are not selling cheap commodities - their prices are high. In stark contrast, Microsoft's licensing model is built for growth... huge growth.

So where does this leave us as customers, consultants, and practitioners? With options.

As competitors consider how you will be competitive within this consolidated BI world.

As customers consider how you will win with either the Walmart's or the up & comers.

And a final up-lifting note on market change for up & comers. A market that is squeezed into a corner has high magnitude potential for paradigm shifts and innovative ideas to alter the status quo... sometimes in a significant way!

Here's a TED video of Larry Lessig telling 3 great stories of change.

Final acquisition - IBM

The story has hit the airwaves. One of the more obvious acquisitions in recent months completes the trend that has taken place in the Business Intelligence industry for several months/years now.

What this means for the industry specifically is too hard to tell at the moment. So as a CIO or business manager will you jump for joy or go running for the door.

IBM acquired the Canadian-made Cognos yesterday for $5 billion. This Wallstreet Journal article states Cognos is No. 3 in the BI industry following SAS (No. 2) and BO (No. 1). The IBM price is slightly less than SAP's BO ($6.78B) and more than Oracle's Hyperion ($3B).

Some may say IBM's hand was forced with the industry consolidation by competitors Oracle and SAP. However I think the IBM-Cognos deal was in the works for some time.

Cognos and IBM have 'played' together on large government projects. IBM tested Cognos tools in their performance lab. IBM and Cognos have done joint whitepapers. IBM consulting services has people focused on Cognos. I think the writing was on the wall -- just when and how much.

Plus you don't make a $5B decision in cash over a couple months because of pressure from your competition. At least I couldn't (my cheques don't have the space to write that many zeroes).

Overall:

IBM is known for going after the "big and scary projects that no one can do" -- in IBM's words. Cognos is known for selling to large companies and governments. Probably a match made in heaven.

But let's take a step back for a moment. We know the acquisitions of BI companies will change the landscape of the industry. BI could simply slide into being another component of ERP vendors. Or BI may continue to stand on it's own as a business improvement driver by gauging performance.

The big picture question is, "with independent BI companies gone, where will BI go?"

Just like most acquisitions, the ecosystem of toolset vendors, consultants, customers, and 3rd party vendors can change dramatically. How? I would like to think our drive to improve BI for the betterment of organizations and people is the underlying goal. But that may get in the way of profits, selling licenses, and implementing large BI projects because they can.

Don't let them tell you "nothing will change; it is status quo" -- it will change.

Don't let them tell you "more benefits exist by being acquired" -- there can be huge drawbacks.

Don't let them charge more because "IBM, Oracle or SAP specialists are doing your BI" -- the business problem and expertise have not changed.

Don't let them tell you "this is the only way to do BI" -- get a second opinion from someone independant.

And welcome to the confusing jungle of seriously large companies where BI is just one line item on their financial plan.

BO takeover made public @ 20% premium

The agreed to takeover of Business Objects at a 20% premium or $6.8B by SAP is now public knowledge. The news articles read as per expected; synergies between their companies will benefit customers. Click on the diagram showing how SAP and BO complement each other.

The question is How will they benefit customers?

It has been an acquisition frenzy lately. SAP purchased OutlookSoft (performance mgmt), Pilot Software (analytics), and now Business Objects (suite of tools). Plus BO recently purchased Cartesis (performance mgmt).

And with all this growth, BO has been spending years integrating the original BO package and Crystal engines. Cartesis hasn't been fully integrated prior to this acquisition so the challenge will be how to sensibly pull together four different products into a clear product offering from SAP.

Then take the overlap with SAP's own Business Warehouse product. From my experience BW had difficulties being received by customers -- the takeovers/acquisitions definitely give SAP a strong BI offering for their business process solutions. They should simply replace their Business Warehouse tools completely with tools that work.

Will SAP embed their acquired analytics into their transactional applications?

Perhaps the question is When. You can expect SAP to provide an integrated SAP/BI solution where the new analytics will be embedded within the business process tools. Which is another challenge. Will SAP embed their new BI tools and still offer a separate BI suite of tools? BO, as with the others, are vendor neutral - meaning the tools can sit on a variety of vendor databases.

And with BO's revenues of $1.25B last year, disupting that source of income just to have BI embedded within SAP so you can sell a packaged solution, would be suicide. Unless the management team thinks the world revolves around business process and is unable to see the vendor-neutral potential of BI.

The merging portfolio's of two multi-billion-dollar companies give this takeover huge complications in vision and direction. I'm sure existing Business Objects customers will be watching closely.

Now that leaves Cognos to be purchased by EMC, HP or IBM. Any friendly bets on who takes Cognos to new heights?

Beating them at "their" game

Hearing someone talk about one of the big BI vendors usually doesn't grab my attention. Except when I hear Cognos being coined the "Elder Statesman of the Canadian software sector"! I was a little shocked that Cognos was considered in those ranks so I listened. Here are a few tid bits:

Meaning: Elder Statesman is any influential person (company) whose advice is highly respected.

Rob Ashe: "almost all of our sales are from outside of Canada."

Acquisition: Cognos has the "intention" to acquire Applix for $339 Mln. The deal has been compared to the Hyperion acquisition by Oracle, in part, because of Applix's financial focus.

Rumour: Business Objects is on the block to be acquired. Oracle passed them up, so now who else is looking for BO?

Looking Over their Shoulder: Cognos. Trying to grow fast enough organically and through acquisitions to hold off the wolfs.

How?: Through license sales, services, and maintenance fees. This year Cognos earned 12% growth on license fees, with revenues of $252 Mln and $87 Mln in license sales.

Cognos is established and was conceived in 1979 where they started with their PowerHouse product; then produced reporting and analytical products, and now flog a full suite of web-based BI tools.

Oddly they recently partnered with Informatica where the deal has Cognos reselling Informatica products. I can only conclude that Cognos is admitting their ETL tool is substandard - why else would you resell a competitor's product? (Great for Informatica who doesn't really have a home.)

In my opinion and all that being said about elder statesman status, Cognos still has high-end license fees and products with a plethora of functionality (does anyone use half the buttons in PowerPlay cubes? Really. Honestly.)

I'm still looking for BI tools with ease of use for business users who don't have to be techies at heart. However today, is anyone beating Cognos at "their" game?