Don't confuse my simplification of BI with lack of value. BI can provide value when implemented correctly. However if you consider at a basic level, reporting displays information -- whether it's from one or more sources.
Now if I parking lot the "methods" used to bring information to the forefront (i.e. ETL) and focus on the ways customers use the results, I can consolidate this down to:
- reports (including prompted, charts and KPI reports)
- OLAP/cubes
In fact, looking at it that way, the acquisitions of BO, Hyperion and Cognos make sense. The acquirers (IBM, Oracle, SAP) simply needed a suite of reporting tools for their ERP applications. Because why would Oracle want their customers using IBM Cognos or SAP BO as the reporting tool on top of Oracle Financials?
Now let's be honest about the backend, ETL. This is data warehousing and DW has had a troubled history of failures. I would guess there are recent failures too. The question to customers is how much are you willing to spend to provide a finite set of inflexible answers based on today's business?
Plus the costs to maintain a data warehouse can be tremendous. In my opinion, a DW loses value a few months after go-live. I remember doing cost/benefit analysis for a government client to determine the value of paying $1m annually to maintain their DW/BI.
There was value in the information but not enough to warrant the cost. Although it was a System Integrators dream project. They didn't want to finish until every system was in the DW. The customer had a feeling of diminishing returns but waited years to do anything about it.
So is there a better way of integrating information? Well there are many techniques that are becoming mainstream, none of which would consider DW/ETL competition. These new technologies share information in real-time or near real-time (no nightly loads or checking the error log in the morning for failed batches).
And where's the new thinking for such things as integrating with information outside of the organization. Integrating and sharing information with partners. I hope BI can become more like Google. New websites are created daily and it only takes a few days before that information can be searched. And probably mostly done through automation. How long does it take for new data and reports to be provided in your BI/DW?
7 comments:
Tom, I completely agree.
One of the challenges is to get access to the data that's sitting in all of the operational systems and them being able to manipulate it with familiar tools.
I think we should be looking at ways that are already familiar and ubiquitous (i.e. Web, Browsers, Search, Excel, etc.)
I like where this discussion is heading. The BI world is definitely changing. SOA, real-time update, visual analytics, and the SaaS model combine to deliver cost-effective, actionable information to all decision makers. The SaaS model also lends itself to combining data from inside the company with external from supply or delivery partners.
Excellent comments guys. I agree BI/DW needs to change. Any thoughts on how useful (or not) traditional ETL, dimensional modeling will be moving into the future?
Tom
Tom,
I think you're right - Business Intelligence has simply become a fancy name for reporting, and the intelligence part never really materialized.
The enterprise reporting tool vendors had better wake up - much of what they do is easily replaced by either a mashup style toolkit, or open source web application stacks combined with widget toolkits like Dojo. For departmental and focused reporting, a 'situated software' style solution is faster and more cost effective, and there are few reports that are relevant to a wide audience.
As for predicting the end of ETL and dimensional modeling, it will go the way of the mainframe. Many will predict it's extinction, but in reality it will be repositioned to do what it was originally intended to do well.
There will always be a need for a stable, reliable, temporal data source for reporting, whether it's a full warehouse, a data mart, or just a master data reference. But that is just one source of data, and today we have many data sources, not all of which justify inclusion in the warehouse. Those sources will probably become 'on demand' rather than traditional ETL.
Your cost comments on data warehouses mirror my experiences. I believe one of the reasons the maintenance costs get so high is because the data warehouse starts be repurposed into an operational reporting system. Everyone wants their own special reports, now!, and in a month the reports are irrelevant. The warehouse is a large system not designed to evolve at that rate, which results in large incremental costs with litle ROI. On the other hand, companies that resist these short term changes to the DW end up with spreadmarts, which doesn't help either.
This is where the new model of situated applications/enterprise 2.0/BI 2.0 will gain traction - small short term projects with immediate value, driven by a combination of reference and on demand data.
Interesting post about one of Neil's great articles. Prompted me to blog a response/extension.
JT
James Taylor
Author, with Neil Raden, of Smart (Enough) Systems
Mikeyp,
By situated software, I assume you mean focused, locally purposed applets addressing the specific needs of that user community (not scalable by the common definitions). This surely is similarly restrictive to, for example, OLAP - where now you have the designers of the 'situated software' pre-determining data needs. Or, you have the free-for-all that machups allow.
With out data quality, and data source control (sourcing the 'right' data from the 'right' source) - this is headed straight back down the path of the many inaccurate decisions made by factually inaccurate spreadsheet users.
I believe BI is changing, and Web 2.0, BI 2.0, mashups, etc. all have a place in information collection and dissemination. But for a company to ensure that factually accurate information in placed in the hands of the people who need it, you need more quality control than I see in the 'buzz' technologies.
And the fact remains, technology aside, that the biggest challenge for most of these technologies - old and new - is enabling someone who doesn't care about the technology to accurately answer the questions they have.
Small, short-term, immediate are all words traditionally associated with projects which receive minimal, to no quality focus - very dangerous if decisions are to be based on them.
Nigel,
Who has been around since before BI was called BI - indeed when it was called Reporting!
Yes I agree too. The data warehouse is very risky (I think more than 50% of projetcs fail).
Probaly local data marts are better option.
Here are some more my oppinions.
Post a Comment