Enterprise-wide data warehousing, duh

From recent talks with organizations on preferred techniques when building their BI/DW visions, I heard a rainbow of responses. I'm sure there are many schools of thought so I spoke with a friend Dave Hewlett, an architect and BI practitioner for years.

"...In my opinion, anything under the category of “data warehouse” absolutely has to be planned enterprise wide. It really doesn’t matter if you are implementing Kimball or Inmon to be honest. The less you plan the more you have to refactor/rebuild as you develop and grow the warehouse. As it gets bigger the refactoring cycles get larger and larger requiring more and more “rebuild from scratch”. It basically turns your warehouse into an upside down pyramid of growing effort. Quick wins up front result in massive complexity down the road."

This means organizations designing their first data mart or silo from one source system or focusing on a single Line Of Business (LOB) are going to cause themselves additional re-work and budget increases while growing their enterprise BI/DW -- not to mention difficulties keeping up with a changing business.

I equate Dave's analogy to building a house. First an architect does up the plans based on your needs, wants, desires. Then a builder plans out the execution and schedules resources. All this before anything is built. You don't design and build one room in the house, then move to the next room to design & build.

That would be one funky house, if it could even be finished!

So why do organizations design one source system or data mart, instead of a big-picture view? Man, what are they thinking...? Well here is what some are thinking:

• It's easier for me to ask for and receive a smaller initial budget. I can ask for more next year when I've proven our team can deliver.
• My risks of making a mistake are reduced. I don't want to promote enterprise BI to management when I cannot be sure we will be successful.
• I want the vendor to prove themselves first. I've heard of failed BI projects before with license and consulting fees going through the roof.
• The consulting firm suggested starting small, then building it out.

These are responses from a few leaders I've spoken with - legitimate, definitely. In my mind, this thinking is similar to building software.

We all have been affected by this vis a vis monthly software upgrades. A software company builds a basic software product and gets it out the door in a rush. Then hopefully with feedback from customers, they continue to release newer versions, each one inching the product closer to the overall product vision.

Incremental building causing redesign as they go. Customers suffer the pain of not having a final product. In BI, some call this stovepipes. Others would say poor design.

So what works best for BI/DW?

1. Up-front design, best practices for enterprise-wide BI.
2. Or incremental design, budget-easing, risk-adverse data marts towards enterprise BI.

You may want to say, "Tom, we could use Master Data Management (MDM) as a way to tie together (conform) our data marts." I'm sure MDM could help in some circumstances but this does feel like a band-aid approach (unless you designed with MDM in mind from the beginning).

I think you need to design & build based on the business need.

If (or should I say when) your business needs to improve performance, are you concentrating on the entire business or a Line Of Business? If you want silos or reports or cubes or KPIs for one Line of Business, you are only focusing on operational improvements for that one LOB.

You're not looking at how that LOB performance supports the organizational strategy and goals. Which, if that is all you need to do, then you're off to the races. Then go deliver!

But... if you feel this is only the beginning and other LOB's would want something similar... the management team may want performance from their perspective... you may share common dimensions across the organization...

...then I would say you should be looking enterprise-wide design.