Anatomy of a Data Warehouse Project
My wife enjoys the TV drama Grey’s Anatomy, and admittedly, I’ve gotten sucked in. However, there is a reoccurring theme with this, and I suppose all, TV dramas of the medical variety. It goes something like this:
Normal Day → Massive Calamity → Dramatic Rescue → Normal Day
If it’s Grey’s Anatomy, throw in a few romantic conflicts, and you’ve got yourself a show. The formula works because everyone loves a drama, as long as it’s not happening to them.
But what if it is? Unfortunately, high drama happens frequently in data warehousing projects because there is a lot at stake: Time, Money, Reputations, Business Impact. The list is endless.
As you know, our message at Datawa.re is simple: Focus on Content, rather than Construction. Both are important, but Content is where the business value is at. That said, we’ve discovered a fascinating (and recurring) scenario that we have characterize as a "Massive Calamity."
The Massive Calamity
Just short of a cars going over bridges or killers lurking around the hospital, here are some of the common symptoms:
- The active data warehouse project is being built by an external BI team.
- The project started a *long* time ago (whatever *long* means to you)
- The agreed-to deliverables seem more distant than ever, and weekly meetings are growing tense
- Conversations about the deliverables have become increasingly nuanced and complex
- Delivery dates are being missed, and the project is due to be competed in 90 days
Essentially, these symptoms describe a situation in which the project owner has lost faith in the direction of the team and/or the ability of the team to successfully (read: acceptably) complete the project. Driving all of this are missed deadlines and a looming due date… and the project or business owner is looking for answers.
This is an unfortunate, yet predictable, outcome because traditional warehouse projects that require opposing priorities vying for attention, and something must *give*. If the infrastructure is compromised, then the project is not going to be supportable. If the business data is compromised, then there's going to be revolt by the business users. Compounding this are frequent realizations that drive business requirement changes. It's, admittedly, a tough balancing act... and it's can be challenging to get right.
The Dramatic Rescue
The most common dramatic rescue is the classic "double-down". Throw in more resources, more time, more money... more prayers. But even the most dramatic rescues have their limits… and the next project will suffer the same fate if you don't understand the how to change the plot line of the story.
Changing the plot line starts with changing the opening scene. If you start with a solid data management platform, you can reinvest the time normally spent building-out necessary infrastructure, and instead invest in understanding the data, better implementing business logic, and thinking beyond the documented requirements to provide a more compelling data solution.
Getting back to the Normal Day
If you find yourself in the midst of a Massive Calamity, Datawa.re can help in delivering the Dramatic Rescue if you have certainty about two things:
- You understand and have your source data
- You understand and have the needed business logic
While these two items seem basic, they are the foundation for more warehouse failures than anything else. If you don't have a handle on both, the prognosis for the patient is, well, concerning. However, if you have a understand both items, Datawa.re can dramatically change your project trajectory. Because we table-drive our framework, leverage 60+ proven stored procedures, and utilize a single SSIS package for flow control. Your source data queries can be inserted almost immediately, and your business logic can quickly be added, and you'll find you're back on track with your project in less time than you'd expect.