To deliver something in BI, one has to write a SQL query or create a cube. But doing this is actually a two step process.
1. you must find the data and the relationships between tables, if they exist.
2. the data usually needs filtering, grouping, and calculations and other things that change it from a list of data, to something that you make decisions with.
Maybe the SQL or MDX can do it, but more than likely, it can’t. If it can’t, then you must design intermediate tables and views and run a series of processes to populate them. If you’re going for a cube, a cube needs to be built or an existing one modified, and this might even be on top of those intermediate tables and views.
The costs of doing all this can be significant, but that’s just a start. The IT deparment will never know when it can drop support for these tables, views, and cubes so it’s probably going to be a permanent investment.
That’s the crux of the problem: The nature of analytics often means that the need evolves. It’s either because the analytics identified the problem or the analytic led to other questions. By their nature, analytics are iterative and sequential. For example:
-
I want to see the fastest growing products but it's more complicated than just telling me one measurement.
-
for each region, I want to see which products were growing faster than the rolling six month average, at least six times out of the last N periods.
-
next, again for each region, of those products that were just discovered, which ones grew by more than twenty five percent from the start of the period to the end of the period.
-
I want to know the intersection of those two sets of products across all regions.
-
By the way, I won’t need these questions answered until next year at this time, although I might change them a bit.
That’s an example of one analytic combined with another, iteratively and sequentially. Neither of which would be possible in MDX or SQL without the creation of a complex underlying data structure. Notice that this isn’t simply an OLAP “drill down” or a “slice and dice”. These are distinct questions, with the answers needing to be merged and intersected. This kind of query would take only a few hours in nextanalytics, or a few months in a BI tool.
The problem for the company who wants to use BI is this: Once those products have been identified, the analytic request isn’t needed for another year. When the request comes through next year, it might be a very different set of questions, so the IT money that’s been spent is wasted. It might even have become part of the infrastructure and can’t be removed without great expense or risk. In the long run, an IT budget is eroded by most analytic requests unless they are the trivial sort you can get from an existing OLAP cube.
The real question is: Why incur such a significant up front and ongoing cost for something that isn’t regularly needed and might need dramatic changes the next time it’s requested? In my opinion, a business intelligence approach is not really a very good fit for analytics, unless it is very clear that the analytic question is permanent and static. Even then, it’s entirely possible that the implementation and infrastructure cost is too high to justify many early stage analytic requests where they're not even sure where the questions will lead yet.
Upcoming blogs will consider the roles of dashboards, charting components, spreadsheets, and writing analytic software from scratch.
