Friday, November 25, 2005

Overcoming (some of) the BI barriers

A new survey from from Gartner has some interesting findings. Business intelligence in its broadest sense has moved from #10 in CIO priorities to #2, quite a jump. Spending in this area is set to rise sharply, with companies on average spending 5-10% of their overall software budgets on BI, but with some sectors such as finance spending 16% of their software budgets on business intelligence (more can be found in research note G00129236 for those of you who are Gartner clients). This is obviously good news for vendors in the space, but it seems to me that CIOs have been very slow to grasp that providing business insight is surely a high priority for their customers. Once the Y2k madness was over and everyone had rushed in their shiny new ERP, CRM and supply chain systems, just what else was it that CIOs should be doing rather than exploiting the wealth of information now being captured by these systems? CIOs are increasingly under pressure to deliver value to their companies, and what better way to do this than by providing improved insight into the performance of the company's operations? Surely there is more bang for the buck for a CIO here than in fiddling with new middleware or upgrading the network, activities in which most business people have little interest and regard as "business as usual". Anyway, the penny now seems to be dropping, so given it is finally on their radar CIOs should also consider what will stop them delivering this value, with its inherent career-enhancing kudos.

The main barriers to adoption of business intelligence, in Gartner's view, are:
  • a lack of skills
  • difficulty in getting business intelligence out of enterprise applications (e.g. ERP)
  • perceived high cost of ownership
  • resistance due to some managers viewing enterprise-wide transparency as a threat to their personal power
I recently wrote on this last point. Let's consider the others.

The skills one is the easiest: "organizations lack the analytical skills, causing them to have difficulty using the tools." The answer is that most people in a business simply do not need a sophisticated analytical tool. It is not their job to be creating whizzy charts or mining through data - most business managers just need a regular report telling them their key performance information e.g. production throughput, sales figures etc. This requires at most Excel, and probably not even that. As I have argued elsewhere, the answer to BI vendors selling you thousand of copies of their software is simple: just say no, to quote Nancy Reagan. In my experience perhaps 5%-10% of end users of data warehouse applications actually need true ad hoc analytic capability - the rest need a report or at most an Excel pivot table. Putting a complex, powerful but unfamiliar tool on business people's desks and then wondering why usage rates are low is a self-inflicted problem.

The second barrier is the dffifcult in getting BI linked to enterprise applications. This is a real issue, with the big application vendors either providing weak capability or, where they do, tying it too heavily to their own data structures. While there is a place for operational reporting, enterprise-wide performance management requires information from a wide range of sources, some of it in spreadsheets and external data. Obsessed with trying to broaden their own footprint, application vendors seem unable to let go and realize that customers have a diverse set of applications and are not going to simply ditch everything they have from everyone else. The answer here is to adopt an enterprise data warehouse approach that is separate from the application vendors and neutral to the applications. Leave the operational reporting to the app vendors if you must, but at the enterprise level you need something that is truly application-neutral. Cutting this link, and using app vendors analytic tools for what they are good at, rather than trying to shoe-horn them into roles they were never designed for, will save you a lot of pain and suffering here.

The third issue is cost of ownership, and here too the issue is very real. Recent work by The Data Warehouse institute (TDWI) shows that data warehouses have very high cost of maintenance and support. Indeed according to major TDWI survey, the annual support costs are 72% of the implementation costs, on average. For people used to traditional "15% of build" costs this may seem outlandish, but it is not. The reason that maintenance costs are often around 15% of build costs for transaction systems is that this is roughly the amount of code that is impacted by change each year. Most transaction systems are operating in a specific area of the business that does not change radically every week, so much of the application code is stable. By contrast, a data warehouse (by definition) takes as sources many different transaction systems, and every one of these is subject to change. So if you had a data warehouse with six sources, each of which had a 15% degree of change per year, then your warehouse is subject to a 6 * 15% = 90% level of change stress. Of course this is too simplistic, but you can see the general idea: with many sources, each undergoing some change, the warehouse encounters more change issues than any specific transaction system. Hence custom-built data warehouses do indeed have these very high lavels of maintenance costs. There is not a lot to be done about this unless you use a data warehouse design specifically built to address this issue, but unfortunately the mainstream approaches (3NF, star schema, snowflake schema) do not.

So to summary, you can take steps to circumvent at least three of Gartner's four barriers. The fourth one involves tackling human nature, which is a more difficult problem that software design or architecture.

2 Comments:

Anonymous Rajeev said...

I think points 2 and 3 could be minimized if you go for the integrated product i.e. SAP or any other product for that matter. Coming from SAP background as well as having seen some of the blunders, at the end of the day it does make sense to have one software rather than having multitudes of them.

• difficulty in getting business intelligence out of enterprise applications (e.g. ERP) – Having the integrated system does take a load off from retrieving the data. Nothing is 100% done but at least there are tools and standard programs which could bring the data into the BI system.
• perceived high cost of ownership – Cost of ownership also get reduced as now you have one less system/software to maintain. The percentage of cost is absorbed by avoiding the duplication of work and sharing the same resources.

One of the implementation I worked, the client decided to write the custom BI product which was supposed to take over SAP reporting as well as from other systems. The blueprint looked great but when it came to implementing the design, the complexity of getting the data from SAP was tremendous and the problems attached to it. This resulted into reducing the scope of the BI system and instead of doing selective data retrieval they ended up duplicating SAP tables in their database. So at the current time the ratio of reports coming out from SAP to BI is 10:1. And, moreover they have team of few dozen maintaining that system just in case…

3:26 PM  
Blogger Andy Hayler said...

Thanks for your comment Rajeev. I think it is a matter of horses for courses. The analytic functions of the ERP vendors are well suited to operational reporting from their own ERP systems, so I agree that for operational reporting from SAP, BW makes good sense. However remember that SAP only covers 30-70% of a company's business model (that is an SAP figure) so you have 30-70% in other systems e.g. marketing systems or whatever. For certain types of analysis (but not operational reporting) you need to combine ERP data with these other systems, and this is where an enterprise data warehouse makes sense. Just as you probably should not try and shoe-horn an EDW into an operational reporting role (as in your example) similarly you should not try and shoe-horn an ERP reporting system into a role to which it is not suited, that of an enterprise data warehouse. BW is not well suited to dealing with lots of non-SAP data, especially where the business hierarchies are very complex. Both types of systems have their place, and can work well together. This is the case in the majority of our Kalido clients, most of whom are huge SAP users.

1:36 AM  

Post a Comment

Links to this post:

Create a Link

<< Home