Why do so many analytics projects fail ?

Whether you are a seasoned BI user or just getting started making sure your organization understands what will it need to succeed is crucial to avoid missteps and pitfalls. A lot of BI implementations fail. Depending on the survey, between 50-80% of BI projects fail for various reasons. It’s astonishing to see how much time and money is spent building solutions that are under-delivering so significantly! Let’s take a look at some of the reasons behind these terrifying numbers. Most of them have nothing to do with the actual technology used, but rather things like:

• bad communication between IT and corporate
• misaligned expectations
• inadequate training
• abandoning old tools too early
• not being able to adapt to the changes

It does not have to be like this. Successful BI projects require a certain change in the mindsets of users in order to shift from pure intuition-based decision making to a more data-driven one. They also need everyone to come together and collectively decide what success looks like and the path to get there. Let’s take a look at some things you will need to bring along on that journey – to make sure you will not become one of the statistics above.

What will you need for business intelligence success

We think you will need these 10 things to claim business intelligence success in your organization

Let’s break it down one by one.

1. Complete buy-in across the organization

BI is for everyone. All departments, all levels of organization all information workers. In order to be successful, it requires a cultural change – making data and information an integral part of your decision making process. This is why you need a complete buy-in from the leaders of your organization. If you aren’t ready to change – you are not ready for business intelligence.

2. Roadmap and strategy

Applying business intelligence is important – the way you do it matters just as much. Having a BI strategy is a must – no builder will start a new construction without understanding what he is building, for whom and why, without a blueprint, permit, plan and a measuring tape.  You define the strategy in terms of a vision, organization, processes, solutions and then draw a roadmap based on your objectives, priorities, cost, and impact.

3. Access to raw data

Data is the raw material of the digital age. It is being created at an ever-increasing rate all around us and the business world is no exception. In its raw form, it typically reflects transactions, activities and low-level objects or entities. Most organizations will have multiple data systems – both internal and external, some of them they own and some created by vendors. They will all contain data relevant to the business and most likely needed for an effective decision making. Making sure you can identify and access all critical pieces of data is critical. 

4. Robust data integration and modelling process

Raw data in its natural state is very rarely suitable for any kind of analysis. Since it originates at a system responsible for one aspect of a business (CRM, Purchasing, Sales, Finance) it will typically cover a subset of what’s needed for analysis. Having access to millions of records of sales transactions without the product catalog or list of customers will not be helpful at all. Therefore raw data needs to be extracted, collated, cleaned, transformed and organized into a more cohesive and organized shape that reflects the core model and operations of your business.

5. Analytical data store

The analytical data store is a necessary foundation of a BI solution. Its task is to hold all the data that was extracted and collected and provide a high performance one stop shop for analytical workloads. The alternative is to use data directly from the applications that create it. This can create many problems including low query performance, overwhelming the source system, inconsistency and complicated data joining required for every analysis. While it does take time and effort to implement an analytical data store, as the diversity and complexity of data increases and your analytical work become more sophisticated – the need for preparing data for consumption becomes even more pressing and you will be glad you did.

6. Data management and governance processes

The goal of data management is to organize and control your data resources so that they are accessible, accurate, reliable, consistent and timely. In reality this is a set of tools, practices, processes and systems that will help you maintain your data and encompasses the entire lifecycle of a data assets. Data management can include many processes including: data stewardship, master data management, data quality, data security, metadata management and more. After all, if data isn’t treated as a valuable material that it is – it will become stale, corrupt, unstable and ultimately completely useless.

7. Bussines oriented semantic layer

Wikipedia defines a semantic layer as “business representation of corporate data that helps end users access data autonomously using common business terms. It maps complex data into familiar business terms such as product, customer, revenue, profit and offers a unified, consolidated view of data across the organization”.

You can think of it as the alignment of business concepts and terms with the underlying data. This way the end-users can have a complete view of data relevant to them without the need to understand how exactly to construct a query. It essentially decouples data from business representation and ensures consistency and coherency for reporting purposes.

While many of the modern business intelligence tools moved to embrace a world without a semantic layer and are promoting a more direct “data discovery” approach, we believe the idea is still relevant today. While we love these tools and use them daily,  they were designed for discovery and aimed at data analysts and data professionals; not necessarily business users. The fallout means business users need to learn fairly complex tools and techniques, need to pick up skills to get a lot better at building queries and will ultimately create their own definitions and meaning – all over the place.

The bottom line is that you will have to define a semantic layer one way or another. If you don’t do it centrally, your end users will in a whichever tool they are using.  While this is a great way to let users explore and discover new ways to use data, for established and core business intelligence needs it makes sense to centralize ad manage this responsibility. It will increase adoption and eliminate the pain of maintenance, increase accuracy and ultimately free up users to do more analysis. At the end of the day, it is still easier to maintain and update a semantic layer of definitions that to update 100’s of reports.

8. Data dictionaries and metadata

A data dictionary is a list of key terms and metrics with definitions, a business glossary. While it sounds simple, almost trivial, its ability to align the business and remove confusion can be profound.

Most businesses have at least one concept, term, or metric that is used or interpreted differently among teams. When this happens, confusion reigns. Decision makers may disagree about what the data show and what actions to take.

This should align with the semantic layer so you can easily find the data you need, understand the underlying definition and use it in your analysis.  In fact, it should be available as a part of your analysis tool (metadata) so you have access to data, understand the definition and assurance it is calculated the way you expected. This eliminates inconsistency, streamlines analysis and reduces potential confusions thus shortening the decision making process.

9. End-user tools for various analytical and reporting needs

A successful BI solution supports the various analytical and reporting styles of the business executives and information workers using the BI system.  This includes reporting, ad hoc querying, online analytical processing (OLAP), data mining, data visualization, and predictive analytics.

These tools must be designed for business users and easy to learn. One of the easiest ways to frustrate the business community is to give them another tool to learn, manage and interact with. The typical business user is pressed for time and is looking for the most efficient way to do their job. Part of that is finding and using information. The best tools might not be the fanciest ones or the ones with the most features and options for visualizations, but ones that offer practical and easy to use way to access useful, relevant and timely information.

In today’s world of myriads of devices and workforce on the move – it also means that the end user applications will need to be able to support the many devices in your organizations in both offline and online mode.

10. Communication, collaboration and training

This is the last but maybe one of the most important points. When it comes to successfully implementing business intelligence solution, everyone must be on the same page. Many times a solution is selected before collaborating with the end users, which will lead to frustrated end users and a solution that does not meet everyone’s needs and expectations.

Involving the final users of your BI solutions is a critical yet many times a missed step.  User input helps change and evolve the BI strategy as users need change.  Project leaders should identify and classify users and define their needs, then continue to train and engage them as the solution evolves.

Identifying champions and power users within groups will help push usage and adoption and these key users can be critical for receiving the much-needed feedback about what works and what does not. No solution is perfect but knowing what aspects to improve on can prevent the worst possible outcome of all – you users reverting to not using the BI system at all and reverting to the “old way” of doing business.

Implementing business intelligence in any organization successfully is no small feat. There are many pitfalls to avoid and areas to watch out for. The 10 points in this article should be a great start to put the odds in your favor of having your project delivered successfully.