Tag Archives: GIS

Where are You on the Analytics Value Chart?

This post is part of a series on how the 9 Laws of Data Mining from Tom Khabaza can be applied to analytics. You can find previous posts here.

Law #8: “Value Law” – The value of data mining results [and analytics] is not determined by the accuracy or stability of predictive models

In analytics, we evaluate accuracy by defining how often a model predicts values correctly.  Stability refers to the ability of a model to predict correctly on a consistent basis when the data sample is changed within a given population. If a model is stable, then the predictions will not change much when the data sample is changed for the same population.  Both accuracy and stability seem essential to the modeling process, so why does the 8th Law tell us that the value of data mining should not be determined by them?

The answer is that the value is not derived from the process, but from the end result. The 8th Law tells us that the value of the data mining process results from 1) the models ability to improve action (more effective business processes) and 2) the models ability to give insight that leads to an improved business strategy (better decisions).

A model that is overly complex may not have as much business value as a model that is less accurate.  The reason for this is that the cost associated with gathering the survey data for the calibration of the model could be very high; whereas a simpler model might lead us to the same business conclusion.  Khabaza encourages us to ask the question: “Is the model predicting the right thing, and for the right reasons?”  This relates back to Law #2 about business knowledge.  We can only determine whether a model is predicting the right thing for the right reasons if we know the context of the business problem that we are trying to solve. In addition, we can only determine the value of analytics if we think about value in terms of risk and reward.


The chart above encapsulates the goal of analytics investment in relation to the value (the desired net return) from the investment.  4CGeoWorks created this chart based on a discussion with Bill Huber, Owner/President of Quantitative Decisions.  Bill and I both strive to guide companies to make good business decisions that are based on good data and solid analytics.  We try to determine where a business falls along the curve and then how to get them to the optimum.

Some companies get lucky.  They spend a moderate amount on analytics and they reap huge rewards.  It is important to remember that there aren’t many companies in this category.  (The probability curve is not in your favor here.)  Getting value from analytics typically takes money and a lot of effort. If a company happens to be lucky in their analytics, then they probably don’t need our services (at least not until Law #9-the Law of Change-kicks in).

There are some companies (mostly very large businesses) that understand the value of analytics and they are willing to spend a lot of money to achieve their goals.  Those who have figured out how to make analytics work for their business have the enviable position of “Everybody Wins!”  Again, the percentage of businesses that fall into this category is likely to be very small.

A survey in 2012 indicated that only about 5% of businesses were using data analytics on a daily basis and considered it to be a core competency despite a majority of businesses using some form of data analytics.  Although that study is several years old, there is no indication that these percentages have changed dramatically. A smaller survey in 2015 of 316 executives of large, global companies found that only 8% of data scientists felt that their use of analytics was “best of breed” despite the fact that 90% of the large companies were using data analytics.

Other large businesses may spend a lot of money, but they don’t get a good return on their investment.  Most likely, they don’t follow the 9 Laws of Data Mining and ensure that the analytics are based upon business knowledge and objectives.  It could be a problem of hiring the wrong people or any number of internal accountability failures that cause the low return on investment.  For many of these companies, they don’t even know they have a problem, so it may be difficult for us to help them. If a company does recognize that they are not getting the best results from their analytics, we can assist in optimizing their approach.

Most small to midsize companies either do not invest in or spend very little on analytics because they don’t fully recognize the value that it could bring to their business.  By only spending a small amount on analytics, the company reduces the risk of a failed analytics project, but as a result they also are likely to derive fewer benefits which may put them at a competitive disadvantage. They could be missing out on returns that far outweigh their investment which would take their business to the next level. A consultant can provide real value for this company in educating them on the potential benefits from analytics and helping them establish best practices to ensure a good return while keeping investments closer to the optimum point.  So, where do you fall on the analytics value chart?

Did Target Miss the Mark? Recent Closings Explained With Analytics

Mega-retailer Target recently announced that it would be closing the doors of eleven locations by February 1, 2015. The company’s press release stated, “The decision to close a Target store is only made after careful consideration of the long-term financial performance of a particular location.”[i] Target’s team of site selection experts have certainly been doing a great job considering their $72.6 BILLION in revenue for 2014. [ii] That being said, what were the root causes behind the closing of these particular stores?

Continue reading Did Target Miss the Mark? Recent Closings Explained With Analytics

Do Your Location Analytics Have Soul?

Hands playing piano. Motion blur

This post is part of a series on how the 9 Laws of Data Mining from Tom Khabaza can be applied to analytics. You can find previous posts here.

The statement that “a piano makes music” is a clear misunderstanding. We all know that the pianist or any other musician is the source of the music and that the instrument is the tool. But in the context of location analytics, this distinction has often been overlooked. With the proliferation of online maps, there is a sense that anyone can create a map and thus every business professional should be an analyst. Business reality is far more complex.

Many location analytics (GIS) implementations fail because they do not have “soul.” Although there can be many contributing reasons, a common fundamental misstep is that some businesses buy software and then fail to put the tools into the hands of a skilled analyst. Even software training may not be enough to overcome the need for an analytical mind that is able to create experiments to combine business knowledge with a properly described problem space and a well-defined model.

A good analyst is like a musician and both are better when they have “soul.” A musician performing a composition interprets the written score and provides the phrasing and dynamics that turn the notes into music. A jazz musician takes this one step further and creates the composition as well as performs the music. The analyst and the musician both define the problem space as well as deliver the “answer” because the result of the analysis must be interpreted in the context of business knowledge, the “soul” of analytics.

How do we create analytics with “soul”? We start with business knowledge and then we generalize or extrapolate from that initial understanding to identify a model that will show us new patterns and insight. Using statistical inference, we assume that a model will produce reasonable answers when it has been applied to a well-defined problem space. We might also assume that if a market analysis methodology is well defined then it should be general enough to be applied across a wide range of market scenarios. But a model that performs well is dependent upon using business knowledge to match the modeling procedures to the problem and we only find a model that performs well through experimentation and exploration of the data. This is the fourth law of data mining:

Law #4: “No Free Lunch for the Data Miner” – The right model for a given application can only be discovered by experiment

David Wolpert and William Macready developed the “no free lunch theorem” for search and optimization techniques. The basic premise is that any two algorithms are equivalent when their performance is averaged across all possible problems. Because of this, it is necessary to build a foundation of problem specific (business) knowledge in order to build the right model for a given problem.

This law highlights the iterative nature of data mining and analytics. Experimentation and iteration are necessary because the problem is not often well understood. If it were, then the analytics would not be necessary. The value of the analysis is that it allows us to uncover things that were previously unknown. It allows us to make connections and associations that we didn’t know existed (and we may not have even known to look for them). Sometimes the proper hypothesis can only be created after a series of experiments and exploratory or descriptive analytics have been completed.

A problem space may not be known or there may be multiple problem spaces and each one needs its own model. Thus, we might start with one hypothesis that we think is valid, but when we learn more about the entire landscape we may have to change both the initial business goal and our evaluation of the results. We can even change the problem space by the way that we do our data preparation work. Because the model results must be evaluated based on business knowledge, we may even find that we need to re-state the business problem after some initial analysis.

Avoid the “God Complex”

Tim Harford is also a proponent of the experimentation or “trial and error” method. To those who think this is merely pointing out the obvious, he reminds us of the danger of the “God complex” that can result from someone believing they already have a complete understanding of the problem space and therefore do not need to embark on the study or explore other models. Analysts must have an open mind. They must seek patterns even when they think they might already know what the pattern will be.

Khabaza notes that there are some cases where the body of knowledge has been well researched and modeled. There may be cases where the business goals do not fluctuate from year to year and where the data is relatively stable so that an acceptable model can be re-purposed year after year. In these cases, the “no free lunch” law may be less important. However, this “free lunch” must be seen as temporary in order to avoid the “God complex”.

Practice Creativity – Trial and Selection

Tim Harford also explains that sometimes a problem is so complex that the only way to generate a successful solution is through the evolutionary process of trial and error, or what we might better term “trial and selection.” This process involves identifying which parts of the model are working and keeping those parameters while varying other parameters until the best model is found. Essentially, this is what a jazz musician does during practice—the trial and selection of combinations of notes. It may seem that a jazz musician simply creates on stage, which to a certain extent, they do, but the combinations are based on many hours of learned sequences (could we say learned creativity?) that can be subject to variation. Without the hours of practice time to gain proficiency in both the technical aspects of the instrument and the selection of chord sequences, no musician should expect to deliver music with “soul.”

Similarly, in order for an analyst to improvise with “soul,” the analyst must practice creativity. The analyst must be able to generate creative questions and then devise innovative experiments and models to test various options to find the best model. Analytics with “soul” have a combination of skill, experience based on trial and selection, and creative combinations of ideas. Those who are content to simply “follow the music” lack a fundamental component of successful analytics.

Sculpting Your Clay: Analytics Lessons from the Amazon

3f02402This post is part of a series on how the 9 Laws of Data Mining from Tom Khabaza can be applied to analytics. You can find previous posts on the 4C Blog.

I grew up in the Amazon. (How and why is a topic for another forum.) Today, I am an analyst. Although there are few similarities between these two worlds, some principles are common to both.

Continue reading Sculpting Your Clay: Analytics Lessons from the Amazon

Brand Rescue

Brand Rescue?

Feature artwork provided by Tim Higgins.

Many may be familiar with the television show “Bar Rescue” currently in its 3rd season on Spike. If you are anything like my fiancé, you find yourself sucked into the difficulties these business owners find themselves in before host Jon Taffer kicks in the doors and begins making changes. In any given episode, Jon does some research by sending “secret shoppers” to get information on the bar. Based upon these initial observations, Taffer then meets with the owner(s) and staff to discuss his findings, and to describe the specific changes that he insists must be made (e.g., management, customer service, work ethic, cleanliness) for it to become a surviving and thriving business. He also examines the bar’s financial records to find possible cost savings. *Some Information taken from Wikipedia

Continue reading Brand Rescue?