All posts by Susan Zwillinger

About Susan Zwillinger

Susan is a certified Geographic Information Systems (GIS) Professional with almost twenty (20) years of experience in the GIS industry. In her role, Susan consults and develops the scope of each project, coordinating proposal development, designing project plans, developing deliverable schedules, defining the market analytics methodologies, and managing the execution of the deliverables that ensure solid business value.

Change the Way You Look at Change

This post is part of a series on how the 9 Laws of Data Mining from Tom Khabaza can be applied to analytics. You can find previous posts here.

Law #9: “Law of Change”: All patterns are subject to change

change

Photo credit: Tedrafranklin, Shutterstock. CC0 Public Domain

All patterns change — not only because the data changes, but because our understanding of the business domain changes.  As Wayne Dyer said, “If you change the way you look at things, the things you look at change.” (emphasis mine)

For example, when we devise a better marketing campaign after completing a strategic plan to increase customers, we may change the customer profile so that the next time that we run the customer segmentation we may get a different result and those results may trigger other marketing changes. Even when the customer profile does not change, our understanding of competitive offers, substitute products, and other market factors can cause us to change how we decide to implement the business process.  Even when the data pattern is similar, we may have new information about the business or the economy as a whole that will affect how we understand the business and this will affect our evaluation of the model.

The job of a data miner and an analyst is never done—there is always something else to study and a new nugget of truth to learn about a market.  However, we can sometimes fall into the trap of doing things because they have always been done that way. So, how do we get ourselves out of a “rut” in analytics?  How do you repeat a process over and over again while still asking yourself about what could or should change?  The answer lies in having a “cheat sheet” of questions that forces you to think about the analysis in a different way, in other words to change the way you look at the problem. . . or in this case, to change the way we look at change.

This time of year is a great time to think about change, so while we are on the subject of “change”, let’s ask ourselves: How do we measure change? Is there a checklist of things that we should investigate when studying change?  Below are six ways to think about measuring change.

1. How Much?

The obvious and first question that we typically ask is the “how much” question.  In other words, what was the amount of change, the raw total difference from one time period to another?  But don’t stop there!  There is so much more that you can learn about change by asking more questions.

2. Rate of Change: To What Extent? What Percent?

How about calculating the percent change from one time period to another?  This helps us to understand when the change happened and what the extent of the change was. You could also analyze the rate of change over different time periods and ask yourself whether there is a pattern in the rates of change.  If you vary the time period that you are studying, you might get different patterns, like the difference between the average change over months versus quarters versus year over year.

3. What is the Average Change for Multiple Change Rates?

If you have lots of data, you could look at the average change over different time periods and then compare the change rates for one period compared to the difference from the mean or median value.

4. What is the Difference from Optimal?

In some cases, the difference from an average won’t mean much, but the difference from an optimal number or the top performer in a category will give you a lot more insight.  Creating an index is helpful in this case because it makes it easy to see how the current value differs from the optimal or desired amount.

5. Related questions: Where? Causation? Multiple Changes?

Once you have measured change, you can ask other questions that are related to the change, like “Where did the change happen?” or “Who or what factors caused the change?” or “Did multiple changes happen at the same time?”

6. Meta questions: The Nature of the Change

Don’t forget to ask meta questions like: What was the nature of the change? Is it beneficial or detrimental? Was the change an anomaly, an outlier, or part of a larger pattern of change?  And finally, you can ask: How long do you expect or predict the current trend to continue?

We all know that change is inevitable, but how we analyze and learn from change is not.  Understanding the relevance of the changing patterns, what the change means for our business, is how we translate information into insight.  Insight helps us develop a strategy and then we can focus on executing the plan with specific tactics.  However, everything starts with the decision to analyze the change and to look at change in a new way.

 

Where are You on the Analytics Value Chart?

This post is part of a series on how the 9 Laws of Data Mining from Tom Khabaza can be applied to analytics. You can find previous posts here.

Law #8: “Value Law” – The value of data mining results [and analytics] is not determined by the accuracy or stability of predictive models

In analytics, we evaluate accuracy by defining how often a model predicts values correctly.  Stability refers to the ability of a model to predict correctly on a consistent basis when the data sample is changed within a given population. If a model is stable, then the predictions will not change much when the data sample is changed for the same population.  Both accuracy and stability seem essential to the modeling process, so why does the 8th Law tell us that the value of data mining should not be determined by them?

The answer is that the value is not derived from the process, but from the end result. The 8th Law tells us that the value of the data mining process results from 1) the models ability to improve action (more effective business processes) and 2) the models ability to give insight that leads to an improved business strategy (better decisions).

A model that is overly complex may not have as much business value as a model that is less accurate.  The reason for this is that the cost associated with gathering the survey data for the calibration of the model could be very high; whereas a simpler model might lead us to the same business conclusion.  Khabaza encourages us to ask the question: “Is the model predicting the right thing, and for the right reasons?”  This relates back to Law #2 about business knowledge.  We can only determine whether a model is predicting the right thing for the right reasons if we know the context of the business problem that we are trying to solve. In addition, we can only determine the value of analytics if we think about value in terms of risk and reward.

Approach-to-Analytics-Spending---Line-Graph-WITHOUT_Title

The chart above encapsulates the goal of analytics investment in relation to the value (the desired net return) from the investment.  4CGeoWorks created this chart based on a discussion with Bill Huber, Owner/President of Quantitative Decisions.  Bill and I both strive to guide companies to make good business decisions that are based on good data and solid analytics.  We try to determine where a business falls along the curve and then how to get them to the optimum.

Some companies get lucky.  They spend a moderate amount on analytics and they reap huge rewards.  It is important to remember that there aren’t many companies in this category.  (The probability curve is not in your favor here.)  Getting value from analytics typically takes money and a lot of effort. If a company happens to be lucky in their analytics, then they probably don’t need our services (at least not until Law #9-the Law of Change-kicks in).

There are some companies (mostly very large businesses) that understand the value of analytics and they are willing to spend a lot of money to achieve their goals.  Those who have figured out how to make analytics work for their business have the enviable position of “Everybody Wins!”  Again, the percentage of businesses that fall into this category is likely to be very small.

A survey in 2012 indicated that only about 5% of businesses were using data analytics on a daily basis and considered it to be a core competency despite a majority of businesses using some form of data analytics.  Although that study is several years old, there is no indication that these percentages have changed dramatically. A smaller survey in 2015 of 316 executives of large, global companies found that only 8% of data scientists felt that their use of analytics was “best of breed” despite the fact that 90% of the large companies were using data analytics.

Other large businesses may spend a lot of money, but they don’t get a good return on their investment.  Most likely, they don’t follow the 9 Laws of Data Mining and ensure that the analytics are based upon business knowledge and objectives.  It could be a problem of hiring the wrong people or any number of internal accountability failures that cause the low return on investment.  For many of these companies, they don’t even know they have a problem, so it may be difficult for us to help them. If a company does recognize that they are not getting the best results from their analytics, we can assist in optimizing their approach.

Most small to midsize companies either do not invest in or spend very little on analytics because they don’t fully recognize the value that it could bring to their business.  By only spending a small amount on analytics, the company reduces the risk of a failed analytics project, but as a result they also are likely to derive fewer benefits which may put them at a competitive disadvantage. They could be missing out on returns that far outweigh their investment which would take their business to the next level. A consultant can provide real value for this company in educating them on the potential benefits from analytics and helping them establish best practices to ensure a good return while keeping investments closer to the optimum point.  So, where do you fall on the analytics value chart?

Do Your Analytics Create Information Or Insight?

This post is part of a series on how the 9 Laws of Data Mining from Tom Khabaza can be applied to analytics. You can find previous posts here.

Law #6: “Insight Law” – Data mining [analytics] amplifies perception in the business domain

The word Insight on a cork notice board

Data mining tools are “perception amplifiers,” increasing our understanding of the business world. Tom Khabaza uses the term “intelligence amplifiers” from the field of Artificial Intelligence. The process of data mining or analytics reveals patterns, context, and connections that we would not normally be able to identify by simply looking at the raw data. The tools allow us to discover outliers, unknown associations, correlations, clusters, complementary processes, new classifications, and even in some cases, an estimate or prediction of why the pattern is happening or what results the pattern might produce.

Khabaza’s 6th Law of Data Mining is called the “Insight Law.” While data mining and analytics tools can’t produce insight on their own, they can lead us to an effective insight. What do we mean by this? Insight, according to Merriam Webster is, 1) “the power or act of seeing into a situation,” or 2) “the act or result of apprehending the inner nature of things or of seeing intuitively.” For the purposes of data mining and analytics, we can define it this way: “Insight is the act or result of perceiving the value of the information for a business.” An effective insight would be when the insight can be turned into action in order to improve a business situation.

Essentially, the data mining process turns data into information, but this is not enough. We need insight to solve business problems, and this is done by creative people, not by data mining algorithms, GIS, or statistics software. In order to understand how we develop insight, I recommend using Bloom’s Taxonomy which is used in the education field to classify different levels of learning and thinking. The taxonomy is represented as a pyramid with the most basic levels of thinking at the bottom of the pyramid. Thinking skills and learning advance as you move up the pyramid. There are currently two versions of Bloom’s Taxonomy. The original version used nouns. The revised version uses verbs and made some slight changes to the connotations of the levels. Both versions have some merit, so they are shown side by side below.

Blooms Taxonomy

Data mining tools are essentially helping us with the lower levels of the pyramid, but we need to progress to the higher levels of the pyramid to gain insight.  In fact, the data mining process leads us through the entire taxonomy of thinking skills.  First, we feed the model data (Knowledge – facts, things that we know) and these are “remembered” in order to produce information or understanding.  Using our analytics tools we can enhance our understanding of the data to apply the results of the models to our hypothesis.  From there we continue to apply and analyze the results until we have a model that best represents the data and the business context.

However, at this point, our computer tools still have some need of improvement.  In the evaluating level we make judgments about the utility of the model for the business purpose. I also like the word “synthesis” at this level because insight is about making connections.  Computer systems can only make connections within the data that we provide.  On the other hand, our minds have a beautiful capacity to make external connections and these become the true meaning of insight.  Finally, if we can determine a positive value for the analytics (the insight), we can then create the plan of action.

This is not to say that a business that does not employ data mining and analytics will not be successful; only that a business that follows the data mining process will have the capability to identify patterns and act upon them before others in the market.  A startup business can be very successful without any formal analytics, but the longer the business exists, the more likely that it will need to change in order to continue its success.  Knowing how to change and making decisions that will be the most beneficial to the business are the result of the data mining and market analytics processes being used by creative minds.

Of course, the existence of the data mining and analytics will not translate into a guarantee of business success on their own. The results will not be useful unless the human problem solver translates the results into a recommendation for an improved business process.  The insight must be interpreted in the context of the business value and then actions need to be taken to ensure that the proper change to the business process occurs.

Pattern Seekers: The Good, The Bad, and The Ugly

HiResLaw #5: “Watkins’ Law” – There are always patterns [i]

First, let’s talk about “The Good.” David Byrne, writing in the introduction to Gareth Cook’s book, The Best American Infographics, 2013, describes the power of the infographic as:

“…an inbuilt ability to manipulate visual metaphors in ways we cannot do with the things and concepts they stand for — to use them as malleable, conceptual Tetris blocks or modeling clay that we can more easily squeeze, stack, and reorder. And then — whammo! — a pattern emerges, and we’ve arrived someplace we would never have gotten by any other means.”

He could just as easily have been talking about the data mining and analytics process, except that the process is much slower and more methodical than the expression “whammo” suggests.[ii]

Continue reading Pattern Seekers: The Good, The Bad, and The Ugly

Do Your Location Analytics Have Soul?

Hands playing piano. Motion blur

This post is part of a series on how the 9 Laws of Data Mining from Tom Khabaza can be applied to analytics. You can find previous posts here.

The statement that “a piano makes music” is a clear misunderstanding. We all know that the pianist or any other musician is the source of the music and that the instrument is the tool. But in the context of location analytics, this distinction has often been overlooked. With the proliferation of online maps, there is a sense that anyone can create a map and thus every business professional should be an analyst. Business reality is far more complex.

Many location analytics (GIS) implementations fail because they do not have “soul.” Although there can be many contributing reasons, a common fundamental misstep is that some businesses buy software and then fail to put the tools into the hands of a skilled analyst. Even software training may not be enough to overcome the need for an analytical mind that is able to create experiments to combine business knowledge with a properly described problem space and a well-defined model.

A good analyst is like a musician and both are better when they have “soul.” A musician performing a composition interprets the written score and provides the phrasing and dynamics that turn the notes into music. A jazz musician takes this one step further and creates the composition as well as performs the music. The analyst and the musician both define the problem space as well as deliver the “answer” because the result of the analysis must be interpreted in the context of business knowledge, the “soul” of analytics.

How do we create analytics with “soul”? We start with business knowledge and then we generalize or extrapolate from that initial understanding to identify a model that will show us new patterns and insight. Using statistical inference, we assume that a model will produce reasonable answers when it has been applied to a well-defined problem space. We might also assume that if a market analysis methodology is well defined then it should be general enough to be applied across a wide range of market scenarios. But a model that performs well is dependent upon using business knowledge to match the modeling procedures to the problem and we only find a model that performs well through experimentation and exploration of the data. This is the fourth law of data mining:

Law #4: “No Free Lunch for the Data Miner” – The right model for a given application can only be discovered by experiment

David Wolpert and William Macready developed the “no free lunch theorem” for search and optimization techniques. The basic premise is that any two algorithms are equivalent when their performance is averaged across all possible problems. Because of this, it is necessary to build a foundation of problem specific (business) knowledge in order to build the right model for a given problem.

This law highlights the iterative nature of data mining and analytics. Experimentation and iteration are necessary because the problem is not often well understood. If it were, then the analytics would not be necessary. The value of the analysis is that it allows us to uncover things that were previously unknown. It allows us to make connections and associations that we didn’t know existed (and we may not have even known to look for them). Sometimes the proper hypothesis can only be created after a series of experiments and exploratory or descriptive analytics have been completed.

A problem space may not be known or there may be multiple problem spaces and each one needs its own model. Thus, we might start with one hypothesis that we think is valid, but when we learn more about the entire landscape we may have to change both the initial business goal and our evaluation of the results. We can even change the problem space by the way that we do our data preparation work. Because the model results must be evaluated based on business knowledge, we may even find that we need to re-state the business problem after some initial analysis.

Avoid the “God Complex”

Tim Harford is also a proponent of the experimentation or “trial and error” method. To those who think this is merely pointing out the obvious, he reminds us of the danger of the “God complex” that can result from someone believing they already have a complete understanding of the problem space and therefore do not need to embark on the study or explore other models. Analysts must have an open mind. They must seek patterns even when they think they might already know what the pattern will be.

Khabaza notes that there are some cases where the body of knowledge has been well researched and modeled. There may be cases where the business goals do not fluctuate from year to year and where the data is relatively stable so that an acceptable model can be re-purposed year after year. In these cases, the “no free lunch” law may be less important. However, this “free lunch” must be seen as temporary in order to avoid the “God complex”.

Practice Creativity – Trial and Selection

Tim Harford also explains that sometimes a problem is so complex that the only way to generate a successful solution is through the evolutionary process of trial and error, or what we might better term “trial and selection.” This process involves identifying which parts of the model are working and keeping those parameters while varying other parameters until the best model is found. Essentially, this is what a jazz musician does during practice—the trial and selection of combinations of notes. It may seem that a jazz musician simply creates on stage, which to a certain extent, they do, but the combinations are based on many hours of learned sequences (could we say learned creativity?) that can be subject to variation. Without the hours of practice time to gain proficiency in both the technical aspects of the instrument and the selection of chord sequences, no musician should expect to deliver music with “soul.”

Similarly, in order for an analyst to improvise with “soul,” the analyst must practice creativity. The analyst must be able to generate creative questions and then devise innovative experiments and models to test various options to find the best model. Analytics with “soul” have a combination of skill, experience based on trial and selection, and creative combinations of ideas. Those who are content to simply “follow the music” lack a fundamental component of successful analytics.