Predictive Analytics Q&A

Posted by David Paradis on April 02, 2010

On Wednesday, March 31^st, Unica hosted the webcast, “Top 5 Ways to Leverage Web Data with Predictive Analytics”, which featured Dr. Eric Siegel, president of Prediction Impact, Inc., and the program chair for Predictive Analytics World. Dr. Siegel was joined by our Akin Arikan, Director of Product Strategy.

Together, they presented an introduction to predictive analytics and examples of real-world success, including a one million dollar success story. Attendees also learned how the use of next generation web analytics and predictive analytics to improve their ability to take actions that increase revenue, as well as:

• Improve customer retention
• Facilitate behavior-based ad selection with AB testing data
• Trigger email follow up campaigns
• Target responders avoid adverse effects

Several great questions were asked during the Q&A session, which the speakers did not have time to address. Together, they created a list of FAQs that has great tips on how to get started with Predictive Analytics inlucing which web analytics tools are the most helpful in supporting predictive analytics and improving sales, and much more.

Q: Where should I go to learn or read more about predictive analytics?
A: We’ve set up a “one-stop-shop” to start you off that includes informative, brief, substantive articles, links to training resources, and links to the main online portals: The Predictive Analytics Guide: http://www.predictiveanalyticsworld.com/predictive_analytics.php

Q: If we’re just getting started with predictive analytics, what is our first step, where do we begin?
A: First decide on the application (value proposition) that looks to be the “lowest hanging fruit” for your organization, such as targeted retention with churn modeling. Then see what relevant data is available and plan a pilot “proof-of-concept” analysis to demonstrate the value and potential thereof. The best guidance for these choices is gained by learning more via the resources linked in my answer to the prior question, and/or seeking expert guidance.

Q: Do you need a Ph.D. to use predictive analytics?
A: Definitely not – many of the most technically-savvy “rock stars” in the industry have no graduate degree. But you do need experienced staff with the analytical know-how (regardless of the software you employ). If you are not ready to invest in a full-time hire, know that such experience often is only needed 1) to guide the overall project on a consulting basis, and 2) to perform the core predictive modeling, which itself can entail a relatively limited engagement. For such external resources, many firms such as mine, Prediction Impact, are available; Unica customers often work with The Modeling Agency (http://www.the-modeling-agency.com/).

Q: Are there more success stories I can access about predictive analytics?
A: The conference Predictive Analytics World’s program is packed primarily with exactly such cases studies – see prior conference programs listed at http://pawcon.com/. I’d be remiss not to also mention that the Unica’s webpage for the PredictiveInsight product links to a number of customer success stories with predictive analytics.

Q: Some authors of excellent web analytics books said that predictive analysis is not going to work with web analytics data because the click-stream of individuals includes too many random page views. E.g. people sometimes mindlessly go to and click around websites. Where is the disconnect between this opinion vs. yours?
A: I actually chatting once with one such renowned author, who qualified what he’d blogged to say “it depends.”  Such back-peddling probably makes sense, seeing as the results presented during this webcast and many others’ case studies demonstrate otherwise. Any kind of time-series such as click-streams or purchase history will include data that does not inform the prediction goal, along with data that does; even without time-series data, there are sometimes several hundred variables (attributes, i.e., columns of the training data). Predictive analytics includes techniques to weed out the things that don’t help predict; in fact, that’s one if this technology’s most central functions.

(More technically, this stage is called feature selection – some predictive modeling methods, such as decision trees, do well even without a feature selection stage, since they essentially do that selection as a part of/side-effect of the modeling process.)

Q: I’m noticing much of the “research” is from years prior to the current economic downturn… How has the current economic recession affected these findings?
A: With a new economic climate, as with any contextual change or even just the passing of months, predictive models need to be refreshed and updated over more recent training data. But the performance and business value of the models does not decrease in this new context (it will change on each iteration, but could go either way). In fact, since predictive analytics can serve to cut costs, it is in many ways seen as all the more valuable during tough economic times.  For more details, see my article, “Six Ways to Lower Costs with Predictive Analytics”: http://www.b-eye-network.com/view/12269

Q: Can you explain how “automatically with decision tree” works?  Is this a program?  Do you put inputs?
A: Decision trees are one of many standard predictive modeling methods – others include logistic regression, neural networks, and Naive Bayes. In all cases, existing software tools have these programmed and ready for you — but you need to get your data together first. In a nutshell, the learning data (training data) must be one row per customer, with all the per-customer variables on each row, as well as the target for prediction on each row, such as whether the customer churned, responded or made a purchase.

For more on how the automatic generation of predictive models such as decision trees works, I’d refer you to the training options and articles listed in the Predictive Analytics Guide: http://www.predictiveanalyticsworld.com/predictive_analytics.php

Q: Is predictive analytics only for B2C businesses?
A: No – everything basically works just the same for B2B. In this case you are making similar predictions that apply to a corporate client rather than an individual consumer.  Of course, this often means there are fewer rows of data — a smaller count of (corporate) customers.  But often there’s plenty of data none-the-less — and the per-customer payoff is usually greater for such businesses.

Q: Is there a minimal email list size necessary for employing Lift Analysis?
A: The count that matters is that of the “minority class,” i.e., the thing that happens less often. For example, for response modeling, if there is a 1% response rate in the training data, there must be at least several hundred – or in some cases, depending, at least several thousand – positive examples of a response.  If there are, of course, there’ll then be plenty of negative examples in the data.

Q: What analytics tool does find the most helpful in supporting predictive analytics?
A: There are a lot of considerations in selecting a predictive analytics software solution. I would refer to the resources above such as the training workshops to get oriented.  BTW, Unica’s PredictiveInsight is the shown as a demo during our online training program, “Predictive Analytics Applied” (http://www.predictionimpact.com/predictive-analytics-online-training.html), alongside another very different tool, and a broad survey of solutions is provided.

Q: My questions are focused on the differences between Unica and other predictive analytic engines.  Can I reach out Eric for answers on the differences?
A: Absolutely – my email address is eric@predictionimpact.com.

Q: If you had a website that was getting lots of traffic, but no sales, what analytics would you apply to try and find out why and how to improve sales from the site?
A: Well this question is best first addressed with standard web analytics, before predictive analytics.  But, predictive analytics could help model what makes the difference – although, technically, if there are literally no (or too few) sales, there won’t be enough positive examples from which to learn (not enough rows in the training data).

Q: Once you’ve implemented a model, are there examples where a model can continue to change based on attribute changes?
A: Yes, many methods such as Naive Bayes are amenable in principle to continuous learning/updates as more data comes in. However, in practice, this is very rarely employed, since the implementation is much more challenging (in fact, most software tools are not set up to support it readily), the analytics more constrained in options, and since offline updates over new data is often or even usually just as effective analytically.

Q: How do you develop a predictive model? Or do you have references you can provide to help develop a predictive model?
A: There’s a heck of a lot to learn about how predictive analytics works. We’ve set up a “one-stop-shop” to start you off that includes informative, brief, substantive articles, links to training resources, and links to the main online portals: The Predictive Analytics Guide: http://www.predictiveanalyticsworld.com/predictive_analytics.php.  See also my answer to the question above, “Can you explain how “automatically with decision tree” works?”

Tags: Predicting Analytics