Prediction versus Causality – What Machine Learning can do and what not?
In recent years, we are witnessing a big boom in machine learning methods. Deep learning enables us to algorithmically write harmonic music or beat the greatest masters in chess.
Machine learning methods are excellent for detecting patterns and forming predictions in a given environment. However, business often needs to know which environment leads to a desired behaviour. For that one needs to answer causal questions: what is going to happen if we change the environment, possibly for states that were previously unobserved?
This talk will highlight the differences between predictive and causal questions, and talk about the possible techniques to solve each. I illustrate the main points on one of our projects, personalised incentive recommendation, where we needed to solve both predictive and causal problems.
Lead data scientist, Emarsys
János Divényi is a PhD candidate in economics at the Central European University (CEU) who works as lead data scientist at Emarsys in Budapest. He writes code in R (and Python), likes to think carefully about causality, and seeks intuitive understanding of complicated stuff. He is an occasional speaker of the local R meetup, and has more than 5 years’ experience of teaching from various institutions (CEU, BME, MCC).