Friday, September 23, 2016

ECML 2016 tutorial on Bayesian vs. Frequentist tests for comparing algorithms

Tutorial went very well. It was a nice experience and we received very positive feedback. If you are interested in the content please visit this page.

Clinton vs. Trump 23th Sptember 2016


I have run again the Python code that computes the worst-case (red) and best-Case (blue) posterior distribution for Clinton winning the general USA election. using fresh (September) poll-data.    At the moment there is a quite large uncertainty but is still in favour of Clinton: the  probability of winning is between 0.78 and 0.95.  If you are interested in the methodology I have used to compute these distributions please see the past posts. If you want to try yourself, Python code and data-poll from fivethirtyeight.com are available in my github (just click on the links).
 

Friday, September 16, 2016

19 September Tutorial at ECML

Working on the slides for our Tutorial at ECML 2016 (Riva del Garda)  


G. Corani, A. Benavoli, J. Demsar.  Comparing competing algorithms: Bayesian versus frequentist hypothesis testing


Schedule

Time Duration Content Details
09:00 15min Introduction Motivations and Goals
09:15 60min Null hypothesis significance tests in machine learning NHST testing (methods and drawbacks)
10:15 25min Introduction to Bayesian tests Bayesian model comparison versus Bayesian estimation
10:40 20min Break Is the coffee in Riva del Garda better than the coffee in Porto?
11:00 35min Bayesian hypothesis testing for comparing classifiers Single and hierarchical Bayesian models
11:35 55min Non-parametric Bayesian tests and presentation of the results of Bayesian analysis Dirichlet process and how to perform nonparametric Bayesian tests
12:30 10min Summarizing! Summary and conclusions