M2R Scientific Methodology and Performance Evaluation
Table of Contents
Sitemap
Scientific Methodology and Performance Evaluation
General Informations
The coordinator for these lectures is Arnaud Legrand. The lecturers are Jean-Marc Vincent and Arnaud Legrand .
Lectures take place on Thursday afternoon from 3:45PM to 5:15PM, generally in Amphi H of the Ensimag (H building).
Here is a map of the campus for those who do not know where the rooms are.The planning with lecture rooms is available at the ADE website (look for PDS then for Scientific Methodology).
The whole content of the lecture including the slides, the sources of the slides, referencees, and helps on how to use R is available at https://github.com/alegrand/SMPE.
Objectives
The aim of this course is to provide the fundamental basis for sound scientific methodology of performance evaluation of computer systems. This lecture emphasize on methodological aspects of measurement and on the statistics needed to analyze computer systems. We first sensibilize the audience to the importance of experiment and analysis reproducibility in particular in computer science. Then we present tools that help answering the analysis problem and may also reveal useful for managing the experimental process through notebooks. The audience will be given the basis of probabilities and statistics required to develop sound experiment designs. Unlike some other lectures, our goal is not to provide analysis recipes that people can readily apply but to make the audience truly understand some simple statistical tools on which they can build further.
Here are links to the previous editions of this lecture: 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017.
Program and expected schedule
See https://github.com/alegrand/SMPE for details of the lectures and additional resources. You can also check the last edition.
5 October 2017: A. Legrand
Reproducible research12 October 2017: A. Legrand
Litterate programming26 October 2017: A. Legrand
R crash course9 November 2017: J.M. Vincent
Descriptive statistics of univariate data15 November 2017: J.M. Vincent
Data presentation23 November 2017: J.M. Vincent
Introduction to probabilities and statistics, confidence interval30 November 2017: J.M. Vincent
Correlation/causation7 December 2017: A. Legrand
Linear regression14 December 2017: A. Legrand
Linear regression21 December 2017: A. Legrand
Design of experiments11 January 2017: A. Legrand
Homework presentations15 January 2017 (Monday!!!): J.M. Vincent
A bit of epistemology…- Homework: Read the presentation of Karl Popper Science: Conjectures and Refutations.
Question: This text has been written before Computer Science was funded. Explain in at most one page, how Popper's theory could apply in the context of Computer Science research now.
Etudiants Intervenants (accès restreint) Salles Equipements Enseignements Autres Autres (accès restreint)
Homework
As explained during the lectures, now that we have covered the whole spectrum of experimental study (preliminary analysis and visualization, modeling, experiment planning, corresponding analysis), I want you to put this into practice. The best way for this is to work in small groups, pick a topic of your choice and just report your activity and possibly conclude. The topic and the conclusions you will reach are of little importance. What's important is that you do it in a clean way and allow others to look into your work.
Here is a pad (https://pad.inria.fr/p/RiWGRqRW2lnZqdUJ) where you will indicate for each group:
- the names of the members
- the topic in a few words
- and the URL where we can access your work (github, rstudio, …).
The deadline is set to January 19th so that we have enough time to read your work. Note that I have added in this pad a few links that some students returned me a few weeks ago. This is what they did in 5-6 hours max so you should have way enough time to do something pretty clean.