M2R Performance Evaluation
Table of Contents
Sitemap
Performance Evaluation
General Informations
The coordinator for these lectures is Jean-Marc Vincent . The lecturers are Jean-Marc Vincent and Arnaud Legrand .
The next lecture will be on Friday 21/10 in room H102 from 13h15 to 16h30.
Objectives
The aim of this course is to provide the fundamental basis for performance evaluation of computer systems. Two approaches are developed:
- performance measurement: based on experimental platforms (benchmarks or owner instrumented code execution), how to analyze data and synthesize performance indexes
- performance modeling: from a description of resources and the behavior of applications, how to predict the performance of the application
Program and expected schedule
Friday 14/10/2011 (13h30-16h45): Jean-Marc Vincent
Introduction, performance indexes, data analysis, modeling and inference.Documents:
slides EP_01_Introduction.pdfReferences:
- Preliminary guidelines for empirical research in software engineering Kitchenham, B.A.; Pfleeger, S.L.; Pickard, L.M.; Jones, P.W.; Hoaglin, D.C.; El Emam, K.; Rosenberg, J.; Software Engineering, IEEE Transactions on , Volume: 28 Issue: 8, Aug 2002 Page(s): 721 -734
- Partie I : R. Jain, The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling Wiley- Interscience, New York, NY, April 1991.
Friday 21/10/2011 (13h30-16h45): Arnaud Legrand
Measurement on computer systems (benchmarking, observation, tracing, monitoring, profiling).Documents:
slides EP_02_Measurements.pdfReferences:
- David J. Lilja. Measuring Computer Performance: A Practitioner’s Guide Hardcover. Cambridge University Press, 2000.
- R. Jain, The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling Wiley- Interscience, New York, NY, April 1991.
Friday 4/11/2011 (13h30-16h45): Arnaud Legrand
Introduction to discrete event simulation and emulation of computer systems. The simulation crux: validation. Introduction to network topology generation.Documents:
slides EP_03_Simulation1.pdf, EP_03_Simulation2.pdf, EP_03_Simulation3.pdfReferences:
- Banks J., J. S. Carson and B. L. Nelson 1996. Discrete-Event System Simulation, second edition. Prentice-Hall, Englewood Cliffs, NJ. GORDON G. 1996.
- R. Jain, The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling Wiley-Interscience, New York, NY, April 1991.
- On Traffic Phase Effects in Packet-Switched Gateways by Sally Floyd and Van Jacobson.
- Difficulties in Simulating the Internet by Sally Floyd and Vern Paxson.
Monday 7/11/2011 (13h30-16h45): cancelled session
Work to do:
Read the following document and make a 1-page checklists for designing/conducting/analyzing your experiments.- Other suggested readings: How to read an article? and How to write a review?
Monday 14/11/2011 (13h30-16h45): Arnaud Legrand and Jean-Marc Vincent
Visualization of distributed systems and Workload generationDocuments:
visualization slides EP_04_Visualization.pdf, workload generation slides EP_04_Workload.pdfReferences:
- Lucas Schnorr's PhD thesis: Some Visualization Models applied to the Analysis of Parallel Applications
- Again, R. Jain's book.
- Books by Edward Tufte
- GGen: Random DAG Generation
Friday 25/11/2011 (13h30-15h00): Arnaud Legrand
Introduction to design of experiments.Documents:
slides EP_05_DoE.pdfReferences:
- Again, R. Jain's book.
- Design and Analysis of Experiments, by Douglas C. Mongtgomery
- One-Factor-at-a-Time Versus Designed Experiments by Veronica CZITROM
- There are three kinds of lies: lies, damned lies, and statistics (Benjamin Disraeli ?). Two articles on the (mis)-usage of statistics in medical science: Lies, Damned Lies, and Medical Science and Do We Really Know What Makes Us Healthy?
Monday 28/11/2011 (14h00-15h30): Jean-Marc Vincent
Bibliography
- R. Jain, The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling, Wiley- Interscience, New York, NY, April 1991.
- Jean-Yves Le Boudec. Methods, practice and theory for the performance evaluation of computer and communication systems, 2006. EPFL electronic book.
- David J. Lilja, Measuring Computer Performance: A Practitioner’s Guide, Cambridge University Press 2005
- R. Nelson, Probability stochastic processes and queuing theory: the mathematics of computer performance modeling. Springer Verlag 1995