ETH Zuerich - Homepage
Systems Optimization (SOP)

Wichtiger Hinweis:
Diese Website wird in älteren Versionen von Netscape ohne graphische Elemente dargestellt. Die Funktionalität der Website ist aber trotzdem gewährleistet. Wenn Sie diese Website regelmässig benutzen, empfehlen wir Ihnen, auf Ihrem Computer einen aktuellen Browser zu installieren. Weitere Informationen finden Sie auf
folgender Seite.

Important Note:
The content in this site is accessible to any browser or Internet device, however, some graphics will display correctly only in the newer versions of Netscape. To get the most out of our site we suggest you upgrade to the latest Netscape.
More information

ETH Zürich - D-ITET - TIK - SOP - PISA
print page
this webpage might no longer be updated more...

Performance Assessment


Performance Assessment

R-Indicator Revised

A revised version of the R-indicator implementation of the performance assessment package published.

According to the Principles of PISA, with PISA PA, one can reliably compare different algorithms on benchmark problems using appropriate indicators and statistical methods. To this end, the communication between the Variator and Selector module is controlled and observed by a Monitor module. The tool flow is depicted in the following figure:

performance assessment

The performance assessment tool suite

Authors: Carlos Fonseca, Joshua Knowles, Lothar Thiele and Eckart Zitzler.

The following package contains the complete binaries for Solaris, Linux and Windows, the source code and some documentation of the individual tools: Performance Assessment Package (tar.gz) or  zip (PLEASE NOTE: The R-indicator implementation contained an error in a previous version.).

Tutorial on Performance Assessment

This tutorial reviews the current state of the art in statistical performance assessment of stochastic multiobjective optimizers, derives general recommendations, and demonstrates the implementation of the suggested guidelines on the basis of a case study (pdf):

    author = {J. Knowles and L. Thiele and E. Zitzler},
    title = {A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers},
    institution = {Computer Engineering and Networks Laboratory (TIK),
    ETH Zurich, Switzerland},
    year = {2006},
    type = {214},
    month = feb,
    note = {revised version}

The tutorial was given at the Third International Conference on Evolutionary Multi-Criterion Optimization EMO 2005 in Guanajuato, Mexico (pdf).


The part of the performance assessment package which is responsible for starting multiple runs and for recording the non-dominated front through the optimization run is called the monitor. In particular, the purposes of the monitor tool are the following, see also Principles of PISA:

Therefore, the monitor is useful also independently of the performance assessment in order to automatically invoke and report optimization runs.

© 2009 Institut TIK, ETH Zürich | Imprint | Last updated: Mon, 15 Jun 2009 18:01 | Valid XHTML 1.0! Valid CSS! Valid XHTML 1.0