mooperformanceassessment

IntroductionReal-world optimization problems most often involve multiple, conflicting objectives. Thus, the development of efficient multi-objective optimizers has become more and more popular lately and a lot of different approaches ranging from quite simple search heuristics to sophisticated evolutionary algorithms are currently known.

Consider a scientists or engineer that has just prototyped a new approach for solving multi-objective optimization problems. Of course, she wants to know whether her own method performs better or worse than those already available, i.e., she needs to assess the performance of her system.

The framework presented here targets the engineer and/or scientists who wants to empirically compare multi-objective optimizers. It is heavily inspired by the performance assessment module of PISA but tries to address some of its main issues that I have had by choosing an object-oriented approach.

Design Principles & IdeasIn general each important core function of the system is abstracted by the definition of an interface:

IClass: Each interface implements this interface, that just consists of one method QString name(). IDataSetReader: This interface abstracts loading of 'raw' objective vectors from any kind of data source, i.e. later on the processing of remote resources or resource bundles might be an interesting an option. Currently a fairly simple implementation just loading a file containing the data is available. IFileNameAssembler: IUnaryQualityIndicator: A unary quality indicator assigns a real number to a set of m-dimensional objective vectors. IBinaryQualityIndicator: A binary quality indicator assigns a real number to two sets of m-dimensional objective vectors. Currently the additive epsilon indicator is implemented and will shortly be followed by an implementation of the hyper volume indicator. IStatisticalSignificanceTest: An interface that allows for different statistical tests. Implementations of this interface are used to verify that differences in (binary) indicator values are significant. IPostProcessor: A high-level interface that allows for the post-processing of assessment data (e.g. indicator values, significance levels), like plot generation, LaTeX-code generation etc..

Please note, that this is currently a playground for my ideas regarding the framework mentioned above. For this reason, the software has to be considered ALPHA :-) So be prepared for quite faulty and unreliable system :-) Of course, if you are willing to help or assist in completing the framework, or have any questions, just drop me a mail at thomas.voss.bochum@googlemail.com

Tags

  performanceassessment multiobjectiveoptimization statistics searchheuristics cplusplus evolutionaryalgorithms qt

Share