Using Benji to systematically evaluate model comparison algorithms

Model comparison is a critical task in model-driven engineering. Its correctness enables an effective management of model evolution, synchronisation, and even other tasks, such as model transformation testing. The literature is rich as concerns comparison algorithms approaches, however the same cannot be said for their systematic evaluation. In this paper we present Benji, a tool for the generation of model comparison benchmarks. In particular, Benji provides domain-specific languages to design experiments in terms of input models and possible manipulations, and based on those generates corresponding benchmark cases. In this way, the experiment specification can be exploited as a systematic way to evaluate available comparison algorithms against the problem under study.