Benchmark for Versioning Systems
A Benchmark for the Evaluation of Model Versioning Systems
Description | Class Diagram | State Chart | Sequence Diagram | Test Reports
Description
In the following we present a collection of test cases aiming at the evaluation of model versioning systems.
Note that this page is subject to constant updates. If you would like to contribute, please send an email to This e-mail address is being protected from spambots. You need JavaScript enabled to view it .
Class Diagram
Add Different Class
Description: Two different classes (different identity) are added to one diagram
Origin Version: diagram containing one class A
Working Copy 1 (V0'): class B is added
Working Copy 2 (V0''): class C is added
Expected Conflicts:none
Expected Merge Result: diagram includes class B and class C
Add Reference
Description: Two different associations are added between two classes.
Origin Version:Two unconnected classes A and B.
Working Copy 1 (V0'):The association x is added between A and B.
Working Copy 2 (V0''): The association y is added between A and B
Expected Conflicts: none
Expected Merge Result:Both associations are included.
Unit of Consistency
Description: This scenario tests the granularity of the conflict detection.Two properties of an attribute are changed independently.
Origin Version A class Person with the attributes name and birthday.
Working Copy 1 (V0'): The attribute birthday is renamed to dateOfBirth.
Working Copy 2 (V0''):The attribute birthday is set to private.
Expected Conflicts:none
Expected Merge Result:Both changes are included in the merged version.
Rename vs. Move
Description: This scenario tests the consideration of refactorings or the correct handling of a move vs. a rename
Origin Version:A class containing at least one operation.
Working Copy 1 (V0'): The operation is moved into a superclass.
Working Copy 2 (V0''): The same operation is renamed.
Expected Conflicts:none
Expected Merge Result:Renamed operation is in the superclass.
Rename Class
Description:The same property is changed in different manners.
Origin Version:A class with name A
Working Copy 1 (V0'):Class A is renamed to B
Working Copy 2 (V0''):Class A is renamed to C
Expected Conflicts: Concurrent Overlapping Changes. Contradiciting, overlapping modification
Expected Merge Result: User decision is necessary
Containment
Description: A containment relationship is added to a class which is concurrently deleted. This scenario tests whether a delete is correctly propagated to contained elements if the containment is expressed in a language specific way and not physically existing in the model.
Origin Version:The origin model contains two classes (Person and Car)
Working Copy 1 (V0'): Deletion of a class.
Working Copy 2 (V0''):Addition of a (language specific) containment relationship for the class the other modeler deleted.
Expected Conflicts: Concurrent Overlapping Changes. Delete Update Conflict including the mark that Engine is deleted logically.
Expected Merge Result:Only the Person Class is kept or both Car and Engine are kept in the model.
Add Same Class
Description: A class with the same name is added in both working copies.
Origin Version:A single class A.
Working Copy 1 (V0'): A class B is added.
Working Copy 2 (V0''): A class B is added.
Expected Conflicts: Overlapping Concurrent Changes. It should be reported that the same model element has been added.
Expected Merge Result: The merge ends up in model with class A and only one class B.
Contradiction in Hierarchy
Description: This scenario tests whether contradicting intentions of the operations are correctly considered. From a syntactic point of view a naive merge would not report any conflicts.
Origin Version: Two classes with the same two methods each.
Working Copy 1 (V0'):Refactoring: Extract abstract super class pulling up two methods.
Working Copy 2 (V0''): Refactoring: Extract abstract interface pulling up two methods but leaving the implementation in the concrete class.
Expected Conflicts: Equivalent Overlapping Changes. It should be indicated that the overlapping express similar intensions.
Expected Merge Result: User has to be asked which operation should be omitted.
Dangling Reference
Description: One modeler adds an association to a class wheras the other removes this class.
Origin Version:Two classes which are not connected.
Working Copy 1 (V0'): The attribute of the Person class are movd to Passport and Person is removed.
Working Copy 2 (V0''): An association is added.
Expected Conflicts: Syntax Conflict. The merged model would contain a dangling reference.
Expected Merge Result:User has to be asked which operation should be performed.
Inheritance Cycle
Description: The merged concurrent modifications result in an inheritance cycle.
Origin Version:Three classes A, B, C where A is a superclass of B.
Working Copy 1 (V0'): A inherits from C.
Working Copy 2 (V0''): C inherits from B.
Expected Conflicts: Syntactic Conflict. Violation of the metamodel should be reported.
Expected Merge Result: One solution has to be selected.
Contradiction in Reference
Description: Combining the changes of both modelers results in a violation of the metamodel.
Origin Version: The classes Car and Engine are related by an association.
Working Copy 1 (V0'): The multiplicity is set to unbound.
Working Copy 2 (V0''): The composition property is set to true.
Expected Conflicts: Syntactic Conflict. It should be reported that the merged version would contain a composition with an multiplicity higher than one.
Expected Merge Result: One change has to be selected. As the modifications also reflect different intensions of the two modlers (types vs. instances of Engine), a redesign might be necessary.
Inheritance of Methods
Description: Due to concurrent modifications, a new class inherits some methods which it did not inherit in the original model.
Origin Version:A model containing three classes where two classes inhert from a common super class. These two classes contain the same method.
Working Copy 1 (V0'): The method contained in both subclasses is shifted into the superclass.
Working Copy 2 (V0''): A new class is added which also inherits from the superclass.
Expected Conflicts:A semantic conflict. Maybe it is not intended that the new class inherits the shifted method.
Expected Merge Result: Depending on the modeler' intension. No automatic decision possible.
Semantics in Associations
Description: Two modelers express the same information in different ways.
Origin Version:A class person is given.
Working Copy 1 (V0'): One modeler adds an association with role parent and children to express that a Person may have some children.
Working Copy 2 (V0''): The other modeler adds two associations, one for the mother and one for the father of a Person.
Expected Conflicts: Semantic conflict. The same facts (namly that the parents (mother and father) of a Person are Persons and that the children of a Person are Persons.
Expected Merge Result: One solution has to be chosen.
Delete/Update Class
Description: The property of a class is modified and the same class is concurrently deleted.
Origin Version:One class Entity.
Working Copy 1 (V0'): The class Entity is deleted.
Working Copy 2 (V0''): The class Entity is renamed to Person.
Expected Conflicts:Overlapping Concurrent Changes.
Expected Merge Result: One solution has to be selected.
Add/Delete Element
Description:The container of a new element is deleted.
Origin Version:One class Personwithout attributes.
Working Copy 1 (V0'): Class Person is deleted.
Working Copy 2 (V0''): An attribute name is added to Person.
Expected Conflicts:Concurrent Overlapping Changes.
Expected Merge Result: One solution has to be selected.
Sequence Diagram
under construction
State Chart
under construction
Test Reports