Sorting Matches and Constraints

[MarkBeB]

This PR tackles #260 by adding two new features to GIPS; match sorting and constraint sorting.

To enable match sorting, a sorter needs to be added via the API (GipsEngineAPI.setMatchSorter(PatternMatch2MappingSorter)) after initialisation but before the problem is built.
Once all the matches have been collected, the match sorter is called for each mapper, together with the matches that the mapper has collected. The returned list of matches is then added as mappings to the mapper in sorted order.

I have implemented a sorter (PatternMatch2MappingSorterByURI) that sorts matches based on their location within the graph. This means that, regardless of the order in which matches are discovered, they can be sorted in the same sequence each time.

Constraint sorting works in a very similar way. It can be enabled by setting a constraint sorter via the API (GipsEngine.setConstraintSorter(ConstraintSorter)). The constraints are sorted before being passed to the solver. ConstraintSorterByName provides an implementation that sorts constraints by name. However, other implementations could group them e.g. by type.

How to test:

• Choose any GIPS project.

• Note: The meta-model must be EMF. SmartEMF can only be supported once PR Feature: URI Fragments (SmartEMF) eMoflon/emoflon-core#161 has been merged.

• Ensure that the LP path is set. The goal is to generate multiple LP files.

  • To do this, change the LP path slightly; do not rename the files, otherwise tracing won't work for the other files.
  • Run GIPS a few times (1–3) and change the LP path after each run.

• We compare the files to see how each run creates a slightly different LP.

  • By opening the input model, we can also see that the same variable across all LP files can represent different nodes (matches) in the input.

• Add both sorters.
  api.setMatchSorter(new PatternMatch2MappingSorterByURI());
api.setConstraintSorter(new ConstraintSorterByName());

• Again, run GIPS multiple times to generate one or more LP files.

  • Remember to adjust the LP path after each run.

• Compare the newly generated LP files with each other.

  • Ideally, they should all look identical. Furthermore, opening the input model should show that each variable represents the same nodes in all LP files.

[maxkratz]

Thank you, @MarkBeB, for your effort in implementing this feature.
I tried it (again, with an adapted MdVNE runner: Echtzeitsysteme/gips-examples#120) but, unfortunately, was unable to generate identical *.lp files. May this be regarding the issues you found in the GIPS mapping indexer?

What I tried:

• Replacing all parallelStream in the mapping indexer(s) with stream. This might also be necessary in order to get a fully deterministic build. Another way to circumvent this would, e.g., be the possibility to completely disable all mapping indexers - the more I think about it, the more I like the idea that we add a switch for this.
• Not rebuilding the GIPSL file/project after changing the LP output path and, instead, using the API to configure new output paths.

Do you have an idea?

[MarkBeB]

Standing issues:

• The LHS terms of constraints are not built in a deterministic way; the process that builds them uses parallelStream whenever possible. To 'fix' this, we could adjust the code generator so that it uses either stream or parallelStream depending on a flag in the preferences, or change the code so that we can switch between parallel and serial processing at runtime. (e.g. a boolean flag).

  • org.emoflon.gips.build.generator.templates.GipsAPITemplate.generateValueAccess(ValueExpression, boolean)
  • org.emoflon.gips.build.generator.templates.ProblemGeneratorTemplate.generateValueAccess(ValueExpression, boolean)
  • org.emoflon.gips.build.generator.templates.ProblemGeneratorTemplate.generateAttributeExpression(AttributeExpression)

• The order in which pattern matches are transformed into constraints is possible not deterministic either. The current process involves obtaining a list of matches per pattern via the eMoflon-API. But afaik, matches are added to the list in the order they are discovered, which is not deterministic.

  • org.emoflon.gips.core.gt.GipsPatternConstraint.buildConstraints()
  • org.emoflon.gips.core.GipsConstraint.calcAdditionalVariables()

• For some reason, running constraints.values().parallelStream().forEach(constraint -> constraint.calcAdditionalVariables()); in parallel can sometimes affect the order of the constraints' terms in the Gurobi model later on. For example, the linear expression -a + b ≥ 0 may later turn into b − a ≥ 0. I couldn't figure out how to retrieve an expression from the Gurobi model to check at what point this (sometimes) happens. My best guess is that this could be related to the order in which variables are defined in the Gurobi model.

  • org.emoflon.gips.core.GipsEngine.buildProblem(boolean, boolean)