Algorithms for music scores comparison and difference visualization

We display here some experiments with a tool for comparing XML-MEI music score files (it aims to be an analog for score files of the Unix diff tool for text files). There are two possible modes for performing the comparison:

A melodic mode, in which scores are interpreted as sequences of notes (triplets of pitch, onset and duration, like in a MIDI file) which are compared. Somehow, this mode detects differences on how the scores sound (during a MIDI play).
A syntactic mode, in which the engraving (graphical) content of scores are compared (including ties, beams, dots, accidents etc). This mode detects differences on how the scores look.

Rameau Dataset

The MEI scores in the dataset were produced by members of the laboratory *REMOVED FOR ANONYMITY* during the *REMOVED FOR ANONYMITY* project. There are built from a corpus of the Bibliothèque nationale de France (BNF, Gallica) containing 21 ouvertures of Jean-Philippe Rameau.
Every entry in the corpus is composed of one page OMRized from the manuscripts, and its manual correction. In our experiments (see the 3 examples displayed below), we compute the diffs between the OMRized version and the manual version, and display them side by side using Verovio. Considering the manual version as a ground truth, the displayed differences can be useful for a fine detection of OMR errors.

Example 1 Rameau: Les Indes Galantes

original manuscript: https://gallica.bnf.fr/ark:/12148/btv1b8451564d/f15.image

Melodic difference

OMRized version

OMR

Manual correction (ground truth)

edited

Remarks about the differences detected (bar numbers refer to the ground truth):

on bar 2, part 4: the pitches of the notes are not the same due to a missing clef in OMR version (the change is marked in yellow in both versions)
on bar 6, part 1: OMR missed a rest which is detected as added (green in the ground truth).
on bar 6, part 4: the first note has wrong pitch in OMR. This is masked as a substitution (yellow). There is also a note missed by OMR at the end of the bar (in green in ground truth).
on bar 7, part 1: rhythmic mistakes in the OMR version.
on bar 14, part 4: one missing note in OMR version (in green in ground truth).

The missed barline between bar 10 and 11 is not marked in melodic mode because it is not a melodic difference. This is also the case of the accidents in bars 15, 16, 18, because the pitches of notes are actually the same in OMR and corrected version (see remark below for the treatment of this passage in syntactic mode).

Syntactic difference

OMRized version

OMR

Manual correction (ground truth)

edited

Remarks (bar numbers refer to the ground truth):

on bar 6, part 1: same differences as in melodic mode for the missing rest.
on bar 6, part 4: same differences as in melodic mode.
on bar 7, part 1: the interpretation of the rhythmic difference differs from the melodic comparison. Note in particular the deleted note and the triplet notation.
the OMR missed the barline between bar 10 and 11 (for all parts). this is interpreted as having the notes of the first bar (first 4 beats) deleted (in red in OMR version) and reinserted (in green in ground truth).
on bar 14, part 4: same missing note as in melodic mode.
in bars 15, 16, 18: the OMR misinterpreted a key signature as an accident on bar 14, and consequently the editor added accidents problems are due to the change of key signature.

The differences of key signature are currently ignored. If a difference of key signature would be detected, then it might seem redundant to mark also the additional accidents because the pitches are the same. However, the syntactic diff tool compares the graphical content of scores, hence an additional accident is considered as a difference (whatever the pitch value).
About the missing barline: the logic of our diff procedure is to consider the parts as sequences of bars, and to perform bar-by-bar comparison based on the content of bars. The barlines themselves are not considered as a content in this procedure. That explains the displayed difference for bar 10.
More generally, following the above logic, the page layout (organisation of bars in staves and of staves in a page) is not compared by our diff procedure. Moreover, the trill marks are currently ignored by our procedure (that will be treated in a next version), as well as the changes of clef or key signature and also text indications.

Example 2 Rameau: Les surprises de l’Amour

original manuscript: https://gallica.bnf.fr/ark:/12148/btv1b84788964/f11.image

Syntactic difference

OMRized version

OMR

Manual correction (ground truth)

edited

The diff tool highlights differences in:

pitch:
- on bar 2, part violin 1
accidents:
- on bars 1, 11, part bass
- on last bar, part violin 2
rhythm (dots & rests):
- on bar 2, part violin 2
- on bars 13, 14, part violin 1
- on bars 13, 14, part violin 2
rhythm (ties):
- on bars 3, 6, part violin 2
missing notes:
- on bar 2, part bass
- on bar 12, part bass

Note that all these differences are both melodic and syntactic differences.
Remark: this example was ran with an option to ignore grace notes.

Example 3 Rameau: Dardanus prologue

original manuscript: https://gallica.bnf.fr/ark:/12148/btv1b8452561g/f7.image

Syntactic difference

OMRized version

OMR

Manual correction (ground truth)

edited

Note the many deletion/insertion of beamings between notes between the OMR and manual version. All these beaming differences are syntactic but not melodic.
For technical reasons, in the actual state of Verovio, we cannot highlight only a part of beaming between 2 notes that was added (see e.g. in part 1, bar 2), and must highlight the entire beam. This is because the entire beaming is a single element in the SVG output that Verovio produces from the MEI score. As future work, we could post-process the SVG output of Verovio for splitting the beaming in multiple parts, in order to highlight exactly (and precisely) the differences.
Note also the (wrong) note added by OMR at the beginning of part 3.

Contact

*REMOVED FOR ANONYMITY*