» Dezrann documentation

Synchronization in Dezrann is done through synchronization points linking symbolic musical onsets (now in quarter notes, that will eventually be displayed as musical time), to audio dates (in seconds).

There can be as few as two synchronization points, one at the beginning, one at the end, or many other points (for every bar, for every note…). There can also be sections with constant tempo with only two (or a few) synchronization points, and more synchronization points on ritardando/accelerando passages.

The first and the last synchronization points do not need to be at the start/end of the audio file or the musical time. Indeed, there may be a score related to only a part of the audio file, such as

• a transcription of a jazz solo in the middle of a jazz piece (Weimar Jazz Database);
• a score for one movement (while the audio/video has the full piece);
• a score with repeats: We now synchronize only the first repeat;

or, conversely, an audio file related to only a part of the score.

For rendered audio, audio software can provide such synchronization data, even when the tempo changes. For any other audio/video file, the synchronization can be produced by audio/score alignment methods and/or corrected or entered by human input.

Synchronization can be done in Dezrann or from external data.

Some examples

• Two lieder by Fanny Mendelssohn
  • Dein ist mein Herz (.json file, only a few points)
  • Mayenlied (.json file, points at each beat)
• Bach, fugue in D major BVW 850
  • Track 1 .json, a few points
  • Track 2 .json, points at each beat

Editing synchronization within Dezrann

Once a piece with a score and audio is on Dezrann, the synchronization editor is available through Edit Piece Synchronization.

In the edit mode, any synchronization point can be added or updated, either on the representation of audio or on the score. In the tap mode, one can tap points at regular intervals such as measures or beats.

The modes can be mixed: The user can thus synchronize a score starting with two start/stop points, adding manually a few points when the tempo changes, tap a few sections where the synchronization is more elaborated, and finally correct some of these points as needed.

Practical guide

0. Keep track of your time. It’s interesting to report what is the average time to synchronize.

1. Synchronize Go to the piece, then Edit Piece Synchronization. Correct the sync point at the beginning and the one at the end (and when the score related to only a part of the audio, the “beginning” or the “end” should be put at the relevant position, “in the middle”).

Then you have two options: - You can add a few points at some places in the score, typically at the beginning of each section, especially when the tempo may change

• You can tap bar by bar (or by other increments). In the tapping mode, space creates a new point, and there are the following experimental keyboard shortcuts:
  • i: ignore next point (the audio does not stop)
  • d: delete last tapped point (the audio does not stop)
  • r: retry last tapped point

In both cases - You may want to add some synchronization points when the tempo changes - But… it has not to be perfect… - Read the recommendations below

Many people target a synchronization by measure, not on every note. And, as anywhere in Dezrann, shift+click on the waveform or on the score sets the player somewhere.

2. Save your work Save your work - on the server, Save Sync to Cloud (you must be logged in to your Dezrann account) - (if you want to have a backup on your side, Export to synchro.json file)

If you take part in a collective synchronization effort, all the synchronization files will be taken from the server and stored in reference places (git repositories and/or Neuma).

Recommendations

• On the score, favor sooner rather than later. When an onset time is not very clear in the audio (e.g. voice or string instrument with slow attack), it is better to put the sync point a bit earlier on the score. In the general case, on the score, sooner is better than later. By doing so, you also avoid truncated beginnings when listening to single bars. This also generates a more “natural” vibe: Cognitive studies show that one may better perceive audio-visual synchronicity when the visual stimuli is about 40ms earlier than the audio one (Van Eijk, 2008).
• Choose the best scale. Use global zoom (with your mouse or pad) to visualize the whole score, or the most important parts (e.g. in orchestral pieces, focusing on string sections usually works well). Then, you can zoom in on the audio spectrum panel by clicking in the top right-hand corner.
• Segment your work. Split the main sections of the piece. Work section by section. Edge cases often happen at the transition between sections (ritardando, silence, repeats…). Adapt the tapping duration between sections if they have different tempo or rhythmic signature.
• Do not fear mistakes. In tapping mode, if you miss a timestamp, continue tapping, possibly using i/d/r or otherwise fixing the mistakes later. The mistakes are usually easy to spot in the audio panel, at places where sync points are less evenly spaced. Once you found a mistake, you can simply delete the defective sync point (if the tempo is stable enough, neighbouring sync points will be enough).

Providing synchronization from external data

The synchronization mechanisms are described in (Garczynksi, MEC 2022). Synchronization points are encoded into synchro.json files linking musical onset and audio time/date in seconds:

[
  { "onset": 0,  "date":   1.60  },
  { "onset": 4,  "date":   9.10  },
  { "onset": 8,  "date":  16.18  },
  (...)
  { "onset": 72, "date": 132.38  }
]

Corpora with existing synchronization should create these files and provide a reference to them through the synchro key in a audio source in their metadata/xxxx.json.

As of 2025, a preliminary support of repeats in available (to be documented).

The Dezrann platform is also able to import/export (and download/upload) synchronizations from the Neuma platform using the time-frame model.