» Dezrann corpus/developer documentation

Encoding music annotations with .dez files

Dezrann uses json files to encode an “analysis”, that is a set of labels describing music annotations. It is intended to be small (usually a few kilobytes). Ideally, it refers to the music notation, referring only to (symbolic) musical time and being independent of any audio file. The .dez format makes no assumption on the annotation concepts, they are left to the user.

Here is such a .dez file:

{  
  "labels": [
    { "type": "S", "start": 8.5, "ioi": 7.5, "staff": "1"},
    { "type": "CS1", "start": 8.25, "ioi": 7.75, "staff": "2"},
    { "type": "S", "start": 0.5, "ioi": 7.5, "staff": "2"},
  
    { "start": 8, "comment": "Second voice enters"},
    { "start": 16 },  
    { "type": "Pattern", "start": 16.5, "ioi": 1.5, "tag": "h", "layers": ["predicted"]},
    
    { "type": "Structure", "start": 16, "ioi": 8, "tag": "codetta"},
    { "type": "Pedal", "start": 114, "duration": 10, "tag": "Tonic pedal"},
    { "type": "Cadence", "start": 64, "tag": "v:PAC"}
  ],
  
  "meta":{
    "date": "2021-01-06T13:50:38.306Z",
    "producer": "Dezrann c6c80150",
    "title": "Fugue in C minor",
    
    "layout": [
      { "filter" : { "type": "Structure" }, "style" : {"line": "bot.1"} },
      { "filter" : { "type": "Pedal" }, "style" : {"line": "top.1", "color": "#bbbbbb"} },
      { "filter" : { "type": "Harmony", "tag": "I" }, "style" : {"color": "#ff7777"} },
      { "filter" : { "type": "Cadence"}, "style" : {"color": 'by-tag'} },
      { "filter" : { "type": "Pattern", "layers": ["predicted"] }, "style" : {"line": "top.2"} }
    ]
  }
}

Another example:

{
 "labels": [
    { "type": "level-1", "tag": "a", "start": 0, "duration": 1},
    { "type": "level-1", "tag": "b", "start": 1, "duration": 1},
    { "type": "level-1", "tag": "a", "start": 2, "duration": 1},
    { "type": "level-1", "tag": "b", "start": 3, "duration": 1},

    { "type": "level-2", "tag": "K", "start": 0, "duration": 2},
    { "type": "level-2", "tag": "K", "start": 2, "duration": 2},

    { "type": "level-3", "tag": "Z", "start": 0, "duration": 4}
 ]
}

An example with lyrics, with line breaks:

{
  "labels": [
    { "type": "lyrics", "start":  6.0, "tag": "Bräch" },
    { "type": "lyrics", "start":  8.5, "tag": "te "},
    { "type": "lyrics", "start":  9.0, "tag": "dich "},
    { "type": "break",  "start":  9.0, "tag": "line"},
    { "type": "lyrics", "start": 10.5, "tag": "mei" },
    { "type": "lyrics", "start": 11.5, "tag": "nem "},
    { "type": "lyrics", "start": 12.0, "tag": "Arm "}
  ]
}

Other examples: (TODO, put LouisC examples with layout block) - Fugue BWV 847 - Die Wetterfahne, from Schubert Winterreise Dataset

labels: list of annotations

The labels list is the only mandatory part of a .dez file. Each label may have the following attributes:

A label with no ioi neither duration is supposed be “instant” (and will be displayed without any width). The ioi may allow better comparison between perceptually similar labels, however the duration is closer to what people usually mean by “duration”.

A label can have both a duration and a ioi. The conversion between the two is sometimes ambiguous, in particular in a polyphonic context (see dezrann#846).

Note that a label can also have graphic attributes (such as line), but this is generally not recommended: One should use instead the meta.layout block as described below.

Thus the labels list should be as independent as possible from any actual implementation of Dezrann or any other visualisation platform.

Labels can be exported from music21 with our music21.schema package. (2022: This has to be updated/replaced with the output of the RepresentingHarmony group).

Musical time

start, duration and ioi are expressed in musical time

Musical time - is usually measured in quarter notes from the beginning of a piece. In Dezrann, they are displayed, by default, in measure/beat/offset, the goal being to adhere as closely as possible to “common musicological practice” on score-based music; - may be any other integer/float representation. For example, “4.65” could indicate “65% of the page 4” in a graphical score - and, when no symbolic source is available, can default to a wall-clock/audio time in seconds

In all the cases, the musical time serves as the central reference for synchronization (Figure 1). For metered music, basic time signatures can be encoded with time-signature (default 4/4), time-signature-upbeat (default 0). Flexible meters are encoded through measure maps.

TODO: Can be encoded as measure/beat/offset in .dez ?

meta: analysis and piece metadata

The meta block list metadata attributes of the analysis and possibly of the piece (#899).

meta.layout: label formatting options

The meta.layout block lists rules defining styling options for the labels, as in { "filter" : { "type": "Pedal" }, "style" : {"line": "top.1", "color": "#bbbbbb"} }.

Each rule has

References