Definition:
Score following is the synchronisation of a computer with a performer playing a known musical score in order to create an appropriate accompaniment.
Basic Description:
According to a.o. Nicola Orio in Score Following: State of the Art and New Developments, during his presentation at the 2003 Conference on New Interfaces for Musical Expression (NIME-03), Montreal, Canada. Orio presented the stages involved in this process schematically by Roger Dannenberg in Recent Work In Real-Time Music Understanding By Computer (1991) as follows:
input processing ----> matching ----> Score location
Whereby the first step is input processing on the performance at which this will be compared with the pre-existing score and the appropriate place within this found (score location).
As picturized by Miller Puckette himself http://www-crca.ucsd.edu/~msp/pdrp/latest/doc/scorefollow/index.htm#sect_scorefollow:
Extended into a more elaborate conception at IRCAM by Orio et all, which continues to insist on the abstract modelling of the score (gestures) through the use of comparing the information with the midi-rendition (Score Following: State of the Art and New Developments):
There is a nice description how this was initially implemented in ISPW-Max/FTS-PD:
Containing the basic elements of:
1-the pitch tracker
2-the score follower
3-the sequencer
This model formed the basis of all score following approaches that were presented at IRCAM and still seems to be based on the assumption of the note-attack state, the note-sustain state and the note-rest state. As put forward by Cont/Schwarz/Schnell in Training IRCAM's Score Follower (2004). In these recent developments there have been the introduction of spectral features in order to track the subsequent stages of the note, but it remains firmly based on the representation of the music as 'score' rather than as 'sound'. Which are both probably, a practical as well an aesthetical decision.
List of score following softwares
- IRCAM works on realtime score following since 1986. (symbolic approach). Software that was originally developed for jMax: the suivi.score~ and the suivi.score objects, as described in: "The jMax environment: an overview of new features" by François Déchelle, Norbert Schnell, Riccardo Borghesi, Nicola Orio, ICMC2000 (ftp://ftp.ircam.fr/pub/ircam/papers-temps-reel/str-icmc-00.pdf), have now been ported to Max/MSP. They work by identifying notes, trills and rests, whether being it pure audio or midi, using a reference from the the similarly named FTM score objects: http://recherche.ircam.fr/equipes/temps-reel/ftm/intro.html Real-time score following has been achieved at IRCAM with wind instruments such as the flute (Noah Noah by Kaija Saariaho, Jupiter by Philippe Manoury, Explosante-Fixe by Pierre Boulez), clarinet and saxophone (Dialogue de l'Ombre Double by Pierre Boulez), trumpet (Metallics by Yann Maresz), with voice (En Echo by Philippe Manoury) and more recently with violin (Anthemes 2 by Pierre Boulez). There is a wide variety of documentation available on the approach by IRCAM at: http://recherche.ircam.fr/equipes/temps-reel/suivi/
- The Schreck Ensemble uses its' own experimental score following software called ComParser (sonic approach). It has been succesfully applied in performances of a piece for violin. The large dynamic range of Poemi Eterni by Hans van Eck however still causes problems: at very silent sections (where the software has to trigger on the last of a series of very soft pppp pizzicati, for instance: it may trigger on any tiny noise,... or, it refuses to trigger at all. Furthermore, recognition is almost purely based on simple spectral match, which makes it too dependant on timbre (ComParser's strength is its' weakness). So the logic of this all should imply that these capabilities have to be expanded, which is now under consideration.
Working Group:
A proposal for the evaluation of score following systems has been put forward by Arshia Cont-UCSD & Ircam- and Diemo Schwarz-Ircam/Realtime Applications Team- and can be found at:
http://www.music-ir.org/mirex2006/index.php/Score_Following_Proposal
A mailinglist at: https://mail.lis.uiuc.edu/mailman/listinfo/mrx-com01
Arie van Schutterhoef
Pieter Suurmond