VirtualBand is a music interaction system based on style modeling. The idea behind VirtualBand is to capture essential elements of the style of a musician – either solo or comping styles - and reuse these elements for musical improvisation.
VirtualBand proceeds in two steps: a recording and a playing phase. First, selected musicians are recorded using a specific procedure. Musicians are given playing instructions so that their recorded material covers a large range of musical situations, as well as transitions between them. These recording are then analysed using signal processing techniques (mostly borrowed from the domain of music information rsetrieval) to build a style database.
In the playing phase, the user (say, a sax player) first selects a song and a set of virtual musicians (for instance a virtual drummer, virtual guitarist and virtual bassist). When the user plays a music generator uses the style database (for each virtual musician) to produce music that fits with the song structure (harmony, metrics). Most importantly, VirtualBand listens to the soloist and analyzes a variety of factors such as volume, density of notes, pitch and reacts with logical and musical responses in real time. For instance, if the user plays louder, then the generator selects parts of the database that are themselves louder, or contain more notes, etc.
A recent development of the VirtualBand software is the Reflexive VirtualBand.
The idea behind Reflexive VirtualBand is to capture on the fly essential elements of the style of the musician playing with it and reuse these elements for real-time interaction. In this way the musician can build up a duo or even a trio, playing with virtual representations of her/himself and without having to previously "teach" their style to the system.
Check the videos on the right, to see what VirtualBand can do with jazz and beyond.
Video extracts of jazz musicians (Jeff Suzda - sax, François Pachet - guitar) interacting with VirtualBand.
Sax player Jeff Suzda playing Funk with virtual drum
In this video, a simple demo with a joystick explains the basic concepts behind VirtualBand. This video has been presented to SIGGRAPH 2012, Los Angeles.
All the Things You Are by Jerome Kern, performed by VirtualBand and François Pachet (guitar).
Reflexive Interactions in a simple blues.
Jazz standard Stella by Starlight, by Victor Young, performed by VirtualBand, Jeff Suzda (sax) and François Pachet (guitar).
Isn't She Lovely by Stevie Wonder, with François Pachet (guitar) and Reflexive VirtualBand.
Pianist Mark d'Inverno playing Solar with himself, thanks to the Reflexive VirtualBand.
Reflexive VirtualBand: Looper
Reflexive Loopers for Solo Musical Improvisation: reflexive interaction explained for the ACM SIGCHI Conference on Human Factors in Computing Systems (Best paper honorable mention award)
Reflexive VirtualBand and pianist Mark d'Inverno in Blue Moon
Pianist Mark d'Inverno playing Blue Moon with himself, thanks to the Reflexive VirtualBand.
Reflexive VirtualBand and pianist Mark d'Inverno in Now is the Time
Pianist Mark d'Inverno playing Now is the Time with himself, thanks to the Reflexive VirtualBand.
Adapative Mashup with Virtualband
Reflexive VirtualBand: beatbox
Beyond jazz: you can use Reflexive VirtualBand to beatbox... with yourself!