Network-centric Music Performance
music, audio systems, networked entertainment
Introduction
The advances in information technology and the
great proliferation of the Internet have changed nearly every
aspect of the work and life of human beings. Despite the
progress in networked entertainment, many music professionals
and enthusiasts are still sticking to the traditional way of carrying
out rehearsals and concerts. Music performance in this way
requires physical presence of the participants and has a number
of inherent limitations. We introduce a novel system called
“Network-centric Music Performance” (NMP) that enables multiparty
music performance through cyberspace. Our target is to
support real-time multi-channel natural audio streaming over
the network, using audio compression schemes that can provide
acceptable audio quality. Besides many challenges to cope with today's
technology, a tight delay bound between the production and perception of
audio is the dominant requirement for this project.
Narration of the Background of NMP
Project Description
The NMP system is bandwidth demanding, highly delay-sensitive
and requires the synchronization of the
audio streams. Hence, the support from underlying end-systems
and the networks is critical. However the current source coding
mechanisms and the best-effort nature of the Internet poses many
challenges to achieve the desired quality of service. We implement
a prototype of NMP using the client-server architecture and exploit end-system’s and
network’s influences on Network-centric Music Performance. This is done in a local area network environment using Linux
PCs (see the provided video clips). The system enables two different application scenarios,
namely, real-time rehearsal and rehearsal on-demand. Real-time
multi-channel audio transport and different audio compression
schemes are supported. Both subjective and objective measurements are conducted to verify if the system
suffices the audio quality level for the target application in such
environment. Scalability tests are carried out to validate whether the system
scales well with the increase of clientele.
NMP Application Scenarios
While most of our work has been focused on system aspect of NMP, we
are working on QoS support through bandwidth prediction and forward
error correction techniques at the moment. We plan to extend the scale
of the application to networks spanning across reasonable larger
physical distances and supporting more simultaneous users. Realistic
network conditions outside the LAN will be considered in the next step
to investigate the performance of the application in larger-scale
networks. Then also delay jitter, its impact and counter-measure
approaches will be more intensively studied. A more sophisticated
evaluation model will be developed to compare different approaches
with higher fidelity.
Detailed descriptions of NMP's architecture and current results are
included in our NIME04 paper
(PDF, 848KB), the IEEE Communications paper (PDF, 167KB),
and the Communications of the ACM paper (PDF, 127KB).
Project Members
Should you have any questions, please feel free to contact , or .
Publications
Theses for this project
Related Work
- William T.C. Kramer, "SCinet: testbed for high-performance networked applications”, IEEE Computer, vol. 35, issue 6, pp.47-55, June 2002.
- Chris Chafe, Michael Gurevich, Grace Leslie and Sean Tyan, "Effect of time delay on ensemble accuracy", In Proc. of the International Symposium on Musical Acoustics, (ISMA2004), Nara, Japan, March-April 2004.
- Fernando Lindner Ramos, Marcio de Oliveira Costa and Jônatas Manzolli, “Virtual studio: distributed musical instruments on the web”, in Proc. of IX Brazilian Symposium on Computer Music, Campinas, Brazil, August 2003.
- Dimitri Konstantas, “Overview of a telepresence environment for distributed musical rehearsals”, in Proc. of ACM Symposium on Applied Computing (SAC'98), pp. 456-457, Atlanta, US, February 1998.
- Young J.P. and Fujinaga I., “Piano master classes via the Internet”, in Proc. of the International Computer Music Conference (ICMC'99), pp. 135-137, Beijing China, October 1999.
- Goto M., Hidaka I., Matsumoto H., Kuroda Y. and Muraoka Y, “A jazz session system for interplay among all players”, IPSJ Journal, vol. 43, pp. 299-309, 2002.
- Xu, A. et al., “Real Time Streaming of Multi-channel Audio Data over Internet”, Journal of the Audio Engineering Society, vol. 48, issue 7-8, July-August 2000.
- Cooperstock J. and Spackman S., “The recording studio that spanned a continent”, in proc. of IEEE International Conference on Web Delivering of Music (WEDELMUSIC01), Florence, Italy, November 2001.
- R. Zimmermann, K. Fu and D. A. Desai, "HYDRA: High-performance Data Recording Architecture for Streaming Media", Chapter 2 in Video Data Management and Information Retrieval, IRM Press, 2004. ISBN: 1-59140-571-8.
- Chafe C., Wilson S., Leistikov R., Chisholm D., Scavone G., “A simplified approach to high quality music and sound over IP”, in Proc. of COST G-6 Conference on Digital Audio Effects (DAFX-00), Verona, Italy, December 2000.
- Nicolas Bouillot, “The auditory consistency in distributed music performance: a conductor based synchronization”, ISDM (Info et com Sciences for Decision Making vol. 13(0), pp. 129-137, March 2004.
- A. J. Aude, “Audio quality measurement primer”, Intersil Corp., Application Note AN9789, February 1998.
- International Telecommunication Union, “Methods for the Subjective Assessment of small Impairments in Audio Systems including Multichannel Sound Systems”, Rec. ITU-R BS.1116-1, 1997.
- International Telecommunication Union, “Method for Objective Measurements of Perceived Audio Quality”, ITU-BS.1387, November 2001.
- R. Steinmetz, "Human perception of jitter and media synchronization," in IEEE Journal on Selected Areas in Communications, vol. 14, issue 1, pp. 61-72, January 1996.
Media and News
-
In June 2007, NMP was demonstrated at the event
'Kompetenztage Kommunikation' with live jam sessions. Recorded media are available here.
- First recording of NMP test sequences of the Trinklied from Giuseppi Verdi's La Traviata. The artists are
from the Staatstheater Braunschweig. Recorded at the Aula of the Technical University
of Braunschweig. AB Stereo recording with a pair of
Sennheiser MKH 800 set to omni-directional pickup pattern.
- Video demonstrating NMP in action (all encoded with XviD and LAME)
- Press Reports
- Coverage (in German) in the local newspaper 'Braunschweiger Zeitung', 31.08.2004
download article as PDF (269 KB)
or visit online
Adopted Tools
Associated Equipment
- Audio Measurement Instrument: Lindos MiniSonic MS-20
- Microphones: Sennheiser MKH800, Neumann KM184
- Sound cards: E-MU 1616M, RME Fireface4, Echo Audiofire 8, Echo Indigo IO
- Headphones: Sennheiser HD650
- Studio monitors: Tannoy Ellipse 8
- Oscilloscope: Tektronix TDS 220, Tektronix 2467
- Sweep Generator: WaveTek 183
- Mixers: Yamaha MG10/2
Acknowledgement
We would like to thank House of Hansen Productions LLC in California, USA for funding the development of this project. This work was sponsored by Sennheiser Electronic GmbH&Co. KG, Germany (special thanks to Dr. Wolfgang Niehoff) and Lindos Electronics, UK (sincere thanks to Chris Skirrow), by providing the MKH800 microphones and DigiSonic DS10. Previously, the Canadian Communications Research Centre (CRC) and OPTICOM GmbH, Germany provided their audio quality evaluation tools CRC-SEAQ and Opera 3.5 respectively.