Research Proposal

Degree of Doctor of Philosophy

Research Proposal
Reverse Physical Modelling of Musical Instruments

David A. Allen-Williams

Student Number: 940503/1

Department of Computer Science

The University of Western Australia

Nedlands, WA, 6907

E-mail: daws@cs.uwa.edu.au

20 August 1997

Proposed Study

The rapid advances in technology within this century have radically altered the way that humans can create and imagine sound. Since the advent of sound representation using electrical signals, the generation of sound is no longer bound to the constraints of mechanical and acoustical systems. Electrical signals can be generated and manipulated using both analogue and digital means, allowing composers to create new sounds never previously heard.

Over recent years, effort has been expended on the digital creation and manipulation of sounds using various synthesis techniques. These techniques can be considered to be based on one of two models:

Signal models describe the sound signal and are generally related to the perception of sound - they modify parameters like frequency and amplitude to create particular sounds.

Physical models use physical parameters (geometric information, materials, masses, forces, et cetera) to model the physical production of a sound through mechanical or acoustical means.

The development and refinement of physical models has been an area of particular interest to commercial manufacturers, since the capability to realistically synthesise musical instruments is vital to the sales success of many electronic synthesisers. A physically modelled ‘virtual’ guitar with specified dimensions, materials, and string tensions (with the method used to pluck or strum them), would ideally produce a sound indistinguishable from a real guitar of the same specifications. The financial influence of these manufacturers is such that a large amount of research has been undertaken in order to develop and refine physical modelling techniques. Software packages have been developed that can take a range of specifications for a virtual instrument, and closely reproduce the sound that the instrument would make if it were to be physically built. For example, a 30 metre long oboe or a wooden tuba could be created, and the sounds of these instruments used in musical composition.

The purpose of my research is to develop methods to "reverse-engineer" physical models of instruments given an output sound. Through analysis of a sound sample, it should be possible to produce the specifications for an instrument that (if physically built) would produce a sound approximating the sample. The applications for this would generally be analytical in nature. The research would enhance the understanding of how physical parameters affect sound production, and could prove an invaluable tool for music historians and experimentalists. For example, analysis of old recordings could help in the identification of what instruments (and their particular makes) were used by individual musicians. Instrument manufacturers could use signal processing on the output from an instrument to get some desired sound, and then use reverse physical modelling to gain an idea of how to achieve that sound through physical alteration of the instrument.

Research Plan

Specific Aims

Given a musical note, humans can usually identify what type of instrument (percussion, string, woodwind, or brass) made that sound. The trained ear could tell you precisely what instrument produced the note, and possibly its make also. However, it is significantly more difficult to train a computer to perform the same task.

My research will initially be aimed at getting a computer to recognise standard instruments given a high quality recording of a single note. Each of these standard instruments would have an associated set of detailed specifications for a physical model, and the sound wave produced using the physical model of the recognised instrument should match the sample sound better than any other instrument.

Once a sample has been matched to a standard instrument, it should be possible to alter the physical parameters of that instrument to minimise differences between the physical model’s output and the sample sound.

If the alteration of known instruments is successful in producing a desired sound output, then it is possible that virtual instruments could be built that do not use a known instrument as their basis.

Methods

At a basic level, it would be necessary to identify the characteristics of a sound that make it recognisable as a particular class of instrument. Once this is known, the sound could be matched to a specific instrument using specially developed heuristics. A large database of high quality instrument samples would be built for this purpose. In addition, a database of physical models for many known instruments would also need to be developed.

Within this instrument recognising process, I will need to account for the different ways a particular instrument can be played. For example, a guitar note can be picked using a plectrum, finger-picked, hammered on, or pulled off. These methods all produce distinctively different sounds (generally in the attack portion of the note), so each instrument in the database may also require a number of different playing methods to be recorded with it.

Alteration of the physical model in order to approximate a desired sound could be done using a number of methods. Possibilities include genetic search algorithms and the use of neural networks.

Originality

As far as can be discerned, all current physical modelling research for musical instruments is concerned with producing a realistic sound from a fully specified virtual instrument. No research has been done to produce the virtual instrument from a realistic sound. This represents a significantly non-trivial difference. Resonance, echoes, and other sound producing mechanisms have been effectively modelled previously, but the complex interaction of these produces a sum output that is very difficult to break down into its component parts. It is this problem that will constitute the majority of the work.

Duration

The research will span three years. Approximately one year will be spent building a large database of instrument samples and physical model specifications - both through acquisition from external sources and through recordings done in the department’s anechoic chamber. Another year will be spent developing sound analysis techniques for the characterisation of instruments. During this time, heuristic methods will be developed for the matching of a sound sample to an approximating instrument within the database. From this point, effort will directed to a physical modelling process - the progressive alteration of an approximating instrument until its expected sound matches the given sample.

Scholars

There are many researchers that have done work on physical modelling and the characterisation of sound. Most of these are related to musical institutes and research groups like the Institute of Research and Coordination Acoustic/Music (IRCAM), the CERL Sound Group at UIUC, the Stanford University CCRMA, and many other university groups. Particular luminaries include René Caussé, Roger Dannenberg, Philippe Depalle, Gerhard Eckel, David A. Jaffe, Jeff Pressing, Curtis Roads, and Xavier Rodet.

Facilities

It is expected that computation can be carried out using the department’s existing infrastructure, with a single workstation (probably DEC- or NeXT-based) devoted to the task. A large hard disk will be required for data storage. Access to translation software (French to English) will be required to interpret the large amount of foreign material related to the subject.

High quality recording equipment will be required for the digital sampling of various instruments. This will involve scheduling a significant amount of time for the use of the department’s anechoic chamber. The expertise of professional musicians will also be required to play instruments using different techniques.

Estimated Costs

Hardware

If no workstations are available to be used primarily for sound analysis, a DEC Alpha station will cost approximately $5000.

$1000 per annum will be required for maintenance of the Unix workstation.

The department owns most of the recording equipment required, but a high quality microphone will need to be purchased. The Shure SM57 or SM58 would probably be suitable, which retail for $195 and $235 respectively. It is expected that the department will be able to collaborate with the music department to organise the loan of instruments and pickups.

Software

Translation software should be available at minimal cost, with a shareware translator likely to be sufficient.

Access to IRCAM’s Modalys software (a physical modelling package) will also be required, the cost of which is unknown at this stage.

Database software will be required - it is expected that the department will already have access to this.

Other Costs

Every attempt will be made to acquire the voluntary services of musicians (again possibly with the collaboration of the music department), with minimal hiring of session musicians - typically at $50 per hour.

$15 US per annum will be required for a student subscription to the Computer Music Journal.

$500 per annum will be required to cover costs including the purchase of books, software, and other small items.

Confidentiality

There are no confidentiality requirements for this research project.

References

G. Eckel. Manipulation of sound signals based on graphical representation: A musical point of view. World Wide Web. http://viswiz.gmd.de/~eckel/publications/eckel92.html.

G. Eckel, F. Iovino, and R. Caussé. Sound synthesis by physical modelling with Modalys. World Wide Web. http://viswiz.gmd.de/~eckel/publications/eckel95b.html.

Institut de Recherche et Coordination Acoustique/Musique. Modalys. World Wide Web. http://www.ircam.fr/produits-real/logiciels/modalys-e.html.

D. Jaffe and J. O. Smith. Performance expression in commuted waveguide synthesis of bowed strings. World Wide Web. http://cmn19.stanford.edu/~daj/bowedstring.html.

Jeff Pressing. Synthesiser Performance and Real-Time Techniques. Oxford University Press, Oxford, 1992. 462p.