Interdisciplinary Theory & Technology

pressure sensor array touchpad user interface (u.s. patent 6,570,078)

Pressure Sensor Array Touchpad User Interface

Demonstration 4: Applications to Music Melodic Keyboards- Vocal Choir Synthesis

As described in U. S. Patent 6,570,078 [1] and associated whitepapers [2]-[4], the pressure sensor array touchpad user interface finds many exciting applications in music technologies. This and the preceding demonstration page consider implementing of a separate touch pad on each key of a traditional Western melodic keyboard as covered in pending U.S. Patent Application 2004/0074379 [5] and described in associated whitepapers [6] and [3]-[4]. The figure below, various forms of which may be found in a plethora of references (see for example [7] and [8] or almost any modern phonetics text), show regions of ranges of the center frequency of the two primary "formants" (vocal resonances) associated with various continuously sounded vowel tones. (The regions here are denotes with the internationally accepted phonetics symbols, while other texts may use English or other language vowel name labels.)




This spatial two-dimensional 'map' of vowel tones may be associated with the two-dimensional surface of a key on a traditional Western melodic keyboard. The geometric center of the area of touch where the key is contacted with a finger can the be used to select and vary a vowel tone. The pressure applied to the key (or vertical displacement of the key) could be used to control the volume of a rendered vowel tone. The synthesis of vowel sounds by two controllable resonant bandpass filters (emulating the formants created by modulations of the shape of the vocal cavity) applied to a pulse waveform (emulating vocal chord vibration) is well-known and can be controlled with such a key. Other well-known techniques involving emulation of nasal frequency response notches and a third formant may also be included to improve the quality of the synthesized vowel tones. Multiple instances of these may be used to create a 'vowel choir' which both produces a richer effect and more forgiving demands on the required accuracy of the individual vowel synthesis system. Alternatively, families of sampled vowel tones (individual or choir) may be used in lieu of synthesized tones, and the surface of the key may be used to gracefully seque between the samples so as to create a smoothly responsive vowel. The animation below shows how the six parameters from each key may be used. Each key can independently produce its own vowel sound (employing left-right, forward-back touch-location on to control formants) with its own amplitude control (pressure or vertical displacement of the key). The other three available parameters (finger rotation angle, left-right finger tilt, forward-back finger tilt) may be used for other aspects of the sound. Although the (left-right, forward-back) touch-location and applied pressure/displacement is readily controllable for each key when a chord is played, holding two or more notes will often limitations the range of independent adjustment possible for the other three available parameters (finger rotation angle, left-right finger tilt, forward-back finger tilt). Thus, in many cases it may be desirable to carefully choose the parameter assignment with this in mind. In the example below, the left-right finger tilt is used to control the pan of the sounded vowel note, the forward-back finger tilt is used to control an octave mix or other male/female vocal attribute parameter, and the finger rotation angle is left unused.



A wide range of other options are of course possible. For example, controlling of the "Male/Female" gender and "Left/Right" pan could be replaced by the controlling of selected overlaid variable bandpass and lowpass filters so as to emulate the effect of selected opening and closing consonant effects. Additionally, the number of parameters used can vary with the number of keys operated. For example, if more than one key is operated, the finger rotation angle parameter is ignored, but with a single finger playing the keyboard it could be recognized and assigned to mixing in of an instrument sound, control of an opening and closing c onsonant effect, etc.

REFERENCES

[1] U.S. Patent 6,570,078, "Tactile, visual, and array controllers for real-time control of music signal processing, mixing, video, and lighting," May 27, 2003

[2] NRI Whitepaper, "Rich Multi-Parameter Touchpad User Interface: Overview: Background, Capabilities, and Application, 2004

[3] NRI Whitepaper, "Rich Multi-Parameter Touchpad User Interface: Technology and Examplary Implementation," 2004

[4] NRI Whitepaper, "Rich Multi-Parameter Touchpad User Interface: Music Applications," 2004

[5] U.S. Pre-Grant Patent Application 2004/0099131, "T ranscending extensions of classical South Asian musical instruments," May 27, 2004

[6] NRI Whitepaper, "T ranscending Extensions of Classical South Asian Musical Instruments,"

[7] U.S. Pre-Grant Patent Application 2004/0074379, "Functional extensions of traditional music keyboards," April 22, 2004

[8] NRI Whitepaper, " Functional Extensions of Traditional Music Keyboards ,"

[9] Moog, Robert A. "The Human Finger - A Versatile Electronic Music Instrument Component", Audio Engineering Society Preprint, 1977, New York.

[10] Snell, John M. "Sensors for Playing Computer Music with Expression,"Proceedings of the Intl. Computer Music Conference at Eastman, 1983.

[11] Wessel, David L., "Timbre Space as a Musical Control Structure," Computer Music Journal, vol. 3 no. 2, pp.45-52, 1979.

[12] Slawson, Wayne, Sound Color, University of California Press, 1985.

[13] Appelman, D. Ralph, The Science of Vocal Pedagogy, Theory and Applications, Indiana University Press, Bloomington, 1967.

[14] Winckel, Fritz, Music Sound, and Sensation: A Modern Exposition, Dover, New York, 1967.

[15] Vertegaal, Roel, "An Evaluation of Input Devices for Timbre Space Navigation," MPhil. dissertation, University of Bradford, Department of Computing, Bradford, U.K.,1994.