pressure sensor array touchpad user interface (u.s. patent 6,570,078)
Pressure Sensor Array Touchpad User Interface
Demonstration 3: Applications to Music Melodic Keyboards: Timbre Control
As described in U. S. Patent 6,570,078 [1] and associated whitepapers [2]-[4], the pressure sensor array touchpad user
interface finds many exciting applications in music technologies. These can involve simply directing the time-varying control
signals (derived from real-time pressure image measurements) to various controllable sound synthesis and audio signal processing
functions, allowing one to "fingerpaint" musical sounds with simultaneous control of lighting effects. Though additional
pressure image processing, the touchpad can emulate a drumhead or other percussion instrument surface, providing elaborate
control over the creation of percussion sounds. Those approaches are covered in pending U.S. Patent Application 2004/0099131
[5] and described in associated whitepapers [6] and [3]-[4], generalizing the sophisticated techniques of the South Asian
tabla and baya. This and the subsequent demonstration page considers another refined application, specifically in
implementing a separate touch pad on each key of a traditional Western melodic keyboard. This is covered in pending
U.S. Patent Application 2004/0074379 [7] and described in associated whitepapers [8] and [3]-[4]. The idea of using finger
contact position on a key of a traditional Western melodic keyboard was explored by Moog [9], Snell [10], and others more
twenty years ago. In one simple embodiment of the material covered in U. S. Patent 6,570,078 [1] and pending U.S. Patent
Application 2004/0099131 [5], arbitrary means of sensing the two directional components (left-right, front-back) of finger
position on a key are combined with a key displacement or other pressure measurement to obtain 3 independently adjustable
control parameters to control sounds generated by the operation of that keyboard key. In [1] it is argued that the
critical value of 3 independently adjustable control parameters is a fundamental minimum for "natural" control of sounds,
typically using one parameter for volume and the other two for aspects of timbre. The minimum of two timbre parameters provide
the basis for the manipulation sounds in a Wessel "timbre space"[11], creating of Schoenberg/Webern "melodies of "tone color'
(as documented and explored in Slawson [12] ), articulating vowel sounds (via relative formants [13-14], etc.
Providing additional independently adjustable control parameters (beyond 3 per key) provides important enhanced capabilities. By implementing a pressure sensor array touchpad on each key, an infrastructure for individually controlling up to 6 independently adjustable control parameters per key can be had. The animation above shows a traditional Western melodic keyboard adapted to include a pressure sensor array touchpad on each key. The keys are depicted in the traditional white and black colors, and pressure images from finger contact as measured by the touchpad on each key are shown in the same gradient color scheme used in the previous demonstration pages in this series. Note as the three pressure images separately move and change form over time, six parameters are derived in real time for each key. In playing a single keyboard note, the finger contact can be quite free in its variation, while holding two or more notes place limitations on how each finger's contact with a key may be independently varied. The figure below, adapted from Wessel's paper, shows an exemplary two-dimensional timbre space for cataloging the continuous-duration tones of a variety of traditional Western musical instruments. Such a "layout" of timbrel variation can be imposed on the surface of each melodic keyboard key. Collections of synthesized tones and other organizational schemes of traditional instruments employ other two-dimensional layouts (imposed on the surface of each melodic keyboard key) and arrangements involving more than two independent timbrel parameters (as explored by Vertegaal [15] and others). Alternatively, the additional parameters may be used for non-timbrel effects such as stereo panning, vibrato, oscillatory timbrel modulation (i.e, "growls" and "shimmers"), etc.
REFERENCES
[1] U.S. Patent 6,570,078, "Tactile, visual, and array controllers for real-time control of music signal processing, mixing, video, and lighting," May 27, 2003
[2] NRI Whitepaper, "Rich Multi-Parameter Touchpad User Interface: Overview: Background, Capabilities, and Application, 2004
[3] NRI Whitepaper, "Rich Multi-Parameter Touchpad User Interface: Technology and Examplary Implementation," 2004
[4] NRI Whitepaper, "Rich Multi-Parameter Touchpad User Interface: Music Applications," 2004
[5] U.S. Pre-Grant Patent Application 2004/0099131, "T ranscending extensions of classical South Asian musical instruments," May 27, 2004
[6] NRI Whitepaper, "T ranscending Extensions of Classical South Asian Musical Instruments,"
[7] U.S. Pre-Grant Patent Application 2004/0074379, "Functional extensions of traditional music keyboards," April 22, 2004
[8] NRI Whitepaper, " Functional Extensions of Traditional Music Keyboards ,"
[9] Moog, Robert A. "The Human Finger - A Versatile Electronic Music Instrument Component", Audio Engineering Society Preprint, 1977, New York.
[10] Snell, John M. "Sensors for Playing Computer Music with Expression,"Proceedings of the Intl. Computer Music Conference at Eastman, 1983.
[11] Wessel, David L., "Timbre Space as a Musical Control Structure," Computer Music Journal, vol. 3 no. 2, pp.45-52, 1979.
[12] Slawson, Wayne, Sound Color, University of California Press, 1985.
[13] Appelman, D. Ralph, The Science of Vocal Pedagogy, Theory and Applications, Indiana University Press, Bloomington, 1967.
[14] Winckel, Fritz, Music Sound, and Sensation: A Modern Exposition, Dover, New York, 1967.
[15] Vertegaal, Roel, "An Evaluation of Input Devices for Timbre Space Navigation," MPhil. dissertation, University of Bradford, Department of Computing, Bradford, U.K.,1994.