Yamaha multi-touch patents - Sand, software and sound

I was on a quest for information about Virtual Resonance Modeling (VRM) when I encountered two interesting Yamaha patents. Both patents have the same obtuse title: “Musical sound information outputting apparatus, musical sound producing apparatus, method for generating musical sound information.” The U.S. Patent numbers are:

U.S. 11,398,210 B2, July 26, 2022
U.S. 11,657,791 B2, May 23, 2023

Both patents, essentially, cover the same technology — a kind of multi-touch keyboard and control scheme for synthesis. Here’s my synopsis.

Black and white keys are divided into two (or three) detection regions each as shown in the diagram below. Each region detects touch and release. Pretty simple, eh?

Yamaha U.S. Patents 11,398,210 and 11,657,791

The sound generator reacts in repsonse to touch and release differently according to key region. Here are a few examples mentioned in the patent:

A touch in white (black) key region Wa (Ba) initiates a tone.
A slide in white (black) key region Wb (Bb) applies an effect to the tone.
A release in Wa (Ba) stops the tone.
A release in Wb (Bb) stops application of the effect.

The word “effect” is meant rather broadly, including vibrato, modulation and so forth. Touch and release generate typical MIDI-like information such as note number and velocity.

Please note that a player can touch more than one key region simultaneously. The player chooses the gesture — with one or more fingers of a single hand or both hands. The sound generator could respond differently as to which region is touched first or held. Similarly, the sound generator could react differently depending upon the order (or temporal relationship) of release.

The inventors describe possibilities afforded by the invention when the sound generator is producing guitar tones. Depending upon touch gestures, the sound generator may produce a fingered tone, a plucked (picked) tone, hammer-on, fret noise or mute. Possibilities abound: one region initiates a single tone, the second region initiates a chord.

Obviously, if the idea works for two key regions, why not three or more?

The actual mechanism for key region detection depends upon the chosen key technology itself. Much of the patent describes implementation with a tablet touch screen, e.g., electrostatic sensing. They also mention a “pantograph-type elevating structure” where the player can depress independently the front and rear parts of a key.

What really caught my eye is the list of inventors: Masahiko Hasabe, Shinichi Ito, Kenichi Nishida, Masahire Kakishita, and Shinichi Ohta. These folks are heavy-hitters and have made many technical contributions to Yamaha products and its patent portfolio. Shinichi Ohta, for example, was the manager in charge of MONTAGE development!

So, make of it what you will. Yamaha has a broad and deep patent portfolio and not all of its patented tech gets into product. However, that is a rather large amount of talent interested in multi-touch, multi-zone keyboard control.