Sending performance data via audio

It’s a challenge to get one’s head around the recent patents filed and obtained by Yamaha. In this post, I concentrate on one kind of communication technology that pops up in several patents.

Most people would like to get rid of the cables in their studio or living room. Radio-based communication technology like wi-fi (e.g. IEEE 802.11), Bluetooth, or Bluetooth Low Energy (BLE) seems like a no brainer for wireless communication. Radio is a bit of a regulatory nightmare for a global electronics corporation, however, because radio gear needs type acceptance and approval from governmental authorities. On a functional level, both wi-fi and Bluetooth communications are subject to interference, conflict and latency.

If Yamaha knows anything, it knows about latency and how latency can adversely affect the generation/transmission of data and sound.

US Patent 8,779,267 describes an approach to CDMA-like (code division multiple access) communication via near-ultrasonic sound (18KHz). Pseudo-random spreading codes allow multiple transmitters to operate within the same frequency band. Thus, multiple musical devices in your living room or studio can communicate with each other at the same time. The sound of ongoing communication — “control tones” in the terminology of the patent — are sufficiently high as to be inaudible to humans. (I wonder what dogs will hear and think? Seriously.)

The patent deals specifically with the modulation and generation of control tones by a synthesizer. The synth CPU borrows one of 32 tone generator channels to generate the control tones to be transmitted. The waveforms for the tones are stored, ta-da!, in wave ROM. Amplitude is constant (no ADSR for you) and the tone is sent only through the left channel to avoid sonic interference with itself through the right channel (avoiding phase cancellation, no doubt).

All in all, this is quite clever. Using a tone generation channel keeps cost low — no specialized modulator. The symbol rate is about 400.9 symbols per second, so transmission speed is not blazing fast. However, the ultra-sonic approach avoids the regulatory hassles and latency of consumer data radio technology.

The application discussed in the patent is the synchronization or display of “musical score data” on a tablet. The synthesizer sends control tones to the tablet telling notation software where it is in a musical score. The low symbol rate should be OK for this kind of application. If you’re curious about this application, then check out US Patent 9,029,676 (“Musical score device that identifies and displays a musical score from emitted sound and a method thereof”).

Six futher “embodiments” of theses idea are described in US Patent 9,006,551. The object of this invention is a musical performance information output device and system which superimposes musical performance-related information (e.g., notes, tempo, expression, etc.) on an analog audio signal without damaging the “general versatility” of the the audio data. The embodiments include:

  • A guitar that derives MIDI messages from string sensors and imposes the MIDI data on the audio signal.
  • A guitar that determines fingering information and sends it.
  • An electronic piano that sends tempo clock by imposing it on the audio.
  • A guitar that controls an effect unit.

And so forth.

Data is superimposed onto an audio signal. The signal can be sent in either free-air (patent ‘267) or over an audio cable (patent ‘551). There are probably limits and restrictions on free-air transmission such as signal strength, interference from ambient noise and so forth. Patent ‘267 assumes that the tablet and keyboard are in close proximity (speaker to microphone). In the case of ‘551, combining audio and data communication over an audio cable at least eliminates the need for a separate parallel data cable.

The normal disclaimers apply: Who knows if this technology will make it into product, how, or when?