Tip-toe through the tech

Posted on January 15, 2017 by pj

Last year ’bout this time, we were all holding our collective breath awaiting the new Yamaha Montage. There are two products which I expect to see from Yamaha sometime in the next one to two years:

The successor to the mid-range MOXF synthesizer, and
The successor to the top-of-the-line (TOTL) Tyros arranger workstation.

NAMM 2017 seems a little too soon for both products. In the case of the MOXF successor, Yamaha conducted marketing interviews during the summer of 2015. I would guess that MOXF sales are still pretty good and no new products from the usual suspects (Korg, Roland) are visible on the horizon. The Krome and FA could both use an update themselves. Not much market pressure here at the moment. (Korg’s NAMM 2017 announcements are, so far, a little underwhelming.)

Read my MOX retrospective and interview follow-up.

I suspect that the Tyros successor is somewhat closer to launch. Speculation has been heated ever since Yamaha filed for a US trademark on the word mark “GENOS”. The word mark was published for opposition on November 15, 2016. “Published for opposition” means that anyone who believes that they will be damaged by registration of the mark must file for opposition within 30 days of publication. If “GENOS” is indeed the name for the Tyros successor, then the 30 day period ending December 15, 2016 is cutting it very close to NAMM 2017. Even more ludicruous if Yamaha were to begin manufacturing products printed with that name for a NAMM 2017 launch. Imagine the scrap if opposition was successful!

For quite some time, I have been meaning to summarize the key U.S. patents that I believe to be GENOS-related. (Assuming that “GENOS” is the name!) I’ve procrastinated because the launch date is most likely fall 2017 at the earliest as previous Yamaha mid- and high-end arranger models are typically launched in the fall in anticipation of the holiday selling season.

A much larger barrier is the task of reading and gisting the patents. Patents are written in legalese and are much more difficult to read than the worst written scientific papers! One of the folks on the PSR Tutorial forum suggested making a list of the top five technologies for the new TOTL arranger. I generally hate the superficial nature of “list-icles,” but the suggestion is a good one. Nothing will get done as long as the barrier is big because I would much rather jam and play! I’m supposed to be retired.

The 2016 Yamaha annual report states that Yamaha want to make innovative products which are not easily copied by competitors. Patents — legally protected intellectual property — are essential to achieving this goal. Generally, a company only applies for a patent on technology in which they have a serious business interest due to the significant cost of obtaining and maintaining patent protection.

So, here are a few of Yamaha patented technologies which could appear in future products — perhaps GENOS, perhaps others.

SWP70 tone generator

This may seems like old news…

The next generation SWP70 tone generator first appeared in the mid-range Yamaha PSR-S970 arranger workstation. The SWP70 made its second appearance in the Yamaha Montage synthesizer. The S970 incorporates only one SWP70 and does not make full use of the chip. (At least three major interfaces are left unconnected.) In keeping with Yamaha’s TOTL design practice, the Montage employs two SWP70 integrated circuits: one each for AWM2 sample-playback and FM. A second sample cache interface on the AWM2 side is unconnected.

The Tyros successor likely will use two SWP70 tone generators, too. The number of available tone generation channels with two SWP70s will be massive (512 channels). Yamaha could opt for a single SWP70 and still outmatch the current generation Tyros 5. Like the Montage, there will be enough insert effect DSP processors to cover each style and user part, as many as two for every part.

It will be interesting to see (and hear) if the GENOS will make use of the second sample cache interface. A second cache would not only support more tone generation channels, but might be necessary for long, multi-measure musical phrases that are needed for full audio styles (discussed below).

The SWP70 flash memory interface follows the Open NAND FLASH interface (ONFI) standard, the same as solid state drives (SSD). ONFI memory devices can be stacked on a bi-directional tri-state bus, so potentially, the GENOS could support a large amount of internal waveform storage. This flash memory will contain the “expansion memory,” that is, physical memory reserved in flash memory for user waveforms. The expansion flash memory expansion modules (FL512M, FL1024M) are dead, Jim.

If you’re interested in Yamaha AWM2 tone generation, here’s a few patents to get you started:

Patent 9,040,800 Musical tone signal generating apparatus, May 26, 2015
Patent 8,383,924 Musical tone signal generating apparatus, February 26, 2013
Patent 8,389,844 Tone generation apparatus, March 5, 2013
Patent 8,957,295 Sound generation apparatus, February 17, 2015
Patent 8,035,021 Tone generation apparatus, October 2011
Patent 7,692,087 Compressed data structure and apparatus and method related thereto, April 6, 2010

U.S. Patent 8,957,295 is the patent issued for the SWP70 memory interface. U.S. Patent 9,040,800 describes a tone generator with 256 channels — very likely the SWP70.

Pure Analog Circuit

This may seem like old news, too, since Pure Analog Circuit (PAC) debuted in the Yamaha Montage.

Pure Analog Circuit is probably the least understood and least appreciated feature of the Montage. It’s not just better DACs, people. The high speed digital world is very noisy as far as analog audio is concerned. Yamaha separated the analog and digital worlds by putting the DACs and analog electronics on their own printed circuit board away from noisy digital circuits. Yamaha then applied old school engineering to the post-DAC analog circuitry, paying careful attention to old school concerns like board layout for noise minimization and clean power with separate voltage regulation for analog audio. Yamaha’s mid- to high-end products have always been quiet — PAC is pristine.

Since the PAC board is a separate, reusable entity, I could see Yamaha adopting the same board for GENOS.

Styles combining audio and MIDI

Yamaha are constantly in search of greater sonic realism. Existing technologies like Megavoices and Super Articulation 2 (Advanced Element Modeling) reproduce certain musical articulations. However, nothing can really match the real thing, that is, a live instrument played by an experienced professional musician. PG Music Band-in-a-Box (BIAB), for example, uses audio tracks recorded by studio musicians to produce realistic sounding backing tracks. The Digitech TRIO pedal draws on the PG Music technology for its tracks. (“Hello” to the Vancouver BC music technology syndicate.)

Yamaha have applied for and been granted several patents on generating accompaniment using synchronized audio and MIDI tracks. Here is a short list of U.S. patents:

Patent 9,147,388 Automatic performance technique using audio waveform data, September 29, 2015
Patent 9,040,802 Accompaniment data generating apparatus, May 26, 2015
Patent 8,791,350 Accompaniment data generating apparatus, July 29, 2014
Application 13/982,476 Accompaniment data generating apparatus, March 12, 2012

There are additional patents and applications. Each patent covers a different aspect of the same basic approach, making different claims (not unusal in patent-land). Yamaha have clearly invested in this area and are staking a claim.

The patents cite four main motivations, quoting:

The ability to produce “actual musical instrument performance, human voices, natural sounds”
To play “automatic accompaniment in which musical tones of an ethnic musical instrument or a musical instrument using a peculiar scale”
To exhibit the “realism of human live performance”
To advance beyond known techniques that “provide automatic performance only of accompaniment phrases of monophony”

Your average guy or gal might say, “Give me something that sounds as natural as Band-in-a-Box.” Yamaha sell into all major world markets, so the ability to play ethnic instruments with proper articulation is an important capability. Human voice, to this point, is limited to looped and one-shot syllables, e.g., jazz scat. The new approach would allow long phrases with natural intonation. [Click on images in this article for higher resolution.]

Currently, mid- and high-end Yamaha arrangers have “audio styles” where only the rhythm track is audio. The patents cover accompaniment using melodic instruments in addition to rhythm instruments. The melodic audio tracks follow chord and tempo changes just like the current MIDI-based styles. Much of the technical complexity is due to synchronization between audio and MIDI events. Synchronization is troublesome when the audio tracks contain a live performance with rubato. Without good synchronization, the resulting accompaniment doesn’t feel right and sounds sloppy.

Accompaniment from chord chart

This next feature will be very handy. U.S. Patent 9,142,203 is titled “Music data generation based on text-format chord chart,” September 22, 2015. If you use textual chord charts (lyrics plus embedded chord symbols), you will want this!

Simply put, the technique described in this patent translates a textual chord chord to an accompaniment. The accompaniment is played back by the arranger. The user can select tempo, style, sections (MAIN, FILL IN) and so forth.

The translator/generator could be embedded in an arranger or it could be implemented by a PC- or tablet-based application. Stay tuned!

Selectively delayed registration changes

A registration is a group of performance parameters such as the right hand voice settings, left hand voice settings, accompaniment settings, and so forth. Mid- and high-end arrangers have eight front panel buttons where each button establishes a set of parameter values (“readout”) when the button is pushed. It’s the player’s job to hit the appropriate button at the appropriate time during a live performance to make voice settings, etc. A player may need a large number of buttons, if a musical performance is complicated.

Usually only a few parameters are different from one registration to the next. Recognizing this, the technique described by U.S. Patent 9,111,514 (“Delayed registration data readout in electronic music apparatus,” August 18, 2015) delays one or more parameter changes when a button is pushed. The user specifies the parameters to be delayed and the delay (such as the passage of some number of beats or measures, etc.) Thus, a single registration can cover the work of multiple individual registrations.

I’ll have to wait to see the final product to assess the usefulness of this feature. Personally, I’d be happy with a configuration bit to keep OTS buttons from automatically turning on the accompaniment (ACCOMP). Sure would make it easier to use the OTS buttons for voice changes.

Ensembles / divisi

Tyros 5 ensemble voices assign played notes to individual instrument voices in real time, allowing a musician to perform divisi (divided) parts. Tyros 5 ensembles can be tweaked using its “Ensemble Voice Key Assign Type List.” Types include open, closed, and incremental voice assignment. U.S. Patent 9,384,717, titled “Tone generation assigning apparatus and method” and published July 5, 2016, extends Tyros 5 ensemble voice assignment.

The technique described in 9,384,717 gives the musician more control over part assignment through rules: target depressed key, priority rule, number of tones to generated, note range, etc. The rules handle common cases like splitting a single note to two or more voices.

These extensions could lead to some serious fun! I didn’t feel like the Tyros 5 ensemble feature was sufficiently smart and placed too many demands on the average player, i.e., less-than-talented me. The rules offer the opportunity to shift the mental finger work to software and perhaps could lead to more intuitive ensemble play. Neat.

Voice synthesis

As I alluded to earlier, arrangers make relatively primitive use of the human voice. Waveforms are usually limited to sustained (looped) or short (one-shot) syllables.

Yamaha have invested a substantial amount of money into the VOCALOID technology. VOCALOID draws on a singer database of syllable waveforms and performs some very heavy computation to “stitch” the individual waveforms together. The stitching is like a higher quality, non-real time version of Articulated Element Modeling (AEM).

VOCALOID was developed through a joint research project (led by Kenmochi Hideki) between Yamaha and the Music Technology Group (MTG) of the Universitat Pompeu Fabra in Barcelona, Spain. VOCALOID grew from early work by J. Bonada and X. Serra. (See “Synthesis of the Singing Voice by Performance Sampling and Spectral Models.”) More recent research has stretched synthesis from the human voice to musical instruments. Yamaha hold many, many patents on the VOCALOID technology.

Patent 9,355,634, titled “Voice synthesis device, voice synthesis method,” is a recent patent concerning voice synthesis (May 31, 2016). It, too, draws from a database of prerecorded syllables. The human interface is based on the notion of a “retake,” such as a producer might ask a singer to make in a recording studio using directives like “put more emphasis on the first syllable.” The retake concept eliminates a lot of the “wonky-ness” of the VOCALOID human interface. (If you’ve tried VOCALOID, you know what I mean!) The synthesis system sings lyrics based on directions from you — the producer.

An interface like this would make voice synthesis easier to use, possibly by novices or non-technically oriented musicians. The big question in my mind is whether voice synthesis and editing can be sped up and made real time. Still, wouldn’t it be cool if you could teach your arranger workstation to sing?

Music minus one

This work was conducted jointly with the MTG at the Universitat Pompeu Fabra. A few of the investigators were also involved in VOCALOID. Quoting, “The goal of the project was to develop practical methods to produce minus-one mixes of commercially available western popular music signals. Minus-one mixes are versions of music signals where all instruments except the targeted one are present.”

This is not good old center cancellation. The goal is to remove any individual instrument from a mix regardless of placement in the stereo field. You can hear a demo at http://d-kitamura.sakura.ne.jp/en/demo_deformation_en.htm.

I doubt if this technique will appear on an arranger; the computational requirements are too high and the method is not real time. However, “music minus-one” is very appealing to your average player (that is, me). My practice regimen includes playing with backing tracks. I would love to be able to play with any commercial tune on whim.

There are patents:

US Patent 9,002,035 Graphical audio signal control
US Patent 9,224,406 Technique for estimating particular audio component
US Patent 9,070,370 Technique for suppressing particular audio component

and there are scientific papers:

“Audio Source Separation for Music in Low-latency and High-latency Scenarios”, Ricard Marxer Pinon, Doctoral dissertation, Universitat Pompeu Fabra, Barcelona, 2013.
D. Kitamura, et al., “Music signal separation by supervised nonnegative
matrix factorization with basis deformation,” Proc. DSP 2013, T3P(C)-1, 2013.
D. Kitamura, et al., “Robust Music Signal Separation Based on Supervised Nonnegative Matrix Factorization with Prevention of Basis Sharing”, ISSPIT, December 2013.

Music analysis

Yamaha have put considerable resources into what I would call “music analysis.” These technologies may not (probably will not) make it into an arranger keyboard. They are better suited for PC- or tablet-based applications.

I think we have seen the fruits of some of this labor in the Yamaha Chord Tracker iPad/iPhone application. Chord Tracker identifies tempo, beats, musical sections and chords within an audio song from your music library. It displays the extracted info in a simple chord chart and can even send the extracted “lead sheet” to your arranger. The arranger plays back the “lead sheet” as an accompaniment using the selected style.

We’re probably both wondering if Chord Tracker will integrate with the chord chart tool described above. Stay tuned.

Yamaha Patent 9,378,719 (June 28, 2016) is a “Technique for analyzing rhythm structure of music audio data.” Patent 9,117,432 (August 25, 2015) is an “Apparatus and method for detecting chords.” I wouldn’t be surprised if Chord Tracker draws from these two patents.

Yamaha has also investigated similarity measures and synchronized score display:

Patent 9,053,696 Searching for a tone data set based on a degree of similarity to a rhythm pattern, June 9, 2015
Patent 9,006,551 Musical performance-related information output device, April 14, 2015
Patent 9,275,616 Associating musical score image data and logical musical score data, March 1, 2016

I’m not sure where Yamaha is going with similarity measures and searching. Will they use similarity measures to selected accompaniment phrases? Who knows?

The work on score display synchronizes the display of the appropriate part of a musical score with its live or recorded performance. These techniques may be more appropriate to musical education and training, particularly for traditional brass, string and woodwind players. Yamaha derives considerable revenue from traditional instruments and this is perhaps a way to enhance their “ecosystem” for traditional acoustic instruments.

Score display is one possible application of Yamaha’s patented technique to transmit performance data via near-ultrasonic sound. The technique borrows one or more tone generation channels to generate the near-ultrasonic data signal. See my earlier post about U.S. Patent 8,779,267 for more details.

So long for now!

That’s it! I hope you enjoyed this brief tour through a few of Yamaha’s recent patent grants and filings.

If you want more information about a particular patent, then cruise on over the the U.S. Patent and Trademark Office (USPTO) web site. Navigate to patent search and plug in the patent number.

The long view

Posted on October 26, 2016 by pj

Here’s some information attributed to Martin Harris from Yamaha. Martin is one of the key sound developers at Yamaha:

Better Pianos
New Strings – 70 piece Seattle Symphony Orchestra Mega
New Orchestral Brass – highly dynamic
New Tuned Percussion – Glock, Xylo, Marimba and Vibes (with motor on)
New Mega guitars – Telecaster with Finger and Plectrum
SA2 Celtic Violin
New Synth Voices
New Classical Choir – Cathedral ambience
New Gospel Choir – Various articulations and Ad libs
New Pop Vocals – 4 session singers, 2 male and 2 female
Singing many dynamics and many articulations (wave cycling)

Montage? No, Tyros 4. The “SA2” should be a clue as the Montage does not provide Super Articulation 2 (SA2) voices.

My purpose here is not to be tricky, but to make the case that sample-based workstations or synthesizers draw from the sound pool that is available at development time, much the same way that hardware designers draw on the pool of available components. Products cannot be composed of imaginary circuits (“sand”), software, and sounds, after all.

To better illustrate this point, here is a rough timeline for the Tyros and Motif product lines with a few mid-range products (S9xx and MOX) thrown in:

             Tyros                        Motif/Montage
----   ------------------  ------------------------------------------
Year   Model     Physical  Model     Physical  Uncompressed waveforms
----   ------------------  ------------------------------------------
2001                       Motif      48MB     84MB 1,309 waveforms
2002   Tyros      96MB
2003                       Motif ES   96MB     175MB 1,859 waveforms
2004
2005   Tyros 2   192MB
2006
2007                       Motif XS  128MB     355MB 2,670 waveforms
2008   Tyros 3   256MB
2009
2010   Tyros 4   512MB     Motif XF  256MB     741MB 3,977 waveforms
2011                       MOX       128MB     355MB 2,670 waveforms
2012   PSR-S950  256MB
2013   Tyros 5   768MB     MOXF      256MB     741MB 3,977 waveforms
2014
2015   PSR-S970    2GB
2016                       Montage     4GB     5.67GB 6,347 waveforms

I included physical wave memory size for each product. I also included the uncompressed total sample size and number of waveforms for each member of the Motif/Montage line.

Clearly, Yamaha know how to ride the memory technology curve. Memory technology has progressed to the point where it is no longer a significant hardware design factor. Rather, the amount of wave memory in a product depends more upon the ability of the sound designers to fill it with quality content and mid- versus premium-product grading (i.e., the target market segment and price point for the model). For example, note that the mid-range S970 has more than twice the physical wave memory than the Tyros 5. Although the “expansion memory” is reserved in the S970’s physical wave memory, the S970 waveform content is substantially smaller than the Tyros 5.

The other characteristic to note is how the Tyros and Motif lines tend to leapfrog each other. Generally, the Tyros line leads the Motif line in physical wave memory and content. This is partly due to the higher memory requirements of SA2 voices, which require many additional articulation samples.

Both the Tyros 4 and Motif XF were released in 2010. Both machines use two SWP51L tone generators. (Newer products like the Montage use the SWP70 tone generator.) The Tyros 4 has twice the physical wave memory capacity with respect to the Motif XF. Yet, the Tyros 4 has sample content which did not make it to a deliverable product in the Motif line until the Montage in 2016: Seattle strings, orchestral brass, Celtic violin, vocals (choir and scat), Telecaster guitar and suitcase electric piano.

Tyros 5 expanded this content in 2013. The Motif XF, on the other hand, received a significant update in January 2014. The V.150 update added the “Real Distortion” effects implemented by the Tyros 5. (A few Real Distortion effects actually premiered in the mid-range S950.) The V1.50 update and the “White Motif” color job were life-extenders for the Motif line. I’ve conjectured before that Montage development was late and this is further evidence.

So, what can we expect in the Tyros successor which I’m calling the “Tyros++”. (Yamaha have trademarked the name “GENOS” which may be the name of the follow-on. Only Yamaha really knows.) Personally, I’m hoping for the new orchestral woodwinds from Montage. These are superbly expressive voices. I’m also expecting improved electric pianos, again, of comparable quality to the Montage.

SA2 voices will probably remain exclusive to the Tyros line. Many folks hoped that Montage would have SA2 and it didn’t. SA2 is an important product differentiator — kind of like the premium “Natural” piano voices are to the Clavinova line. I suspect that FM voices will be a differentiator for the premium Montage line in years to come as well. Yamaha tends to think of these three product lines as distinct, so cross-over is carefully controlled and limited.

All of this talk about samples and wave memory size is overly reductionist. The three main (DMI) product lines — Tyros, Motif/Montage, Clavinova — have distinct personalities and features. Motif/Montage is a synthesizer for stage and production studio. Clavinova is primarily a home or church piano. Tyros serves double duty as a home keyboard and as a workstation for performing professionals. (Oddly, many USA customers scoff at this latter role.)

Although these are all fine instruments, the personalities have quirks. Upper-range Clavinovas are Tyros-in-disguise except for multi-pads, third RIGHT voice (i.e., only two voice layers in the right hand), and no expansion memory. Tyros does not have the deep editing or modulation features of the Motif/Montage. The Motif and Montage — strangely! — do not have a tonewheeel organ mode. This latter omission is hard to understand since the Montage competes against other “stage” products like the Korg Kronos and Nord Stage.

Having compared voice programming between PSR-S950 (Tyros 3 without SA2 voices) and MOX (Motif XS sound set), the product lines are voiced (programmed) differently. Motif/Montage effect programming has a harder edge than the Tyros, which is oriented toward oldies, pop and jazz standards. (Yes, Virginia, the Tyros does have latent EDM potential to be tapped.) If the Tyros++ includes the orchestral woodwinds, for example, they will probably be programmed rather differently than Montage. Tyros++ four-part divisi ensembles with the new orchestral woodwinds would be simply brilliant. Can’t wait to see and hear what happens!

One finally editorial comment. The world is filled with product reviews. Publications like Keyboard magazine, Electronic Musician, etc. focus on individual products and rarely present a deep, long-term perspective on products. Sound On Sound reviews occasionally give historical background — usually for esoteric, retro studio pieces. As consumers, we need the long view in order to make the most informed choice.

Rainy day ramblings

Posted on October 22, 2016 by pj

A rainy New England day and the leaves and pine needles are piling up. Can’t do much of anything outdoors today, so off to GC. (Not that I really want to do yard work.)

No real agenda. I’ve been thinking about Yamaha Montage vs. Tyros 5++ vs. Nord Stage 2 ex vs. Electro 5d. That’s all “long term” as I’m having a lot of fun and staying busy with the S950 and MOX.

I really could use a “lap piano” for rehearsals. (A distant relative of “floor melodica?”) My body ached so much last Wednesday before rehearsal that even an eight pound Korg Triton Taktile was too much to schlep. So, I sang with the group, hoping to internalize the melodies of the new music for the week. This isn’t such a bad idea in any case, since it’s good form to sing along in one’s head while playing — improvisationally or not. A good reminder that, yes, hymns actually have words.

So, the issue of mini-keys rises from the grave like Joan Crawford. About one month ago, I sought and found a Yamaha Reface to try again. As it seems for most interesting music tech, one needs to drive a zillion miles or take two or more trains to find and play Reface, Montage, Korg Arp Odyssey and so forth. And thus it was to play a Reface DX. I had a fair chance to plink away and the DX provided a wide range of solid sounds. But, still, no love for the Reface mini-keys. I simply cannot imagine playing a Reface at rehearsals and even remotely enjoying the experience.

Today’s journey was inspired by a favorable review of the new Korg MicroKorg S in Sound On Sound magazine. What a pretty picture it is; Korg’s industrial design may ape Arturia, but they took the best! The review mentioned the larger mini-keys (what an oxymoron!) of the Microkorg XL+ and I decided to find a comparably equipped Korg.

Happily, today’s trial was the Minilogue, which proved to be a fun time indeed. It’s got a pretty sweet sound for an inexpensive polyphonic analog synth. With the right programming, I could even warp the Minilogue into a “lap piano” good enough for rehearsals. A built-in speaker a la the new Microkorg S would be nice. However, I could easily run it into the JBL Charge 2 that serves as the battery-powered amplifier for the Triton Taktile 49 (my usual rehearsal ax). It’s a shame that the Minilogue isn’t battery powered, too, as it would make a terrific portable instrument.

The Minilogue’s oscilloscope is a real treat and is totally entertaining. It’s also a reminder that I need to add a mini-/micro-oscilloscope to the dining room lab one of these days. The oscilloscope display is a small OLED screen much like the screen in the Triton Taktile.

The Minilogue’s keys are far more playable than the Reface. The keys are longer than typical mini-keys and the black keys (sharps and flats) are narrow. This combination makes for a surprisingly effective keyboard design. I wouldn’t want to play a gig with these, but they are suitable for plinking out melodies and such at rehearsal. (See this article at Synthtopia for a good analysis of the Minilogue’s key size.) Several other Korgs have the same key design: the Korg Arp Odyssey and the “Natural Touch” microKEY, to name two.

I’ll say this for Korg. They may miss the mark sometimes, but these folks are actively innovating at a fast pace!

After messing with the Minilogue, I revisited the Nord Stage 2 ex. This is a fine instrument and is in the same premium range as the Yamaha Montage. Having also revisited the Montage in recent weeks, the Nord’s string and woodwind voices just don’t come up to the same level as Montage. The Montage voices live and breath. Although the Nord is quite good, these voices sound like “sample playback.” Kudos to Yamaha.

I will have more to say about Montage in a forthcoming post. In short, is it time to spring for Montage or wait for the successor to the Tyros 5 (“Tyros++”)?

Time for a cuppa…

New Yamaha workstation at NAMM 2016?

Posted on December 11, 2015 by pj

True gearheads are already making predictions and plans for 2016 Winter NAMM, January 21-24, 2016. Winter NAMM rumors abound including “Montage,” the rumored name for the rumored new Yamaha synthesizer workstation.

See the list of new waveforms in the Montage and read my initial review of the Montage8. Update: May 10, 2016.

Find the latest links, pictures, rumors and facts here . Update: January 21, 2016.

Check out some new thoughts about the rumored workstation and preliminary comments . Update: January 18, 2016.

Many folks — myself included — anticipate the release of a new Yamaha synthesizer workstation at the next NAMM. Much has been made of the registered trademark “Montage.” I don’t really care too much about what they call it, as I care about what it will do.

Last month, I posted two articles about the new Yamaha tone generation chip called “SWP70”:

This chip made its first appearance in the new PSR-S770 and PSR-S970 arranger workstations. Lest anyone scoff, the S770 and S970 produce Motif-caliber sounds including the REAL DISTORTION effects added to the Motif XF by the v1.5 update. The previous tone generator (SWP51L) is used throughout the mid- and upper-range Yamaha keyboard products including Clavinova, MOX/MOXF, Motif XS/XF, and Tyros 4/5. The number of tone generator chips varies by product specification and, most notably, sets the maximum available polyphony. A new tone generator chip is a pretty big deal since it will have an impact on all mid- and high-grade electronic instruments across product lines.

My earlier article about the SWP70 is written from the perspective of a computer architect and is way too nerdy for normal people. 🙂 Let me break it down.

Musicians using VST plug-ins within a PC-based DAW are familiar with the concept of sample streaming. In the quest for greater realism and articulation, sample libraries have become huge. These libraries simply cannot fit into fast random access memory (RAM) for playback. As a work-around, a software instrument reads samples from a drive-based library on demand and only a small part of the entire library is resident in RAM at any given time. The process is often called “sample streaming” because the software instrument streams in the samples on demand from a large fast secondary memory like a Solid-State Drive (SSD). The Korg Kronos workstation caught everyone’s attention because it incorporates an x86-based software system that streams samples from an SSD. (For Kronos-related articles, look here and here.)

The SWP70 combines streaming with tone generation. It does not, however, use an SSD for storage. Rather, it subsumes the functionality of the SSD. A moment to explain…

An SSD consists of three major subsystems: SATA controller, temporary storage cache (RAM) and one or more NAND flash memory chips. The NAND flash memory chips typically adhere to the Open NAND Flash Interface (ONFI) standard. This allows expansion and standardized configurability. The SATA controller exchanges commands and data with a computer using the SATA bus protocol. The temporary storage cache holds data which is pre-read (cached) from the NAND flash chips. Caching is required because random access read to NAND flash is too slow; sequential paged access is much faster. Data must be prefetched in order to achieve anything like SATA 1 (2 or 3) transfer speed.

The SWP70 subsumes the SSD functionality. It has its own memory controller and has a side memory port to its own RAM for caching samples. The SWP70 reads samples from its ONFI-compatible NAND flash memory bus and stores the samples in its cache. The tone generation circuitry reads the samples from the cache when it needs them. The SWP70 solution is, effectively, sample streaming without the added cost and latency of SATA bus transfers. The samples coming into the SWP70 from flash are compressed, by the way, and the SWP70 decompresses them.

The SWP70 will very likely make an appearance in the new Yamaha synthesizer workstation. The S770 and S970 do not make full use of the SWP70, so we have yet to see what this chip is fully capable of. We can definitely expect:

Much larger wave memory (4GBytes minimum)
Greater polyphony (256 voices or more)
More simultaneous DSP effects (32 units or more)
The demise of the expensive expansion flash DIMMs

I would simply love it if the new workstation implemented some form of Super Articulation 2 voices (now supported by Tyros 5). The raw resources are there.

User-installed expansion memory may be a thing of the past. The current DIMMs plug into a two channel, full parallel memory interface. That interface is gone and the SWP70 communicates with flash NAND through an ONFI-compatible interface. The Motif and Tyros follow-ons will likely reserve space for user samples and expansion packs in built-in flash memory just like the new mid-range PSRs.

What does Yamaha intend to do with all of this polyphony? Current high-end models like the Tyros 5 use two tone generation chips. Yamaha could replace both chips with a single SWP70 and pocket the savings.

Another possibility is to provide advanced features for musical composition that combine MIDI and audio phrases. Here is a list of technologies covered by recent Yamaha patents and patent applications:

Beat detection and tracking
Chord detection
Synchronized playback of MIDI and audio
Combined audio/MIDI accompaniment (time-stretch and pitch-shift)
Object-oriented phrase-based composition on a time-line
Accompaniment generation from chord chart
Display musical score synchronized with audio accompaniment
Phrase analysis and selection (via similarity index)
Near ultra-sonic communication of control information
Search for rhythm pattern similar to reference pattern

A few of these technologies are covered by more than one patent — recurring themes, if you will. I could imagine a screen-based composition system that combines audio and MIDI phrases which are automatically selected from a database. The phrases are transparently time-stretched and pitch-shifted. Some of the compositional aids may be implemented in the workstation while others are tablet-based. The tablet communicates with the workstation over near ultra-sonic sound (no wires, no Bluetooth, no wi-fi, no time lag).

Sample-based tone generators already perform pitch-shifting. That’s how a single sample is stretched across multiple keys. A musical phrase can be pitch-shifted in the same way. As to time-stretching, stay tuned.

Some of these features, like accompaniment generation from a textual chord chart, are more likely to appear in a future arranger workstation product. Making product-specific predictions is a risky business, especially if you want to get it right!

Yamaha — the business — is keenly interested in growth and expanding markets. Management sees opportunity in growth markets like China. The need to combine audio phrases with MIDI is driven by non-Western music: time signatures other than 3/4 or 4/4, different scales, different playing techniques and articulations. These concerns are perhaps more relevant to the arranger product lines. However, phrase-based composition that manipulates and warps audio and MIDI transparently is a basic feature of many DAWs. (Think “Ableton Live.”)

One final theme seems to recur. Yamaha appear to be interested in analyzing and accompanying non-keyboard instruments. The market for guitar-driven accompaniment is much wider and deeper than today’s arranger workstations and is a lucrative target.

Here are links to a few earlier articles, including speculation about the new Yamaha synthesizer workstation:

These articles link to further background information. Of course, we’ll know a lot more once Winter NAMM 2016 is underway!

The SWP70 tone generator

Posted on November 13, 2015 by pj

As I mentioned in an earlier post, the Yamaha PSR-S770 and PSR-S970 arranger workstations have a new tone generator (TG) integrated circuit (IC) — the SWP70. (“SWP” stands for “Standard Wave Processor.”) The SWP70 is a new TG family in a long line of Yamaha tone generators. The SWP70 replaces the SWP51L, which has been the mainstay in recent generations of Tyros, upper range PSR, Motif, and MOX series workstations.

The SWP70 has much in common with the SWP51L, but also some very significant differences. The SWP70’s external clock crystal frequency is 22.5792 MHz versus 11.2896 MHz for the SWP51L. This funky looking clock rate is a multiple of 44,100 Hz:

    22.5792MHz = 44,100Hz * 512

Samples are transferred to the DAC, etc. at a multiple of 44,100 Hz (Fs). Thus, it makes sense to derive Fs and its multiples from the chip-level master clock. The higher crystal frequency and faster memory read clocks lead me to believe that the SWP70 is clocked twice as fast as the SWP51L.

I am comparing SWP characteristics as deployed in the S970 (SWP70) and the S950 (SWP51L) workstations. This keeps the basis of comparison even although many characteristics (clock rates, DSP RAM size) are the same in higher end models like Tyros 5 or Motif. Higher end models employ two SWPs in master/slave relationship and both SWPs share the same wave memory. For more information about the PSR-S970 internal design, look here.

Five interfaces are essentially the same as the SWP51L:

CPU interface: Communicate with the Main CPU (e.g., Renesas SH7731) via the parallel CPU bus.
Serial audio: Send/receive audio data to/from the DAC, audio ADCs, and main CPU.
Clock interface: Synchronize serial audio data transfers (generate multiples of Fs).
DSP SDRAM interface: Store working data for effect processing.
EBUS interface: Receive controller data messages (e.g., pedal input, keyboard input, pitch bend, modulation, live knobs, etc.) from front panel processors.

The DSP SDRAM is the same size: 4Mx16bits (8MBytes). The SWP70 read clock is 95.9616 MHz, while the SWP51L read clock is 45.1584 MHz. This is more evidence for a higher internal clock frequency.

The Tyros 4, Tyros 5 and S950 have an auxiliary DSP processor for vocal harmony. The microphone analog-to-digital (ADC) converter is routed directly to the auxiliary processor. Prior to these models, the microphone ADC is connected to the tone generator. With the SWP70, the S970’s microphone ADC is once again routed to the SWP70 and the auxiliary processor disappears from the design. Thus, vocal harmony processing (fully or partially) is located in the SWP70. See my post about SSP1 and SSP2 for further details.

The biggest change is the wave memory interface.

A little history is in order. The SWP51L (and its ancestors) were designed in the era of mask programmable ROM. I contend that tone generation is memory bandwidth limited and the earlier interface design is driven by the need for speed. The SWP51L (due to its evolved history) has two independent wave memory channels (HIGH and LOW). Each channel has a parallel address bus (32 bits) and a parallel data bus (16 bits). The two channels account for over 100 pins. (System cost is proportional to pin count.) The user-installed, 512/1024MB flash DIMMs plug directly onto the two channels.

The SWP70 wave memory interface takes advantage of new NAND flash memory technology. The interface is described in US patent application 2014/0123835 and is covered by Japanese patent 2012-244002. I analyzed the US patent application in an earlier post.

The SWP70 retains the HIGH port and LOW port structure. Each port communicates with an 8Gbit Spansion S34ML08G101TFI000 NAND flash device. Address and data are both communicated over an 8-bit serialized bus. This technique substantially decreases pin count and the resulting board-/system-level costs. Smart work.

I did not anticipate, however, the introduction of a new parallel memory interface called “wave-work”. The wave work interface communicates with a 16Mx16bit (32MBytes) Winbond W9825G6JH-6 SDRAM. The read clock is 95.9616 MHz.

The purpose of the wave work SDRAM is revealed by US Patent 9,040,800. This patent discloses a compression algorithm that is compatible with serialized access to the wave memory. The wave work SDRAM is a cache for compressed samples. The characteristics of the Spansion memory device give us a clue as to why a cache is required:

    Block erase time               3.5ms    Horrible (relative to SDRAM)
    Write time                     200us    Terrible
    Random access read time         30us    Bad
    Sequential access read time     25ns    Very good

As the patent explains, two (or more) samples are required to perform the interpolation while pitch-shifting. If there is only one tone generation channel, access is paged sequential. However, random access is required when there are multiple tone generation channels. (The patent mentions 256 channels.) Each channel may be playing a different voice or a different multi-sample within the same voice. One simply cannot sustain high polyphony through random access alone. The cache speeds up access to recently used pages of uncompressed samples.

The wave work interface takes additional pins, thus adding to board- and system-level costs. The overall pin count is still lower when compared to SWP51L. The penalty must be paid in order to use contemporary NAND flash devices with a serialized bus. This is the price for catching the current (and future) memory technology curve.

A few SWP70-related printed circuit board (PCB) positions are unpopulated (i.e., IC not installed) in the PSR-S970. There is an unpopulated position for a second Winbond W9825G6JH-6 wave work SDRAM which would expand the wave work memory to 32Mx16bit (64MBytes). A larger cache would be needed to support additional tone generation channels. Perhaps only half of the tone generation channels are enabled in the mid-grade PSR-S970 workstation.

There is what appears to a second separate wave work interface that is completely unpopulated. The intended memory device is a Winbond W9825G6JH-6, which is consistent with the existing wave work interface.

The PSR-S970 also has a stubbed out interface that is similar to the DSP SDRAM interface. The existing DSP SDRAM signals are labeled “H” for HIGH while the unused interface is labeled “L” for LOW. Perhaps only half of the hardware DSP processors are enabled for the mid-grade S970, waiting to be activated in future high-end Tyros and Motif products.

I refer to future high end products by the names of the current product lines. Yamaha may choose to rebrand future products (e.g., the much-rumored “Montage” trademark).

The Spansion S34ML08G2 8-Gb NAND device is Open NAND Flash Interface (ONFI) 1.0 compliant. The S34ML08G2 device is a dual-die stack of two S34ML04G2 die. The 8-bit I/O bus is tri-state allowing expansion e.g., multiple memory devices sharing the same I/O bus and control signals with at most device enabled at any time. The SWP70 has additional chip select pins that would support this kind of expansion. The current expansion flash DIMMs will no longer be needed or used.

In this note, I concentrated on observations and fact, not speculation about future products. I’ll leave that fun for another day!

Sand, software and sound

Electronics and computing for the fun of it

Tag Archives: Montage