MySong, from Microsoft Research, makes your singing sound a lot better than it really does

Microsoft Research MySongCue Scoble tears. This is one hell of a technology from Microsoft Research that I think will have a huge impact on how amateurs and hobbyists write music. The technology and software prototype is called MySong.

In a nutshell, the software records your singing (preferably in tune) through a microphone, and it systematically generates an instrumental accompaniment for your song. The quality is even comparable with a professional accompanist, not to mention the cost and time involved.

If you don’t believe me, have a look at this quick 5-minute demo with loads of practical examples.

[flv:MySongCHI2008.flv 630 420]

The technology was developed by Ian Simon from University of Washington in collaboration with Dan Morris and Summit Basu from Microsoft Research. They’ve published all of their research and methods in a paper (PDF) of the same name to present at the CHI Conference 2008 in a couple of months in Italy.

Without getting into too much detail, the technology uses the Hidden Markov statistical model which has been ‘trained’ by preprocessing a database of nearly 300 musical lead sheets. The sheets come from a variety of sources, genres and popularity to give a broad set of melody and chord sequence combinations. When it comes to the end user, it uses this abundance of data to find chord sequences which work best for each segment of melody. Presumably that means more ‘training’ can be done to improve the output even more.

The paper also points out “there is not a single correct accompaniment for a particular melody; chord selection will vary among musicians and genres, and a single musician may recognize many appropriate chord sequences for a single melody.” As a result, the software also gives the end user some room for creative adjustments in the form of “jazz factor” and “happy factor”. Both of which have a reasonable effect on the resulting accompaniment as demonstrated in the video.

If that wasn’t enough already, the researchers sought reviews from 30 independent musicians asking them to rate the different accompaniments produced by three different systems from one vocal input. They compared the result of MySong with hand-crafted chords and as well as Band-In-A-Box, the “state-of-the-art” commercially available software which they recognize to be the only automatic accompaniment generation system.

Here is just one example of the twelve they’ve ranked. I’ll leave you to judge for yourself.

Original input [flv:mysong/input.8.mp3 300 0]
MySong [flv:mysong/song.8.0.mp3 300 0]
Band-In-A-Box [flv:mysong/song.8.1.mp3 300 0]
Hand-crafted [flv:mysong/song.8.2.mp3 300 0]

Like many cool technologies at Microsoft, this remains still a research project. Whilst there’s no written indication of a commercial product as a result of the technology, but I don’t see why they wouldn’t or even just release the software as is.

As for future work, they say “additional development will focus on improving and diversifying the audio generated by MySong; the system is already able to supply chords interactively to a pattern based arrangement tool, which results in compelling audio output. Several study participants indicated that MySong would be of significant value for learning music theory; we are thus excited about exploring educational applications of this technology.”

77 insightful thoughts

  1. Wow, pretty cool. MSR really is an amazing part of MS. They dream it, and make it! I think going forward, 10-15 years down the line, it will be MSRs projects raking in all the revenue.

    1. hello can you help me to Gat
      MySong, from Microsoft Research, makes your singing sound a lot better than it really does softwar
      pless thank you

  2. This software as is, with a much nicer GUI, would be perfect as a Vista Ultimate Extra, and can really help selling Vista copies.

  3. Haha I loved the usage scenarios & the credits music. I agree, sounds like a great Ultimate Extra. πŸ™‚

  4. Sounds cool!

    I wish that there was something where you could hum or sing a song and find relevant results for the song you hummed/sang.

  5. From a musicians perspective this was just bad. The chords are predictable, even for a non-musician. The only thing worth noting is the pitch analyzer – which have been with us since the early Windows days. The jazz factor thingy made it a really bad software. “Let’s use dim-chords, even if they may not match”.

    No thank you.

  6. To my untrained ears all the accompaniments sounded the same. Maybe if they switched the General MIDI sounds around a bit it would be more compelling/convincing.

  7. Let me guess: Morten is the type of person that watches people play Guitar Hero and snarkily suggests that they learn to play a real guitar.

  8. @Fred: Sorry, I’m with Morten. And to use the Guitar Hero example, Guitar Hero and Rock Band took great pains to ensure the music sounds like the original.

    The problem with this is, it doesn’t work right. The chords on “The Way You Look Tonight” don’t connect musically — and they’re wildly inferior to the original. What the demo really illustrates is that you can reproduce some basic harmonic rules, but not come out with something at all comparable to what a human would come up with.

    Okay, I do see the idea of allowing amateurs to put together harmonies. But they need more control than two sliders. In fact, the beauty of sitting for hours with your guitar or keyboard is you eventually put together something meaningful. Mostly what this does is rob the musician of that choice.

    I wrote some other ideas here:
    http://createdigitalmusic.com/2008/02/29/mysong-your-own-virtual-tone-deaf-accompanist/

    And, truly, this isn’t to be snobby — I think the untrained, amateur musician is going to have a better experience playing around with an instrument, or something like GarageBand if they can’t play, than they are with this. And you have a *different* experience when you learn to do something. Guitar Hero doesn’t replicate playing a guitar — that’s not the point. But this *is* trying to replicate something, and it isn’t succeeding. It’s a worthy task, and you learn a lot from what doesn’t work and what does, but it’s a long, long way from working the way it promises.

  9. For simple songs, I guess this is a pretty good program. But for more complicated ones Band in a Box (which I use) is much better. Though in the example the chords chosen by BiaB weren’t all matching, it was better and less predictible that MySong.
    Oh and @Orion, you might like http://www.songtapper.com

  10. LOL. I absolutely love Emily – she’s the best of all of them. Congrats on the new website too, change the font to verdana though.

  11. I like the new theme Long, though its going to take some time to get used this smaller content area (well, I guess I could just close Favorites Center too…)

  12. This is stupid. It sounds like a technology from 1995. It’s like the “make your own music” program in Photo Story 3.

    I love to write and produce music, and I’m going to go to college for music production (in 4 years). I can’t stand when companies come out with lame programs for “Music Production.” If their going to make a program, make it like Garageband on the mac.

    I don’t like Garageband very much, but anyone that owns a mac can make music or podcasts. Garageband has really good loops, synths and effects, and you can choose to make your own melodies or not. In this it analyzes the pitch of the vocal (like melodyne (celemony.com)), and makes chords that work with your “song.” Technically it’s not your song. The same thing with loops (except for drum loops).

    It’s just my opinion..

  13. What is the point of this article as it doesn’t even have a download for a demo program at the least….Yet another Tesla like promotion…

  14. FYI, there are ways of producing music easily without a Mac. Steinberg’s Sequel is a very powerful alternative that’s cross platform, and there are entry-level versions of software from Cakewalk, Sony, etc. But then, yeah, that does beg the question of why you need the software to harmonize for you. It’s an interesting idea, but in practice doesn’t really make things any easier.

  15. The new website design sucks. Either stick with the old one or modernize the new one. It’s not visually appealing.
    Just an advice no offense. U have the right to do whatever u want. After all ur website.

  16. The design of the website is okay, certainly needs improvement. The older one looked more professional I think.

  17. Love the new design Long, very clear and easy to use (and aestheticly appealing)

    also, you need to put the “Genuine istartedsomething” button back on at the bottom

  18. Make it look visualy cooler, and you’ve got fans. I listened to all the samples, and I barely got much of a difference between them. MySong sounds promising.

    But why do they always show off their stuff from Research? MSR is neat, but outside of MS fans, nobody hears much about it. And other people come out later and make similar-like things, and by the time MS rolls it out the door, they look like old news. Kind of dumb if you ask me. They should just finalize it right, and then show it off.

  19. OMG I want it!!!!!!, very cool and fun software, release this as a Vista Ultimate Extra!!!! I REALLY WANT IT!!! πŸ˜€

  20. That music is quite predictable and rather cheezy.
    It DID sound a bit better than band in a box though.

  21. Its nice idea, but sounds like it needs more work. All the songs really sound the same but just played at different speeds.

    Looking forwad to the final release with some other tunes and instruments

  22. Great article, but I just needed to point out that I am a music producer, and there is a program called Antares AutoTune which does the same thing. (Ever heard T-Pain? He can’t sing without it!) πŸ™‚

  23. Pingback: technabob
  24. I initially clicked with enthusiasm, and then disappointment filled me as I realized that it doesn’t do ANYTHING but pick chords and play them all back using the same rhythms and patterns. If I paid an accompanist to help write a song and this is what they came up with, I’d suggest they go back to 5th grade music class.

  25. Pretty cool, imho. But after listening to a few examples, all of the generated accompaniments start to sound like each other. A great next step for MySong would be to enable variety in rhythm and tempo of the accompanying chords, not just pitch. From the whitepaper:

    “Chords are generated at a fixed interval that corresponds to a specific number of beats at the known tempo”

    …which is what gives most of the tracks a bit of a lifeless, metronomic feel.

  26. Frankly, I dont like the whole idea at all. The beauty of creating music is that you are creating it, not some machine. It just takes the whole feeling out of it. Besides, all the computer is doing is telling which note the person is singing, and setting a standarized set of chords to it, no ingenuity involved.

  27. I was programming stuff like this in 1995 using MIDI. Never got results close to real musicians/composers so I gave it up. based on the gven examples, MSR’s work is impressive.

  28. Honestly , the program is just as brainless as someone with no musical ability trying to write music. If you cant write music, and have no musical ability whatsoever , you shouldn’t try . ITs like someone getting a program that makes their drawings into architectural blueprints. It won’t make you an architect . That takes skill and dedication.
    If you have to use a program like mysong to write your music for you, you probably shouldn’t quit your day job, and PLEASE whatever you do, don’t try to get in to the music industry,
    there’s already enough talentless clones flooding the airwaves with garbage. Mysong would only contribute to that .

  29. Wow, this is amazing for what it does. It blows the one in Garage Band to pieces. It even blows away the live accompaniment. I love the varied chords it generates considering they are a whole lot more interesting than the ones in most or even all pop songs today. I would most certainly use it with my music composition students, to show them how many different chord options are possible with a given melody, and to teach them how to hear which chords on the circle of fifths it is choosing… and why they sound the way they do.
    I absolutely love it! I would buy it for my music school as a teaching aid in a heartbeat! A+++++
    PS: For those who think it is so “canned” or “stupid”, lets see you come up with a better program!

  30. This is another example of the “I need it now and I need to be able to do it now” attitude that permeates our society. If you bothered to even take music lessons for 1 year, you could do everything that this program claims to and would most likely do a better job considering the chord progressions this program puts together are questionable. As a very accomplished pianist, this program in no way emulates what a real accompanist would do for you.

    And p.s. I love rockband and guitar hero

  31. Surely it’s PhotoBooth for singing.. and isn’t that the point? (and, if not, shouldn’t it be?!)

    People love playing with PhotoBooth – gurning gleefully at their Macs with their friends, taking snapshots to put into their own little hall of mirrors. It’s pointless, but it’s fun for about ten minutes and gives you a nice warm & fuzzy feeling about using a Mac.

    Windows can feel too worthy at times, bogged down in business justification, ROI and design-by-committee and a few “gimmicks” like this in the consumer-oriented “Home” editions would do no harm I’m sure.

    It’d be great if it could do a spot of auto-tuning to a ‘best guess’ melody to suit a chord pattern, or even change the rhythm of the singing to suit a particular style, but I’m sure nobody’s intending it to replace years of study and natural talent… those suggesting it is are, quite frankly, just sounding a tad arrogant and elitist.

  32. Anyone notice how this new “Microsoft” product sounds almost the exact same as the “state-of-the-art” software? This situation seems oddly familiar. hmm…

  33. Ahhh!!! Kill it with fire! No one could possibly think that horrible vocals somehow sound fantastic with cheesy chords in the background. That first chick who murdered Sinatra – blasphemy!

  34. Is amazing how people confuse the purpose of the software, even thinking is made to make professional music! for god’s sake! is a software for amateurish and karaoke people!

  35. I am impressed. It kept the integrety of the submitted piece unlike BinB which warped the original. It is good enough to use as an early introduction to songwriting. It would improve as it is worked on. When you start working on a piece of music, how often do you change it, how often do you forget some of the magic you create in earlier renditions. This would be a fun program to have. I could never live with BinB. Keep working and good luck.

  36. hey this shane and i love this idea i want the software, the only think that i see that i have a problem with is the instrument, is it only piano? what if someone wants like a techno sound or a rock distortion sound to chords of the song they want to make.

  37. Wow, I have been looking for a software like this from colege. Now , 10 years later, I bought a Yamaha and I try to put chords to my songs but its a real headeak, and even if other software like BIAB are helping, for someone like me, whit no musical studies this software will be a bless and I will be able to fulfill my dream of making a CD whit my music and show it to my frends and family. So please contact me when this software is released (even in the beta or demo version ) or if u need testers for this software, if I can, I will be glad to help.
    Many of the people and frends I know will be intrested in this software.
    I am intrested to try even the version of Mysong that you used for examples.
    Thanks, and hope to hear of Mysong release soon.
    Till then I will work on the lyrics and melody of my songs. Bye

  38. is the mysong software on sale yet? if so where can i buy it? I have been writing songs for years but can not play an instrument or write music so this would be ideal. can anyone advise.

  39. I think they should make this no download and i think it should be free because if its not a guitar would probably be cheaper πŸ˜›

Comments are closed.