Maybe additionally highlight the notes that are currently played. And have you tried scrolling the sheet instead of moving the 'now' indicator? And who are you and why are you doing this? Add an about section.
Yup, I can do the highlighting fairly easily, that's a good idea!
Scrolling is easily doable too, but I'm not sure that's the best way for all situations. I might provide it though, and then let the user choose depending on their situation.
About me: a wannabe classical composer / tech geek ;)
(and in the event people are actually interested in this idea becoming more than just a concept, I would be very happy to provide a more elaborate About section)
Even a "right now this is just a little concept I worked on in my free time, hope you find it useful. Tweet me suggestions @..." would be enough on /about.
Some of Sibelius's core developers we let go recently. Steinberg hired them and they're writing a new music notation application from scratch. They're still in the very early stages of development. (see http://blog.steinberg.net/)
I like it. I have a few suggestions (full disclosure: I haven't played music since high school).
It would be nice if the key and time signature could remain onscreen as the sheet scrolls.
I think my preferred solution for scrolling would be to have the position marker move until it hits the center, then have the position stay centered while the sheet scrolls until it hits the end. I'm not sure what the correct way to handle repeats in my scenario would be--perhaps have the sheet stop scrolling when it hits a repeat.
I think having the key / time signature "stick" makes a lot of sense... I will work on that. The only thing to look out for is key signature / time signature changes, and make sure those get captured too and that they stick...
As for scrolling, it seems like that is a feature that some people might prefer. I will most likely add that as a feature to it.
I think having the page stay still for the majority of the time (like it is now) is better than to have it constantly moving, but when you do change pages, scroll to the new position (ease-in, maybe ease-out too).
Removes the disorienting jolt of having everything just change, but also not always moving and being a distraction
I really like this and would use it at a concert, but the page turning is very jarring. Having the time sig fixed while the sheet scrolls like some sort of symphony hero would be pleasant.
Hell, you could even color code the instruments subtly with some light shading. Maybe make that a checkbox option.
Well it seems that consensus is that the current page turning is not optimal... Perhaps scrolling is the solution. I will definitely invest some time in that to improve it.
Also, regarding the time and key signatures, I very much agree with you, they need to "stick"...
As for color coding, is the purpose to make it easier to follow a certain instrument?
As for the page turning - I think you could alleviate a lot of the issues by just slowing it down somehow. I can't remember if it was animated, but, if not, I think it would be less jarring if there was an animated scroll effect, starting slowly and then getting very fast, to cue the viewer that we're about to change pages. Similar to how the iPhone transitions screens, but maybe a bit slower since with the iPhone the user initiates the action and is therefore more prepared, perceptually.
Got it. It is is currently scrolling (but very fast; I think I set it to 100ms for every page turn), so I could certainly slow that down. I will experiment a little bit and see how it looks. Thanks for the suggestion!
The colors would be both aesthetically pleasant (i'd go with pastels) and make it immediately obvious what you're looking at. You could also put the labels next to the note chart on the left.
I think highlighting a particular instrument or two (that you are either playing or focusing on) is a good idea and could be implemented as a checkbox option... so thanks for that suggestion.
I love (classical) music and as a music hobbyist I try to dig more into sheet music for a while. I would also pay for some kind of premium service (maybe great content or listening the sheet music in combination with a famous record).
Anyway, I subscribed to your mailing list, please just keep on rocking.
PS: And yes, I own Logic and could play sheet music in that way. But I do think you folks will prepare something more which a hobbyist can enjoy!
I think so too :) Thanks for the subscription and the nice words! I definitely want to hear your ideas, so please feel free to get in touch (contact information listed under http://notezilla.io/library)
As a guy who is learning the piano but doesn't know how to read sheet music very well, I'm really excited about this as a learning tool. Would it be possible to show the letter notes next to the notes on the sheet music? Also, how easy is it to incorporate new pieces... could I potentially submit sheet music that I have purchased and you could convert it for me for a fee?
If you're wanting to get better at reading sheet music you really should not have notes being shown. It will be harder to read but that just means you need to start simpler and build up over time. If you get used to reading notes it will always be a crutch you have to work with, and instead of sight-reading the notes themselves you'll be more accustomed to sight-reading the letters.
You might be right about that. I still find though that the most daunting part about learning something new for me is grinding through writing the notes in on the sheet music then "sounding it out" going back and forth from my computer/ipod/ipad getting the timing right. This tool seems to wrap it all up into one neat learning process, perhaps one day after using it enough with "notes on" I could uncheck that box and go pure sight reading.
Yes, I am exploring adding more functionality to it, and one thing that I'd like myself (and I think, you too), is piano tutoring.
For example, it could play a certain section once, then repeat it, but without sound, while the student plays, and then continue in a loop like that until the student feels comfortable with that particular section.
I'm sure there are many other things that could be done in that realm too, so feel to tweet or email me anything that comes to mind (@notezillaio / notezilla.io@gmail.com)
As for incorporating new pieces, it's definitely technically doable and it seems that that's something people are interested in. There might be certain copyrights issues that have to be ironed out first though... ;)
Minor bug, Beethoven's 5th was off by a measure for me. (Goldberg variation was spot-on).
Also, right/left arrow keys aren't skipping measures (as the keyboard page says), but are skipping whole sets of measures.
But overall, as a musician, I actually found it rather distracting to watch the line try to follow the notes exactly, as the line constantly sped up and slowed down due to rubato, etc. -- indeed, in particularly fluid pieces, it can even be possible that the right and left hands don't match up precisely in actual performance.
It might actually be more clear, and far less distracting, if you followed the time signature for the beat, and simply highlighted all notes in the current beat within a light-pink rectangle overlay, and then that box would instantly disappear, and the next one appear, upon the next beat.
That way, we can focus with our ears on the actual movement in the music, without being distracted by visual movement, but still have a perfect visual indicator of where we are.
But anyways, great work and great idea! Kudos, well-done!
Oops! I forgot to change the definition of the keyboard controls... it's actually meant to turn a whole page so I just need to change the definition.
(just moving by one measure is more easily done by clicking in the score...)
Thanks for correcting that :)
As for Beethoven's 5th, it should self-correct, but I have noticed that in some browsers it doesn't quite work... What browser and hardware are you using? I will look into this further.
I have heard from several others that the current way of moving the cursor isn't optimal. Your solution could work. I'm going to explore a few ways and probably offer some new choices to the user. Your suggested solution would be very easy to implement, so most likely I will add that.
Tried it again and the Beethoven's now fine. (Chrome, OSX.) The Beethoven audio file had taken about a minute to load though, maybe it had to do with that. Anyways, probably just an edge case.
One minor "bug" / missing feature; I have two monitors at different resolution; when moving the browser (maximised) from my laptop monitor (1440 x 900) to second display (1280 x 1024) the staves are put on a different line to the instrument names. If I refresh the browser on my second monitor then move to my laptop screen all works fine though.
The same issue exists if I resize my browser; i.e. currently window.onresize events aren't handled.
Great work though - and nice choice of pieces too.
Really nice. Now I know the meaning of those squiggly lines. I also like the sparse full screen layout.
A few minor things: While jumping to time points works, scrubbing does not. Instead, the notes image gets dragged. You could embed the image as a background-image instead of an <img> to prevent that. Or catch the dragstart event on it and return false (http://stackoverflow.com/questions/4211909/disable-dragging-...). A custom Rails error page or better error handling would also be nice (http://notezilla.io/bla). And there are a few console.log's left (TIME!!!! :)
Thanks for pointing that out, I didn't anticipate this level of interest so I suppose I forgot to take out the console.log stuff. Will do now. Glad it wasn't anything embarrassing ;)
As for scrubbing, it was a conscious decision to keep it fixed for now, but I might consider changing that in future version... and also need to fix status pages, etc.
Yes, I match it by hand, but it's fairly easy to do using some javascript that I wrote...
I have been wanting to work on this idea for quite a while in my spare time, particularly to enable scrolling and keeping the signatures in place while the sheet scrolls.
This looks prettier than the MuseScore web interface.
Please tell me you will have some way to use this to display a score written in some open format. and/or interface with MuseScore or some other software.
Most likely that will be very hard to do, at least for orchestral pieces, which are typically recorded using overhead microphones (ie, not with separate tracks for each instrument).
For pop music, it's definitely doable as long as I can get access to the individual tracks. It's actually something I have been wanting to do, so thanks for the suggestion.
I'm actually using LilyPond already to create the sheet music, so if I do add the possibility for users to upload their own scores, it would most likely be with LilyPond...
Although the user would still need to partition the score (ie indicate at what time a certain note occurs) if the music was to be synced up to it.
It isn't terribly labor intensive but still requires a little bit of work... so maybe, if that's what people want!
Oh hey, sorry, just realized that you are actually the original author. See my comment above suggesting MusicXML instead, since it's more widely used and machine-readable. As for specifying times, you can use the note lengths and tempo markings to generate a rough timing and then allow the user to tweak it to fit the music.
But this is awesome by the way, great job! I'd really love it if user-generated content is supported. I have a bunch of lilypond transcriptions from the Real Book that I'd like to upload.
Cool, let's stay in touch because I also think that user-generated content is the way forward, and I agree about MusicXML.
Just need to be clever about it and make sure it's not too arduous a task for the user to upload something. Please feel free to email me or tweet at notezilla.io@gmail.com / @notezillaio and we can stay in touch / discuss further, would love more input!
Actually, now that I think about it, it might be better to support MusicXML, which is more machine-readable (you just need an XML parser). Lilypond and many other music notation systems like Sibelius and Finale can export to it.
It's a demo, but you may find some UI ideas interesting, for example synchronization and navigation through a video, flowing score, hiding the instrument parts that are not playing for a while, and assembling a score from small pieces with measure images hosted on multiple subdomains to improve the fetching performance in the browser.
I have considered that, actually. There are certain advantages with it - although, one thing that I like in particular with hosting the audio myself is that for example, I can slow it down to 25% speed and still have it sound great! Not possible with Youtube at the moment (if you slow down their videos, the results are deplorable).
Interesting demo! I definitely like the hiding of the instruments not playing.... also, lots of people seem to prefer a flowing score.
As for fetching more quickly, yup, also worth addressing.
I prefer the entrancing visualization done by this guy: http://www.youtube.com/watch?v=rRgXUFnfKIY (his YouTube channel has much more). As a semi-trained musicians, I do see value in showing the full score, but purely as an audiovisual experience I think removing some of that information and just using colors for instruments and y-axis for relative pitch is more mesmerizing.
In music sequencer a.k.a. DAW software (Logic, SONAR, Cubase, etc.) this is usually called "piano roll" notation. Because it's essentially a player piano roll -- you know, the original lossy compression. ;)
Such software usually offer 3 views of the same underlying music data: piano roll, staff/sheet notation, and an event list.
Wow, that's the best I've felt all day. Truly lost myself in the piece trying to follow the red line and tease the individual notes out of my perception.
I will definitely work on getting that line synced up. On my computer (Mac OS X / Chrome), it's perfectly synced, but it seems that that isn't the case for some people.
FYI, I'm on a 2011 Macbook Air running Safari. It's good enough that I enjoyed the experience. Honestly, I was never absolutely certain that the red line was off. Just suspicious. :)
Wow this is so great. As a musician listening to a classical piece (well and an engineer too) I often think, "How does this work? I hear a clarinet here, is that what it is or something else?" So yeah, very cool.
It's mostly manual work at this point, although it's not too bad because of some javascript that I wrote that makes it really straightforward to partition a score into time intervals...
And yes, I might actually write a blog post about this so make sure you have signed up on the site :)
First off, really nice job. Secondly, if you're not aware, audacity has a segment labeling feature - (I'm just throwing this out here because it might be easier than what you're doing). Select time boundaries and [ctrl+b] will label it. Labels can snap to abut each other. Each label might represent the span of a beat. The labels can be exported as an easy to parse .txt file and can be used to determine cursor position as a function of time.
Sonic Visualiser [1] can---or, at least, it makes an attempt to.
It's a highly useful tool for the analysis of music recordings.
I, too, am highly interested in the extent to which matching recordings to scores can be automated, as are a lot of music theorists. I hope you can publish a write-up of the process for syncing the audio and the score.
If there is a significant human spend required, then developing an open tool for community contributions would greatly speed up the creation of a corpus of scores.
I think a good case could be made for such an automatic system... although it's probably a somewhat too complex / labor intensive task at this particular stage for this particular project!
As for the write-up however, the process I currently use is very simple (and it's pretty much all "manual labor").
To partition a score into time intervals, I have a couple of lines of javascript that record the time the audio is at, together with where in the score I click (and where I aim to click on the note currently being played).
It doesn't take too much time and effort to partition a score this way actually, unless the piece is slightly complex. Beethoven's 5th took a little more effort than Air for example ;)
Hope this helps for now! And I am planning to publish a more detailed write up of everything, so please stay expect that in the next month or so (easiest to hear about via Twitter or by submitting your email to the site).
I wouldn't recommend using this structure directly since it requires the project to have a specific structure. There are some scheme callbacks in the writeSpaceTimeDumper function which you can use directly to extract the information that you can use.
I agree, this is super cool. Really simple and elegant. For something along the same lines in IOS app form, I found UK developer Touch Press' The Orchestra does something similar but also mixes with a video feed of the Orchestra itself, and a running commentary http://www.touchpress.com/titles/orchestra/
Thanks for the heads up, I will fix it this weekend along with a few other things - I didn't anticipate this level of interest so there were a few details (including that one) that I had overlooked ;)
Listening to classical pieces with the score was one of my favourite things to do when I was at music school. One thing that would be cool would be an overlay of the clef, time signature and current (declared hah) key on the left hand side that updated as everything changed. Good for context as you're following along.
Hey Adrian, I accidentally discovered your site just a few days ago and I think it's great! Neat solution to use Youtube to pre-empt copyright problems as well.
I'll send you a tweet or email so we can stay in touch!
131 comments
[ 4.6 ms ] story [ 212 ms ] threadMaybe additionally highlight the notes that are currently played. And have you tried scrolling the sheet instead of moving the 'now' indicator? And who are you and why are you doing this? Add an about section.
Yup, I can do the highlighting fairly easily, that's a good idea!
Scrolling is easily doable too, but I'm not sure that's the best way for all situations. I might provide it though, and then let the user choose depending on their situation.
About me: a wannabe classical composer / tech geek ;)
(and in the event people are actually interested in this idea becoming more than just a concept, I would be very happy to provide a more elaborate About section)
Even a "right now this is just a little concept I worked on in my free time, hope you find it useful. Tweet me suggestions @..." would be enough on /about.
http://notezilla.io/about
Cheers
http://musescore.org/
It would be nice if the key and time signature could remain onscreen as the sheet scrolls.
I think my preferred solution for scrolling would be to have the position marker move until it hits the center, then have the position stay centered while the sheet scrolls until it hits the end. I'm not sure what the correct way to handle repeats in my scenario would be--perhaps have the sheet stop scrolling when it hits a repeat.
As for scrolling, it seems like that is a feature that some people might prefer. I will most likely add that as a feature to it.
Thanks!
Removes the disorienting jolt of having everything just change, but also not always moving and being a distraction
Personally, I honestly appreciate the fact that the score is not scrolling continuously! I will try to have it not jolt though.
Hell, you could even color code the instruments subtly with some light shading. Maybe make that a checkbox option.
Well it seems that consensus is that the current page turning is not optimal... Perhaps scrolling is the solution. I will definitely invest some time in that to improve it.
Also, regarding the time and key signatures, I very much agree with you, they need to "stick"...
As for color coding, is the purpose to make it easier to follow a certain instrument?
[edited for clarity]
I love (classical) music and as a music hobbyist I try to dig more into sheet music for a while. I would also pay for some kind of premium service (maybe great content or listening the sheet music in combination with a famous record).
Anyway, I subscribed to your mailing list, please just keep on rocking.
PS: And yes, I own Logic and could play sheet music in that way. But I do think you folks will prepare something more which a hobbyist can enjoy!
For example, it could play a certain section once, then repeat it, but without sound, while the student plays, and then continue in a loop like that until the student feels comfortable with that particular section.
I'm sure there are many other things that could be done in that realm too, so feel to tweet or email me anything that comes to mind (@notezillaio / notezilla.io@gmail.com)
As for incorporating new pieces, it's definitely technically doable and it seems that that's something people are interested in. There might be certain copyrights issues that have to be ironed out first though... ;)
Minor bug, Beethoven's 5th was off by a measure for me. (Goldberg variation was spot-on).
Also, right/left arrow keys aren't skipping measures (as the keyboard page says), but are skipping whole sets of measures.
But overall, as a musician, I actually found it rather distracting to watch the line try to follow the notes exactly, as the line constantly sped up and slowed down due to rubato, etc. -- indeed, in particularly fluid pieces, it can even be possible that the right and left hands don't match up precisely in actual performance.
It might actually be more clear, and far less distracting, if you followed the time signature for the beat, and simply highlighted all notes in the current beat within a light-pink rectangle overlay, and then that box would instantly disappear, and the next one appear, upon the next beat.
That way, we can focus with our ears on the actual movement in the music, without being distracted by visual movement, but still have a perfect visual indicator of where we are.
But anyways, great work and great idea! Kudos, well-done!
(just moving by one measure is more easily done by clicking in the score...)
Thanks for correcting that :)
As for Beethoven's 5th, it should self-correct, but I have noticed that in some browsers it doesn't quite work... What browser and hardware are you using? I will look into this further.
I have heard from several others that the current way of moving the cursor isn't optimal. Your solution could work. I'm going to explore a few ways and probably offer some new choices to the user. Your suggested solution would be very easy to implement, so most likely I will add that.
Thanks again!
One minor "bug" / missing feature; I have two monitors at different resolution; when moving the browser (maximised) from my laptop monitor (1440 x 900) to second display (1280 x 1024) the staves are put on a different line to the instrument names. If I refresh the browser on my second monitor then move to my laptop screen all works fine though. The same issue exists if I resize my browser; i.e. currently window.onresize events aren't handled.
Great work though - and nice choice of pieces too.
A few minor things: While jumping to time points works, scrubbing does not. Instead, the notes image gets dragged. You could embed the image as a background-image instead of an <img> to prevent that. Or catch the dragstart event on it and return false (http://stackoverflow.com/questions/4211909/disable-dragging-...). A custom Rails error page or better error handling would also be nice (http://notezilla.io/bla). And there are a few console.log's left (TIME!!!! :)
Did you match the music to notes by hand?
As for scrubbing, it was a conscious decision to keep it fixed for now, but I might consider changing that in future version... and also need to fix status pages, etc.
Yes, I match it by hand, but it's fairly easy to do using some javascript that I wrote...
http://pianobooster.sourceforge.net/
http://sourceforge.net/projects/midisheetmusic/
I have been wanting to work on this idea for quite a while in my spare time, particularly to enable scrolling and keeping the signatures in place while the sheet scrolls.
Kudos to Heroku!
Please tell me you will have some way to use this to display a score written in some open format. and/or interface with MuseScore or some other software.
Alternately, it would be possible to just upload the pictures of the score directly in .png format onto noteZilla.
Would it be possible to isolate each instrument? Turn each one on/off?
For pop music, it's definitely doable as long as I can get access to the individual tracks. It's actually something I have been wanting to do, so thanks for the suggestion.
That way you could allow users to upload their own songs.
I'm actually using LilyPond already to create the sheet music, so if I do add the possibility for users to upload their own scores, it would most likely be with LilyPond...
Although the user would still need to partition the score (ie indicate at what time a certain note occurs) if the music was to be synced up to it.
It isn't terribly labor intensive but still requires a little bit of work... so maybe, if that's what people want!
But this is awesome by the way, great job! I'd really love it if user-generated content is supported. I have a bunch of lilypond transcriptions from the Real Book that I'd like to upload.
Just need to be clever about it and make sure it's not too arduous a task for the user to upload something. Please feel free to email me or tweet at notezilla.io@gmail.com / @notezillaio and we can stay in touch / discuss further, would love more input!
Well you could automate it -- simply implement 100% accurate polyphonic pitch detection. That should take, what, a weekend? ;)
We've built a similar thing at http://particellissima.net at the Classical Music Hack Day in Vienna.
It's a demo, but you may find some UI ideas interesting, for example synchronization and navigation through a video, flowing score, hiding the instrument parts that are not playing for a while, and assembling a score from small pieces with measure images hosted on multiple subdomains to improve the fetching performance in the browser.
And of course more music would be great!
Interesting demo! I definitely like the hiding of the instruments not playing.... also, lots of people seem to prefer a flowing score.
As for fetching more quickly, yup, also worth addressing.
More music? You can be sure about that ;)
Thanks.
Such software usually offer 3 views of the same underlying music data: piano roll, staff/sheet notation, and an event list.
Here's the example I was using: http://notezilla.io/p/bach/air-on-the-g-string
[edit: I agree with crazygringo that it can be distracting if you are worried about the red line being slightly off.]
I will definitely work on getting that line synced up. On my computer (Mac OS X / Chrome), it's perfectly synced, but it seems that that isn't the case for some people.
Thanks for the feedback.
I think Chrome for whatever reason will make it "less off"!
Either way, it would be cool to see a writeup about how you implemented this.
And yes, I might actually write a blog post about this so make sure you have signed up on the site :)
As for Audacity, can it recognize the beats automatically?
It's a highly useful tool for the analysis of music recordings.
I, too, am highly interested in the extent to which matching recordings to scores can be automated, as are a lot of music theorists. I hope you can publish a write-up of the process for syncing the audio and the score.
If there is a significant human spend required, then developing an open tool for community contributions would greatly speed up the creation of a corpus of scores.
[1] http://www.sonicvisualiser.org/
As for the write-up however, the process I currently use is very simple (and it's pretty much all "manual labor").
To partition a score into time intervals, I have a couple of lines of javascript that record the time the audio is at, together with where in the score I click (and where I aim to click on the note currently being played).
It doesn't take too much time and effort to partition a score this way actually, unless the piece is slightly complex. Beethoven's 5th took a little more effort than Air for example ;)
Hope this helps for now! And I am planning to publish a more detailed write up of everything, so please stay expect that in the next month or so (easiest to hear about via Twitter or by submitting your email to the site).
Thanks for your interest and suggestions!
I've used a similar process to generate a video with a score: http://www.youtube.com/watch?v=OnoBD_QSskE.
Just checked out your video + followed you on youtube! Looks very cool as well!
You might of seen it in the Apple ads
it's $13.99, although to try something similar, I got Beethoven's 9th Symphony for free first http://www.touchpress.com/titles/beethovens9thsymphony/
"This website lets users to bla bla bla..." (cause og:description).
Just something to note.
Super nice work though.
Demo: http://www.soundslice.com/demo/
I'll send you a tweet or email so we can stay in touch!