Show HN: A platform for remote piano lessons based on the Web MIDI API (keyboardconnect.com)
I'm building a video conferencing app designed to facilitate better remote piano lessons. My hope is to solve a lot of the challenges piano tutors and new students deal with when taking lessons through Skype/Zoom. It leverages WebRTC's data transmission to send media, MIDI state, and (eventually) send sheet music files and other musical data.
I'm surprised with how well the MVP has worked and would love to hear any feedback or suggestions!
110 comments
[ 5.2 ms ] story [ 152 ms ] threadThis will probably require assigning the received MIDI to a different channel than 0 to allow simultaneous play without conflicting note offs and sustains.
At least a YouTube demo video, or ideally direct access to a real demo interface would be more helpful.
I thought Getting Started might help, but that forced me into a sign-up flow that I don't want to invest in without more information about the outcome.
(I learned on a physical piano in person, so I can't speak personally about learning on a MIDI controller, but it sounds plausible to me.)
Also, there are plenty of electric pianos, like the Wurlitzer or Rhodes, that have leas than 88 and only semi-weighted keys.
Of course, whether you would want to use the OP's app in some giant cathedral is another question.
organs have always had their own feel and plenty of greats have proven they sound great.
And what comes out of the speakers is the only thing that matters.
Playing the piano means different things to different people.
It just demonstrates the way in which people gatekeep music.
If your goal is to learn music theory or produce music or anything other than learn to play piano, the mpk is fine. No gatekeeping here.
If all you have is a digital keyboard, don't let that stop you for learning it!
Also, when recitals came around it was a big shift to playing on a grand piano with real weighted keys, and that, coupled with the nerves of playing in front of an audience, often led to mistakes.
I think the weighted keys help you feel more physically connected to the instrument with more of your body. Unweighted keys require the same involvement as typing on a computer keyboard.
…and who’d have any fun with that?
Unweighted is fine if you're making synth sounds. And of course for understanding the things you've listed.
But you can't actually play emotionally expressive piano music on them -- not in the style of classical or jazz. If you tried to play the Moonlight Sonata first movement, it would sound terrible, because the dynamic shadings couldn't be done.
I'm not really sure how you've determined what "core skills" are. Sure, if your kids are only going to spend a couple years learning the very basics, and then move onto other things, then it's fine.
But the heart and soul of piano music is in the precise touch to generate the dynamic sensitivity. If you want them to learn how to be emotionally expressive through music, playing either classical or jazz, weighted keys are essential. It's not piano otherwise -- it's a synth.
(And going from unweighted to weighted isn't trivial. It's an entirely different muscle memory that needs to be developed. They're fundamentally different instruments.)
Yes you can, and yes it is more-or-less trivial. With a small amount of deliberate practice, you can learn to produce a full and finely-gradated dynamic range on either a piano-weighted or a semi-weighted keybed. It's a one-dimensional mechanical skill that just isn't particularly difficult relative to, say, an oboeist's reed control or a violinist managing the very complex relationship between bow and string. If I say "you just don't press quite as hard" it'll sound like I'm being glib, but that's literally all there is to it, because physics.
Strings players choose from a wide range of string tensions and woodwind players choose from a wide range of reed stiffnesses, based purely on their own personal preference. Some prefer something soft and pliable that responds to the lightest touch, others prefer something that fights back when you dig in. It's an accident of history that mechanical pianos fall into a relatively narrow range of weightings, not a deliberate choice on the part of pianists or piano makers - the range of options are limited by the mechanics of an escapement and hammer.
Pianists overwhelmingly prefer weighted keybeds out of habit, but more generalist keyboard players will often prefer a semi-weighted keybed for versatility. You can play gigs or sessions with either and you're the only one who's going to notice. Calling either choice wrong is just dogma.
> Yes you can, and yes it is more-or-less trivial. With a small amount of deliberate practice, you can learn to produce a full and finely-gradated dynamic range on either a piano-weighted or a semi-weighted keybed. It's a one-dimensional mechanical skill that just isn't particularly difficult relative to, say, an oboeist's reed control or a violinist managing the very complex relationship between bow and string. If I say "you just don't press quite as hard" it'll sound like I'm being glib, but that's literally all there is to it, because physics.
Pretty much every piano teacher on the globe would like to have a word with you. The idea that it's a "one-dimensional mechanical skill" is greatly misleading when it has to do with joints in the fingers, wrist, elbow, shoulder, even your spine. Positions of all those things, varying degrees of muscle tension. And it's incredibly difficult -- pianists spend years improving their touch, and do things like Alexander lessons to eliminate muscle tension that interferes. "Don't press quite as hard" is incredibly complex, physiologically. (And oboeists are only playing one note at a time, violinists one or two -- pianists need to learn to independently control the force of all 10 fingers independently, sometimes all at once!)
But if you want to understand it in just a one-dimensional way, you can. Imagine that the full range of force a finger can produce is mapped from 0 to 100, and humans are sensitive to the degree they can adjust that by 1. Now imagine a weighted keyboard is responsive to the values between 10 and 60 -- that's 50 levels of sensitive gradation. An unweighted or semi-weighted keyboard is more like 25 to 35 -- merely 10 levels. Because we have so much less control, the software makes sure that the extremes are clipped -- you can't play as quietly or as loudly. Your dynamic expressiveness is severely limited. If you want to say "because physics", that's your "because physics".
You can measure this in MIDI outputs, actually. A good weighted keyboard will give you a wide range and an experienced pianist can consistently hit the same values without much noise. Whereas unweighted either gives you a much smaller range of values, or you can crank the output range up with software settings but discover that there's either a ton more signal noise because we don't have that fine control over our muscles, or else you discover that the intermediate MIDI values aren't even being used because the keyboard sensors don't support it (e.g. they only support 8 values).
> Pianists overwhelmingly prefer weighted keybeds out of habit, but more generalist keyboard players will often prefer a semi-weighted keybed for versatility. You can play gigs or sessions with either and you're the only one who's going to notice. Calling either choice wrong is just dogma.
That's false. Pianists don't prefer weighted out of habit, they prefer it because it's required for greater expressiveness, which classical/jazz requires.
The "more generalist" keyboard players you're referring to doing "gigs or sessions" are generlly not aiming for dynamic expressiveness -- they're part of a band or providing basic accompaniment to singers doing pop songs, and so forth. And that's fine -- unless it's Norah Jones-type music, you're not usually asking for much expressiveness from the piano in pop music. (She is much more jazz, after all.)
But that isn't "more generalist", it's pop. They're not practicing or performing Beethoven or Chopin, because it doesn't work except on a weighted keyboard. Nor are they doing jazz piano. You can't do it. Not well, anyways.
And you claim that "calling either ...
Unweighted keyboards tend to have simple springs to return the keys to resting position, and more importantly, the cheapest instruments tend to only have note-ON;note-OFF characteristics due to being unable to sense any variation in how a note is played. These instruments playing all notes at the same volume is a natural consequence of inadequate sensors.
Returning to your sample values, MIDI 1 only directly supports 256 levels of volume, I think? This 8-bit level of variation is far lower than what any normal human sensory input can be expected to discern. MIDI 2, which was still very rare when last I paid heed, is a substantial improvement for potential expressivity. But even many years later, most consumer-grade instruments and playback devices only supported MIDI 1, sometimes with extensions such as GS or XG. The most expressive of the pro or semi-pro instruments that I've experienced have only seemed fully capable when using their internal proprietary enhancements; when outputting to a MIDI 1, the results were much less impressive.
As an aside, I've played only one piano whose keys felt as lightly weighted as my first semi-weighted keyboard and I found it a delight to use. Fully expressive, and less effort for the same effect as other pianos. I felt like I could play it for hours without tiring. By comparison, keys on all other pianos I've tried now feel quite heavy. I wish I could have afforded that one at the time. Sadly, memory of its make and model are long garbage collected.
Of course there is -- it's not a technical limitation of the instrument, but of our physiology.
If there's more weight resisting, this gives us a wider range of control, for the reasons I explained. Of course only to a point -- once it gets too heavy that a light touch can't move it, or the heaviest touch becomes too fatiguing, you've gone too far. But pianos are weighted in a way that approximately maximizes our level of expressive control without being fatiguing.
So yes, semi-weighted keyboards inherently have less expressive range. I mean sure, not if you're a robot playing with infinitely fine motor control. But for actual human beings, semi-weighted are absolutely less expressive. There's no getting around it.
And with regards to the one extremely light-touch piano you found a joy to play on -- I'm quite sure that it couldn't have been played extremely softly in an even way. That's just the inherent tradeoff. If you're playing music than never goes to ppp then maybe you didn't notice. But I don't need to see the piano myself -- this is all just physics and physiology. It's inherent to how it works.
I know I can play with high musicality on almost any keyboards with velocity, because I was blessed to have learned to use bad instruments. But, it doesn't compare to the depth of the sound generated by all the moving parts and interactions happening in a real piano. Not only the sounds, but also the sheer weight of the keys.
Most* keyboards/vsts are just triggering a (pitch-shifted, looped) sample at a given note and then doing that for n notes and that doing an additive sum.
That is definitely not what occurs in a piano though. There you have the 3-dimensionality of the physical world, like the way waves are traveling through distance and shape. When sounds' harmonics interact, resonant nodes in overtone sequences can trigger each other to resonance, which can trigger other resonances throughout the tone. Maybe you know the feeling of depressing the sustain/damper pedal while sitting in front of it and giving the instrument a smack (or holding down the keys you are not playing and doing it). Or running your nail or a pick over all the low notes with sustain.. like you are in a cave.
In midi/digital, there's the fact that dynamic is usually gonna be 8-bit. Just because midi did that and it made sense at the time, other keyboards and VSTs mostly follow suit. I'm surprised this gets generally passed over. Obviously there's more than 128 strengths of note in real life.
But all that said I think it's possible to learn keyboards/music theory/songs/playing on a non-weighted keyboard, but false to say that digital/non-weighted is equivalent to acoustic piano. But you only really need that for really dynamic music like Jazz, Classical, Instrumental et al. But it feels so very wrong to play that kind of music on bad keyboards.
* Roland V-Piano, and PianoTeq, as well as many I'm unaware of do in fact use physical/acoustic modeling as opposed to triggering samples, but it has not been predominant even among high-end digital instruments
It’s not hard to develop that finger strength, just need give kids the opportunity
Nylon has a very different touch and tension than steel.
You could start a student on electric or acoustic guitar that wants to learn classical guitar by why would you just not get them a classical guitar instead?
Same thing with piano to me. If you want to eventually learn to play some Chopin there is no reason you would not start with weighted keys.
The modern piano was also not even popular until the late 1700s.
But weighted keys are crucial for building coordination. The resistance of the keys helps you calibrate the connection between your touch and the sound produced. You want both kinesthetic and aural feedback.
All digital pianos have USB midi, and you can easily pick one a used one with weighted keys for a few hundred on craigslist.
My Yamaha U1 piano also came with a box that lets you interface over MIDI.
To that end, I think a crappy plastic MIDI keyboard can do a better job than a steinway if it is easier to acquire and accommodate. Being at hand is more important than being just right at this stage.
I say this as a music major, but it is not in any way a universal opinion to be sure.
The big advantage is that you don't need to tune them twice per year and that extra budget is either savings or you can spend it on more lessons. A physical piano is nice (and even those can have 'MIDI out' if you look around for a bit or are prepared to do some DIY) but really not a must. I have both here and spend much more time on my digital just because it is far more convenient, I can practice on it at any time of day even when the kids are sleeping and the feel of the keyboard is as good as the real thing (Yamaha P515, not the cheapest but very good quality).
I would suggest recording the videos from the player's point of view. It adds a layer of abstraction to have to reverse the image when reading it as a student.
Unless you are also encouraging your students to learn to read a pianist from an audience's perspective.
A few immediate thoughts:
As a student, I don't care about a two-sided platform (which is what it looks like you're trying to build). I want to sign up for vetted piano lessons through your site, and purchase a USB midi adapter that you recommend. As a short-term MVP, perhaps point people to a known-working adapter on a site with an affiliate program. Long-term you might consider finding a company willing to sell one white-label and put your logo and website on it, but that's a lot less important than everything else you're building so put it off for a while.
Also, for the first handful of lessons and for subsequent practice, it'd be awesome if you had automated, pre-recorded lessons available, to get the basics down, or to train things like reading sheet music at speed.
The big choice for the beginner (or anyone) is how many keys do you want, and have space for? It goes all the way from 25 keys up to 88.
Then going up in price, models will start to include (in random order) old 5-pin MIDI in and out, velocity sensitive keys, (semi-)weighted keys, aftertouch, inputs for foot controllers, pads/knobs/faders for controlling a DAW, built-in sequencers, external sync in/out, CV and gate outputs for interfacing with modular gear, etc..
Good keyboards aren't expensive any more, the simplest Yamaha, Casio and Roland that qualify for serious study are all around $500 if you shop around a bit and if you go for second hand can be had for half that. You'll very quickly match that in lesson costs so if you can afford the one you probably can afford the other.
You don't need sequencers, aftertouch, built in DAWs or controlling an external one etc, those are not aimed at piano students. Can be fun but complete luxury.
It's now connected via a cheap USB adapter to my Chromebook (biggest issue: latency of the - mostly Android - apps) planning to connect it to my Linux box instead.
https://68kmla.org/bb/index.php?threads/usb-adb-converters-t...
https://www.reddit.com/r/synthesizers/comments/3wefun/midi_i...
Does it work on an iPad?
Keyboard Connect is still in an early stage, so we don't support tutors advertising on the platform yet. Thank you for these suggestions!
I'm happy to help test if you need a hand.
I transmitted a frontal video of the player from the laptop cam, a top-down view of the center section of the keyboard (the cam angle was not wide enough for the whole), and the virtual keyboard from Synthesia. I had three different layouts of these elements in OBS, that could be switched by pressing the space bar. On the audio side I sent a mix from the laptop mic for voice plus the direct audio from the e-piano.
Served us well for the time.
One constraint that improved solutions probably don't have now was that teachers could not decide on a video conferencing software, so my setup-up needed to be independent. This almost made OBS a no-brainer, because of the virtual cam support.
This represents a challenge for tutors and could be discouraging for new pianists who just want to try lessons without committing to buying a digital/acoustic piano are supporting AV equipment.
Also, I'll add modules to Keyboard Connect that support tasks tutors/students would normally have to organize on their own (lesson planning, file sharing, practice tools)
The app also includes preprepared videos and lessons (scales, arpeggios etc.) which interact with the hardware component.
My son's been using it with a tutor remotely and it's been working great! Not only that, but he can use the app and the snake to self-guide his practise during the rest of the week.
And cost is probably due to first gen, yeah. Plus, hardware development is pricey, I think.
I've been developing a webMIDI app myself for the past couple of years [0] and adding a multiplayer feature is something I've been eyeing for a while. May I ask how good the latency is?
Some feedback on the landing page:
- As others already pointed out, a video example showing the app would be really helpful.
- I think allowing visitors to try out the piano and only prompting to login when trying to access the remote features would work best
- All those buttons that are clickable/selectable but don't do anything when clicked were a little confusing.
On the app: - Connecting the keyboard worked flawlessly (and without any permission prompt). But I'd add some indication that a MIDI device is connected
- Pressing the keys using my mouse doesn't produce any sounds
- Pressing keys on my MIDI keyboard (Yamaha P120) will cause the keys to be selected indefinitely, so I can only ever press them once. Somehow the noteOff events don't seem to be getting registered. Make sure you also register noteOn events with velocity = 0 as a noteOff event (although I don't believe that that's the issue here) and I would suggest calling noteOff yourself if you register a noteOn event for a note that's already playing. That way you can prevent keys from getting stuck.
- I wouldn't hide keys automatically based on screen size. It does look better but imagine someone trying to play this on a tablet/phone where they can't change the scaling
Again, really cool concept and good luck with further development! Feel free to reach out if you want any help :-)
[0] app.midiano.com
(Where's my drum machine when I need a badoom psssssh?)
I see you're also using Vexflow. Are you using a library to go from Midi to Vexflow format? It's a rabbit hole I sometimes wish I hadn't entered :)
@OP Can really recommend VexFlow [0] when you do get to implementing sheet music.
[0] https://github.com/0xfe/vexflow
There are some aspects of that that I want to revisit such as the automatic recognition of decorations (for instance: trills) so that they can be shown properly but these are tricky problems and I want to avoid spending more time on pianojacq.com than on using it.
Lately I've added an ear training module but I'm not completely happy with it yet (hit Bb0 to enable a hidden menu if you want to play around with it).
I haven't measured the latency directly. But, I did try this with my sister who lives across the US while also on a phone call with her. When she speaks or plays a note, I hear it through the app sooner than over the phone. So it's at least better than that!
The video loop, as it currently is, makes if feel YouTube-y or a formal, business-like Zoom call, while the really special thing of this very interesting platform is the connectedness of tutor and student.
Awesome platform!
There is no tracking on the site so I quite literally have no clue how many people use it and because it also works as a download there are plenty of offline users as well.