283 comments

[ 2.1 ms ] story [ 246 ms ] thread
Author here. Happy to answer any questions or take any feedback about this post.
Dude, thanks so much for this. I have a loud machine I’m using for ML stuff and I’ve been reluctant to move it to the basement as I didn’t want to have to run downstairs with a monitor for the odd issue. I also looked at KVMs and was annoyed at the prices. I have a R Pi sitting here I was looking to do a project with. This is perfect!
Oh, that's great! So glad it was helpful!
How would you remotely connect to something like this ? using something like ngrok ?
Yeah, I haven't looked into that much because everything is on my local network. ngrok looks like it would work. One of my users is looking into remote.it, which seems like a similar service.
I work for remote.it. If you or your users need any help setting up remote access to the TinyPilot let us know. This looks like a great application. We can make ssh/VNC/http on the Pi available remotely over internet as a free service no problem.
Oh, that's awesome. Thank you!
or VPN on your router? or just get a VPS and run wireguard.
Would a Pi Zero W work for this, or do you need a full-size Pi 4? I would think that with the HDMI dongle doing the hardware MJPEG encoding, a Pi Zero might suffice. It'd be really neat to A) bring the cost down another $30 and B) be able to package everything into a tiny 3D printed case.
Thanks for reading!

>Would a Pi Zero W work for this, or do you need a full-size Pi 4?

I tested Key Mime Pi (the keyboard-only version) on the Pi Zero W and it worked great. I haven't tested TinyPilot on the Pi Zero W, but I think it will work. The video encoding is happening in the HDMI dongle, so it doesn't hammer resources much. I don't have a USB to microUSB adaptor on hand, but I'll order one now and see if it works.

>be able to package everything into a tiny 3D printed case.

Yeah, that would be neat! One of the biggest issues I have with TinyPilot's setup right now is that it looks kind of like a bunch of random hobbyist parts stuck together. A 3D printed case would be good and wouldn't have the heat issues that Pi 4 has.[0]

[0] https://www.jeffgeerling.com/blog/2019/best-way-keep-your-co...

You might also be able to use the "standard" radio kit metal boxes - tap some standoffs, carefully drill some holes for wires etc. Dunno which works out better for the world - ordering a pre-fab metal box that you have to do some work on (or perhaps use https://www.frontpanelexpress.com/ to get a customised enclosure), or printing something from plastic. Might depend on production volume, metal used etc.
Do you happen to have any idea, whether older RPIs like 1 or 2 could work for this fast enough? Have some of those laying around.
The video capture part should work, but I don't believe they support USB OTG, so they wouldn't be able to impersonate the keyboard.

The official documentation[0] says all Pi devices have USB OTG, but I'm not sure if there's something else about the Pi 4 and Pi Zero W that allows USB gadget mode because I've never found anyone talking about using it before recent generations of Pi. This StackExchange answer says the A and the A+ had it, but that the 2 doesn't.[1]

[0] https://www.raspberrypi.org/documentation/hardware/raspberry... [1] https://raspberrypi.stackexchange.com/questions/71613/how-to...

I'd expect the Pi Zero W to be fast enough to pass through the MJPEG stream and impersonate the keyboard, which should cut the cost by $20+ plus however much you save on not needing a fancy case.
Yeah that's true. I stopped thinking about the Pi Zero W early in the project because I assumed it would choke on video encoding. When I discovered that the HDMI dongle was doing the heavy lifting, I forgot to revisit that assumption. I'm going to order an USB to micro-USB adaptor to test it out because it would slim things down significantly.
Woops, forgot the major issue-- the Pi Zero only has one USB port (the other is just for power), so you couldn't get video frames in at the same time as you pretended to be a keyboard.
The Pi Zero's power port is actually a USB OTG port. I used it in my precursor project to forward keystrokes:

https://mtlynch.io/key-mime-pi/

On the web page you explicitly say to connect it to the port labelled "USB", which is not the power port.
Whoops, you're right. I misremembered. It's power on the Pi 4 and data on the Pi Zero.

Thanks for the correction!

(comment deleted)
Any Pi with just one USB port (zero, A, A+, 3A) is capable of USB gadget mode because there is no hub between the SOC and the port.

Those with multiple ports have a hub in between and are not capable of gadget mode.

The pi 4 is an exception, it is capable of gadget mode on its USB-C power input, not via the normal ports as there is a hub in between there too.

I think you might run into a lack of connectivity on the Pi Zero W. Since the only data USB port has to run in device mode, I would assume you can't connect the HDMI capture device.
The immediate issue would be the Pi Zero W only has one usable (micro)USB port, the other port is Power In only. If you run it for OTG, you can't use it for the dongle.

There would be ways around this though, perhaps using a cheap microcontroller for the USB HID stuff.

Yeah I forgot that only one of the USB ports actually does data. I guess you could use a Teensy that interfaces through the GPIO pins and pretends to be the keyboard and mouse.
That's what I'm thinking, or worst case, one that communicated via USB to TTL adaptors. Or an ESP-based board that connects over WiFi. Or BT-enabled board.

Combined with a HDMI switch, you could then control multiple devices with just one Pi+Cap device.

I've been looking to do something similar for a while, albeit possibly with Windows as that's where I've (very rusty Delphi) programming experience.

Cool project, but what about power management? What happens if the PC being controlled crashes and does not let you reboot it using keyboard shortcut?

I'm might be damaged by testing and working with embedded development.

If you can plug stuff into it, just use the KVM from a laptop and sit by it.
Plug it into a smart outlet running tasmota.
Thanks for reading!

>Cool project, but what about power management? What happens if the PC being controlled crashes and does not let you reboot it using keyboard shortcut?

Yeah, that's one of the limitations. TinyPilot can't control power for the target computer.

I don't know of any way to manage power without connecting the Pi to the target machine's motherboard, but that increases complexity significantly. I'm using this for a homelab, so it's easy for me to walk over and cycle power if that ever comes up, but for this to support more remote scenarios, I'd need to address this.

hook up a gpio pin to a reset switch jumper somehow?
APC produces (or rebranded) power control modules you can control over Ethernet. They are probably expensive.

There are most likely cheaper products which would work. You should probably implement some kind of power control interface that could be used. Calling a shell script would be enough and some example implementations provided by your users. :) It doesn't need to be complicated.

Perhaps a fake USB device that issues power or sleep button requests could work. I had limited success with this on Linux using an HID keyboard emulator. However this won't work on a machine that's too far gone to respond to USB, and that's also an issue with the physical power button as well. Perhaps an AC latching relay would work to cut power.
Wake-on-Lan can handle powering on. It should be possible to directly connect a RPi GPIO pin directly to the 0.1" headers on the motherboard.. the power and reset buttons are just closing a circuit to pull high/pull low an input pin. Use a 1k resistor between the Rpi and motherboard headers for protection.
A horrible way would be to set the target to power on after power loss. Plug the target into a WiFi-controlled smart plug, like the Belkin Wemo. In the event of catastrophic hard lock, power cycle it at the wall.

A better way would be a relay and microcontroller, which you could also potentially use for the USB HID stuff too.

In theory, you could have a HDMI switch, a single Pi and HDMI grabber, and just a microcontroller board for each target machine that could have power button control, keyboard emulation etc built in.

Can highly recommend the TP-Link HS100 for this - it's "just" a wifi power switch. Change your motherboard settings to "power on after power loss" and you've got your remote reboot.
If you need access to the power and reset, you can replace the respective buttons with a cheap relay switch controlled by GPIO. If you know what you're doing you can also solder your own circuit using opto-isolators instead of relays, to reduce the power draw on the Pi.
This isn’t a bad choice:

https://wiki.52pi.com/index.php/DockerPi_4_Channel_Relay_SKU...

It’s $17 for four relays, expandable to 16. If you wire it “normal closed” then the relays will only draw power while it’s resetting the other machine.

However, the relays are only rated for 3 amps. (360W at 120V). Also, the board is labeled backwards so, the NO port is normal closed, and NC is normal open.

Has anyone found something that’s similarly plug and play with a higher amperage rating?

Edit: this would work as a switched AC outlet (to replace a four outlet PDU) or for hot wiring the power / reset buttons. You could do both buttons and the AC line and have a relay left over.

I’d feel funny running AC that close to the lower button wires though (if cross those wires, it will certainly let the magic smoke out of your motherboard).

It could work on a power line, but I'd seriously recommend against it for a beginner project. Anything over 50V requires proper safety procedures.
This is epic, I run a small computer cluster using workstation hardware (AMD 3970X's) and no IPMI boards exist. If I get some time I might build 7 of your little system to manage them all. Thanks!
Awesome! I'd love to hear how it turns out.
Really nice, exactly what I need just now as I'm setting up a remote workstation/part-time server at my families farm, but did not have a motherboard with IPMI/iKVM at hand.

Oh, and thanks for using Proxmox VE! Was positively surprised when I saw our installer flash up out of nowhere :)

Wow, thank YOU for making Proxmox! I'm switching over after three years of Kimchi[0], and Proxmox does everything so much better.

[0] https://github.com/kimchi-project/kimchi

Glad to hear that you like it!

For (integrated) mouse support you could take a look at a project from a good friend and colleague: https://github.com/Blub/netevent

Allows passing along arbitrary input events over the network and more.

Thanks! I'll check it out.
This isn't an area I know anything about but I really enjoyed your write-up (I still don't know what KVM stands for! But I get that you wanted to act as a physical keyboard and display over IP). From a quick skim of your site it looks like you've worked at large companies and now are doing some independent projects. Do you think you'll go back to working for a company (perhaps a smaller one?), or do you see yourself working independently for many years? I ask because (a) I'm also casting about at a similar point in my career doing independent projects, but (b) I've never worked at a truly large company, and (c) (someone like) you seem(s) like you'd be a great colleague to work with, and (d) my vague ambition is to work for a small company with skilled colleagues that I can learn from and to work on "technical" projects as opposed to product development but (e) I guess, when I see talented people such as yourself apparently rejecting the conventional job market, I partly infer that they've looked at what's on offer and declined and thus that I'm deluding myself in thinking that I'll find something in the conventional job market that I really want.
Thanks for reading!

>I still don't know what KVM stands for!

Oh, keyboard, video, mouse. The article does define it, but it's a little buried.

>From a quick skim of your site it looks like you've worked at large companies and now are doing some independent projects. Do you think you'll go back to working for a company (perhaps a smaller one?), or do you see yourself working independently for many years?

I'd love to continue working for myself forever. My ideal would be if I could find a business sustainable enough that I can hire 2-5 developers and other teammates to work with me. If I did have to go back to being an employee, I'd probably lean toward something smaller.

It's very much about your temperament, what type of work you enjoy, and how much you value flexibility and autonomy. A good way to test the waters is by building a side business and see if you enjoy it more than your job. You can also read/hear more about the lifestyle in places like Indie Hackers[0], Starter Story[1], and WIP[2].

But I agree with you that you pass up certain kinds of growth by working on your own. I'm happy that I had ~10 years of experience with big companies before doing my own thing. I learn a lot on my own, but I don't think I could have learned a lot of my software "craftsmanship" kind of skills (e.g., designing for maintainability, creating repeatable, well-documented processes) had I not worked for Google and Microsoft.

[0] https://indiehackers.com

[1] https://starterstory.com

[2] https://wip.chat

Have you looked at a solution for controlling the power on/off yet? This was something I was considering to do with a Raspberry PI and a relay and I figure it is the final part of the solution for when the machine inevitably crashes. Admittedly if its just across the room it was less of a concern but I have had the need to restart while I was away before and it would be nice to be part of the solution.
This is a pretty simple thing to achieve electrically, just plug something instead of the power button inside the computer. You likely just need one mosfet. If soldering is too uncomfortable, you can find relays to control the main power supply (requires screwing a power supply cable and a couple dupont connectors).

If you don't want to touch anything related to electronics, you could rely on enabling a watchdog on boot-up. Or control if from a smart power supply (you likely have a UPS plugged in, for instance). Use the power after power loss BIOS feature, or WoL to wake it up.

If this just requires USB and HDMI, it should theoretically work identically a for Mac Mini or even something like a Fire TV, yes? Is there an upper limit on resolution and frame rate for the dongle?
>If this just requires USB and HDMI, it should theoretically work identically a for Mac Mini or even something like a Fire TV, yes?

I believe so, but with the caveat that I haven't tested it. Does Fire TV take USB keyboard input?

I have successfully captured video from a Roku Premiere[0], even though it's HDCP protected. The LKV373A couldn't capture anything out of it, but the $11 dongle did just fine. But that was just video capture. I didn't have a way of sending input to the Roku.

>Is there an upper limit on resolution and frame rate for the dongle?

Yes, the limit is 1280x720px @ 30FPS.

[0] https://twitter.com/tinypilotkvm/status/1277994838210875392

I have a Fire TV brick (not stick) and it has a USB A port that I've successfully used with a mouse at least. I don't have a keyboard handy to test.
I love how this is a project which puts together several different projects which are themselves the product of significant amounts of effort: RPi, the HDMI-USB dongle, the browser, and uStreamer. I think we are in for an interesting decade or so of people plugging things together like this.

I'm curious why you use the browser instead of ffplay for decoding and display. I'd guess ffplay would be faster - but maybe not?

For one, it wraps the screen and input into one window. Using two would be unwieldy.
and today I was wondering how to solve the video problem! that's great timing.
I was wondering if you could try my usb-ip software for the hdmi dongle. https://www.virtualhere.com (ill give you a license key) i think this is quite useful for my customers and its something i was looking for. Can you pass the dongle directly through to your pc (no need for intermediate software) and see what the performance is like.
Thanks for reading!

I'm not sure that VirtualHere would work in this scenario because it looks like it requires code on the client computer. I'd like to encapsulate all the software on the Pi so that the client computer doesn't need anything more than a web browser. The other tricky issue is that uStreamer is optimizing the stream to drop frames from the dongle when it detects that the browser is falling behind, so we'd lose that functionality.

Fantastic project. Amazing how you overcame each obstacle, and very well written to boot. Kudos.

Quite funny how you brought 1000 ms latency down to 200 ms. I wonder how this compares to iDRAC and that Raritan device.

Thanks for reading!

Yeah, it was kind of like the perfect difficulty level for this kind of project. If it was two months of work to get it working at all, I probably would have gotten discouraged, but it worked out that every couple weeks I was rewarded with a breakthrough that substantially improved the latency until it was finally a practical tool.

I don’t know about iDRAC but I used raritan a bit for some home projects and it’s not super fun. I couldn’t connect it on HDMI when the server was already running, it had to be connected before the boot. For both VGA and HDMI, it sort of works, there is a clunky but functional java desktop application, the screen is often off by a dozen pixels (part of the screen hidden) and the pointer too.

For new servers, I just buy a supermicro motherboard with IPMI and stop worrying. You can set up an IP based firewall within IPMI, so that only a handful of IPs can even connect, plus a very strong password, and I am not too worried about exposing it to the WAN (I know it’s not a great idea but I don’t know any alternative that doesn’t require another U in a datacentre)..

I've used virtually all of the iDRACs. They're probably in that 200ms area (based on my gut feeling w/ no empirical measurements). They've been fairly responsive in my experience.
Out of curiosity do they bring any functionality that this device does not?
The most interesting stuff is generally:

-Remotely mounting images (ISO/FS/IMG/etc) on the host for bare-metal installs or any other purpose

-Hard power options for unresponsive servers or emergencies

-Heavily integrated diagnostics (fan RPMs, thermals, RAID status, etc etc) with eg. snmp output

Cool, I assume with enough willpower those first two could be solved with a pi too ;)
If we go that route, all three of them could (for example via fan sensors). Whether it's worth it is a different question :)
> -Remotely mounting images (ISO/FS/IMG/etc) on the host for bare-metal installs or any other purpose

For what it's worth, that stuff has always been incredibly unreliable for me. Enough so that old school PXE booting was the only way to maintain a server pool, couldn't get them to consistently reimage through the proprietary stuff.

The remote power control is very nice, and I didn't see anything on this page about that. (Though now I'm just imagining the "useless PC" [1] that just tells a smart power plug to turn itself off.)

[1] https://en.wikipedia.org/wiki/Useless_machine

I never had a single issue with it if I mounted it from an onsite drive. Mounting from home in my den is a bit more iffy and pretty slow. PXE is definitely better, but the iso method is nice for when that's not an option.
I use a RPi to control smart home components including wifi controlled smart plugs. You could use one as a hard power off, and WoL as a soft power on.
It can one-click update all firmware for bios, network cards, storage controller etc
I manage a machine that has an AST2500 (competitor to iDRAC in smaller vendors like Gigabyte, it rocks), and I didn't notice latency, so it's probably under 200ms.
Most iDRAC type products actually use that same Aspeed ASTxxxx with vendor firmwares. If a server reports "Matrox G200e" as display adapter it's it.
That MacroSilicon MS2109 capture is super interesting! $11 shipped for 1080p30 capture is unbeatable. I wonder what kind of host SBC is going to be necessary to build a clone BlackMagic ATEM Mini without choking up USB bandwidth.
Ah! That's the name of it. Thanks! I didn't know how to tie it to any kind of identifier.
A USB 2.0 device can use up to 480Mbps and MJPEG is on the order of 10s of Mbps (estimated), so I don't think you're anywhere near needing to worry about bandwidth.

Your biggest complication will be real-time re-encoding and compositing with effects.

I can't believe it until I see it. I've seen USB EHCI choke up from just three webcams.
I'm all in favor of cool projects, and I have done my fair share of useless experiments. But wouldn't it just be simpler to attach a dumpster-sourced monitor and keyboard to the server in question for the 3-4 times a year you brick it? To me the idea of a locally placed 1:1 KVM "switch" (huh? it's not switching anything if it's 1:1) is crazy. It makes sense if it is remotely located, but not if it's within your own home.
Of course it would be simpler, but it's also addressed multiple times in the blog post why he decided to do it anyway.

I'm glad he did because it looks like a fantastic project.

Remote monitors like this are occasionally used in offices so that powerful workstations can be put in racks. I'm not completely convinced it is a great idea, even for that application, but it does get used in those instances quite a bit.
Makes sense if you want the flexibility to do it from anywhere over IP, even across the house.
Every time I update windows and reboot, I never know if the server is bricked or if it is just windows taking ages to update.

Also if you use TPM / full disk encryption for a server in colocation, it only takes a firmware update for TPM to request to reenter the key on reboot.

Thanks for reading!

>But wouldn't it just be simpler to attach a dumpster-sourced monitor and keyboard to the server in question for the 3-4 times a year you brick it?

Maybe it's just personal preference, but I'd much rather plug in a Raspberry Pi than a whole monitor and keyboard. My server is tucked away in the corner without a good desk surface to place a keyboard and monitor.

With the Pi, I can just plug it and go back to my desk and do everything through the browser. It's also nice for when I have to Google things about why it's not booting, I can just do it from another browser tab rather than physically walking back and forth between computers.

not sure why you talk about "switch"? The word never appears in the article?

And at such a no-brainer price (especially if you already have a Pi), kind of worth it just to avoid the hassle of moving a screen around.

I think the confusion is that the original comment was worded as if I called TinyPilot a switch, but I think you're actually saying you think it's strange that Raritan calls their device a switch, despite the fact that it's only 1-port.
This is a game-changer! Thanks for all your work on this.

Do you have any plans to allow controlling more than 1 computer remotely? I'd love to implement this in my homelab.

>This is a game-changer! Thanks for all your work on this.

Thanks for reading!

>Do you have any plans to allow controlling more than 1 computer remotely? I'd love to implement this in my homelab.

Right now, I can't think of a way to do this. The Pi only has one USB OTG port, so I don't know of a way to share that among multiple machines.

A friend suggested an interesting solution of chaining TinyPilot together with a "dumb" multi-machine KVM that supports switching via hotkey. So it would go:

TinyPilot -> legacy KVM -> server1, server 2, ...

You could still use a web interface and hop between servers using keyboard shortcuts that the legacy KVM understands.

That's a really great and simple idea, actually. I might give this a try.
Funnily enough I'm waiting for my hdmi to usb capture card to arrive so I can play on my PS3 while also having the option of switching back to the desktop instantly.
That has to add a lot of latency.

Although honestly I think some of the latency complaints by gamers are a little much. Personally I have a Bluetooth mouse/keyboard connected to my desktop with a super duper cheap projector pointed into the living room and can play FPS games on it just fine.

It has a lot of latency indeed. I tried the raspberry as a Steam Link alternative, but as far as I know the ethernet port is not suitable (?). I've also read that it actually gets worse if you plug in any USB peripherals. Correct me if I'm wrong though.
Yes the Ethernet port is on USB so you’re sharing (the already small) bandwidth between that and all the peripherals.

This may have changed with the pi4 though.

It probably won't work. Though I think it could still be a fun project. Maybe I'll use it in the same way the author does in the end.
I just finished building a Pi KVM about a month ago using project https://github.com/pikvm. Has been great so far.
Yeah, I'm really impressed with that project and love Max's work! There's a bit of a barrier to entry though, as it requires you to compile the OS yourself. My goal with TinyPilot was to create something that's easier for newcomers to install and simpler for me to reason about.
Any plans to add power control to tiny pilot?
This looks awesome. I was looking for a solution like this last year. I budgeted a few hundred bucks but couldn't find anything. Ideally would have a way to handle a few different boxes.

I did see something called Intel AMT, part of Intel vPro. https://www.howtogeek.com/56538/how-to-remotely-control-your... Does anyone have experience with this? (I have AMD box so doesn't help me) - actually looks like AMD has similar https://developer.amd.com/tools-for-dmtf-dash/

It works, however you need a non-free special client called VNC Viewer Plus [1]. Afaik it is Windows only. You can enable regular VNC protocol capabilities for AMT[2], but then you can't use the interesting stuff anymore.

[1] https://www.realvnc.com/en/products/viewerplus/ [2] https://blog.michael.kuron-germany.de/2011/10/using-intel-am...

//edit: link [2] added.

This seems to show a reasonable option from Linux: https://www.cyberciti.biz/faq/remotely-access-intel-amt-kvm-...
I've used instructions similar to those to get AMT KVM working with a normal VNC client, it works fine. The only catch is that the AMT seems to intercept all traffic to port 5900 coming in through the NIC, not just that addressed to the IP of the AMT. It's only the "legacy" port 5900 VNC which is affected, the other ports such as 16992 for AMT's web admin are unaffected.
You can use meshcommander[1], which has a nodejs version that runs under linux and mac.

Then you just have to pay attention to your certs (if you are using https), because browser policy will apply.

[1] https://www.meshcommander.com/meshcommander

vPro is error prone and may require multiple setup attempts. If you want to use it on anything but the same LAN there is a built-in VPN. If you can find anything about the AMD version I'd be interested, they don't seem to publish anything unless you are a partner.
Here are some AMD tools, docs are thin on the ground. [0]

I have a boatload of Intel vPro machines spread throughout multiple countries (or the basement) and connect to them both via internet (mutual TLS authentication) and VPN. Never had any reliability issues at provisioning or operating. I had an old AMT v2.1 machine that was a bit unreliable, needed to be manually powered on once in a while or the remote connection wouldn't work.

I wholeheartedly recommend Meshcommander [1] for this.

[0] https://developer.amd.com/tools-for-dmtf-dash/

[1] https://www.meshcommander.com/meshcommander

AMT is great. I use it with https://www.meshcommander.com/ - though learning how to provision the machine correctly in Admin Mode so it doesn't pop up a pin-entry screen on the remote end is "fun"!
I just set up AMT for a bunch of servers.

It is possible but extremely tricky to set up AMT headless on a self built Intel. You basically need to press keyboard keys at the right time. Too challenging without a display.

Devices don’t load AMT configuration from USB by default unless a vendor configured them to. In which case the vendor just ships you AMT enabled in the first place.

Better to just image the disk on a different computer.

Is it trustworthy, or full of backdoors?
If a machine/server doesn't have built in IPMI, I am using an Adderlink Digital ipeps and control it that way 1). Apart from the built in RealVNC server, it also has HDMI video, USB and virtual media support among other things. I have no relationship with Adderlink, but I am a happy user of this product. Not cheap, but worth the price IMHO.

1) https://www.adder.com/en/kvm-solutions/adderlink-digital-ipe...

At the ~$700 price tag I found for it, I feel like you're missing the spirit of the post.

You could build ~7 devices for as many machines using the article's method.

I was commenting on this thread, where the commenter said: "This looks awesome. I was looking for a solution like this last year. I budgeted a few hundred bucks but couldn't find anything."

So I only wanted to point out that there are good solutions out there, even though they might not be cheap. The solution have saved my bacon, and has been worth the money.

Looking at the original project, authentication, mouse support and audio are not supported (maybe in the future?), so if those things are important; a solution like the ipeps might be worth it as well.

But your mileage may of course vary . . . :-)

Edit: Grammar

Sorry to ask but what's it cost to buy a KVM over IP device?
You're not going to find a decent one sub $500. Used devices can be had for less, but most of the older ones I've seen use terrible Java-based clients that are a major pain to get running on modern browsers. For multi-device KVMs you usually have to figure in the prices for the individual interface module dongles that plug into each computer, too.

StarTech makes a USB-to-KVM that looked promising to me, but it's still >$400: https://www.startech.com/Server-Management/KVM-Switches/Port...

Thanks and that StarTech doesn't have HDMI. Maybe its cloud VMs that are removing the need for KVM over IP. I still want it.
I wouldn't say cloud VMs have removed the need, they've just made infra admins have to live without it. For the most part it's fine on well supported and tested OSes but before the container/kubernetes revolution I managed some personal VMs with esoteric distros in various clouds and so many things have to go right for that to work. A bad update or install of grub, initrd, dhclient, a handful of system services and maybe systemd, or sshd and the machine is effectively dead. You have to attach the disk to another machine to debug it.

A more extreme example was when I went down the rabbit hole of trying to set up remotely unlocked full disk encryption on a cloud, just to see what was possible. This was maybe 4-5 years ago. This involved a pre-boot environment that would run from memory only, packaged into an initramfs and used dropbear as an ssh server to accept a connection.

I really, really wish cloud VMs had virtual KVMs. It turns out, it's really easy to mess this up, and when you do recovery is Non Trivial.

The best I found was that Azure's cloud allows you to connect (read/write!) a virtual serial port via web interface so you can use the Linux serial console to recover. That was a game changer for playing with this. If dropbear failed to come up, I at least got a shell into the pre-boot environment.

Google Cloud also provides read/write serial ports over SSH: https://cloud.google.com/compute/docs/instances/interacting-... -- but I think AWS still doesn't.

One thing I've ran into was root not having a password since the cloud images are typically SSH key auth only. That was annoying when trying to fix something simple like iptables or entering maintenance mode for a disk check.

Thank you! I wasn't aware this was possible back then. I'm glad they implemented this.
This looks great. To be useful in the datacenter environments I work in such a device would need VGA capture, though. Most servers don't have HDMI ports in my experience.

Anybody know of a good, cheap V4L2 USB-based VGA capture dongle?

Could always add another dongle with something like this: https://www.amazon.com/dp/B07BKYBKGP maybe?
Most obvious choice, but the USB port might not be able to provide enough power. You could splice in an external adapter with a bit of work, though :)
I've used one of these a couple of time and they work great, not yet hooked up into the HDMI cap device but can test if anyone is interested.
Oh nice I will have to look into this as I was considering Using a dual pc set up for streaming and this would be great for controlling the second pc (mac mini)

I want to have main pc for running ROLLd20 and a daw - then use a mac mini to run the processed audio from my DAW into hangouts.

This is so I can properly play in audio and , music cues as well as altering my voice to suit the character eg a galdos style for a robot

You could probably do this with the loopback features of a podcast style interface and not need a second computer. The one I use is the MOTU M4, the M2 is a little cheaper.

https://m.youtube.com/watch?v=IKjKymjc_R8

Yes I have looked at virtual loop back like Voicemeeter and might be able to do it with the routing aspects of the focusrite software.

It is of course the sort of thing that should be baked into the OS.

Thanks for reading!

I'm not an audio guy, so I can't comment on how well it would suit that scenario, but it sounds like a neat pipeline, and I'd love to hear about it if you end up building it.

I was more a better way of remote controlling the mac vnc works but its a bit crap.

The main idea was for GM's to use mood music to set the tone or for audio effects the classic red alert should for a sf based game.

What I wanted it for was real time voice changing for character voices - basically the flanged goa'uld or the glados formant shifting robot voice or the classic Dalek voice

Great little project! I had no idea capture cards got that cheap
Thanks for reading! Yeah, my understanding is that these capture cards are fairly recent. I think my timing was pretty lucky because a lot of earlier projects had to do HDMI->S-Video and S-Video->USB[0] to keep costs down because there weren't any inexpensive HDMI->USB options at the time.

[0] https://github.com/Fmstrat/diy-ipmi#setting-up-the-hardware

If you're looking to buy the card in the article, I just got one of these https://www.aliexpress.com/item/4000958505670.html which looks to be exactly the same card, and doesn't require scouring eBay. I haven't done much with it yet, but can confirm that it arrived to me in the UK with no issues, and seems to work from a quick test.
I have given up on KVMs altogether and I am using the following approach now:

- Most monitors have multiple digital inputs. Connect each computer to one of them

- Use a USB switch keyboard and mouse

- Before switching the USB switch, press the "lock screen" key combination and make sure locking the screen drops the video signal

- After switching the USB switch, pressing shift should bring up the login dialog and re-activate video output. Since the monitor has just lost the signal in the previous step, it will scan the inputs and switch to the desired signal.

I did this for a while, the problem is that there's really no such thing as a high-quality USB switch. They are all garbage and mess with USB devices in some way (at least at the consumer level).
I switched from a KVM to a USB switch a few years back, and haven't had any real issues with it. I don't think it was expensive either, just a standard USB 3 switch with only 2 ports. Occasionally it takes more than a few seconds after pressing the button for the switch to happen, but that's about it.
This is my experience as well. I currently use a USB switch that I got off of Amazon for about 15 bucks. It typically "just works." Press the button, wait 2 seconds and my input devices have switched.
Care to share it? The one I have will work for a few switches, then both Windows/Mac will eventually stop recognizing the port its connected to until a reboot.
I have been using one from IOGEAR (SKU GUS432CA1KIT) for the last 6 months and I haven't had a single problem with it.

I use it to switch all of my peripherals (keyboard, mouse, wireless headset dongle, webcam, and mixer) between my desktop and my MacBook several times per day and have not had a single failure yet.

One I use is a brand called "Plugable". I don't think they still make the model I have, but they have a newer version of the same: https://plugable.com/products/usb3-switch2

I've used it with Windows 10, MacOS and an RPi, with no issues.

I have an old Belkin USB 2.0 switch from god-knows when that works beautifully. I use it to share a hub with a keyboard and mouse attached. I’ve given up trying to find a 4K60 x2 DisplayPort KVM that doesn’t suck.
I have this one[0] that works great.

Wendell at Level1Techs does good work and there's even newer ones that can do 8k30.

[0] https://store.level1techs.com/products/kvm-switch-dual-monit...

I couldn't find the details on their site, but do you know whether the 3.5mm audio jack is 3 or 4 pole? I've found that most KVMs have the 3 pole jack, meaning it just supports audio out, but I need 4-pole so my headset microphone works.
You can get USB audio dongles that have that. Which could then switch on the KVM instead.
I have been using this at 4k60 (but 1 monitor) flawlessly for 8 months.

https://www.startech.com/Server-Management/KVM-Switches/2-po...

Seconding a recommendation for Startech's KVMs. They have an excellent USB stack which handles complex USB hierarchies flawlessly -- I've successfully used mine with chained hubs, weird HID devices like Apple keyboards, and non-HID devices like disks, audio interfaces, and even stranger things.
> I did this for a while, the problem is that there's really no such thing as a high-quality USB switch. They are all garbage and mess with USB devices in some way (at least at the consumer level).

A lot of these things seem to "help" by inserting their own USB host device that acts as a proxy for attached USB peripherals. This allows them to do (ostensibly) useful useful things such as intercepting keypresses and responding to them (e.g. by switching to another input).

Don't get one of those, get a physical switch that just connects the leads to the correct port when pressed. I got this USB 2.0 switch (all you need for input devices) for $12 [0]. I'm sure there are others available with more ports, but this is all I needed.

[0] https://www.amazon.com/gp/product/B01HV1N674

Yeah I second this. I have a physical usb switch to move keyboard and mouse over. It works flawlessly.

Though, I also use a KVM to switch over the video but it's separate.

The USB side always worked fine for me. The problems tended to be mostly video related.

I have a HDMI KVM switch that's supposed to be 4k@60. One PC won't detect it as such; I can make a custom 1440p at 42Hz mode but nothing higher. The other will send 4k/@60, but the monitor will do one of 4 or so failure modes:

* Entire screen compressed to half width: 50% * Correct behaviour: 20% * Correct display, occasionally flipping to black screen: 10% * Gatbled screen: 5% * No synch: 15%

Switching input is a game of toggling back and forth until it synchs correctly.

None of this happens with a direct connection.

The switch also doesn't seem to detect when I set up a keyboard macro for the switch-input sequence (control-control-1/2/3/4) but that's not nearly the dealbreaker.

I'm not sure if it's the switch or the monitor being relatively intolerant of the switch's behaviour, but I don't want to toss around $100-300 on spares to diagnose this.

That's handy for the KVM part, but this is mostly about KVM over IP.
I feel like the ideal here would be for there to be a Thunderbolt/USB4 display which has multiple Thunderbolt "source" ports, and also USB-C connectors for peripherals, where the display itself is acting as a USB controller available over Thunderbolt-PCIe, with the USB-C sockets attached to said USB controller. Change the input on the monitor, and the USB-controller PCIe card in the display would be hotplugged out of one computer and into the other.

Even more ideally, the display would also have a built-in Bluetooth controller that stays active regardless of the USB controller's attach state, such that Bluetooth peripherals could be paired to the display itself rather than to the OS (i.e. you'd manage the pairings through the OSD of the display); and then these devices would be presented through the display's USB controller as always-on direct-attached USB devices — much like VM hypervisors present host-attached Bluetooth HID devices to their VM guests. (As a side-benefit of that, as long as the computer's BIOS understood Thunderbolt well-enough to display anything during boot, then even Bluetooth peripherals would also work during boot.)

I've just given up on Bluetooth for rapid device switching. It's too time consuming to be figuring out which device your headphones are attached to & disconnecting/reconnecting to resolve, especially e.g. when someone calls you. Wires are very, very easy to read and debug.

I have AirPods but will only pair them to my phone.

AirPods are the first blutooth type device that I switch all the time. If I am hooked up to my phone and connect on my Mac, instantly switches, if I am paired to my Mac and get a call on my phone and accept it, it opens a menu asking if I want to use airpods, tap that, instant audio / mic. The switching is so seamless and it makes the tech phenomenal for me.
I just got a bluetooth filco keyboard which you can swap between four devices using ctrl+alt+fn, [1-4] ... I've currently got my work laptop, rpi, ipad and home laptop all programmed in.
The LG ultrawide monitor with Thunderbolt that I have from a couple years ago (34UC97) mostly does this — It has a two-port USB hub which I use for my mouse and keyboard, which it routes either through the Thunderbolt interface to my MacBook or a separate USB cable to my PC, depending on whether Thunderbolt or one of the DisplayPort/HDMI display inputs is selected.

(I'd guess their newer models work similarly but I haven't confirmed.)

I just bought a PiBookPro (1) that I discovered on Twitter. It’s a laptop shell for RPi at $80. Kind of a wild backstory but hard to beat that $

1) Http://pibookpro.com

(comment deleted)
the best thing in this article is the immediate link named "I don’t want your life story; just tell me how to build it "
Haha, that was inspired by my experience of searching cooking recipes online.
Which begs the question - which cooking recipes start off with a biography of the author? Now I want to see those!
Oh, they all kind of do that. "Life story" is an exaggeration, but most recipe authors provide a lot of fluff backstory to a recipe before sharing the instructions because Google seems to favor longer content in search results. Here's a not so egregious example on a blog I follow, but it's ~8 paragraphs of fluff before they explain the recipe:

https://www.ketoconnect.net/protein-waffle-recipe/

Really neat project. Small optimization, you can use the RPI’s gpu accelerated h264 decoding and free up a lot of CPU:

Change:

    ffmpeg -vcodec libxh264
To:

    ffmpeg -vcodec h264_mmal
Thanks, good tip! I ended up not using ffmpeg after I discovered uStreamer, but this is a good trick to have in my back pocket.
Interesting! I wonder if it'd be possible to skip the capture card and have the pi act as a USB graphics adaptor.

Most firmware environments probably don't support them I guess.

I fear that's gonna end up as one of those "we transport HDMI over an USB-C cable without making it actually be USB" things that require hardware support.

(And the "USB monitor" projects meant to be a second display need drivers, seem to all be way too proprietary, and won't work for early boot.)

I wager no firmwares support them at all. Those work only after whole system boots up and loads appropriate drivers.
The hardest part of this seems to be the V in the KVM and the solution of the author's is pretty neat. But I think things could still be improved by dropping his keyboard/mouse setup and just switching to the standard: barrier/synergy, https://github.com/debauchee/barrier
Thanks for reading!

I'm not familiar with Barrier, but is my understanding correct that it requires software running on the target computer? If that's true, it wouldn't work for things like installing a new OS or managing BIOS because you'd have to wait for the computer to boot up fully before you're able to enter keyboard or mouse input.

While Barrier works well, TinyPilot is more capable. TinyPilot gives you full control of the remote machine, including BIOS screens. With Barrier you rely on having Barrier running on both machines, so you can't use it for things like reinstalling an operating system. Or fixing a machine that won't boot.
This is pretty cool (for managing servers cheaply), but I remain disappointed that remotely controlling computers over a network with low latency is still a difficult task. I've been trying to find a way to use a bunch of machines remotely over my LAN (for gaming / media playback on multiple devices), and have been somewhat disappointed at the quality / latency of the various options I've played around with. It's especially disappointing when you have 10gbps networking, and sending uncompressed 1080p60 video over over the network with a couple ms of latency should be trivial, from a bandwidth perspective. Obviously this is a different problem than what you're trying to solve (remote management, so you have to do HDMI + USB stuff in hardware, adding some latency), but it's still annoying that there aren't great lower latency software options.
Steam link does it, and fast enough to play games with no noticeable latency.

Not sure what the magic is, though.

I haven't tried Steam Link yet, but I might give it a shot. I was originally looking for purely remote desktop stuff for a combination of managing computers and gaming, so I was mostly testing stuff not specifically marketed to gamers. Ideally I'd also like something that does software<->hardware (at higher latency obviously), so I can stream older computers or game consoles to a variety of devices as well.

I did end up trying Parsec, since I saw some people recommending it, and that was terrible. Latency was fine (~10ms software <-> software) and video quality was alright, but audio quality was terrible despite being set to the highest option - which is especially bad, because there's no option to send uncompressed audio, and whatever "optimization work" they were doing to reduce bandwidth made their entire product unusable, despite the fact that there's no need to try saving 1mbps over a wired LAN.

the cool thing about this solution seems also, that it conveniently alleviates the "need a screen"-problem of the windows-screencapture-approach.
the latency is absolutely noticeable, the human brain adapts quite well though. i've switched mid-playthrough of ori and the blind forest (highly recommended btw) from a link to native 144hz and was blown away how easy the game has become :)

the point still stands: most games are very playable via a steam link.

SPICE (virt-viewer) performs well enough for my every day web browsing with VMs. The performance is annoying but adequate over wifi/WAN.
I'm curious how well this works over a LAN - over a WAN, my expectations are way lower because of actual limitations from going over the internet, so I'm fine with some latency / quality loss.
Sorry, I was trying to imply that it does work great over a wired LAN. I run my everyday browsers on a separate physical machine, communicating over gigabit ethernet. Videos with sound generally play fine, although I often youtube-dl longer ones for usability's sake. The main source of flakiness I've had is from opening too many tabs in low-memory VMs.
Have you tried out https://github.com/moonlight-stream/moonlight-qt ?

If all your host machines are Windows 8+, then Parsec is fantastic and has clients for every major OS other than iOS.

I looked into Moonlight, but don't have any nVidia cards laying around here to test with.

I tried Parsec on one machine and was super disappointed. Video quality and latency were fine, but audio sounded like crap, despite streaming at 512kbps. Since there's no option to stream uncompressed audio, and no debug options to figure out why it sounded so terrible, I had to write off their entire product as unusable.

This is some great timing, I might need a KVMoIP setup soon and I couldn't find anything cheap. Very nicely written article, thanks!
Awesome, glad it was helpful! Thanks for reading.
This is a great writeup.

I wonder if this can be extended to connect multiple servers to a single Pi.

This guy has apparently never heard of lynx browser. He wasted $40 and a lot of time because of that!
How does lynx browser help you adjust the BIOS or fix the boot process of a remote machine?
Hey OP, I was reading your linked HomeLab article and you mentioned you'd rather have gotten a mobo with integrated GPUs next time.

Actually, motherboards do not include integrated graphics, it's all in the CPU/APU. The mobo you linked would not provide graphics either.

AMD unfortunately does not have a high end APU released to consumers right now (there are some 4000 series, but only to OEMs I believe). However, for a HomeLab setup, you might find a cheap GPU to be useful for many things (including hardware video transcoding).

For years and years, many motherboards did have graphics sets integrated in. These days, you'll still see that on a lot of server-class motherboards.

e.g. https://www.supermicro.com/en/products/motherboard/M11SDV-4C... -- the line about AST2500 BMC graphics? That runs a VGA port.

I'm guessing the rationale for this is that even if the server's CPU can support onboard graphics, you wouldn't want to change the CPU's load profile just by plugging in a monitor to debug something? Or even to support CPUs that have no onboard graphics support?
Mostly the second -- the high end Intel and AMD CPUs don't have integrated graphics, and nobody wants to spend precious PCIe slots on a graphics card that won't be used except at install time and emergencies... or is more useful as a GPU slot that never produces video output.
> Actually, motherboards do not include integrated graphics, it's all in the CPU/APU.

You should probably qualify that as current generation. There's plenty of last gen Intel and AMD solutions with onboard dedicated graphics.

I think the advent of GPU acceleration in servers and APUs for low end machines has impacted demand for these motherboards in current gen products.

Thanks for reading and for the note! I didn't realize that about display not being available on motherboards generally.

I just built a new homelab server and ended up buying a separate GPU again. My new mobo is the SuperMicro MBD-X10DAL[0], which has no onboard display.

[0] https://www.supermicro.com/en/products/motherboard/X10DAL-i

I have MSI 450 MITX board and a Ryzen 3600 that is doing NAS/homeserver duty and it refused to POST without a GPU plugged in. Given its MITX and it only has a single slot needing it for PCI-E is a real pain, I intended to use it for SATA ports so it hurt my plans quite a bit.

Its motherboard dependent but its definitely something to check with AMD Ryzen motherboards.

Hm, I guess I lucked out then. I bought the cheapest motherboard I could find (at the time) for my 3600, an ASRock B450M Pro4. It makes a bunch of annoying beeps, but doesn't stop booting.
Second this. I got an unraid server working headless with an ASRock B450M ITX board (Fatal1ty) but it's still a real pain having to put in a GPU to boot into BIOS. Was hoping this could fix that for me :(
Yeah, it's a real pain in the ass. I miss the workstations of yesteryear where you could do everything with just a serial console. Another example of technology actually getting worse over time.

Even hardware that is made to be run headless ("server-class") is hopelessly broken. They give you a proprietary graphical console that isn't even just a VNC server but has to be accessed with a web browser. And, if the server is more than a few years old, some deprecated, unsupported plugin like Java or .NET will be required. I have an old Dell R710 whose remote console I will never be able to access, because it only works with Java 6 (or so I'm told, I'm certainly never trying that; I wouldn't even know where to begin to try to set that shit up.)

for some asrock mobo, headless is supported. my Fatal1ty B450 Gaming-ITX/ac is one of them.
I got a cheap Lantronix Spider KVM from eBay, it works quite well. Mine was an older VGA model but you can use it with a DP or HDMI to VGA adapter.
I also have a couple of Lantronix Spiders. I use them at remote client sites.

Some of the models have video-passthrough, which is an awesome feature. The ability to redirect an SMB share or local disk image to an emulated disk over the network is also a great feature for installing an OS.

I recently had seen some article on CNX about those cheap USB video adapters and wondered about doing something like this to replace these old Spiders.

There is definitely a market for a low-cost product here. It doesn't have to be perfect. As the article highlights, the pricing for this kind of product is just stupid.

A major con of the enterprisey ilom systems (such as the idrac) is their atrocious security track record. You are basically giving up your "the network is untrusted, I can survive its compromise" badge if you plug in one of those.
Well those ports should never face the internet anyway. Most servers will have a dedicated (physical) port you use for IPMI or whatever -- vlan that and only allow access from your VPN. If you're extra secure you can full on disable the switchport until you need it.
Make sure in the BIOS to disable fallback to one of the other ethernet ports. Quite a few IPMIs will listen on eth0 if it looses the dedicated IPMI port link by default.
This fail-open "should" is bad besides for the obvious reasons, also because it'll be extra ops complexity compared to a secure kvm widget that you don't have to handle with kid gloves.

(And thirdly because of the sibling comment noted footgun.. or silent foot-boobytrap more properly)

The problem is BMC has an astounding array of features[1] that are worth the operational complexity. This isn't just KVM like in OP's post... being able to remote mount images is a godsend when you're provisioning a server or diagnosing hardware issues or doing a BIOS update on the other side of the globe (with your other alternative being shipping a flash drive[2], then paying $200/hr for DC remote hands to plug it in for you).

[1] https://www.supermicro.com/en/solutions/management-software/...

[2] don't even try to talk about PXE booting if you've never tried to get DHCP+BOOTP to work over a WAN

I think you can do all those with iPXE, works well over wan. As a bonus you can get your images over https and not insecure tftp.