This is awesome! Wonder how it compares with mine (github.com/billyeh/termchat). Would love to share notes with the maker, since audio was something I did not even consider while I was making it. The image processing part is fun, though!
I helped create it at a recent hackathon. the ascii is rendered by calculating the intensity of any given pixel, mapping that intensity to a character based on how much that character fills up space ('@' would be bright and '.' would be dark) and then approximating the color of the pixel to fit within the 256 available
I've noticed your implementation uses websockets, which are built on top of TCP. we used UDP and encapsulated everything is a pretty simple to use (and easily the most well documented part of the project) p2plib.c. the fear was that video and audio can get backed up if we were to use TCP. so audio and video were sent via UDP packets and rendered every time a UDP packet was recieved.
You can go a bit further to get significantly higher color depths using ASCII block characters 176, 177, 178. By using the foreground and background ANSI colors and blending them by using these three characters (25%, 50%, 75%) you can achieve some quite realistic images. If you're more interested in resolution, using half block characters like 220 and 223 can allow you to double your vertical resolution at the cost of not being able to use the color trick.
I made something similar to this using a combination of 256 color mode and unicode halftone characters for "dithering" between the foreground/background colors.
Another thing I'd like to try to get a bit more resolution is to perhaps use the braille character set in combination with 256 color mode: https://github.com/asciimoo/drawille
Also, some terminals are starting to support 24bit color (iTerm2 nightlies) which could pretty drastically improve the possibilities of terminal based video: https://github.com/frytaz/txtcam/tree/color :)
I've tried doing this before but wasn't fully satisfied with my color lookup (very basic - really, even a hash table would do the way I made it..) https://github.com/minikomi/ansipix
Do you have any good examples of calculating the right foreground/background colors to use?
Wouldn't something like this be easier to encrypt for transmission than normal video? Plus, in the case it comes up, "that ascii representation doesn't look anything like me and you can't prove it was me."
Why would it be any easier to encrypt? It's data just the same.
I wouldn't rely on terrible quality video for plausible deniability but if you want to there are of course many ways to make your webcam feed shitty looking.
Easier in the sense of less data, which would make it faster. One of the issues with encrypted video chat is that most implementations have a max amount of people you can chat with because of the computing overhead.
Text is extremely compress-able. Admittedly, this is not common text, and is many "snapshots" of the whole screen, but I believe LZ-based compression would still be extremely effective on this data. The OP above said it can only be one of ten characters, and one of 256 colors, and a 100x40 grid, so it would likely require similar data rates to get this level of quality from a video stream, I would think, since the 10 characters provide more visual information than a simple colored pixel of that size could convey.
The text here may compress well, but only because the original video has been "decompressed" into an incredibly inefficient encoding in the first place. If you want to transmit this efficiently, then downscaling the original video and encoding it efficiently using a standard video codec and rendering it to text client side would be vastly more efficient.
well it is 100x40 in number of characters and there are 10 different characters that can take on one of 256 colors. that's 4000 x (8 (256 colors can be represented in 8 bits) + 4 (10 characters in 4 bits)) = 48Kb per frame, with about 20 fps. so maybe 120 KB/s?
Thanks. It would be better to have some navigation functions like what you can do on Windows with the default image preview program, so that users can browse through a series of images without having to convert them one by one.
I recently started looking into building a similar web cam project using WebRTC, except sending ascii images over the datachannel instead of encoded video. So, I was pleased to see this project making it happen. You could easily implement this with XMPP/SIP over WebRTC.
That's interesting. But if you already have a graphical browser which supports WebRTC, why do you still need the ascii? Or is there a terminal browser with WebRTC support?
Bandwidth: I'm often in areas where bandwidth is a premium but I'd still like to video chat with family. And besides, I absolutely hate dropped frames, rebuffering etc etc: I just want to engage in a conversation with friends and I don't care that it's HD video. Applications like Skype don't let you drop the bandwidth (for reasons I don't understand) and dropping it in WebRTC makes the compression almost unreadable. As this project has shown, ASCII video can really be quite legible and probably a lot more forgiving for lower frame rates.
That's pretty awesome, now if I can fins a decent terminal browser that supports javascript and vim keybindings and the equivalent of chromes devtools, I can spend all of my time in the terminal (save design of course, but that wouldn't even make any sense). It's probably unreasonable to think that I'll fin that, but I'll keep looking.
Unfortunately from what I understand, IPv6 has the option for NAT. Once it goes mainstream we'll probably find ourselves right back in this situation with multitudes of users stuck behind firewalls as second class netizens.
47 comments
[ 3.0 ms ] story [ 101 ms ] threadI've noticed your implementation uses websockets, which are built on top of TCP. we used UDP and encapsulated everything is a pretty simple to use (and easily the most well documented part of the project) p2plib.c. the fear was that video and audio can get backed up if we were to use TCP. so audio and video were sent via UDP packets and rendered every time a UDP packet was recieved.
https://github.com/dhotson/txtcam
Another thing I'd like to try to get a bit more resolution is to perhaps use the braille character set in combination with 256 color mode: https://github.com/asciimoo/drawille
Also, some terminals are starting to support 24bit color (iTerm2 nightlies) which could pretty drastically improve the possibilities of terminal based video: https://github.com/frytaz/txtcam/tree/color :)
Do you have any good examples of calculating the right foreground/background colors to use?
http://aa-project.sourceforge.net/aalib/
http://caca.zoy.org/wiki/libcaca
and possibly other clones too. Might help improve the visuals.
I like it.
I wouldn't rely on terrible quality video for plausible deniability but if you want to there are of course many ways to make your webcam feed shitty looking.
Note you can tunnel arbitrary ports in ssh for data if that's what you're thinking of.
sudo apt-get install libncurses5-dev
sudo apt-get install libpulse-dev
One request - what's the bitrate? (I know it's probably possible to measure in 1 line of shell)
"VIDIOC_STREAMON: Invalid argument libv4l2: error turning on stream: Invalid argument
Can't seem to get it to work trying anything on github etc and want to get enough around to file a real issue.
Nice use case for IPv6.
I remember watching the world cup on an ASCII terminal
:)
I made a similar project to this (but without sound).
https://github.com/kfei/sshcam
Which uses SSH connection, 256-colors, Unicode block characters and CIE94 algorithm to compute color distance when mapping colors to 256 palette.
Watch it in action: https://www.youtube.com/watch?v=pAa-pGda9kY