This is a simple Web GUI for puppeteer[0]. The interesting things about it are:
- It works like a very basic (circa 1994 ?) web browser.
- It runs in your normal web browser, even with JavaScript switched off.
If it became popular I would consider expanding the puppeteer API capabilities. Currently it only supports things that enable interaction with the embedded page (the web page running in the headless Chromium instance that puppeteer is connected to).
I achieved the no-client-side JS requirement by solving the following problems:
1. How to transmit clicks without capturing click events?
The button[type="image"] HTML tag transmits X, and Y coordinates when you click it on form submit. Making a full width and full height image button solves the requirement of transmitting pointer position (on click at least, and that limitation is good enough for our simple use case) without JS.
2. How to dynamically update the viewport (the image of the web page running in the headless browser) without client-side JavaScript?
The image/mjpeg MIME type is supported by browsers and implements a simple "server push" type of streaming using a HTTP keep-alive connection. It is not a new feature. I think I've heard MJPEG is how many web-viewers for CCTV or security cameras have been created for a long time.
3. How to combine sending pointer coordinates on a viewport, with a viewport that continually updates with any changes from the server (in other words, a client-side image that updates when the server side headless browser's page changes)?
Well, I combined them together in a way I find pretty cool, beautiful and elegant. And it's basically all thanks to how cool HTML is. The crucial line is:
<image type=button src=viewport.mjpeg>
So in that crazy way, a combination of two old-school HTML technologies (rarely used in the mainstream these days): button images, and MJPEG -- gives us a foundation for basic application virtualization. I love that. I love that the 'old web' can do something so cool and powerful, as well as loving that these two old-school (relatively) neglected (again, in the mainstream of current web dev) technologies can be combined together in one line of code to do something really cool and powerful.
Also, for bonus points, can anyone guess how I solved the, following challenge:
4. "Yes, but how do you know, how big the client's screen is without any client-side JavaScript?"
This was probably the most difficult one. Tho I'm sure some of you (hello, designers) will probably immediately see the result and answer, perhaps. What I figured out was I could use a stylesheet functioning as basically a giant "switch" statement by using media-queries "screen dimension breakpoints" to wrap an image request to the server. What that means is that only the matching media-query is executed, and then information is transmitted to the server by way of the request for an image that contains the approximate screen dimensions in the request URL. So the server can know how big the clients screen is, and it can then instruct puppeteer to set the browser screen dimension to be the same.
The relevant line (on the server, to generate the CSS stylsheet)[1] looks like:
function ViewportProbes(state) {
const BP = [];
for( let w = 300; w <= 1920; w+= 32) {
for( let h = 300; h <= 1080; h+= 32) {
BP.push({w, h});
}
}
const MR = BP.map(({w,h}) => `
@media screen and (min-width: ${w}px) and (min-height: ${h}px) {
body {
background-image: url("/set-viewport-dimensions/width/${w}/height/${h}/set.png")
}
}
`);
return MR.join('\n');
}
So all in all that's how I combined MJPEG, button images, and CSS media queries to do a very basic application virtualization, remote browser, web GUI for puppeteer that do...
Judging from your other comments I'm sure you have some experience in this space and your own views on the different options available.
VNC makes sense for a lot of reasons. Lots of established servers. and clients. Even a solid, free and performant web client--hello Apache guacamole.
The differential rectangle streaming method that VNC uses, where it sends only the layer or the smallest bounding rectangle that's actually changing, greatly improves bandwidth performance. VNC is also application agnostic: it's just about abstracting the viewport streaming and interaction wire protocol away from whatever is being displayed on the viewport--as far as I know. VNC has also been proven in a number of production deployments. I'm not an expert but you could probably also use VNC with xvfb or other late lightweight virtualization components that don't require a full desktop setup. I like VNC and I think it's cool.
And even though I don't think you're asking about why not use VNC for this particular project on this show hn because I think you're speaking to the larger context of streaming applications but even given that I'll just make a point about that here: the point of this little show is to do it in the simplest way possible using old HTML technology and no client-side JavaScript, plus an off-the-shelf service side instrumented browser in the form of puppeteer. I mean you could read that as like there's a theme to kind of do a project of a remote browser in the simplest way possible, and so if you consider like a thematic series of such projects, another possible member of that series could be the simplest remote browser setup achieved using vnc. That could be cool... I haven't thought about that or that thematic series before considering your question in this context.
But this show was more about exploring the possibilities of picking off the shelf puppeteer, off the shelf ancient web technologies and having like the minimal code necessary to provide a sort of MVP remote browser. Also there's a hint, a sense or a flavor of sort of retro tech aesthetic by using really ancient old school tech like mjpeg and the button image.
But aside from the context of this particular show I try to address your question in a more general sense for application virtualization or browser virtualization.
When I started my remote isolated browser project I looked at using VNC but if I recall correctly I think two things dissuaded me at the time from pursuing it further.
The first was I sensed that I wanted or that I would want and need more control over the actual protocol both of streaming frames to me and sending interactions to the server. I sensed that I would want, or like, to reproduce as faithfully as possible a browser experience with sound and multiple tabs and file upload and download and... And also that I would want the flexibility to style and design and alter and add to the functionality and the chrome of--ie the user interface of--the browser however the hell I wanted. Now maybe you could set up VNC with chromium headless and use it with xvfb so you could still just stream the viewport and then provide your own browser Chrome for the tabs the Omni box and so on. But I don't know and also I was just sure that I would want a lot of control which I would be able to have where I to be able to fully control what actions were possible in that protocol. So I wanted a way to fully access the capabilities of the Chrome remote debugging protocol which I was confident could give me a good basis for virtualizing a browser that was running remotely.
The second thing is the dissuaded me from pursuing VNC further at that time was VNC at least the clients and service that I looked into did not support things like file upload or download or audio transmission and those were things that I considered would be essential at least at some point to have or at least to have the option of having I consider that to be absolutely essential so I would not pursue...
3 comments
[ 0.15 ms ] story [ 33.2 ms ] thread- It works like a very basic (circa 1994 ?) web browser.
- It runs in your normal web browser, even with JavaScript switched off.
If it became popular I would consider expanding the puppeteer API capabilities. Currently it only supports things that enable interaction with the embedded page (the web page running in the headless Chromium instance that puppeteer is connected to).
I achieved the no-client-side JS requirement by solving the following problems:
1. How to transmit clicks without capturing click events?
The button[type="image"] HTML tag transmits X, and Y coordinates when you click it on form submit. Making a full width and full height image button solves the requirement of transmitting pointer position (on click at least, and that limitation is good enough for our simple use case) without JS.
2. How to dynamically update the viewport (the image of the web page running in the headless browser) without client-side JavaScript?
The image/mjpeg MIME type is supported by browsers and implements a simple "server push" type of streaming using a HTTP keep-alive connection. It is not a new feature. I think I've heard MJPEG is how many web-viewers for CCTV or security cameras have been created for a long time.
3. How to combine sending pointer coordinates on a viewport, with a viewport that continually updates with any changes from the server (in other words, a client-side image that updates when the server side headless browser's page changes)?
Well, I combined them together in a way I find pretty cool, beautiful and elegant. And it's basically all thanks to how cool HTML is. The crucial line is:
So in that crazy way, a combination of two old-school HTML technologies (rarely used in the mainstream these days): button images, and MJPEG -- gives us a foundation for basic application virtualization. I love that. I love that the 'old web' can do something so cool and powerful, as well as loving that these two old-school (relatively) neglected (again, in the mainstream of current web dev) technologies can be combined together in one line of code to do something really cool and powerful.Also, for bonus points, can anyone guess how I solved the, following challenge:
4. "Yes, but how do you know, how big the client's screen is without any client-side JavaScript?"
This was probably the most difficult one. Tho I'm sure some of you (hello, designers) will probably immediately see the result and answer, perhaps. What I figured out was I could use a stylesheet functioning as basically a giant "switch" statement by using media-queries "screen dimension breakpoints" to wrap an image request to the server. What that means is that only the matching media-query is executed, and then information is transmitted to the server by way of the request for an image that contains the approximate screen dimensions in the request URL. So the server can know how big the clients screen is, and it can then instruct puppeteer to set the browser screen dimension to be the same.
The relevant line (on the server, to generate the CSS stylsheet)[1] looks like:
So all in all that's how I combined MJPEG, button images, and CSS media queries to do a very basic application virtualization, remote browser, web GUI for puppeteer that do...Judging from your other comments I'm sure you have some experience in this space and your own views on the different options available.
VNC makes sense for a lot of reasons. Lots of established servers. and clients. Even a solid, free and performant web client--hello Apache guacamole. The differential rectangle streaming method that VNC uses, where it sends only the layer or the smallest bounding rectangle that's actually changing, greatly improves bandwidth performance. VNC is also application agnostic: it's just about abstracting the viewport streaming and interaction wire protocol away from whatever is being displayed on the viewport--as far as I know. VNC has also been proven in a number of production deployments. I'm not an expert but you could probably also use VNC with xvfb or other late lightweight virtualization components that don't require a full desktop setup. I like VNC and I think it's cool.
And even though I don't think you're asking about why not use VNC for this particular project on this show hn because I think you're speaking to the larger context of streaming applications but even given that I'll just make a point about that here: the point of this little show is to do it in the simplest way possible using old HTML technology and no client-side JavaScript, plus an off-the-shelf service side instrumented browser in the form of puppeteer. I mean you could read that as like there's a theme to kind of do a project of a remote browser in the simplest way possible, and so if you consider like a thematic series of such projects, another possible member of that series could be the simplest remote browser setup achieved using vnc. That could be cool... I haven't thought about that or that thematic series before considering your question in this context.
But this show was more about exploring the possibilities of picking off the shelf puppeteer, off the shelf ancient web technologies and having like the minimal code necessary to provide a sort of MVP remote browser. Also there's a hint, a sense or a flavor of sort of retro tech aesthetic by using really ancient old school tech like mjpeg and the button image.
But aside from the context of this particular show I try to address your question in a more general sense for application virtualization or browser virtualization.
When I started my remote isolated browser project I looked at using VNC but if I recall correctly I think two things dissuaded me at the time from pursuing it further.
The first was I sensed that I wanted or that I would want and need more control over the actual protocol both of streaming frames to me and sending interactions to the server. I sensed that I would want, or like, to reproduce as faithfully as possible a browser experience with sound and multiple tabs and file upload and download and... And also that I would want the flexibility to style and design and alter and add to the functionality and the chrome of--ie the user interface of--the browser however the hell I wanted. Now maybe you could set up VNC with chromium headless and use it with xvfb so you could still just stream the viewport and then provide your own browser Chrome for the tabs the Omni box and so on. But I don't know and also I was just sure that I would want a lot of control which I would be able to have where I to be able to fully control what actions were possible in that protocol. So I wanted a way to fully access the capabilities of the Chrome remote debugging protocol which I was confident could give me a good basis for virtualizing a browser that was running remotely.
The second thing is the dissuaded me from pursuing VNC further at that time was VNC at least the clients and service that I looked into did not support things like file upload or download or audio transmission and those were things that I considered would be essential at least at some point to have or at least to have the option of having I consider that to be absolutely essential so I would not pursue...