HN Explorer

Show HN: I saw this mind-blowing experiment, so I made a simple version of it (github.com)

403 points by momciloo ↗ HN
Two browser windows (acting as socket clients) communicate their:

- Screen dimensions - (screen.width, screen.height)

- Window dimensions - (window.innerWidth, window.innerHeight)

- Window X/Y position - (window.screenX, window.screenY)

...or whichever calculation works best for you.

The original work by Bjorn Staal https://twitter.com/_nonfigurativ_/status/172732259457002734 used localStorage, but I found sockets more fun, because if tweaked a bit, this can be shared with friends.

Here's a demo of how it works and the codebase: https://github.com/Momciloo/fun-with-sockets/

138 comments

[ 4.3 ms ] story [ 225 ms ] thread
doctorhandshake
Cool! Feels like whichever window has focus, that window’s rectangle should draw on top.
benj111
Isn't that just to illustrate that it isn't just transparency?
mholt
More like this:

- https://twitter.com/steveruizok/status/1727625036159234555

- https://twitter.com/_nonfigurativ_/status/172732259457002734...

- https://twitter.com/wesbos/status/1727755227766350031

- https://twitter.com/wesbos/status/1727722358465523721

bionhoward
Wow this seems profound, I didn’t even know you could do that, that could be the future of augmented reality if you think about it, layers! Great job!
simbolit
Could you please expand on your "if you think about it"?
bionhoward
sure, if you think about the long term usage of augmented reality then we will need to have multiple layers of content {rationale=because we need to focus on one thing in the context of multitasking, because we want to reskin the environment while still maintaining the content, and to avoid unsafely obscuring the field of view during ambulation and manipulation tasks}
jonhohle
Is it really different from any multiplayer game? Two view ports, shared coordinates, independent renderers. The “trick” is that the view ports are overlapping, but I would imagine sendMessage might be faster and not require any server round trip.
bionhoward
I was just trying to be nice, sheesh. happy holidays
aantix
Or a postMessage could be used if one tab is opened from the other.
rtcode_io
Use BroadcastChannel; you don't need no sockets

https://developer.mozilla.org/en-US/docs/Web/API/Broadcast_C...

[deleted]
(comment deleted)
bigyikes
Nice, my first thought was to use LocalStorage. Didn't know about BroadcastChannel.

Sounds like OP wanted to use sockets for fun though.

>The original work by Bjorn Staal used localStorage, but I found sockets more fun, because if tweaked a bit, this can be shared with friends.

Who among us hasn't overcomplicated things for fun? :)

kaycebasques
Very cool. What's the working title for this interface / experience?
placebo
made something similar 14 years ago...

https://youtu.be/bEwsuWNxdno

domain isn't mine anymore...

HatchedLake721
Oh wow, from December 2008.

Here's to no one trying to patent it now!

whenlambo
What API allowed that back then?
placebo
It was an API I developed using ActiveX on Microsoft's Internet explorer and NPAPI on other browsers
worldsayshi
I don't see what is technically new about this? What API has been missing to do this?

Tcp and window.screenTop and screenLeft should get you there right? Hmm, yeah, 2008 is a long time ago though..

munch117
Am I the only one that's horrified that window.screenX and window.screenY exist? This is not information that I want to provide.
progbits
Using the demo in https://developer.mozilla.org/en-US/docs/Web/API/MouseEvent/... it seems Firefox pretends the window is always at (0,0). So that's nice.
Cyphase
You likely have `privacy.resistFingerprinting` set to true.
[deleted]
(comment deleted)
fyokdrigd
as everyone should!

people forget lesson from recent past. i don't know when google strong armed w3c to include this back... but we had this since ie4 days and then disabled everywhere since it was abused left and right for click jacking attacks.

xeckr
I just checked, it is possible to override them, which causes them to stop updating when you change the your window's location on the screen.
Andrews54757
Yep. There are some ridiculous web APIs out there that expose all sorts of things you'd think should remain private. Take the Battery Status API for example [1].

[1] https://developer.mozilla.org/en-US/docs/Web/API/Battery_Sta...

geoelectric
IIRC, that one and a number of other lower-level “WTF” APIs were introduced for Firefox OS. They made a little more sense in the context of providing an interface for mobile phones using the browser stack.
[deleted]
(comment deleted)
zer0kool
Sorry, i'm a little lost. I tried thinking about it, but I can't think of any nefarious attacks someone could do with screenX/Y. Why is that such a risk? What could they possibly learn from that other than screen size?
jackewiehose
Because fingerprinting. Shit like this is the reason you are supposed to use the Tor browser only with a tiny window.
spiderice
> you are supposed to use the Tor browser only with a tiny window

I hadn't ever heard of this. Here is a statement from the Tor project for anyone else wondering what this is about:

> We automatically resize new browser windows to a 200x100 pixel multiple based on desktop resolution which is provided by a Firefox patch. To minimize the effect of the long tail of large monitor sizes, we also cap the window size at 1000 pixels in each direction. In addition to that we set privacy.resistFingerprinting to true to use the client content window size for window.screen, and to report a window.devicePixelRatio of 1.0. Similarly, we use that preference to return content window relative points for DOM events. We also force popups to open in new tabs to avoid full-screen popups inferring information about the browser resolution. In addition, we prevent auto-maximizing on browser start, and inform users that maximized windows are detrimental to privacy in this mode

pixel8account
Fingerprinting by screen size is real. I've checked one day, and it turned out my browser viewport size when maximized is super unique (like <0.1% users unique). That's mostly because I use sidebery (tabs on the left instead of the top) and don't have a (visible by default) bookmarks tab, but wow I didn't expect to be so obvious for advertisers by just reading my page-usable screen size.
munch117
The fingerprinting potential may well be limited, but it's the principle of the thing: The browser should not provide unnecessary information about its surroundings.

It would be another matter if this was a very useful feature, but I don't think that it is.

Here's one use for it: Ads could sense that they are off-screen and withhold content until the ad is moved back in view. I could easily see screenX/Y being used for that. I mean, I hope not, but how am I going to stop it?

fyokdrigd
knowing where the window is, you can generate something the user will click on a known position and then at the right time you trigger something like an administrator escalation privilege confirmation dialog and the user clicks that instead.

click jacking is always fun.

no_time
W-why is window LOCATION exposed to websites? Apparently even in FF private browsing...

At first I thought this is an electron only thing. I get it that we gave up on fingerprinting resistance but this is like pissing on it's grave.

EDIT: privacy.resistFingerprinting set to true fixes the coords to 0

a_wild_dandan
I thought you meant literal `window.location` and was confused, but yeah `window.screenX/Y` is a pretty suspect feature, now that you mention it.
crazygringo
I'm as surprised as well.

I'm trying to guess why, and can only imagine that at some point the idea was to support multi-window webapps, like maybe you'd have separate windows for documents and different tool palettes, and they'd want to know each other's locations to try to prevent overlaps? I mean, back in the day, framesets were another idea that experienced some popularity before being totally eclipsed by <iframes> -- all of this back in the era when everything was based on the multiple-windows paradigm rather than the multiple-tabs paradigm. Window.screenLeft apparently originated in IE [1].

Just a guess though. Very curious if there's any legitimate reason for it to continue to exist, or whether W3 ought to deprecate it.

[1] https://developer.mozilla.org/en-US/docs/Web/API/Window/scre...

[deleted]
(comment deleted)
thayne
On Wayland, a native window can't even get that information. At least not in a way that works across composiotors.
andybak
The original seems to be on LinkedIn. Does anyone else find it weird when geek content is posted to LinkedIn? It's the last place I'd think to look. Am I missing out or is this an outlier?
cpach
For sure. It’s approximately the first time I’ve seen something like that on LinkedIn.
zerojames
I work in computer vision and like hearing about the latest thing. I have been finding LinkedIn increasingly useful as a way to learn about machine learning.

There are _a lot_ of posts that I have to scroll through of little to no value, but every so often I see a post by Yann LeCun or the lead developer advocate at TensorFlow or a paper someone has liked that catches my eye or a new model release.

"Machine learning LinkedIn", as it were, is a real thing; not sure whether this applies to other disciplines.

Geee
Yann posts mostly on X.
[deleted]
(comment deleted)
pixel8account
I've recently started posting my content on LinkedIn (usually just linking to my blog or publication with a short explanation). Main reason is that I'm a bit unhappy with the new Twitter (a place where I used to share the most), approximately nobody reads Mastodon so I can't stop there, and all least I already have LinkedIn so it's low cost to share there.
jocaal
The available API's on the web are very questionable. Feross has a web security lecture series on youtube where he showed off the following abomination. Enter at own risk.

https://theannoyingsite.com/

lgats
warning, this is really annoying. imagine you're infected with a ton of adware in the early 2000s you will be logged out of many accounts

here's someone testing it out on Windows a few years ago, but I assure you even in the latest chrome on mac, it's at least as bad https://youtu.be/estvbRSj4LU?si=gq6-l4pWLC9yXqap&t=109

dylan604
Why would you keep minimizing things instead of closing them? The video just looks like one of those meant to frustrate the viewer into wanting to do it themselves. I blame the creator of that video as much as the site itself. The site itself served a purpose. The person running that video has a much less clear motive
bbarnett
I think if you close a window, another pops up. Or two.

You should go to this site and test this.

dylan604
It spawned an app outside of the browser. That's the window being minimized that I meant. I'm not an unawares person to the site's purpose. You're being a bit disingenuous with your reading of my comment
PakG1
I tried it on Safari private window. It was absolutely horrible. I hesitate to guess what would have happened if I didn't keep clicking cancel whenever it asked if I wanted to allow downloads, etc.
38
uBlock Origin has prevented the following page from loading:

    https://theannoyingsite.com/
Because of the following filter:

    ||theannoyingsite.com^$document
Found in:

    uBlock filters – Badware risks
[deleted]
(comment deleted)
gmueckl
Does this work on Wayland?
HellsMaddy
Tested on Sway with both Firefox and Chromium. Resizing works (no surprise) but the position doesn't work. window.screenX/Y always return 0.
account42
Good. Websites have no business knowing anything of the world outside the viewport.
itronitron
This, or something similar, would be nice for managing layers in a paint program such as Krita, Inkscape, or Gimp. Could be implemented simply as tabbed panes within the encompassing application window, and whatever tab is selected is the layer that is active for the editing actions.
hot_gril
This reminds me of a project I did in school, a multi-user web browser. You could both control the same window's size, scroll position, and other state on each other's screens. The intended usage was having 2+ windows side by side.
butz
I'm wondering about practical application of this demo...
worldsayshi
Maybe making colour mixing ven diagrams?
sntran
I have always been fascinated by those sci-fi movies in which a person would swipe up on their tablet, and the current window would fly to another monitor.

This maybe be something that can achieve that.

bartvk
It would not surprise me if the original practical application is long gone, but it's now used to fingerprint browsers for ad serving purposes.
[deleted]
(comment deleted)
ww520
That's a fun use of LocalStorage.

I used the same LocalStorage sharing technique to update the target window when the setting had been changed on a separate browser window. Just listen for the storage.onChanged event for the update.

abid786
I don’t think this is using local storage. There’s a web sockets based server that is sending clients the location of the other boxes as they are moving
sprobertson
Might as well link to the original / "stolen from" author's equivalent demo with Three.js (open this in multiple overlapping windows) - https://bgstaal.github.io/multipleWindow3dScene/ - and repo - https://github.com/bgstaal/multipleWindow3dScene/
[deleted]
(comment deleted)
jdthedisciple
3 days old, twitter one seems 4 days old, who stole from who?

Anyway nice repo, thanks for sharing!

sprobertson
The links I posted are from the original Twitter guy but a simplified version he made to open source it
vunderba
That's a great demo! I'd be curious to see how this would work with multiple monitors.

Also, thank you for voluntarily acknowledging that you were directly inspired by somebody else and giving them credit - the software industry could use more people like you.

stavros
What, thieves?

Make your stuff from zero, like the rest of us!

zxexz
Pretty sure this is tongue-in-cheek, given the commenter (pretty high profile dev who creates a lot of cool stuff and does not shy away from using other open source code!) :)
kh0st
that was super cool of you. Before letting people make the incorrect assumption, you hit em off at the pass. good people right there
stavros
Aw I thought it would be obvious :( Thanks for clarifying, of course nobody makes anything without being inspired from something that already existed, at least to a degree.

I guess it's hard to accurately convey sarcasm online, thanks again!

Sirizarry
No worries, it was pretty obvious to me
bipson
Standing on the shoulders of giants
p-nerd
Really great. The web is really going forward
bottled_poe
Seems like this was all possible 10+ years ago with existing APIs at that time. I don’t see what’s so special tbh.
Agingcoder
Could someone explain what this means ? ( I’m not even sure I understand the gif on the GitHub page ) I just see windows seemingly sharing data.
NL807
I think it's some kind of multiple client, single server setup. The windows are the individual clients, which pass screen geometry info to the server. The server then serves content to clients with different layout that makes content appear as a single object across multiple windows.
[deleted]
(comment deleted)
et-al
The core of the trick isn't the window-window communication as much as using browser APIs to expose the window coordinates.
Filligree
But… why does it lag? Shouldn’t this be trivial enough to be instant?
bawolff
I wonder if it would be more instant if they used postMessage api instead of going over network.
4death4
Any local network usage should be nearly instantaneous.

My personal theory about the lag is that positioning the window is not synchronized to the JS thread, so the window moves before the JS thread is aware, which means you see the square move after the window has already moved.

bawolff
Ah, my bad. I didn't read the code and just assumed there was a far away server.
rco8786
Going over local network is more or less instant
taberiand
Probably because of websocket network lag and also, in my anecdotal experience, window position updates are sometimes strangely jittery
mrits
Why is it over a websocket? Can't it just share local data?
EGreg
those were my thoughts too

You can use Broadcast Channels as well as Service Workers, even WebRTC would use a loopback interface!

Zecc
This ought to work across different browsers though, for what it's worth. But ultimately it's just a fun experiment.
raincole
I might be wrong, but I think the point is to simulate a server-client architecture (to show that it works even for an actual web app).
trevor-e
It says so right in the OP

> The original work by Bjorn Staal used localStorage, but I found sockets more fun, because if tweaked a bit, this can be shared with friends.

akdor1154
Background windows/tabs getting a lower priority maybe a factor.
Geee
Querying window position is laggy unfortunately. It's a browser feature.
ogurechny
Well, it's in a browser. Between trivial mouse move and trivial object coordinate change, there are numerous system and application layers to relay event to your interpreted (or compiled) code, and then to deliver state change to screen.

Even if we move to native world, making two different windows do the same instantly is not always trivial. For example, there have been articles about GUI toolkits making it impossible to have smooth window resizing. If we make everything ourselves (say, immediate drawing in response to WM_PAINT), we can only hope that the system notifies windows quickly, and that performance available is enough to pump bitmaps to screen. Or, better, we can try to use system-wide software or hardware compositor, add layers for individual objects, and only change their coordinates. Then we rely on compositor being good enough to handle 60/120/144/whatever updates per second when it's needed.