Some of you may have seen this before under its previous incarnation of "Texttop". Then it was just a hack, but I got some great feedback, so I've spent most of the last 12 months turning it into something serious. It's morphed into more than a mere TTY gimmick. That UNIX philosophy of text being the "universal interface" has somewhat unexpectedly risen to the forefront, such that Browsh is now essentially a text browser engine, serving to both TTY clients and, somewhat ironically, other browsers - see https://html.brow.sh Being able to render any modern site (even WebGL for example) to text seems like an appropriate return to the web's origins. The obvious benefit being to those less fortunate that have slow and/or expensive bandwidth.
Also, I'd be interested in any opinions here about how to financially support this project. At the very least I'd just love to somehow make this my job.
My guess is to try upselling features like anonymous browsing, ad-blocking, etc. Other ideas would require discovering use-case scenarios for corporate situations: can it be used as a cms-front end or a way to focus on text for localization? a11y testing? You never know.
What do you think of its potential for use in regions with slow and/or expensive Internet (South America seems to be the worst off here), where I could come to some arrangement with a local NGO/GO to fund server costs and development?
You absolutely could, Opera browser was doing something (I think they proxied images and similar files) before it gets to someone's screen. If you can compress large images (and you'd have to know they're images cause you gotta render them right?) in such a way that they're not visually distorted it would be worth paying for.
The harder thing is trust, trusting you to not change any images. On the other hand that'd make for a fun VPN service for general browsing (but again then it comes back to trust) but if you target it as a browser plugin bundled VPN service that saves on bandwidth you could hit more of an audience and use that income to fund Browsh. I figure you dont want to get too far outside of that scope though (development wise) since it's a lot more work.
I also like the other ideas of using some of the Browsh capabilities to scrape the web, maybe as a paid API, sure you can write a script to do that, but if you can curl / postman an API that does it for you and returns what you need, then why wouldn't you? Trick is can an API be done just perfectly enough? How do you tell it to give you whatever part of the screen with whatever requirements? It could sound simple but get complicated quickly enough. If it's too complex maybe a compression VPN is a better target.
Yeah the Opera-style compressing VPN idea is definitely high up on my list. That's why I released Browsh along side text.brow.sh and html.brow.sh At the moment they only return static content just as CDN would, so there's no privacy concerns, but to make those services truly VPN-like then they'll need to allow logins, sessions, etc, which then very much raises privacy concerns. And there's just nothing Browsh can do to assuage that, other than good branding, because Browsh by definition has to read every character on page in plain text.
I've got a good feeling about text.brow.sh, it's so simple. As you say I just need to work on the consistency of what it returns.
Charge for plugins? For example, user macros, user styles, integration of other apps, advanced bookmarks, etc. Sell merch, create your own awards for lo-fi websites and charge people to nominate their sites. Hope it works out for you!
I am very familiar with uBlock and such. I have a local DNS at home which blocks a lot of stuff for the whole family at home without using plugin-blockers. My wife don’t know a life with ads and is often surprised when visiting friends: “what a nightmare of distraction”
What I mean is an app that connects to an instance (vm, server) a brow.sh.
I would like to install this in a lxc container and connect to it from my mobile via VPN and use HTTP-proxy settings or and with an specific brow.sh-app for my mobile. Could this work?
It's be great if html.brow.sh could become a sophisticated proxy that stripped the web down to an absolute bare minimum. Yes I think travelers are the most likely to pay for it, but I think the poorer people of less developed nations would be the largest sector of users.
Currently though html.brow.sh is non-interactive, as soon as I start letting people logging in it opens up a whole bunch of privacy concerns!
Thanks for the answer. Well, created a Linux container (Ubuntu 17.10) using lxc. Installed browsh. Works very very fine. Same issues as others found:
* Forms not working in Firefox but works fine in the terminal. Google search...
Some more testing:
- use nginx as a proxy for browsh
- configure the Firefox instance (install uBlock Orgin, remove history and set other privacy related configurations and hope it works.
I could see this being part of a lightweight browser UI testing framework.
There's already ways to drive a headless UI (Selenium, puppeteer), but they require automation testers to interact with page elements via the "normal" DOM which can be quite painful in terms of creating automation tests - DSLs to drive DOM automation are necessarily huge.
I could envision browser automation via browsh using a very small DSL e.g. browsh.url() to navigate to a URL, browsh.click() to click on an element, browsh.text_exists?() to check whether text exists, browsh.link_exists?() to check whether a clickable link exists
I've thought of that too. The trouble is that Browsh's text renderer is more of a science than an art. It's quite hard to actually get a guarantee of the same text twice, which is a pretty important condition of testing.
But yes, I'd love for Browsh to be used in such a way!
Guaranteeing the same text twice might not be so important. There are pixel comparison testing tools for popular browsers. Your advantage is probably speed. Would cypress end-to-end tests run way faster in Browsh than in a graphical browser? If so, you've got something of value to big companies. You might want to partner with or contract for Cypress.
EDIT: Since Browsh depends on Firefox I'm curious whether it's actually faster than a graphical browser.
As an anecdote, somebody at my workplace has just spent a day tearing their hair out trying to figure out how to get our selenium tests running automatically on a headless machine, because the resolution flat refused to go above 1024x768. Even then, they're slow and heavyweight.
A very fast browser not tied to a graphical display with the bonus of being able to put error screenshots into all of our existing tooling could actually be quite beneficial here, whether the text is character-identical every time or not.
For sure that's definitely an option, for both the SSH service and the html.brow.sh service. As people get to learn about Browsh let's see if there are any takers.
> Also, I'd be interested in any opinions here about how to financially support this project. At the very least I'd just love to somehow make this my job.
I think before you can answer that question you have to figure out if there are enough geeks like you who think this is a useful enough tool to run / use regularly. They'll likely pay you $. Alternately, find use case where even just a few users / companies would pay $$$ each for some thing that it does that you can't easily get anywhere else and helps get work done.
For me it looks cool, and i'll give it a try, but i'm more likely to stay with my normal browser because with plugins like vimium i don't have to reach for my mouse very much. That's actually one of the reasons i use the command line so much. Make it work without constantly using my mouse and I'd be far more likely to use it. Once i'm using it regularly, then I'd be starting to think "I wish it would..." and then making those wishes come into practice is something i'd support. This is kind of a weird / esoteric thing that would just gradually improve so... maybe Patreon is a better route for getting support than normal software / feature / plugin sales.
personally, IF I find myself using it regularly, i think i'd be more likely to support you via a Patreon type regular donation.
Thanks for your advice, it's really helpful. I agree, I think it has a large gimmick factor, for which I don't think there's a huge amount of $ (though it does help spread the word). You have a good point about finding a niche in a large company. As others have mentioned this looks like it would be to do with endeavours out in the field so to speak, where the Internet is slow and/or expensive. So eg; military or shipping operations.
So at the moment I guess it's just best to hope as many different people and organisations see Browsh as possible and seeing what sticks and where.
I think your niche might be companies or professionals that need to communicate via satellite while 'out in the field'. For my job I need to take an Inmarsat BGAN terminal with me on trips into the backcountry so I can pull up a data connection wherever I happen to be. Obviously the data is very expensive. To limit costs, I check my email via SSH using Pine and basic web browsing with Lynx. Regular web browsing in Chrome is slow and very expensive. Your browser represents a huge step up from Lynx.
The maritime industry could find a large benefit from using your software. A friend of mine is a marine electrician who often works on large commercial boats up in Alaska. Some of these boats will pay in the neighborhood of $7000 per month for satellite service and unless you're the captain you get very limited access time. I'm sure they'd be interested in cutting their web browsing costs.
That's an excellent point. Reduce cost of "$xxx" to $x" by filtering / proxying through legible text. And yes. A return to the web's roots. I applaud the author b/c I have constantly wanted to do this same hack, but would never have done it to this level.
> Wouldnt it make more sense to build this as a cloud service?
Definitely. Maritime companies and backcountry users probably won't be interested in messing around with a nifty open source tool nor the hosting infrastructure it'd require. They'll want this built out as a secure easy to use service.
@tombh Feel free to get in touch if you're interested in building or optimizing for satellite users. I'd be happy to provide some beta test feedback using my equipment.
I've been messing about with a toy search engine for a long time, and starting to tie it into my Pinboard bookmarks (6,000 and counting!), and the most irritating part of the whole thing for a while has been extracting useful text from modern websites.
I'd considered using a headless browser setup but I never got 'round to it.
If you structured this as a paid service where I could make some number of requests for some amount of money and get back basically what you've got now in monochrome mode, I'd probably sign up pretty quick.
This is sort of relevant to some other recent HN discussions around the trouble with large-scale crawling and the trouble with so much of it being restricted to Google now, so I don't think I'm the only one.
I've thought about this too, because https://text.brow.sh certainly does make scraping websites easy, you just need to `curl` and `grep`. But the trouble is, as I mentioned elsewhere in the thread, Browsh's text renderer is more of an art than a science, it doesn't guarantee consistent results for every render. Whereas of course downloading HTML and using something like Beautiful Soup does give a much better guarantee of consistency.
I don’t think that lack of consistency should stop you though. Ok maybe it wouldn’t be suitable for testing things but there are other uses for a headless browser
The web 1995-2005 or thereabout will be the best documented period of humanity and it is saddening to see how payload has been replaced by cruft to the point that we might as well have remote PDF viewers.
Has further notes that don't just overlap with the lemonade-stand thing. Unfortunately, Snowdrift.coop is still not available to use itself at this point.
This is not my favorite monetization strategy, but if you had enough people using your browser you could serve text-based ads in place of the now-pixelized image ads on the screen. To test this model right away, you could contact a developer-focused ad network like Carbon (https://carbonads.net/)
Another insidious idea is to pitch this browser to companies as a way to reduce wasted time on social media and time-wasting websites - employees could be trained on how to quickly extract text information from a website that is relevant to some business goals, without getting distracted by the time-wasting opportunities that a browser provides.
What I would love to see (but where far fewer promise of riches lay) is to use this as part of some great developer tool - this seems it could be a more intuitive way to test an interface than Selenium. Until then I look forward to personally supporting and using your awesome project!
This approach has already crossed my mind, and its not my favourite either. But if it enables me to make Browsh my job then I'd be open to it. I actually already self-advertise Browsh at the bottom of every https://html.brow.sh page. So I'd certainly be open to placing a simple ad link there at least.
Ha, the "reduce wasted time" idea is interesting one!
I got a wild idea that I want to do myself for funding a hobby of mine. I am developing a similar software compared to yunohost or cloudron. The way I want to fund my hobby is by afflilate marketing. I am developing a solution for hosting, so it might make sense for someone to use it via digital ocean. All I got to do is provide a link as a their affliate member (which I am as of now not apart of) and my user gets a discount, digital ocean gets a customer, and I get about $20 dollars I think. There are several other programs like this by various other companies. What I think you might do with this idea is offer an affiliate deal for your users for isps or organizations that want to provide super low bandwidth services. I read recently there are still millions of modem users in the USA because of the various remote locations can't afford to bring a network in and no political will to do it either. Good luck to you, I think you got a good idea. I am not able to get it to work on docker at this time but I'll be keeping an eye on it.
I've had the idea to make a webextension that transparently proxies all top-level URL bar requests to html.brow.sh, that way you can click on links in other applications that open in your browser and instantly get the Browsh version. I think it could a fantastic resource for those in the world that have slow and/or expensive internet.
First though I really need to improve html.brow.sh
I wonder if it would be a benefit for EU people who just want to read the site and not get a modal that tells them if they don't accept privacy invasions you can't read the page.
I'm getting tired of writing down GDPR violations to report.
I don't see how you would end up in a GDPR violation scenario for doing it. I suppose the more likely scenario would be Company X wanting people to agree to give up data, and make it hard to access site without giving up data, but browsh makes it easy. So then the company complains that browsh is essentially enabling theft of their resources, I don't think that will work very well though.
actually one problem you have here, and you're probably realizing from the responses, is a problem I had with a personal project that could be used to build lots of things that people wanted, but in itself was not a thing people want. In order to monetize you need to build one of the things people want, but in doing so you put focus into one area and stop improving the generic base of everything, as well as not producing many little toy examples of the possibilities of your technology.
While your point stands, you may be interested to know that a company actually successfully sold a patent for jpeg at some point, and made about $100M doing so. So I wouldn't say nobody made money on jpeg !
But it's also interesting for the OP, maybe there's something patentable in there ?
If you're referring to Forgent: they were patent trolls and their patent was invalidated due to prior art. Unfortunately not before a bunch of companies paid up.
Fraunhofer is a better example of an institute that made good money on tech patents with merit, as well as 3M and lots of other tech companies.
But as a rule these things are hard to monetize and while I'm not happy that it will be hard to monetize the work that went into this particular project at the same time I think it is good that patents are most likely a closed avenue.
Patents are - in my opinion - a net negative and the sooner they are abolished the better.
No, thank you. That must have been a very long and hard slog to get this to work as well as it does and unfortunately this is the only way that I have to show my appreciation of that. (Been there, done that.)
Ha, you're the first person to mention that. It's probably one of the hardest things I've ever done in my life, both technically and in terms of stamina. So I very much appreciate that someone else can understand that.
Stupid question: how do you follow links without a mouse? e.g. with Lynx you'd use the arrow keys / enter - this doesn't seem to work here, and there's nothing obvious in the keybindings document.
Regarding financing, perhaps you should consider collaboration with the Tor Project.
Many Tor users disable Javascript, because Javascript sometimes has anonymizing vulnerabilities. But disabling Javascript cripples many websites.
But I can type html.brow.sh/https://example.com via my Tor browser and get a non crippled website, even with Javascript disabled. This is a very practical use case and I think many will want to pay you to make it work better.
What you need to know about Tor is that the network consists of clients and relays. Clients are mostly users with browsers. Many of which disable JS, because JS can de-anonymize sometimes. This breaks some websites.
Previously I could not visit CNN.com via a JS-disabled Tor browser, but now I can type https://html.brow.sh/https://edition.cnn.com/ and Viola! (Well, it's still a little crippled but much better than a blank).
So, brow.sh benefits JS-disabling users. The Tor project receives donations and they dedicate them to enhance Tor. And, I think this is a neat enhancement. While I am not totally familiar with their money handling, I wouldn't be surprised if they are willing to funnel some of the funds to various side projects that improve the Tor experience.
In summary, the use case you advertise is minimal bandwidth for clients, but you missed the privacy use case, which someone may want to fund.
I think your specific wording does not ring the bells for the casual Tor user (hence "why not lynx"). Perhaps you should clarify the precise use case: No-JS users can see JS-dependent websites simply by URL prefixing. They get the modern web without activating JS locally.
> I'd be interested in any opinions here about how to financially support this project. At the very least I'd just love to somehow make this my job.
One of the cases where developers still use text based browsers is in attempting to check accessibility of websites. For many large enterprises, there's a legal or contractual requirement to ensure a certain level of accessibility for users with disabilities.
Yeah it's a bit weird with it's fuzzy graphics. I'm not quite sure the best way to use this, but one of the most common forms of "blindness" is not a total lack of sight, but very poor eyesight.
It can also work for general usability testing: after testing my personal site, I realized that with the graphical fuzzing that this offers, even I couldn't identify what my site's main navigation buttons (made in CSS, no less!) said or whether they were even links. This let me see graphic through the lens of people that cannot see graphics well, so yay!
It's close, but it doesn't actually output the escape codes. It would be cool if there were different formats besides text and html. Maybe vt100.brow.sh, ansi.brow.sh, etc. That way when I curl it from a terminal it renders like the actual terminal client.
Add terms that commercial use of the site requires an account setup and some pricing info. They can always host it themselves, but like many things, most people would rather pay someone else to do it.
Can it be made not to show WebGL, embedded video, and so forth? I enjoy a very serene internet using w3m set to monochrome, with mouse and images turned off. Every now and then it's necessary to use a graphical browser, and it's the sensory equivalent of being woken up by a toddler at 5:30 on Christmas morning.
What you call "serene," I would call "austere." That's not meant as a denigration, mind you: I'm very curious as to your viewpoint here. What do you enjoy about such an experience?
I like it for reading. I like how all the text is displayed the same, in a fixed width font I'm used to looking at. There's nothing to distract me or get in the way.
Without those, nobody buys anything and companies go bankrupt. Hopefully you make up for your behavior by consuming more than average and watching Fox News.
The major benefit is that I don't enjoy an "experience", I just read the content and leave. So much of the modern web is built specifically to prevent that.
I find the ordinary internet terribly overstimulating. It's a constant din of people screaming for my attention. Even relatively sober sites often have distracting designs that make it hard to focus on the content. Text mode is calming, it cuts straight through the clutter. In text mode, authors have to distinguish themselves by saying something interesting, and it's a lot easier to decide whether that's the case when they have nothing but words with which to make their arguments.
Is there a way to point it to a different Firefox installation path? It only looks at Program Files{x86} and then quits as it doesn't find the FF installation at Program Files.
> Browsh consists of a minimal Golang CLI client and a browser webextension. Most of the work is done by the webextension. When the CLI starts, it looks for a compatible browser (currently only Firefox) and starts it in headless mode. Once the browser has started it opens a remote debugging connection and installs the extension.
elinks and lynx are completely different browsers. Lynx is actively developed and can link against recent versions of OpenSSL, GnuTLS, and LibreSSL. The latest release came out yesterday: http://lynx.invisible-island.net/release/
Lynx can render HTML5 web pages. Not sure what WebGL has to do with a web browser that is meant for terminal emulators. CSS is not supported in text browsers in general (not all of them are full-screen terminal applications) because it makes no sense - for decent text rendering, the browser needs to ignore style sheets and render in a way appropriate for text output.
Yes you're right, it's unfair to lump elinks and lynx together. It's before Browsh I would use elinks all the time as it seemed the most modern.
Surely though just the sheer fact of CSS's `hidden` declaration has to be taken into account by any browser, text-based nor not? I've came to the conclusion that there just simply isn't a subset of modern web standards that you can adhere to and still use the web today. Our best option is just to piggy back off one of the major browsers that does all the hard work of keeping up with the rapidly changing technologies.
I like the idea behind your wrapper but I think you’re mistaken regarding hidden. Things are hidden from browsers for a lot of reasons but a text browser should probably safely ignore them all as content and presentation are different.
I mean for example one of the big problems Browsh has is not rendering text that web developers have hidden in the DOM to be displayed on some kind of mouse interaction. So Browsh has to ask JS to query the `display: hidden`, `z-index`, etc values to ensure that it doesn't render text that is not intended to be displayed until after an interaction.
Displaying hidden text is one of the places where text browsers can "fix" broken web pages. There are broken web pages that hide the text you want to read until you click on some JavaScript modal, or some "Click to continue reading" link/button that breaks scrolling. w3m displays hidden text and greatly improves readability on many web pages.
You can argue that using the CSS display property is a bad hack around the poor performance of adding/removing DOM elements, but the general problem of trying to automatically adapt a GUI application to terminal display does not change (having the GUI be emulated in HTML that is meant to be displayed in the GUI of the web browser just adds a layer of indirection, difficulty, and inefficiency). On the other hand, web pages that are really hypertext documents wrapped in useless and inaccessible JavaScript and CSS can have all of the event handlers and CSS properties ignored, and become legible again. This is something that text browsers do well, and why text browsers continue to be relevant today.
> w3m displays hidden text and greatly improves readability on many web pages
Are you saying that ignoring CSS rules for hidden content improves readability? I find this very hard to understand. For example something like "Thanks for submitting this form" appearing on page load is not an improvement to readability.
apart from ssl, there are slight differences: last time I checked, "lynx" will do a http/1.0 request. Ubuntu nginx-common config suggestions do not ship gzipped content below http/1.1 if enabled without change (gzip_http_version 1.0). "w3m" is the same. "links" does http/1.1 without explicit runtime argument. I need to measure if this makes a difference in the general web (or with a day of my browsing history). http/1.0 sure does receive compressed content with "accept-encoding: gzip", but I'm curious if the http-version will make a big difference. Firefox will give you http/2.0
I stumbled on this detail when I was capped to 2 kbytes/s and tried different cli browser. I applaud the authors effort. The docs should mention the bandwith benefits are gained primarily if you have a remote to ssh into where you can setup a firefox install.
I was wondering about this. If everything goes through an already installed browser, then what is the advantage that browsh has over a minimalistic fork of Firefox with e.g., lower bandwith features?
For this to actually give a bandwidth advantage I think the browser would have to be running on a server somewhere... so the browser has become the server.
I guess the point is to install this on some random server somewhere that you can then SSH to. It really does seem like a good way to have a speedy browser experience over even insanely slow network connections.
Does one then need a Firefox installation on the server, or can it be through a local Firefox installation?
Edit: according to comments below the headless browser (i.e., the Firefox installation) runs on the server. After this, the smaller bandwith data gets sent directly over SSH.
Terminal.app doesn't support true colour :( So it can't properly take advantage of the UTF-8 half block trick to get 2 pixels of colour from each cell - thus creating the banding effect you see.
I think the use-case is that the user is running their headless browser on a remote server with a good internet connection. Then they open an SSH tunnel to that server from their local machine which has low bandwidth--and the only thing that needs to be received by the local machine is the browsh rendering of the webpage.
If you weren’t the fine upstanding person you are, you’d have all the web traffic of users at your disposal: banking, secure interactions with healthcare providers, credentials to Hacker News, the whole nine yards.
With access to my email, you could probably reset a handful of my passwords to various services that don’t support dual factor auth, and you could probably discover what services I subscribe to.
I mean, I wouldn’t want you to have access to my email, but I would much rather that than a permanent man-in-the-middle web client.
It should be quite doable to spin up a container/VM on demand. I'd probably look at lxd/lxc or bsd jails for this (both with zfs for storage) - or if there now are any real ways to run containers under hw virtualization - maybe that.
I don't think I'd look too hard at lxd or freebsd as you already have a docker setup.
But hw isolation might be worth investigating - as others are saying - hostile access to a web browser, including webmail etc - is pretty dangerous. And plain docker never had a good story wrt secure isolation.
Apparently there was "hypernetes", now stackube - for combining VM runtime and kubernetes:
As for lxd/freebsd jails and zfs - both offer very nice and easy to grasp environment for isolated services - and both should end a little more isolated than a typicaldocker setup (some services running as root in container, no additional lxc restrictions).
But all things considered, if you already have k8/docker set up to give every user a separate, possibly ephemeral container... Infrastructure is probably not where I'd devote most time. It should work well enough as is.
Now that you describe it, yes indeed I can see the problems. I can't think of any other precedent, as most other proxies are least protected by HTTPS, wheres a Browsh service is literally reading every character on a page in plain text! So there's need to be a great deal of trust. I wonder if it's just too much to ask of people, especially where money is involved.
Curious: why headless firefox and not headless chrome? Also: 'its main purpose is...to reduce bandwidth.' How does this reduce bandwidth? Is the assumption that you would deploy browsh on a server and ssh/mosh against that, and the bandwidth savings are to the client? (But full bandwidth usage to the server)?
He said Chrome, though. Or is there no headless Chrome but only Chromium? Can't say I've ever heard of headless Chrome, though, so I guess that's why I'm being downvoted.
Curiously, Chrome doesn't support extensions in headless mode. And Browsh needs to run some custom CSS and JS to help with parsing, so Chrome was a no go. You can inject CSS and JS using the RPC though, but that RPC standard (https://remotedebug.org/) isn't yet supported by Firefox. So at the time I felt that using the webextension standard was a more cross-browser investment. Perhaps I should revisit that decision.
And yes you're right about the use of a remote server to make use of the bandwidth savings.
That sort of thing is actually causing spikes in load on the cluster. I think the quickness of the requests must skip the cache. So it's a bug really, I just need to detect recursive URLs and extract the deepest link.
I can see that Browsh's injected JS isn't able to hide your text to take the screenshot, which in turn means that Browsh can't parse the text. There could be a number of reasons for this, but it's very much Browsh's responsibility here, not yours. I've added your site as an issue on the repo: https://github.com/browsh-org/browsh/issues/75
So it's modern in that it won't run on anything but popular platforms, and text-based in that it shows you text using a huge, multi-gigabyte program. Looks like links / lynx aren't going to be replaced any time soon.
The modern age requires a multi-gigabyte program to use the Internet, that's just our reality. So the whole point of Browsh is that you can now offload this from your local machine (eg. to a remote VM) and re-experience the net as pure text, how it used to be.
I use several clients that do not support SNI and one workaround is to connect through a program that does support it, e.g. haproxy, socat, etc.
Privacy/censorship conscious users may dislike SNI, SSL/TLS implementors are now trying to "fix" it, and in fact most SSL/TLS-enabled websites do not require SNI to be sent (popular browsers send it anyway). If requested, I can post stats on whether SNI is required for any list of websites. I have already done this a couple of times with the list all sites currently posted on HN: only a minority require SNI.
You might want to consider adding support for displaying proper terminal graphics, instead of using pixelated ones (which I assume are done using unicode block drawing symbols?).
I'm actually already pretty decided against this. Simply because I won't the basic engine to be as text-focused as possible, as then we have a universal source of the modern web in pure text format, that multiple other clients (terminals being only one) can make use of. And besides graphics are orders of magnitude more bandwidth heavy than text, which somewhat defeats the purpose of Browsh.
Rendering images using unicode block symbols does not make them text, it just makes them pixelated. And you can always down sample images to reduce the bandwidth cost and still get a much better rendering than you achieve currently. For example you can convert the images to indexed 256 color compressed PNG and transmit that, which would reduce bandwidth by a factor of ~4x while still giving you much better rendering.
But anyway, it was just a suggestion, it's your project, you should feel free to do what you think is best for it.
Reminds me a bit of reading about (I don't recall actually using one) web accelerators, or proxies that would retrieve and strip down content requested by a low powered/bandwidth client. I think these have pretty much disappeared over the last 10 years as bandwidth increased and mobile devices became more powerful.
Added: I guess https://www.opera.com/turbo still provides this feature. Is there anything comparable that can be self-hosted?
Yes, sorry about that. I'm still getting used to Golang. So I need to consistently catch panics and I ensure the right ANSI escape sequence gets sent to the terminal before it finally exits.
Ha, that's a very long story. The gist though is that some custom CSS forces a given webpage into a strict, monospaced, mono-sized grid. Then JS queries DOM text nodes for their contents and precise positions. Then using the standard rules of text flow the exact position of every character can be fairly reliably derived.
There really just needs to be a terminal API for true pixel graphics. Something like sixel but modernized and cleaned up. There's no need to hack in graphics on top of Unicode block characters, which negates backwards compatibility anyway.
This is nearly what I'm talking about but it requires the terminal emulator to understand the image format. I guess that makes sense for transfer efficiency, and if you want a pixel-level API you could just transfer BMPs.
329 comments
[ 2.9 ms ] story [ 288 ms ] threadAlso, I'd be interested in any opinions here about how to financially support this project. At the very least I'd just love to somehow make this my job.
The harder thing is trust, trusting you to not change any images. On the other hand that'd make for a fun VPN service for general browsing (but again then it comes back to trust) but if you target it as a browser plugin bundled VPN service that saves on bandwidth you could hit more of an audience and use that income to fund Browsh. I figure you dont want to get too far outside of that scope though (development wise) since it's a lot more work.
I also like the other ideas of using some of the Browsh capabilities to scrape the web, maybe as a paid API, sure you can write a script to do that, but if you can curl / postman an API that does it for you and returns what you need, then why wouldn't you? Trick is can an API be done just perfectly enough? How do you tell it to give you whatever part of the screen with whatever requirements? It could sound simple but get complicated quickly enough. If it's too complex maybe a compression VPN is a better target.
I've got a good feeling about text.brow.sh, it's so simple. As you say I just need to work on the consistency of what it returns.
Thanks for your thoughts!
That would work nicely as a hosted service with an app. Like a more aggressive version of opera's bandwidth reduction.
The military might have similar problems.
It's not really ready fir consumers yet though.
I like the awards for lo-fi websites, I could have a showcase section of the website.
What I mean is an app that connects to an instance (vm, server) a brow.sh.
I would like to install this in a lxc container and connect to it from my mobile via VPN and use HTTP-proxy settings or and with an specific brow.sh-app for my mobile. Could this work?
Currently though html.brow.sh is non-interactive, as soon as I start letting people logging in it opens up a whole bunch of privacy concerns!
* Forms not working in Firefox but works fine in the terminal. Google search...
Some more testing:
- use nginx as a proxy for browsh - configure the Firefox instance (install uBlock Orgin, remove history and set other privacy related configurations and hope it works.
Keep up the good work!
What made you think about using Nginx as a proxy?
You can already install Firefox extensions: https://github.com/browsh-org/browsh/issues/45
There's already ways to drive a headless UI (Selenium, puppeteer), but they require automation testers to interact with page elements via the "normal" DOM which can be quite painful in terms of creating automation tests - DSLs to drive DOM automation are necessarily huge.
I could envision browser automation via browsh using a very small DSL e.g. browsh.url() to navigate to a URL, browsh.click() to click on an element, browsh.text_exists?() to check whether text exists, browsh.link_exists?() to check whether a clickable link exists
But yes, I'd love for Browsh to be used in such a way!
EDIT: Since Browsh depends on Firefox I'm curious whether it's actually faster than a graphical browser.
A very fast browser not tied to a graphical display with the bonus of being able to put error screenshots into all of our existing tooling could actually be quite beneficial here, whether the text is character-identical every time or not.
Given the rest of your sentence, I think you mean the opposite.
I think before you can answer that question you have to figure out if there are enough geeks like you who think this is a useful enough tool to run / use regularly. They'll likely pay you $. Alternately, find use case where even just a few users / companies would pay $$$ each for some thing that it does that you can't easily get anywhere else and helps get work done.
For me it looks cool, and i'll give it a try, but i'm more likely to stay with my normal browser because with plugins like vimium i don't have to reach for my mouse very much. That's actually one of the reasons i use the command line so much. Make it work without constantly using my mouse and I'd be far more likely to use it. Once i'm using it regularly, then I'd be starting to think "I wish it would..." and then making those wishes come into practice is something i'd support. This is kind of a weird / esoteric thing that would just gradually improve so... maybe Patreon is a better route for getting support than normal software / feature / plugin sales.
personally, IF I find myself using it regularly, i think i'd be more likely to support you via a Patreon type regular donation.
So at the moment I guess it's just best to hope as many different people and organisations see Browsh as possible and seeing what sticks and where.
I've got Patreon, Paypal and Bitcoin setup already: https://www.brow.sh/donate/
The maritime industry could find a large benefit from using your software. A friend of mine is a marine electrician who often works on large commercial boats up in Alaska. Some of these boats will pay in the neighborhood of $7000 per month for satellite service and unless you're the captain you get very limited access time. I'm sure they'd be interested in cutting their web browsing costs.
So you connect (using ssh or chrome) to a dedicated host which sends you a digest of the page...
You can also "zoom in" on elements if you want higher resolution.
As a potential user, what are your opinions ?
(Sort of like the Amazon Silk browser..?!)
Edit: phrasing, and also looked at other comments
Definitely. Maritime companies and backcountry users probably won't be interested in messing around with a nifty open source tool nor the hosting infrastructure it'd require. They'll want this built out as a secure easy to use service.
@tombh Feel free to get in touch if you're interested in building or optimizing for satellite users. I'd be happy to provide some beta test feedback using my equipment.
I'd considered using a headless browser setup but I never got 'round to it.
If you structured this as a paid service where I could make some number of requests for some amount of money and get back basically what you've got now in monochrome mode, I'd probably sign up pretty quick.
This is sort of relevant to some other recent HN discussions around the trouble with large-scale crawling and the trouble with so much of it being restricted to Google now, so I don't think I'm the only one.
So much for 'the semantic web' I guess.
A handy guide to financial support for open source.
https://github.com/nayafia/lemonade-stand
https://wiki.snowdrift.coop/market-research/other-crowdfundi...
Has further notes that don't just overlap with the lemonade-stand thing. Unfortunately, Snowdrift.coop is still not available to use itself at this point.
However I think there is very little real use for image to text-as-image conversion.
[0] https://en.m.wikipedia.org/wiki/AAlib
I didn't know about this utility. Awesome it does sound too. Thanks for pointing it out.
Another insidious idea is to pitch this browser to companies as a way to reduce wasted time on social media and time-wasting websites - employees could be trained on how to quickly extract text information from a website that is relevant to some business goals, without getting distracted by the time-wasting opportunities that a browser provides.
What I would love to see (but where far fewer promise of riches lay) is to use this as part of some great developer tool - this seems it could be a more intuitive way to test an interface than Selenium. Until then I look forward to personally supporting and using your awesome project!
Ha, the "reduce wasted time" idea is interesting one!
Thanks for your support :)
First though I really need to improve html.brow.sh
I'm getting tired of writing down GDPR violations to report.
But it's also interesting for the OP, maybe there's something patentable in there ?
Fraunhofer is a better example of an institute that made good money on tech patents with merit, as well as 3M and lots of other tech companies.
But as a rule these things are hard to monetize and while I'm not happy that it will be hard to monetize the work that went into this particular project at the same time I think it is good that patents are most likely a closed avenue.
Patents are - in my opinion - a net negative and the sooner they are abolished the better.
Many Tor users disable Javascript, because Javascript sometimes has anonymizing vulnerabilities. But disabling Javascript cripples many websites.
But I can type html.brow.sh/https://example.com via my Tor browser and get a non crippled website, even with Javascript disabled. This is a very practical use case and I think many will want to pay you to make it work better.
Previously I could not visit CNN.com via a JS-disabled Tor browser, but now I can type https://html.brow.sh/https://edition.cnn.com/ and Viola! (Well, it's still a little crippled but much better than a blank).
So, brow.sh benefits JS-disabling users. The Tor project receives donations and they dedicate them to enhance Tor. And, I think this is a neat enhancement. While I am not totally familiar with their money handling, I wouldn't be surprised if they are willing to funnel some of the funds to various side projects that improve the Tor experience.
In summary, the use case you advertise is minimal bandwidth for clients, but you missed the privacy use case, which someone may want to fund.
Edit: grammar fixes.
I think your specific wording does not ring the bells for the casual Tor user (hence "why not lynx"). Perhaps you should clarify the precise use case: No-JS users can see JS-dependent websites simply by URL prefixing. They get the modern web without activating JS locally.
One of the cases where developers still use text based browsers is in attempting to check accessibility of websites. For many large enterprises, there's a legal or contractual requirement to ensure a certain level of accessibility for users with disabilities.
You might not see the need for this, but others will. I could see it being used in some back end workflow.
`curl https://text.brow.sh/https://news.ycombinator.com/item?id=17... | grep snarfy`
Add terms that commercial use of the site requires an account setup and some pricing info. They can always host it themselves, but like many things, most people would rather pay someone else to do it.
I like it for reading. I like how all the text is displayed the same, in a fixed width font I'm used to looking at. There's nothing to distract me or get in the way.
Without those, nobody buys anything and companies go bankrupt. Hopefully you make up for your behavior by consuming more than average and watching Fox News.
The major benefit is that I don't enjoy an "experience", I just read the content and leave. So much of the modern web is built specifically to prevent that.
Is this relying on an external server to work?
You can use the free live demos though: `ssh brow.sh` and https://html.brow.sh
So a browser wrapper.
But even that is wrong. Development in fact continued for eight years until July 2017.
* http://repo.or.cz/elinks.git/shortlog/HEAD
Lynx can render HTML5 web pages. Not sure what WebGL has to do with a web browser that is meant for terminal emulators. CSS is not supported in text browsers in general (not all of them are full-screen terminal applications) because it makes no sense - for decent text rendering, the browser needs to ignore style sheets and render in a way appropriate for text output.
Surely though just the sheer fact of CSS's `hidden` declaration has to be taken into account by any browser, text-based nor not? I've came to the conclusion that there just simply isn't a subset of modern web standards that you can adhere to and still use the web today. Our best option is just to piggy back off one of the major browsers that does all the hard work of keeping up with the rapidly changing technologies.
You can argue that using the CSS display property is a bad hack around the poor performance of adding/removing DOM elements, but the general problem of trying to automatically adapt a GUI application to terminal display does not change (having the GUI be emulated in HTML that is meant to be displayed in the GUI of the web browser just adds a layer of indirection, difficulty, and inefficiency). On the other hand, web pages that are really hypertext documents wrapped in useless and inaccessible JavaScript and CSS can have all of the event handlers and CSS properties ignored, and become legible again. This is something that text browsers do well, and why text browsers continue to be relevant today.
Are you saying that ignoring CSS rules for hidden content improves readability? I find this very hard to understand. For example something like "Thanks for submitting this form" appearing on page load is not an improvement to readability.
https://bugzilla.mozilla.org/show_bug.cgi?id=1372998
https://ftp.mozilla.org/pub/firefox/releases/
And it worked on a VPS with nothing X related installed
[0] https://pastebin.com/eNvM4YSK
Edit: according to comments below the headless browser (i.e., the Firefox installation) runs on the server. After this, the smaller bandwith data gets sent directly over SSH.
What are you seeing when the server isn't working?
Is there a reason why Terminal.app might not work?
How does it reduce bandwidth exactly? It still has to grab all the html/css/js for the site being rendered.
As well SSH/Mosh access there's a HTTP service, that currently only outputs static, noninteractive HTML and basic graphics. For example: https://html.brow.sh/https://news.ycombinator.com/item?id=17...
So if OP goes down this road then I think he should allow users to access machines.
Do you think such a service would be significantly different to an email service in terms of privacy concerns?
If you weren’t the fine upstanding person you are, you’d have all the web traffic of users at your disposal: banking, secure interactions with healthcare providers, credentials to Hacker News, the whole nine yards.
With access to my email, you could probably reset a handful of my passwords to various services that don’t support dual factor auth, and you could probably discover what services I subscribe to.
I mean, I wouldn’t want you to have access to my email, but I would much rather that than a permanent man-in-the-middle web client.
Maybe something like:
https://github.com/rkt/rkt/blob/master/Documentation/running...
for VM backed containers - but I'm not sure if it's considered stable and/or secure.
But hw isolation might be worth investigating - as others are saying - hostile access to a web browser, including webmail etc - is pretty dangerous. And plain docker never had a good story wrt secure isolation.
Apparently there was "hypernetes", now stackube - for combining VM runtime and kubernetes:
https://kubernetes.io/blog/2016/05/hypernetes-security-and-m...
https://github.com/openstack/stackube
As far as I can tell, this allows the mix of k8 style pod/container management and VM level isolation:
https://stackube.readthedocs.io/en/latest/stackube_scope_cla...
As for lxd/freebsd jails and zfs - both offer very nice and easy to grasp environment for isolated services - and both should end a little more isolated than a typicaldocker setup (some services running as root in container, no additional lxc restrictions).
But all things considered, if you already have k8/docker set up to give every user a separate, possibly ephemeral container... Infrastructure is probably not where I'd devote most time. It should work well enough as is.
Curious: why headless firefox and not headless chrome? Also: 'its main purpose is...to reduce bandwidth.' How does this reduce bandwidth? Is the assumption that you would deploy browsh on a server and ssh/mosh against that, and the bandwidth savings are to the client? (But full bandwidth usage to the server)?
And yes you're right about the use of a remote server to make use of the bandwidth savings.
https://html.brow.sh/https://html.brow.sh/https://html.brow....
Damn, I can't believe I missed that! I fixed it in the code and will deploy soon with some other fixes.
https://html.brow.sh/https://milankragujevic.com/vip-drop-i-...
So it's modern in that it won't run on anything but popular platforms, and text-based in that it shows you text using a huge, multi-gigabyte program. Looks like links / lynx aren't going to be replaced any time soon.
The modern age requires a multi-gigabyte program to use the Internet, that's just our reality. So the whole point of Browsh is that you can now offload this from your local machine (eg. to a remote VM) and re-experience the net as pure text, how it used to be.
In fact, just checked on Ubuntu 16.04 -- both links (2.12) and lynx (2.8.9...) work fine with https://google.com and https://news.ycombinator.com for example.
https://bugzilla.redhat.com/show_bug.cgi?id=1527856
So, according to your own link, yes it should support SNI.
I use several clients that do not support SNI and one workaround is to connect through a program that does support it, e.g. haproxy, socat, etc.
Privacy/censorship conscious users may dislike SNI, SSL/TLS implementors are now trying to "fix" it, and in fact most SSL/TLS-enabled websites do not require SNI to be sent (popular browsers send it anyway). If requested, I can post stats on whether SNI is required for any list of websites. I have already done this a couple of times with the list all sites currently posted on HN: only a minority require SNI.
Please don't do snarky tropes here.
See https://sw.kovidgoyal.net/kitty/graphics-protocol.html
But anyway, it was just a suggestion, it's your project, you should feel free to do what you think is best for it.
Added: I guess https://www.opera.com/turbo still provides this feature. Is there anything comparable that can be self-hosted?
Heads up: on my Mac, browsh isn't properly resetting the tty when it crashes.
Comes up with BrowserModifier Win32/Unwaders.B!ml detected.
Do you still get the warning with the latest v1.2.3? https://www.brow.sh/downloads/
Trojan:Win32/Fuery.B!cl | Severe
https://github.com/daleroberts/bv
https://github.com/daleroberts/itermplot