Not really as customizable though, ublock has useful features like build your own filter and pick/create to block annoying webpage widgets. I do wonder if the python based aspects of qutebrowser might give a large surface to fork it and make this type of "hacker's browser" out of it though.
This probably wouldn't be possible due to browser sandboxing.
You could however, use a tool like External Application Button [0] in order to send the URL of the page to a script/program, which in turn can make the body of the page parsable by downloading the URL it gets, automating some action on it. I use it pretty often to automate downloads with yt-dlp or opening a picture directly in GIMP from the browser, but since you're invoking a bash/python/whatever script, the possibilities are basically endless.
> You could however, use a tool like External Application Button [0] in order to send the URL of the page to a script/program, which in turn can make the body of the page parsable by downloading the URL it gets, automating some action on it.
This works less well than it used to now that CMSs and blogs are now SPAs that render no text without javascript being enabled.
Yes, that is a shame, but that can be worked around by sending it to something like PhantomJS, headless Chromium, or any other scriptable browser that will actually run the JS on the page and render it in the background. Very heavy as you're basically running a full browser in the background, but totally doable.
You could probably add this to Nyxt in a solid hour or two of configuration hacking.
The "configuration" is just more code that gets compiled in on loading. It's a complete first class citizen unlike extensions for most software which is sandboxed in a separate language and can only see certain APIs.
In Nyxt, StumpWM, Emacs, and other Lisp software you can literally put bugfixes in your config, redefine existing functions, and generally do whatever the hell you want.
That is good! I have been creating some add ons for firefox and what i dislike the most is that i can't just easily modify the UI of the browser itself.
I hate that i can't change stuff even if i am only creating add-ons for myself...
And then call the function like C-space buf-text-to-pipe-cmd and specify the commands to pipe to. Output will appear in a new buffer that can be further used as the source for piping.
DOM handling would be a tad more laborous, but you can pick DOM elements from output of nyxt:document-model with CSS selectors using clss:select.
Keyboard-driven UI on Android? Well, for the brave and true hackers (if you don't connect a keyboard to your android device which you can ... but why having an android device then?)
Can you clarify what you mean? It can certainly be run on hardware that you call a server, just would depend on if you're running a graphical environment, it's not able to run in a terminal emulator afaik.
If you want a remote browser capable of running on a server (with the client, browser UI in a regular web page) check out BrowserBox: https://github.com/dosyago/BrowserBoxPro
I expect you can run it in headless mode (new --headless switch), start a Swank server (the backend of Slime, the backend of the most used CL "language server"), and connect to the Lisp process from home, through a SSH tunnel. And control the browser from the Lisp REPL, or a Lisp script, like we would with any running Common Lisp program.
Hmm, not sure it's for me (a bit too much hacker) but I'd definitely like a highly customizable, module based browser with a pure shell by default. Something like Visual Studio Code but with less cruft from the get go.
I've never thought about it until I saw this! Now I'm really irked about there not being a solid, mature browser based on extensibility. Where the base is basically an address bar and history awareness (i.e. back/forward) and even bookmarks is a module, so that you don't even need an integrated solution but can instead rely on e.g. Raindrop.io as your bookmarks manager. Tab management another one, so for example you could have _only_ a powerful vertical tabs mode if that's all you need.
You know what, it feels like this should have happened 20 years ago and it would be as popular as Vim or Emacs now, but somehow never actually did. It would be the ultimate response to Chrome, Edge, even Firefox feature creep, forcing its developer to commit fully to web renderer excellence (speed, RAM, web standards) and maybe some optional official modules if you want them.
Contemplating my navel, I propose that it didn’t happen because the purpose of an editor and browser are so different. (Realistically the driving factor is that an editor is much more useful for making money.)
I commented here [0] and am extremely interested in discussing BrowserBox with you. This is something I've brought up over and over, I'm an experienced web dev and generalist but simply couldn't tackle on my own.
Curious about your roadmap with dosyago and BrowserBox, where our goals converge and diverge and how we can help each other. Feel free to reach out via email, it's in my bio.
While it may not be as technically elegant as using lisp, it greatly increases the count of folks who can contribute and simplifies the code with a single language for both front and back.
Secondly, while, in general terms, JavaScript is not as performant as Java or C++, for asynchronous real time operations, such as mostly used by this application, the event loop in node, a layer over io_uring, is a highly highly efficient method to perform these.
In other words, we get to trade simplicity for raw performance to ease maintainability, and less complex code means less security risk and bugs. In fact, the bottleneck in this application for performance is not JavaScript but network link latency, and the browser and the hardware that runs on.
I use to build my own like this with the VB web browser control. I had tons of automation built in for a lot of tasks but it becomes too much to keep up with. I've looked into headless chrome but it's not as cool as working in the Visual Studio and being able to pause and just develop on the fly.
Honestly it was incredibly cool but websites just get more and more complicated. From like 2004 to maybe 2014-2016 it was great. Somewhere after that, I'm not sure, something changed, maybe me, but it just became too much.
The web standard really has exploded since that time range you mentioned. Nowdays there's a native browser API for text-to-speech. It's easier to generate speech for an article you wrote than it is to style an option in a native <select> element
There are always going to be limitations to extension-based approaches but Firefox with tridactyl is most of the way there capability-wise IMO (https://github.com/tridactyl/tridactyl). I use that plus tree-style tabs and find it excellent. There's always some extensibility limitations that break the integration though, like extensions getting disabled on mozilla domains.
I love projects like nyxt and respect their priorities, but without big-player extension support it's usually a no-go for me. Still, I'll be interested to see the ideas they develop trickle out into the rest of the power-browser ecosystem. I especially like that lossless tree history – history management is a very under-explored UX area IMO
Oh thank you so much! Some of these exclusions make some sense to me but having my ctrl+n mashing interrupted by support.mozilla.com was fairly frustrating. Also security wise if I install a malicious extension with permissions to abuse these domains I'm already a goner haha
> I've never thought about it until I saw this! Now I'm really irked about there not being a solid, mature browser based on extensibility. Where the base is basically an address bar and history awareness (i.e. back/forward) and even bookmarks is a module
If you guys are really into the idea of an extendable virtualized browser API, an extendable browser that you can hack on, and customize programmatically fully (including all the browser UI, the so-called “chrome”), then my BrowserBox project might be for you.^0^1^2
That open extendable browser is actually my vision of BrowserBox and I think it’s a really powerful thing. For example: I imagine being able to create an "extension store" using the more powerful set of APIs enabled by this (compared to the more limited APIs available to regular browser extensions). Similar to the idea you related where bookmarks is a module--which is something we've wanted to do for a long time--but not sure about the API contours!
The other ideas you talk about are all great, too: a UI that can be fully customized (vertical tabs!) or even tree tabs / history. Or all kinds of crazy things. Browser UIs are pretty uninnovative mostly, but a fully customizable UI could really open that up.
Modules ideas (that in some sense we've been exploring at Dosyago over the last few years, through DiskerNet, client projects, GraderJS and so on):
- web scraping / automation script builder
- vim shell
- vertical tabs UI
- bookmarks
- public / shared bookmarks
- full text searchable bookmarks (DiskerNet)
- NodeJS shell (GraderJS)
And I think BrowserBox may have the right model for that extensibility (maybe not totally right but the idea of: instead of using a WebView tag you’re using the much more extensible and powerful set of APIs for browser instrumentation and automation in other words, the "remote debugging protocol" APIs) to create the surface of extensibility. And our company has already built the the application with the browser UI at the front, and the instrumented browser engine at back to enable that building out an API. Our system is already a fully functional browser: with multiple tabs, back/forward history, but no other modules!
The main modules we have built so far have been: web scraping / automation script building for clients. We've experimented with other modules, but the majority of the work we need to do is around carving out the contours of that API. What surface do we provide in order to make modules really great? That's the key question and one we are looking for help answering.
Having that API for the browser to allow like a modules type thing is something that’s been on our plans for a long amount of time we always thought it was a good idea, but we’ve never been sure if there was much demand for it. Like you say here this Nyxt project really highlights the fact that this is something that really is a good idea or at least it seems like it and definitely people seem like they want it. There's still a lot of problems with this approach, but I believe they all have achievable solutions!
While our project may not be as technically elegant as using lisp, but I think it has the prosaic advantage of: having a simple client/server architecture (browser UI at the front, instrumented engine at the back); using a highly extensible set of powerful API‘s that are on essentially standard track with the WebDriver specifications (making, at least in theory, the future ability to swap out the browser engine between Chrome, Firefox, Safari, or whatever else that supports WebDriver); and I think most importantly--it’s written in JavaScript, Node.JS, HTML, CSS--greatly widening the field of potential contributors
I’m encouraged today by this project and the thread you have started to push our BrowserBox project more in this direction and I would like to encourage people to come on and check it out and because of that we are opening up our projects today to ...
uzbl (https://www.uzbl.org/) used to do that but it seems it was too heavy in practice.
Today I'm more interestea in turning the web into a more textual format to integrate it in acme (http://acme.cat-v.org/) which is already built around modularity. Making the web a content provider and letting me interact withit the way I want
We basically had this with old Firefox, didn't work out. We kinda have and had this with all the small custom browsers, like Qutebrowser or Nyxt here, doesn't work either. Customizability is a liability, it's work. It comes with many problems, responsibility, work for the users, and not many are willing to take this. Chrome, Firefox and the others are more or less the best we can get regarding balancing customizing, stability, security and user-friendliness.
I mean even in Editor/IDE-Space the popular tools are not the ones who give you the most customizability, but those who give you enough customizing, while not bothering you with the rest, and instead focusing on stable powerful features. Maybe if Coding AI becomes stronger, we can reach this utopia of perfect customizability without all the problems.
It's not what I expected from their webpage, I think their Github explains it better -
Nyxt [nýkst] is a keyboard-driven web browser designed for power users. Inspired by Emacs and Vim, it has familiar keybindings (Emacs, vi, CUA), and is infinitely extensible in Lisp.
It's not for the OSINT community, is how I'd read it.
"Hacker" as a term has and seems to continue to hold a dual meaning--the penetration testing, blackhat / whitehat, OSINT meaning, OR the creative software engineer, clever maker, resource-constrained artful solver-of-problems.
That Nyxt is targeted at "hackers" therefore requires some disambiguation. It is not primarily for the former camp, and is largely for the latter, it appears.
I've always thought of developing a Hacker's browser. One where there are no restrictions of what it can do. For example, you can't write a chrome extension to spoof the Referrer header on requests. The Chrome browser is more locked down than you think.
There is also Chromium which you can build and patch if you really need low level changes to the browser not achievable from JS, but nothing immediately comes to mind to necessitate that.
It doesn't look easy to change and even then, there is some doubt whether it works with newer Chrome versions. Anyways, the Chrome developers are purposely making hard if not impossible to change the Referrer and other headers. Part of the blame is to comply with standards but there should be some flag to turn off that behaviour.
Did a quick run with it and turned on vi mode from the settings menu.
The first thing I found awkward about it is that prompt fields are always initiated in normal mode. So if you want to execute a command with : or interact with an element using f, you have to manually activate insert mode first before being able to enter text into the prompt.
I'm sure there's a way to change this since the entire thing is in Lisp, but it sure is an odd default.
if it's a fresh field, I think I might turn out to prefer them always initialised the same way for the sake of my muscle memory.
Now, that may not be true of -me- in practice, but it's definitely possible for whoever did implement that part to genuinely prefer it that way.
Of course, if the implementer wasn't a vi user, who fucking knows, vi is kinda baroque as fuck if you run face first into it without preparation so "the person who picked that default was trying to do vi users a solid but had to guess at some bits" is totally viable too.
Does it do regex search? I so often need to do a foo|bar in a page.
Back in the days, before Firefox switched to the Google Chrome model of extensions, there used to be a really nice advanced search addon. Nicely integrated into the UI.
WebKit and Webengine. And the versions of these renderers that this and similar projects like qutebrowser use are just too full of bugs to be a main browser. That’s why I don’t use them, although I would really love to.
I’ve always assumed the major browsers had all sorts of security stuff going on behind the scenes, and using a small alternative like this (or suckless surf) was just asking for trouble. But, I don’t have any real knowledge about security stuff.
How misinformed is my take? Are these kinds of browsers OK? I’d like something light, if possible.
Lightweight web browsers don't have as large of an attack surface, particularly if they eschew JS, cookies, etc or you disable them. Nyxt is alpha software, so YMMV until it's more feature complete, and independently audited to correctly implement WebKit and WebExtensions. Run Tor Browser without JS in a VM for security+privacy.
For those wondering whether this is a new browser engine, the FAQ says:
"Nyxt is web engine agnostic. We utilize a minimal API to interface to any web engine. This makes us flexible and resilient to changes in the web landscape. Currently, we support WebKit and WebEngine (experimental (Blink))."
tl;dr; no. It's just another browser UI, albeit a very different one.
Cool stuff. We all need to escape the oligopoly of Chrome and Safari. Consider making a build for Android and consider inserting a JS polyfill layer that could stub any weird API with an extension, without rebuilding the browser (side effect: it's an ultimate adblock solution).
I personally use Orion Browser (macOS) and Homerow (macOS) and it gets me in a similar situation, without the buffer stuff, and with a more traditional setup. I can recommend that if you're looking for an alternative.
There was a programming language taxonomy thread posted a few weeks ago and I seriously think if you want to ‘git gud’ you should dabble in all of the families.
Since we are on the topic of hackers browsers, does anyone know if Firefox can be made to split pane (both for seeing the same page side by side and also doing it for two different pages)? Having done that with file browsers and IDEs, I wonder how come it wasn't a main feature? Nobody needed to compare pages side by side?
Thanks - this is useful indeed but it is sort of a hobbled view - I don't have access to the extensions I have installed in the side view. E.g. the main view shows the page in dark mode via dark reader but the side view doesn't have access to the extension so it has white background which is very jarring (better than nothing though). I wish they allowed a full-on split - almost like having two browser windows (with a common menu/bookmarks bar that acts upon the "active" pane).
260 comments
[ 3.1 ms ] story [ 251 ms ] threadDoes this support profiles by any chance? I currently use chrome profiles to separate work and personal
Looking forward to giving it a spin
|grep ...|awk ...
And the text of the document gets processed. But if you want to mess with the dom you could:
Html.Body | sed ... | jq ...
You could however, use a tool like External Application Button [0] in order to send the URL of the page to a script/program, which in turn can make the body of the page parsable by downloading the URL it gets, automating some action on it. I use it pretty often to automate downloads with yt-dlp or opening a picture directly in GIMP from the browser, but since you're invoking a bash/python/whatever script, the possibilities are basically endless.
[0]: https://github.com/andy-portmen/external-application-button
This works less well than it used to now that CMSs and blogs are now SPAs that render no text without javascript being enabled.
I’m curious about this comment. Surely this should be possible using accessibility hooks for a normal browser or in Nyxt via scripting, right?
The "configuration" is just more code that gets compiled in on loading. It's a complete first class citizen unlike extensions for most software which is sandboxed in a separate language and can only see certain APIs.
In Nyxt, StumpWM, Emacs, and other Lisp software you can literally put bugfixes in your config, redefine existing functions, and generally do whatever the hell you want.
DOM handling would be a tad more laborous, but you can pick DOM elements from output of nyxt:document-model with CSS selectors using clss:select.
https://lispcookbook.github.io/cl-cookbook/debugging.html#re...
I've never thought about it until I saw this! Now I'm really irked about there not being a solid, mature browser based on extensibility. Where the base is basically an address bar and history awareness (i.e. back/forward) and even bookmarks is a module, so that you don't even need an integrated solution but can instead rely on e.g. Raindrop.io as your bookmarks manager. Tab management another one, so for example you could have _only_ a powerful vertical tabs mode if that's all you need.
You know what, it feels like this should have happened 20 years ago and it would be as popular as Vim or Emacs now, but somehow never actually did. It would be the ultimate response to Chrome, Edge, even Firefox feature creep, forcing its developer to commit fully to web renderer excellence (speed, RAM, web standards) and maybe some optional official modules if you want them.
Curious about your roadmap with dosyago and BrowserBox, where our goals converge and diverge and how we can help each other. Feel free to reach out via email, it's in my bio.
[0] https://news.ycombinator.com/item?id=36008340
Here's some other projects you might have inspo or crossover with and some discussions you might find possible collaborators in:
Bonsai https://news.ycombinator.com/item?id=28446147
Mullvad Browser (security oriented FireFox-based) https://mullvad.net/en/browser
Browsh (purely text-based browser) https://news.ycombinator.com/item?id=17487552
Ladybird (browser from scratch for SerenityOS)
Kosmonaut (browser from scratch in Rust) https://news.ycombinator.com/item?id=24170201
Refresh (concepts for a new kind of web browser) https://news.ycombinator.com/item?id=17638477
Really though? Why
While it may not be as technically elegant as using lisp, it greatly increases the count of folks who can contribute and simplifies the code with a single language for both front and back.
Secondly, while, in general terms, JavaScript is not as performant as Java or C++, for asynchronous real time operations, such as mostly used by this application, the event loop in node, a layer over io_uring, is a highly highly efficient method to perform these.
In other words, we get to trade simplicity for raw performance to ease maintainability, and less complex code means less security risk and bugs. In fact, the bottleneck in this application for performance is not JavaScript but network link latency, and the browser and the hardware that runs on.
Honestly it was incredibly cool but websites just get more and more complicated. From like 2004 to maybe 2014-2016 it was great. Somewhere after that, I'm not sure, something changed, maybe me, but it just became too much.
Some recent HN comments of mine:
https://news.ycombinator.com/item?id=34855750
https://news.ycombinator.com/item?id=33541173
I am 100% down for building this, open source, if I had a team to help me see it through as it's just too much for one person.
Do you have any dev experience / interest in helping create such a thing?
I love projects like nyxt and respect their priorities, but without big-player extension support it's usually a no-go for me. Still, I'll be interested to see the ideas they develop trickle out into the rest of the power-browser ecosystem. I especially like that lossless tree history – history management is a very under-explored UX area IMO
Mozilla believes that it creates a security risk and that you shouldn't do it.
[1]: https://www.ghacks.net/2017/10/27/how-to-enable-firefox-webe...
If you guys are really into the idea of an extendable virtualized browser API, an extendable browser that you can hack on, and customize programmatically fully (including all the browser UI, the so-called “chrome”), then my BrowserBox project might be for you.^0^1^2
That open extendable browser is actually my vision of BrowserBox and I think it’s a really powerful thing. For example: I imagine being able to create an "extension store" using the more powerful set of APIs enabled by this (compared to the more limited APIs available to regular browser extensions). Similar to the idea you related where bookmarks is a module--which is something we've wanted to do for a long time--but not sure about the API contours!
The other ideas you talk about are all great, too: a UI that can be fully customized (vertical tabs!) or even tree tabs / history. Or all kinds of crazy things. Browser UIs are pretty uninnovative mostly, but a fully customizable UI could really open that up.
Modules ideas (that in some sense we've been exploring at Dosyago over the last few years, through DiskerNet, client projects, GraderJS and so on):
- web scraping / automation script builder
- vim shell
- vertical tabs UI
- bookmarks
- public / shared bookmarks
- full text searchable bookmarks (DiskerNet)
- NodeJS shell (GraderJS)
And I think BrowserBox may have the right model for that extensibility (maybe not totally right but the idea of: instead of using a WebView tag you’re using the much more extensible and powerful set of APIs for browser instrumentation and automation in other words, the "remote debugging protocol" APIs) to create the surface of extensibility. And our company has already built the the application with the browser UI at the front, and the instrumented browser engine at back to enable that building out an API. Our system is already a fully functional browser: with multiple tabs, back/forward history, but no other modules!
The main modules we have built so far have been: web scraping / automation script building for clients. We've experimented with other modules, but the majority of the work we need to do is around carving out the contours of that API. What surface do we provide in order to make modules really great? That's the key question and one we are looking for help answering.
Having that API for the browser to allow like a modules type thing is something that’s been on our plans for a long amount of time we always thought it was a good idea, but we’ve never been sure if there was much demand for it. Like you say here this Nyxt project really highlights the fact that this is something that really is a good idea or at least it seems like it and definitely people seem like they want it. There's still a lot of problems with this approach, but I believe they all have achievable solutions!
While our project may not be as technically elegant as using lisp, but I think it has the prosaic advantage of: having a simple client/server architecture (browser UI at the front, instrumented engine at the back); using a highly extensible set of powerful API‘s that are on essentially standard track with the WebDriver specifications (making, at least in theory, the future ability to swap out the browser engine between Chrome, Firefox, Safari, or whatever else that supports WebDriver); and I think most importantly--it’s written in JavaScript, Node.JS, HTML, CSS--greatly widening the field of potential contributors
I’m encouraged today by this project and the thread you have started to push our BrowserBox project more in this direction and I would like to encourage people to come on and check it out and because of that we are opening up our projects today to ...
Today I'm more interestea in turning the web into a more textual format to integrate it in acme (http://acme.cat-v.org/) which is already built around modularity. Making the web a content provider and letting me interact withit the way I want
I mean even in Editor/IDE-Space the popular tools are not the ones who give you the most customizability, but those who give you enough customizing, while not bothering you with the rest, and instead focusing on stable powerful features. Maybe if Coding AI becomes stronger, we can reach this utopia of perfect customizability without all the problems.
Nyxt [nýkst] is a keyboard-driven web browser designed for power users. Inspired by Emacs and Vim, it has familiar keybindings (Emacs, vi, CUA), and is infinitely extensible in Lisp.
It's not for the OSINT community, is how I'd read it.
That Nyxt is targeted at "hackers" therefore requires some disambiguation. It is not primarily for the former camp, and is largely for the latter, it appears.
There is also Chromium which you can build and patch if you really need low level changes to the browser not achievable from JS, but nothing immediately comes to mind to necessitate that.
Genius! I’ll be trying it out
The first thing I found awkward about it is that prompt fields are always initiated in normal mode. So if you want to execute a command with : or interact with an element using f, you have to manually activate insert mode first before being able to enter text into the prompt.
I'm sure there's a way to change this since the entire thing is in Lisp, but it sure is an odd default.
Now, that may not be true of -me- in practice, but it's definitely possible for whoever did implement that part to genuinely prefer it that way.
Of course, if the implementer wasn't a vi user, who fucking knows, vi is kinda baroque as fuck if you run face first into it without preparation so "the person who picked that default was trying to do vi users a solid but had to guess at some bits" is totally viable too.
FTFY.
Back in the days, before Firefox switched to the Google Chrome model of extensions, there used to be a really nice advanced search addon. Nicely integrated into the UI.
So, '/' then '\r' and your regex then 'n' or 'shift+n' for next/previous.
How misinformed is my take? Are these kinds of browsers OK? I’d like something light, if possible.
"Nyxt is web engine agnostic. We utilize a minimal API to interface to any web engine. This makes us flexible and resilient to changes in the web landscape. Currently, we support WebKit and WebEngine (experimental (Blink))."
tl;dr; no. It's just another browser UI, albeit a very different one.
I remember checking it out, being disappointed about its not-great availability for MacOS, and thinking that I should learn lisp.
There was a programming language taxonomy thread posted a few weeks ago and I seriously think if you want to ‘git gud’ you should dabble in all of the families.
https://news.ycombinator.com/item?id=35813496
When you see how to implement a REPL in Lisp, it will blow your mind.
Yes. Nyxt just had a major 3.0.0 release, and we had a thread about that: https://news.ycombinator.com/item?id=35869378
[1] https://blog.mozilla.org/en/uncategorized/its-a-new-firefox-...