I wanted to share my side-project: gPTY. Started off as an idea to combine Godot and Rust in a project (two stacks I wanted to use more to learn more). The base inspiration was tmux - simply allow spawning multiple PTYs and then let the user grid/tile them how they see fit.
But since we have the Godot game engine at our disposal, we can do some more interesting things, like add an FPS counter, and then subsequently also let people set their preferred FPS (the idea being the potential lower power draw if someone's running it on a laptop on battery power vs someone running it on a desktop with high/native FPS). In its current state, with me using Oh-my-Pi a lot, it's evolving into a terminal workspace that can be used for orchestrating autonomous AI agents by way of dogfooding (or you know, just run herdr inside of gPTY - it's the better orchestrator and just good software - I found it after starting this project, and now I'm finding myself using it a lot).
Also, we're not limited to just terminals. Since we have Godot, we have basically a 2D (and potentially a 3D) canvas to play with. We can already full-screen the app for "zen" mode, no taskbar, no distractions. TUI die-hards can have their media or other apps entirely in terminal panes.
There has been some ground-work on getting Markdowns displayed properly done and I want to work on some kind of Wiki framework for local knowledge-management next, then create more types of panes (think native audio/video on a media pane, that sits alongside your terminal pane), and some simple 2D games (like snake) to prototype. More details are on the ROADMAP.
What's not easy (and probably won't happen) is a browser. Having done a couple of (small) projects using Electron already, the temptation to ditch Godot/Rust (learning curve) did come up (and also the ecosystem, the ease with which I could pull components and use web technologies - development velocity would definitely be higher there). But on the flipside, given all of the available LLM and AI support that we are privileged to have today, I figured the velocity should be comparable depending on how much I leaned on those. And lean I did.
Godot/Rust seemed the better call to me and my intent anyway - going with the 'it's not just the end but the journey that matters' philosophy. So yes, there has been heavy use of LLMs & AI to generate a lot of the code. But I do review and steer actively, not relying solely on vibes, and there's a few bits here & there that have been human authored.
There are definitely a lot of polish and QoL items that need to land to make the end user experience better, but in the meantime, let me know your thoughts and/or concerns!
Yeah, the README doesn't have screenshots; but the docs site does - https://godot-pty.github.io/gpty/ - 1 for now, but each release's blog entry gets the main app page + one for any salient features.
I'll add that in; maybe defaulted to folded on the README, a gallery of app screens or snippets. I need to spend some time on the UX, but yeah, screenshots is something I should've also added to the README.
This is such a cursed combination... I really don't understand what you get from Godot that benefits you for building a TTY "multiplexer."
Wildly vibe-coded to the extent that it's more misguided than Doom on a pregnancy test...
Also looking at OP's notes here, there doesn't even seem to be a need for this to exist, "FPS counter" ..? Perhaps, I don't know, fake terminals on a spaceship or something. Yueash...
You get continous rendering in a loop instead of updates only when something changes. It's like our laptop batteries are too big and we need to find something to deplete them.
lmao how is this on the front page at the 2 spot? People upvoting based on seeing buzzwords I assume? It's cool that small projects can get recognition here but this is not useful software
jesus christ i don’t get this attitude at all. you do realize that software is like 100x easier to make than it used to be right? the world is severely idea-constrained in a way that it hasn’t been since, what, 1992 with the web? i love robust, “hard problem” engineering more than your average bear, but we are so much further down the alan kay “the right pov is worth 60iq points” road than we have ever been in my lifetime, and this is a very non-obvious, creative mashup of things that i directly found inspiring. not because i think the project is perfect or genius or extremely well-executed or anything like that— because it is a creative mix of things, and we live in an age where we can be exponentially more creative with what kinds of atoms we can smash together than ever before.
I'll be honest with you: I'm not going to use what you have made. I don't mean that offensively, it's just not what I want to use for my workflow. Keep making tools you enjoy using though. That's the spirit of the computer.
That said, for your browser problem, what if you embed the browser inside of godot? Chromium embedded framework does this. I was able to get this to work for me to see Chrome inside of my godot exported projects. https://github.com/dsh0416/godot-cef I believe this is only for desktop exports though, no html5/mobile builds.
Thanks for your honest response. It's okay. At its core, it's a fun hobby/side project, not my day job. And I like it. But I'm realizing maybe this wasn't the platform to post on.
The browser is probably a non-issue, more a passing thought now. I definitely do not want to bundle or bring in Chromium. If it was a deal-breaker, I would've just gone the Electron route. Appreciate you sharing your WebView extension though, looks neat!
No, this is the platform to post on. You hacked something neat together and posted about it. I really liked how you made it agent first and gave it an MCP server so you can give any agent a tmux like multiplexer to use as subagents.
So many times I see projects here I don’t really need, but I always learn something from their readme and code. A lot of time I learn something I can apply to my project. In this case, I use Node-PTY for my terminal control plane but when I run more than 50 agents, it crashes. Maybe using your pty will let me run more agents.
Thank you for your kind words, that's generally how I see other's projects too.
I haven't performance or load tested, it's on the backlog; so I'd be curious on whether this works for your use-case. And if not, feel free to log an issue!
The concepts/reg-ex redirect is how I was imagining the autonomous agent automation. It does need to be fleshed out a bit more, but essentially, it was an easy way to let a process to know what's happening in another process so it could react appropriately. The e2e tests are a good use-case, but not something I've looked into yet (in theory, the framework should hold up for it)
Original intent was just a hobby/side project. As it grew, I thought maybe posting about it would be alright in case people find the idea interesting.
While I appreciate the creativity around using a game engine for non gaming applications, this does not seem like a particularly good fit.
If most of your problem is 2d window/text stuff, the web browser is probably the best foundation to work with. UI and hud functionality in COTS game engines is light years behind what chromium and friends can do today.
Agree on Chrome being top-dog in this space. Electron would've been the easy (pragmatic?) choice. But the design decision was to use Godot and not create yet-another-Chrome-Electron app. It's certainly not pretty to look at in its current state.
This is something mainly working on it on my free time so figured why not see how far I could push things.
yeah, personally i think branching out and using something like a game engine for something like this is a fairly inspired choice. inspired meaning that it literally inspired me to immediately think “oh yeah, there really is a wider universe of UI stuff that can push good fps and not be electron”. so just for that alone i appreciate this project’s existence. :)
i will definitely be curious to see how you find that decision to be working out for you as the project evolves— will definitely be keeping my eye on it! actual refreshing, interesting take!
Godot is a pretty monolithic engine and I don't know if you can really compile it down to a reasonable size for non game use cases. The scripting language is pretty slow and I remember coding in C++ was painful even after 4.0. I think you're better off with Raylib, Bevy or plain SDL.
Thanks, wasn't familiar with these, I'll look into them though the project direction is unlikely to change to swap Godot out. As of now, the binaries are in the 60-80 MB range; so not tiny - but not something I'd consider massive for what it is.
Usage has been relatively snappy (could be better, I suppose, I need more time). Definitely not sluggish.
A concern is that RAM usage is on the higher side (this is something I'll look at on the next release).
This is just straight up false. You can compile subsets of the engine even in Godot 3. Godot 4 has made this a first class citizen.
C++ coding is just fine. It's not painful. It can be a bit of boilerplate to expose c++ member fields and functions to the rest of the engine but you get some great power out of that. Just use compile your own module right into the engine. Don't use gdnative.
As someone who's codes a lot of c++ in Godot I really am not sure what your talking about here.
SDL isn't even in the same dimension of usability compared to the Godot editor.
On top of that, if you wanted to godot-rust is a extremely well supported - we've used it extensively and have nothing but praise. There are several other language bindings with varying levels of support too, no need to stick with C++ if that's really their line in the sand.
Appreciate the time you took to point this out. Yes, the docs as they are now are LLM written. It might help a bit more if you can point out to a particular section or file that stands out. If not, no worries, I will plan on doing a manual sweep across all .mds for better (clearer) readability.
This exists because OP wanted to learn godot and didn't look around at prior art enough to discover herdr first, so that's probably the top line take away for "should I use this".
If you squint, Godot has a cross platform hardware accelerated GUI. The editor is itself, in a sense, a Godot app and has builds for VR Headsets, Android, web apps, all the usual platforms. The extension api works pretty good too for Rust, and Rust itself has a fantastic cross platform and WASM story. It might be cursed, but it can get complicated stuff out the door and in anyone's hands fast.
Can you add a "why?" / "why now?" section?
Don't be another project that is a bunch of how that is leaving out to explain why it exists or exemplify what classes of problems it unlocks solutions for (applicability).
It sort of goes over the "why?"/"why now?"; but TLDR; Wanted to use Godot & Rust in personal project(s) (no pressing need for it) I saw Cate & wondered whether I could use this tech. stack to replicate, instead of web tech. So that's where it all started. I know that if you want to live in the terminal, there's tmux and herdr, and I'm not (also can't) compete with them. But I can surface a terminal that then lets TUI folks stay in their preferred apps. The residual why/why now are largely shaped by what I want (feature-wise) to put into the tool.
And the next big feature I want is a personal knowledge management. Maybe it looks like a complex beast; but the core idea driving my features are what I want on a daily basis without switching context as much. Sorry, I know it's not a great answer, bit rambly.
Fyne.io in Go also doesn't want to do a browser, but that is basically how to build developer mindshare. (Kind of like deciding whether or not to implement a default data grid in a UI toolkit.)
I tried different terminals and multiplexers for agent development: tty7, Warp, Herdr... Now my favorite is tty7. It does exactly what i wanted from a modern terminal, what Warp should have become.
I've done a lot of things reusing/modifying the internals of alacritty [1] to put terminals in things. TeX in the terminal [2], a terminal in minecraft [3], Alacritty in Godot [4], and not pictured, alacritty in a swift plugin for macos/ios/ipados all at various levels of completion.
Terminals and web browsers are our main UI drivers right, so yeah, it makes sense to have a super app that provides them both in the best way. Embdding a terminal inside a web app is annoying because of Ctrl-W and other issues, and embedding a web broswer inside a terminal is even more weird. So we wind up having Wayland compositors do the work to combine them in one environment.
I tried to do this with a simple wgpu + winit app, servo, and alactitty_terminal, and it kinda worked, but both terminal and web browser rendered poorly.
I made an internal GUI tool for company in Godot. It uses GDExtension to interact with OS systems, namely serial/COM ports. Yes, you can make quite good desktop GUI applications in Godot, with mostly its default toolkit!
Also, exploring minified Godot template compilation led to me distributing a 24 MB portable package (10MB zipped). Astonishingly efficient.
Godot as an application platform and not just a game engine is a really interesting idea. Given Godot’s editor is built as a Godot engine game, it has a very rich UI system, and it is pretty easy for agents to work with.
50 comments
[ 0.69 ms ] story [ 1182 ms ] threadBut since we have the Godot game engine at our disposal, we can do some more interesting things, like add an FPS counter, and then subsequently also let people set their preferred FPS (the idea being the potential lower power draw if someone's running it on a laptop on battery power vs someone running it on a desktop with high/native FPS). In its current state, with me using Oh-my-Pi a lot, it's evolving into a terminal workspace that can be used for orchestrating autonomous AI agents by way of dogfooding (or you know, just run herdr inside of gPTY - it's the better orchestrator and just good software - I found it after starting this project, and now I'm finding myself using it a lot).
Also, we're not limited to just terminals. Since we have Godot, we have basically a 2D (and potentially a 3D) canvas to play with. We can already full-screen the app for "zen" mode, no taskbar, no distractions. TUI die-hards can have their media or other apps entirely in terminal panes.
There has been some ground-work on getting Markdowns displayed properly done and I want to work on some kind of Wiki framework for local knowledge-management next, then create more types of panes (think native audio/video on a media pane, that sits alongside your terminal pane), and some simple 2D games (like snake) to prototype. More details are on the ROADMAP.
What's not easy (and probably won't happen) is a browser. Having done a couple of (small) projects using Electron already, the temptation to ditch Godot/Rust (learning curve) did come up (and also the ecosystem, the ease with which I could pull components and use web technologies - development velocity would definitely be higher there). But on the flipside, given all of the available LLM and AI support that we are privileged to have today, I figured the velocity should be comparable depending on how much I leaned on those. And lean I did.
Godot/Rust seemed the better call to me and my intent anyway - going with the 'it's not just the end but the journey that matters' philosophy. So yes, there has been heavy use of LLMs & AI to generate a lot of the code. But I do review and steer actively, not relying solely on vibes, and there's a few bits here & there that have been human authored.
There are definitely a lot of polish and QoL items that need to land to make the end user experience better, but in the meantime, let me know your thoughts and/or concerns!
Repository: https://github.com/godot-pty/gpty Docs/Blog: https://godot-pty.github.io/
I'll add that in; maybe defaulted to folded on the README, a gallery of app screens or snippets. I need to spend some time on the UX, but yeah, screenshots is something I should've also added to the README.
Wildly vibe-coded to the extent that it's more misguided than Doom on a pregnancy test...
Also looking at OP's notes here, there doesn't even seem to be a need for this to exist, "FPS counter" ..? Perhaps, I don't know, fake terminals on a spaceship or something. Yueash...
There's a new one every day posted on r/tmux
You get continous rendering in a loop instead of updates only when something changes. It's like our laptop batteries are too big and we need to find something to deplete them.
So we get an abundance of (useless) software.
That said, for your browser problem, what if you embed the browser inside of godot? Chromium embedded framework does this. I was able to get this to work for me to see Chrome inside of my godot exported projects. https://github.com/dsh0416/godot-cef I believe this is only for desktop exports though, no html5/mobile builds.
Only downside is you have to use Chromium ;)
The browser is probably a non-issue, more a passing thought now. I definitely do not want to bundle or bring in Chromium. If it was a deal-breaker, I would've just gone the Electron route. Appreciate you sharing your WebView extension though, looks neat!
So many times I see projects here I don’t really need, but I always learn something from their readme and code. A lot of time I learn something I can apply to my project. In this case, I use Node-PTY for my terminal control plane but when I run more than 50 agents, it crashes. Maybe using your pty will let me run more agents.
I haven't performance or load tested, it's on the backlog; so I'd be curious on whether this works for your use-case. And if not, feel free to log an issue!
Original intent was just a hobby/side project. As it grew, I thought maybe posting about it would be alright in case people find the idea interesting.
If most of your problem is 2d window/text stuff, the web browser is probably the best foundation to work with. UI and hud functionality in COTS game engines is light years behind what chromium and friends can do today.
This is something mainly working on it on my free time so figured why not see how far I could push things.
i will definitely be curious to see how you find that decision to be working out for you as the project evolves— will definitely be keeping my eye on it! actual refreshing, interesting take!
Usage has been relatively snappy (could be better, I suppose, I need more time). Definitely not sluggish.
A concern is that RAM usage is on the higher side (this is something I'll look at on the next release).
C++ coding is just fine. It's not painful. It can be a bit of boilerplate to expose c++ member fields and functions to the rest of the engine but you get some great power out of that. Just use compile your own module right into the engine. Don't use gdnative.
As someone who's codes a lot of c++ in Godot I really am not sure what your talking about here.
SDL isn't even in the same dimension of usability compared to the Godot editor.
For everything else, https://gameaccessibilityguidelines.com/full-list/ is a great checklist.
It sort of goes over the "why?"/"why now?"; but TLDR; Wanted to use Godot & Rust in personal project(s) (no pressing need for it) I saw Cate & wondered whether I could use this tech. stack to replicate, instead of web tech. So that's where it all started. I know that if you want to live in the terminal, there's tmux and herdr, and I'm not (also can't) compete with them. But I can surface a terminal that then lets TUI folks stay in their preferred apps. The residual why/why now are largely shaped by what I want (feature-wise) to put into the tool.
And the next big feature I want is a personal knowledge management. Maybe it looks like a complex beast; but the core idea driving my features are what I want on a daily basis without switching context as much. Sorry, I know it's not a great answer, bit rambly.
1. https://alacritty.org/
2. https://blog.fangorn.io/tex-in-terminal
3. https://anvil.fangorn.io/fangorn/huorn-minecraft
4. https://blog.fangorn.io/alacritty-in-godot
I tried to do this with a simple wgpu + winit app, servo, and alactitty_terminal, and it kinda worked, but both terminal and web browser rendered poorly.
Also, exploring minified Godot template compilation led to me distributing a 24 MB portable package (10MB zipped). Astonishingly efficient.
Godot as an application platform and not just a game engine is a really interesting idea. Given Godot’s editor is built as a Godot engine game, it has a very rich UI system, and it is pretty easy for agents to work with.