Show HN: Haystack – an IDE for exploring and editing code on an infinite canvas (haystackeditor.com)

546 points by akshaysg ↗ HN
Hi HN, we’re building Haystack Editor (https://haystackeditor.com/), a canvas-based IDE that automates the boring stuff (plumbing, refactoring, and finding code) so that you can focus on the exciting parts of software development! You can see a quick overview of Haystack at https://www.youtube.com/watch?v=c2uZnR5D_cc!

(It's currently only on Mac OS but we're working on Linux and Windows. Edit: just added a Linux download!)

Haystack was born out of our frustrations with working in large and mature codebases, specifically with navigating and editing functional flows (e.g. the code flow for adding an item to the Amazon shopping cart).

Oftentimes dealing with such flows would involve navigating a maze of files and functions, and making any edits would involve a lengthy process of doing corresponding downstream/upstream plumbing.

Haystack attempts to address this in the following ways:

    1. It allows you to explore your codebase as a directed graph of functions, classes, etc on the canvas. We feel like this better fits how your mind understands your codebase and helps you find and alter functional flows more intuitively. We especially want to utilize this for pull request reviews!

    2. It has a navigational copilot that makes edits across files or functions much easier. After you make some changes, Haystack will try to predict your next action and create functions/methods or refactor upstream/downstream code for you. Haystack will surface these speculative edits on the canvas in a way that you can easily dismiss or incorporate them, allowing you to make large changes with a few clicks or keystrokes.

    3. Haystack will utilize natural language search so you don’t have to play “Where’s Waldo” to find a functional flow in your codebase. This is coming soon!

We’re still pretty early in development and we really want to perfect the experience of navigating and editing code on a canvas. Any feedback would be much appreciated!

PSA: Since Haystack is a VS Code fork, you should be able to move your extensions and keyboard shortcuts. Please let us know if you have any issues with this!

226 comments

[ 3.7 ms ] story [ 330 ms ] thread
You should definitely check out Understand https://scitools.com/features, if you haven't already.
I did not know about Understand. This is really cool -- thank you! I really like how Understand enables you to understand how many upstream dependencies you may need to change when refactoring.

We hope to push it to the next level by helping do those refactors for you in an understandable way!

You can certainly push it a different direction with accessible pricing models. No matter how cool / useful Understand actually is $1,000 / year for an indie dev is too high.
This is cool.

Now that we have LLM models we should really investigate going back and forth between model based representations and text based representations, i.e. allow people to edit both and see the changes reflected in the other.

We write model based code all day, but probably 75% of my time is spent dragging things around and making them look pretty with no net change in the final generated C code. Can't skip that part of the job because the whole point of a model is to make the code easy to understand, but it seems like a good application for an AI.

Very cool concept! Will it be possible to make "boxes" with labels/titles so different frames can be grouped? And will zooming of the canvas be supported?
Ooh grouping is a cool idea we didn't consider! Curious to hear what you want out of grouping -- I can imagine it's useful to label functionalities and move different editors at once, but would love to hear from you.

You can zoom in/out of the canvas with CMD/CTRL + scrolling! I do think this can cause blurry text, which we definitely aim to fix

Windows 11 seems to have a auto grouping feature baaed on what windows are tiled
Oh nooooo, this is MacOS only? This is amazing and I need this so badly.
I am so sorry! Our immediate next priority is to get this on Linux and Windows. May I ask which OS you use so we can get it done now?
I'd use this on linux as soon as it's available
Just added a Linux download!
I use Windows primarily. I don't expect you to rush anything, but definitely looking forward for a multi-platform release.
Just added a Linux download!
This fits my mental model when coding so well. I love it!
So it’s a more fancy/cpu intensive neovim without the learning curve?

I don’t know, looks cool. But seems like it would get annoying after a period of time. Too much dependency on a mouse for me.

You brought up a great point regarding mouse dependency. The end goal is to make this keyboard navigable (I myself don't like taking my hands off the keyboard), but we also need to do it in an intuitive/easy-to-use way which will be tough.
Does neovim have a canvas feature? In what way is it similar to Haystack, apart from also being an editor?
Namespace conflict, and yes it does matter. Just as "mamba" was an anaconda package manager re-implementation before it got commandeered by the LLM research crowd, now Haystack is being stolen away from Deepset: https://github.com/deepset-ai/haystack

https://haystack.deepset.ai/

And this is on top of it being stolen away from the rightful owners: hay farmers.
They don't make stacks anymore; these days we have balers (and rollers) and therefore move the food (over km) to the livestock rather than the livestock (over m) to the food.
This is why we have trademark systems. Name conflicts are allowed for commercial use if the purpose and presentation are clearly delineated from competitors. For non-commercial use... who cares? No one is "stealing" words.
Our intent is not to steal anyone's name, but I hope that by being a very different product nobody confuses us with this Haystack (or the older Haystack that other commenters pointed out).

I do think the idea of a "needle in a haystack" is fairly ubiquitous and makes sense for a ton of contexts to be fair.

I downloaded this and opened up a React based project that uses functions and not classes. I'm not seeing the sideways facing arrows to show the call stack of the function I'm looking at and have my cursor in - the functions that call the function I'm looking at and the functions that reference the function I'm looking at.

It seems the sideways arrows in the demo only appear for class based methods and not plain functions :(

Overall I love the idea and have wanted a graph visualization of a codebase showing every function call, who calls it, and who it is called by which this seems to do but for classes. Hope to see this working for functions as well soon! Great work.

Sorry for the frustrating behavior. It should actually work for plain functions and I just tested with a TSX component we use. I think something went wrong here.

Are you using JSX components? I'll try to repro this on my end.

I'm using a mix of TSX/JSX/TS/JS. I wasn't seeing it working for any of the functions (component or plain JS) I have unfortunately. Will keep an eye out for any updates you make to resolve this!

Awesome project and great work on it! Congrats on the launch :)

I posted this previously, but this, and notebook style output, then flat textfile view would be the sweet spot for universal development. I vasilate between notebooks and vscode to develop most of my projects.
Congrats on the launch. I've actually been going in a totally separate direction lately, it helps with my focus and ADHD tendencies to jump around and get overwhelmed.

1. One window. 2. Minimal everything. 3. Most importantly, no tree navigation.

If I need to jump to a file, I either use recents menu in Intellij or use file markers to jump to it, means I have to remember the filemarker and have intent to go there (I'm using IdeaVim plugin for vim hotkeys). This, to me, make programming much more focused and pleasurable. This is part of my evolution from multi screen to widesreen and then back to a single screen.

For large and mature codebases like you mention, I just use LLM to guide me around.

Makes sense! Curious how you use LLMs to guide you around?

In my own experience, I ask the LLM where a functional flow is and it does a great job of creating an entrypoint, but I frequently have to figure out the finer details myself and sometimes the LLM points to something related, but not quite what I was looking for.

> For large and mature codebases like you mention, I just use LLM to guide me around.

How? What do you call large mature codebases? In my experience LLMs already make many mistakes and assumptions just in one file?

I think the title should make it clear it’s macOS only, before getting my hopes high to test it out, it looks interesting nonetheless.
Very sorry about this. We hope to get in a Linux product soon (building now, fingers crossed).

Thanks Dan for the edit!

Added a Linux download if that's the platform you're using!
Wow, that was quick, appreciated! I just downloaded it and it works perfectly (Debian).
In my minds eye I've been yearning to see an IDE that focuses on a zoomable interface.

This looks like a good start.

> We do not store or log your code anywhere. We do send snippets of your code to Open AI as input for our navigational copilot. We'll add an option to disable this soon.

I look forward to checking this out once that option is in place.

Agreed. That should have been a opt-in from day one!
Had the same reaction. I’m willing to support development for a tool like this, but would prefer to do it more….directly.
Yeah, can't exactly try this at work as is.
Added an opt-out for the navigational copilot on Mac -- just bring up the command pallet (CMD+SHIFT+P) and use the "Opt out of navigational copilot" command. Coming to Windows/Linux soon!

Note that we only send data to Open AI if you have a diagnostic indicating that you're calling a function that doesn't exist.

Looks a little like Labview without the data flow features. Interesting.
I don't and never have used Lab view, but I am interested in adding features to better visualize data flow. I feel like this can easily err on the side of being confusing/annoying, but do you like Labview's data flow features?
I don't mean the lack of data flow like a bad thing. I think your idea might be better, since data flow aspects might be difficult to organize in a 2d plane. It gets cluttered fast.
I'm not sure about the AI features but the concept of working with code on a canvas and having some sort of flowchart UI has been a dream of mine for some time. I mainly do data processing and draw lots of flowcharts for communicating pipelines and also for myself. Having a development UI which goes in that direction somewhat would be great for handling unwieldy codebases written by non-CS folk.
Is a tool like LabView what you have in mind?

https://www.ni.com/en/shop/labview.html

Yeah kind of. I have worked with LabView and liked the flowchart UI for handling lab instruments, but for somewhat complex algorithms it got unwieldy really fast.

I have the feeling LabView is a good tool for designing control panels and visual tools in general ("front-end"). It is less suited for implementing back-end algorithms. I guess I wish for a similar tool which can also handle the back-end (in the context of my data processing job).

The problem is that I don't know myself what UI I would actually want for a "canvas-based editor". Some mixture between a visual flowchart and being able to edit code in each box? I'm obviously not the first person to think about it and it seems to be a non-trivial problem.

Curious if the problem you're encountering is visualizing the data flow vs. the control flow?
(comment deleted)
It would be awesome if we can sketch diagrams and code in the same canvas! This would make visual documentation much easier.
you can do asciidoctor, plantUML kroki in vscode so maybe its present mode can already be used in there like that?
I've been wanting to try something like this for a long time! Is there any way to sign up for updates?

(In particular, I'm waiting for the option to turn off the AI stuff, at least for when I am trying things out.)

I especially love the ability to zoom into a function and hide the rest of a file. I've wanted that sort of thing a surprising number of times. I'm thinking one could also use this to virtually rearrange a single file by pulling the functions in that file that you are working on next to each other easily?

In addition to the editing interface, I think there are probably a lot of visualization opportunities in editors:

- what's the interaction between my code and third-party dependencies? - what are the dependencies/call graph of my code? or perhaps just a portion of it? - call graphs can be nested — perhaps you care for a dependency graph at the class or module level, or perhaps you want to "drill down" into the call graph inside an object

In all these cases, being able to seamlessly transition between the visualization and editing — like you seem to be aiming towards — seems like the killer feature.

And, of course, there are lot of other interesting visualization opportunities once you have any sort of graphics in your editor:

- heatmaps of code churn - heatmaps of code performance - tracing variable usage - and, of course, lots more (https://adamtornhill.com/articles/crimescene/codeascrimescen...)

One last thought: if you haven't read _The Programmer's Brain_, you should at least listen to this podcast. https://se-radio.net/2021/06/episode-462-felienne-on-the-pro... There are a bunch of editing ideas related to how our brain works that I haven't seen well supported in an editor yet. You took one step in that direction — maybe there are some more opportunities?

Looking forward to what you come up with!

Unfortunately we haven't built a clever way to sign up for updates! I'll think of the best way to do this (eventually we'll push updates directly in the product).

Working on adding the copilot opt-out as we speak!

I love the ideas here and I would like to discuss some of these with you. If it's OK with you, I am going email/DM you!

In general, we would love to engage in conversations with folks (feedback@haystackeditor.com)

Feel free to email — it’s in my profile. :-)
- Demo made me want this instantly -- looks better than tabs

- Realized its VSCode -- started downloading

- Stopped the download when I saw AI -- cannot work on company code

Will definately try this when AI is disabled.

Added an opt-out for the navigational copilot on Mac -- just bring up the command pallet (CMD+SHIFT+P) and use the "Opt out of navigational copilot" command. Coming to Windows/Linux soon!

Note that we only send data to Open AI if you have a diagnostic indicating that you're calling a function that doesn't exist.

Nice! I think this is a step in the right direction of the future of IDEs. So tired of switching between dozens of tabs which are either sorted by name (fixed and useless) or usage (chaotic) in common IDEs like Intellij and VSCode.
it brings me a thought: is it just me who kind of hate the "super-modular-files-with-10-lines" way of doing things today? would love to get back to the way of doing things in the past, like super well-commented files, with hooks here and there to help you navigate, single file for single contexts...
Depends on the situation.

E.g., if a project's general rule is one file per class, I prefer consistently following that rule, even of it makes some files be really short.

Looks interesting, but the contrast between white background and actual windows is preventing me to use it. How can it be changed?
Apologies for this! We'll make the canvas match VS Code's theme eventually.
This is really neat, quite a paradigm shift from the IDEs we’ve been using for decades
I spent the last five+ years full-time deep diving the infinite canvas UX problem space if you happen to be hiring ;)
We are not but I'm happy to discuss! Feel free to email feedback@haystackeditor.com
[flagged]