HN Explorer

Codex – Find and Replace for Code (codex-kappa-dusky.vercel.app)

42 points by gushogg-blake ↗ HN
Most code editors' find & replace features are still very close to the original design intended for text documents, so they become unwieldy when you need to match across newlines and indentation for example, or when a parse of the code is necessary to capture a particular expression.

Codex (https://codex-kappa-dusky.vercel.app/) is an attempt to rethink what find & replace should look like in a modern code editor. It defines a simple but powerful syntax for describing code modifications, combining plain text, regular expressions and Tree-sitter queries, along with sensible handling of newlines and indentation*.

It can be used just like regular plain text find & replace, but allows freely mixing in regexes and Tree-sitter queries as more flexibility is needed.

It introduces "line quantifiers" for matching a bunch of lines at the same nesting level, so basic structural changes can be achieved without even using a query (see the JavaScript function example in the link).

I designed Codex with a specific use case in mind (the one I show in my demo video: https://www.youtube.com/watch?v=MQ_N0-AJ2Qg), so any suggestions for other things it should support would be much appreciated, as well as general feedback.

*Indentation is relative and space/tab agnostic.

20 comments

[ 0.33 ms ] story [ 56.7 ms ] thread
ttcbj
This is interesting, but to be honest it’s more learning than I personally want to take on for this use case.

One feature that I miss from code editors is “record actions”. The last time I remember having it was maybe in visual studio around the late 90s. It would record whatever you did (keystrokes/copy paste/find replace/etc). At that time, I found that I could often address use cases similar to those I think you are targeting by thoughtfully recording the right sequence of actions.

sixstringtheory
Xcode does this but only for the last 10 or so queries. I wonder if that is somehow configurable in an undocumented user default…
gushogg-blake
Have you tried Komodo Edit? It has Macros which is what I used to use for this kind of thing.
gushogg-blake
Could you explain in more detail what makes it too much of a learning investment for you/what might improve it? It's always hard to see which bits are non intuitive when you've designed something.
jacobsenscott
"record actions" - most editors call these "macros". I would think most good programming editors support this. I've used vim and emacs and they both do.
joshka
Meta-note: do you have any concerns about overloading the name "Codex"? There are quite a few software products and companies named this.

This is pretty cool. I wonder though if the tree sitter query syntax is the right thing to directly expose. This is definitely useful beneath the hood for people writing specific refactoring patterns, but it would seem overkill for general dev. Is there some syntax that looks closer to regexes for this (tree sitter inside regex rather than regex inside tree sitter)?

I wonder if attacking this problem at one level up say the LSP level could be a good idea.

rockwotj
> Is there some syntax that looks closer to regexes for this (tree sitter inside regex rather than regex inside tree sitter)?

I wonder if you could add escapes for specific syntax features, something like:

\z{FunctionName=Foo}

[deleted]
(comment deleted)
gushogg-blake
Yeah that is definitely a concern, I mostly just needed a name so went with it - another canidate was CodeExp going with the regex/p analogy. But definitely open to something else entirely.

Yes a simpler syntax would be good - I was surprised at how complex the query got for the use case I designed it for. I do like the separation of queries and regex though, and keeping the whole structure as regex and queries inside plain text, as it starts you off in the simplest mode (just text) and lets you add flexibility as needed (although it does mean some chars have to be escaped).

sixstringtheory
Regex search and replace is table stakes for a code editor. All the way back to ed ;-)

Tree-sitter queries is an interesting addition. That and bazel build system queries seem like a step up in editing and compiling code.

gregwebs
This looks neat, how do I try it out? Semgrep is nice for simple modifications but poor at multi line. Comby works well a lot of the time but I really need something more sophisticated. More sophisticated often means language specific but I would like to have a language independent tool like this.
gushogg-blake
The latest version is in my editor project at https://gitlab.com/gushogg-blake/edita-release. It will apply changes but it's just the minimal functionality at the moment, there is no feedback after you press the Replace all button. My process is to use the editor and write whatever I most need on it right now so not sure when I'll get round to fleshing it out.

Ideally other editors will take inspiration from it and add their own implementations. The code is available for reference here and I'd be happy to go over it in detail with anyone who wants to: https://gitlab.com/gushogg-blake/edita-release/-/tree/main/s....

zephraph
ast-grep is another tool along these lines that's worth checking out. Closer to semgrep than comby, but it's improving fairly rapidly.
fizx
I've often wanted to extend the https://selectorgadget.com algorithm to be able to do find & replace on code ASTs, instead of the DOM tree.

It'd be pretty cool to be able to find-by-example, selecting snippets of code and having the algorithm tell you what they all have in common.

counttheforks
Cool! Are you familiar with jetbrains's structural search & replace? The way it works is quite intuitive. Might be worth mentioning as a comparison.

https://www.jetbrains.com/help/idea/structural-search-and-re...

gushogg-blake
Oh wow, yeah that is similar. I'll add a comparison. Thanks!
SebastianKra
I wish I could find proper documentation for it. I still haven't fully grasped how placeholders work, and thus often run into unexpected behaviour.