HN Explorer

Show HN: HiSHtory: Your shell history in context, synced, and queryable (github.com)

179 points by ddworken ↗ HN
hiSHtory is a better shell history. It stores your shell history in context (what directory you ran the command it, whether it succeeded or failed, how long it took, etc). This is all stored locally and end-to-end encrypted for syncing to to all your other computers. All of this is easily queryable via Control-R and via the hishtory CLI. This means from your laptop, you can easily find that complex bash pipeline you wrote on your server, and see the context in which you ran it.

74 comments

[ 3.7 ms ] story [ 119 ms ] thread
nikolay
How does this compare to Atuin [0]?

[0]: https://atuin.sh/

ddworken
Ah, I hadn't come across Atuin before! They're pretty similar, but I think one significant advantage of hiSHtory is the "Custom Columns" support where you can populate custom columns with the output of an arbitrary shell script. For example, you can use this to collect the current git branch as an additional column.
ellieh
Atuin* also allows for opt-in sync, easily hosting your own server, and switching search context for your shell history (ie whether you want to search the current directory, session, host, or all history on all hosts). We also have a bunch of different search modes that query history differently (fuzzy, prefix, etc)

But otherwise yes, the tools are pretty similar! Nice to see another in the space

*: context, I'm the maintainer so may be biased ;)

sapphire_tomb
Is there an arm64 version of the docker image for the atuin server? I'd really like to be able to deploy this to my microk8s cluster.
nikolay
I know it's a challenge, but, please, add deletion! Thanks!
ellieh
First part of deletion went out in v12, we should wrap up the rest in v13!
nikolay
Hmmm, Homebrew still has v11.
nikolay
Homebrew just released v12 today! Yay!
rem7
I wrote something similar that I use across 2 laptops, so it stores it in AWS. The gist of it is firehose to S3 then use Athena to query. My zsh shell sources a script that has functions that uses the AWS cli to post data or query.
dopidopHN
Out of curiosity: why not straight to s3?
LifeIsBio
I've been using Resh[0] for the past 6 months or so. A rich and queryable shell history is a massive boost in day-to-day productivity. The syncing described here is a pretty cool feature.

[0]: https://github.com/curusarn/resh

ddworken
In addition, hiSHtory also supports fish for anyone who uses fish!
[deleted]
(comment deleted)
flas9sd
directory context is quite an argument, I assume I can jump back into the directory with the line both - not just get the cli+args?

(Btw I read the util name in Sean Connerys voice)

ddworken
Yup exactly! And you can even collect other custom metadata to make it easier to jump write back to where you were.
sooheon
There's an app for Macs with iCloud sync as well: https://loshadki.app/shellhistory/
ddworken
Ah very cool! One advantage of hiSHtory over this is that hiSHtory also supports Linux (since it isn't bound to iCloud sync).
[deleted]
(comment deleted)
dorian-graph
This looks great. Thank you! If Atuin or HiSHtory would only let me sync through iCloud or something else like Synching.
encryptluks2
I'm confused why you need python 3 to install a Go app?

> curl https://hishtory.dev/install.py | python3 -

ddworken
If you'd rather `go install` it, that works too! The advantage of this is that it doesn't require users to have go installed. And if you look at the script, all it does is call an API to determine the latest pre-built version for the current OS and install it.
ropeladder
Looks nice! Do all of these shell history replacements completely override native history functionality or just add on to it? (and if I delete something using `hishtory redact` does it delete it from my native shell history too?)

Also just a heads up, I find the demo gif to be unintelligibly fast.

ddworken
It just adds to it, your standard shell history is still there and completely unaffected by this tool. And if you prefer using your native shell's control-r you can also do that.

> if I delete something using `hishtory redact` does it delete it from my native shell history too

It does not (though if you think it should, please open an issue and I'm happy to add this!)

> Also just a heads up, I find the demo gif to be unintelligibly fast.

Ah thanks! Updating.

dopidopHN
Hi, Sometimes things slips into shell-history, it’s convenient to be able to redact it.

Is the history visible and editable by the user ? ( does not have to be overriding a existing command )

ddworken
Yes! See the `hishtory redact` command to do exactly this.
yewenjie
I have been using Atuin for sometime, but I must say, though that looks cool, the fuzzy search results out-of-the-box are useless to me.

I used to use something before, probably McFly, that had way better search results.

https://github.com/cantino/mcfly

https://github.com/ellie/atuin

[deleted]
(comment deleted)
Shank
> atuin

> You may use either the server I host, or host your own! Or just don't use sync at all.

I sincerely appreciate the fact that you provided these as alternatives, because wow, offline only, with no synchronization? That's honestly really nice to see compared to this tool.

Fileformat
I do something similar but much, much simpler with a 4-line `precmd` [1] in my .zshrc file. It just records timestamp, directory and command to a file per day. Then you can use standard grep & standard tools to find things.

[1] https://til.marcuse.info/bash/bash-infinite-history.html

[deleted]
(comment deleted)
angrais
This is the default on ohmyzsh :)
cshou
Looks cool. Is it possible to disable sync? For security reasons, I would rather keep everything local
[deleted]
(comment deleted)
ddworken
See https://github.com/ddworken/hishtory/issues/4

This isn't supported quite yet, but is the next item on the road map.

ddworken
Though keep in mind that all history entries are end to end encrypted, so even with syncing enabled, your shell history isn't accessible to the backend.
insane_dreamer
It would still reassure some people who are skittish about that sort of thing; I can think of people I would otherwise recommend it to who would would automatically say no if it syncs.
ddworken
Agreed! I'll make sure to get this implemented ASAP. :)
pmontra
I didn't investigate how to run the backend server (is its address configurable?) but if it can be self hosted it could be run locally. The advantage is that VMs and maybe even containers can send their history to the local server.

If it can't be self hosted, despite end to end encryption it's a big no no for me.

Edit. I read the code

  func getServerHostname() string {
    if server := os.Getenv("HISHTORY_SERVER"); server != "" {
      return server
    }
    return "https://api.hishtory.dev"
  }
imiric
Same. This tool would've been much better if it didn't reinvent syncing, functioned entirely offline, and just focused on the core problem of contextual shell history. There are plenty of syncing tools that handle security well, and the author could've recommended some that integrate well with this project.

As it is, this is a blocker for me even trying it out. Even if it could eventually be disabled, I don't want it to have network access at all.

mdmglr
I didn't see the install.py script in the repo. Would be a good idea to include it.

What was much more interesting to me is how this tool hooks into the shell to record commands. This can be found here: https://github.com/ddworken/hishtory/tree/master/client/lib/...

henrydark
Does it handle multiple writers on the same machine? For example two people ssh-ing with the same user to a remote machine with hishtory
ddworken
Yes! Though they'll have a single shared history log since there is no way to distinguish between them.
lolive
Anyone/anything solving the issue of bash histories in multiple (intricated) tmux sessions is an INSTANT win !!!!

[Q: can this tool help? As far as in understand, it seems that YEEEES!!!]

UI_at_80x24
I solved that particular issue by using this in my .bashrc

#write to history file at each shell prompt

    export PROMPT_COMMAND="history -a; history -c; history -r; $PROMPT_COMMAND"
(Unless I read your comment wrong.)
ddworken
Yup exactly! This way you'll never again lose a history entry.
msravi
Just tried this, and it appears to cause a very significant lag (~2 sec) after each shell command. Removing the sourcing of the config from bashrc and opening a new shell immediately gets rid of the lag. Maybe it's because I have HISTSIZE set to a large value...
pastage
Sending your command history automatically over the network is always an huge security risk. This tool atleast have a security section in the readme, but nothing about operational security and how to guard against hacks of one synced agent.

I will try it because the history tool looks like an interesting take, but the security model is not ok for multi computer use.

ellieh
I think my main concern here is the fact that you're putting secrets in your shell history!
dopidopHN
Mistake happen.

Yesterday I was tired and working on a friend serveur. The ssh prompt was different and I entered the password from a account while the remote port has hang up on me. So effectively typing “password” straight on my shell.

His password is in my history now. I removed it but that type of thing happen all the time. ( Not that much in a professional settings, password tends to fade away )

capableweb
Pro-tip: if you wanna hide stuff from your shell history, enter a space/blank before the actual command.

Not that it probably would have helped in this situation, but maybe useful for others.

Usually if I have to enter secrets in my shell, I do something like this in order to save the useful stuff to history but hiding the secrets:

- ` export PASSWORD=hunter2` (notice the space before "export")

- `service-cli login username $PASSWORD` (no space before, gets saved to shell history)

Cockbrand
While ignoring commands preceded by a space character is absolutely one of my very favorite shell features, it usually isn't enabled by default (YMMV).

HISTCONTROL=ignorespace # bash

setopt hist_ignore_space # zsh

These go into ~/.bashrc or ~/.zshrc, respectively. zsh is really smart about this and always enables you to retrieve the immediate previous line with arrow-up, no matter whether it's got preceding whitespace or not.

400thecat
why use `export` ? That is a terrible idea.
ddworken
+1 to prefixing with a space. hiSHtory supports this out of the box so that anything prefixed with a space isn't recorded (even if your shell doesn't do that by default!). And if you do ever mess up, `hishtory redact` can be used to delete history entries.
remram
Other options:

- read -s PASSWORD

- service-cli login username $(cat)

At least you probably don't want to use `export` when setting the variable.

Cockbrand
I've certainly entered passwords at the shell prompt by accident. This doesn't happen often, but it sure does.
ilyt
Some apps only accept password in commandline.

The better written ones might accept it in env but that will also stay in history.

But if someone have access to your home they can put keylogger in the .profile/.bashrc so it's not really relevant...

ellieh
Sure, but something like this also works well:

`the-command --password $(read -s;echo $REPLY)`

ilyt
yeah now instead of history you can get the app's password via `ps aux` :D

also you still have the `export REPLY=password` in history.

sure you can source it out of file but, well, that file have same permissions (or looser! if you forgot to change) as your .bash_history file.

`cat password|command` is the safest one (or, well, something 12 factor app clowns hate, a config file)

[deleted]
(comment deleted)
dabeeeenster
Is it doing anything clever for the search? Based on path context etc?
angrais
How does this differ from fzf, peco or autin?

A table comparing these and existing tools would be ideal to show what's unique about this one.

zemnmez
david is one of the smartest people I've ever worked with at Google. It's good to see him here :)
happens
So, this syncs through a hosted server somewhere? Can I host my own? That information should be front and center imo, I didn't see in anywhere in the Readme.
dpz
This is interesting but i'm not interested in the sharing of the history, and i'm not sure what this offers that using fzf with zsh doesn't
xrd
Is my new Mycroft AI device capturing my thoughts and sending them on to a Borg-like hivemind? I was just thinking of something like this the other day and here it is on HN!

All jokes aside, from reading the threads, I see there are other alternatives, most interestingly atuin and fzf. I used fzf for a while, but didn't use the history features. Atuin looks amazing, and installs using nix-shell. My litmus test for whether something is safe to use is nix-shell these days; I'm not excited about running a curl script into python as my starting point!

Does anyone have references on whether atuin and fzf can be used together, or should you only use one or the other? I liked fzf but the encrypted remote sync feature of atuin looks really fascinating.

xk3
I use syncthing instead. But if I switch computers quickly there are conflicts which I resolve like this:

    function sync_history
        cat ~/.local/share/fish/fish_history.sync-conflict-* >>~/.local/share/fish/fish_history
        rm ~/.local/share/fish/fish_history.sync-conflict-*
    end