Ask HN: What are some tools / libraries you built yourself?

364 points by graderjs ↗ HN
Many times you just want to plug something in. PostgreSQL, Node.JS Express, Java Spring, numpy, Three.js. There's many examples where the already existing solution fits well.

Sometimes that's not good enough tho. What are some tools, libraries or services you built, are they open-source and why weren't you satisfied using what already existed?

616 comments

[ 3.0 ms ] story [ 335 ms ] thread
Quepid (http://quepid.com) and Splainer (http://splainer.io) both are tools I created a long time ago to work on improving search result relevance in Solr or Elasticsearch. Both are open source.

I literally started coding Quepid next to a colleague complaining about search so I could help them. It’s basically a kind of test driven approach to continually tuning and proving you’re not harming existing use cases too much. The first version that day was super hacky :) but it worked: it was a “single page server side application” ;). Just dumping a huge HTML page from flask about how our search was doing based on stakeholder feedback.

Nothing out there existed that did what I needed. Tooling around search engines for relevance just wasn’t great. In part because it wasn’t paid as much attention to ~10 years ago when I worked on creating these tools.

Cool! :) I'll definitely take a look at this. I'm working on a search thing now and need some way to measure and improve search result to query relevance. Thanks for posting! :p ;) xx
There is a new, emerging job role, the "relevance engineer", who ensures that search results stay relevant as content sets changes.

In London, the annual Search Engine Solutions meeting (each autumn/fall) organized by the BCS IRSG (Chartered Institute of IT's Information Retrieval Specialist Group) provides a forum for search practitioners to exchange best practices.

Here's the report from the 2020 meeting: https://irsg.bcs.org/informer/2021/01/search-solutions-2020-...

Talk proposals, demo proposals, tutorials etc. welcome.

I wanted a static site generator[0] in the Node.js ecosystem but without React. Or, that's what I told myself- I really just wanted to build a static site generator for fun. Really enjoyed working on it and it's feature-complete enough to run my blog and a few blogs/portfolios my friends run [2]. Learned a lot about publishing and updating NPM packages.

[0]: https://github.com/MH15/neanderthal [1]: https://matthall.codes/ [2]: https://kwest.haus/

(comment deleted)
Mamba (https://mamba.black). It's a blockchain development framework. It's like Ruby on Rails but for smart contracts development.

The story goes like this....

I was writing a book called "Hands-on Blockchain for Python Developers" in 2018. At that time, the only Pythonic blockchain development framework which supported web3.py and Vyper (a smart contract language that has syntax similar to Python) was Populus. A week after my book was published, Populus was killed by its developer.

So I decided to create one to support Vyper language.

However, I need to tell you that there is another blockchain framework that supports Vyper and web3.py which is more popular than my framework. It's Brownie-eth. If this tool existed in the first place, I wouldn't create Mamba. But hey, such is life. Now I have to raise my child, Mamba. ;)

I built Aper (https://aper.dev/) becomes I wanted to write a multiplayer game (https://redwords.paulbutler.org) for my family to play remotely and realized that what I wanted could be generalized as a state machine that is synchronized across clients. By generalizing it, I could use the same synchronization infrastructure to power other games and simple apps.
This is pretty cool. In my experience, the biggest pain point is that game clients are written in netstandard2 C# in Unity, while servers are written in not that. So bridging that gap pays huge dividends.
Thanks! It’s still very much a work in progress, but it’s been fun.
I was dissatisfied with SemVer and it turns out that most people are using it wrong. So, I created ChronVer[0] (Chronologic Versioning). There's a JS/Node plugin I made that will automatically increment your package.json version, and someone else made a Rust crate!

[0]: https://chronver.org

Interesting idea but it's probably only suited to end users of applications, not for npm libraries as the landing page proposes.

The reason sem-ver is better for libraries is that it indicates the compatibility of different version's API's. It doesn't matter when the version was released, just if it has breaking changes, or is just a patch release. Sem-ver is the simplest solution to this problem.

I built a python tool that connects to a datawarehouse or SQL scripts and produces an interactive dependency diagram as a Sankey chart, showing all objects and the data flow.

Used only on Redshift and Postgres, but it should work with other DBs with minimal changes.

The company I was working for had staging, transformation and reporting layers in their warehouses, but no nice way to illustrate the objects in documents. Support was manged offshore so the interactive diagrams helped with handover.

The redshift space is really lacking capable tools, which is why I decided to create this.

I wrote a tool to tail CloudWatch logs called saw: http://github.com/TylerBrock/saw

I was so fed up trying to debug lambdas and the other AWS offerings that a tool like this to see program output in near realtime is indispensable. I use it all the time.

Is it bug free? No

Did i write tests for it? Also no.

Is it useful? Yes.

I built Ward (https://github.com/darrenburns/ward), a Python test framework.

I have issues with the way the pytest fixture system works (parameter names matching function names), the readibility of the output, and some other things.

It started as a little learning experiment and has turned into something I've been building into what I hope will (and already is in some aspects) be a viable contender to pytest.

I want a web interface that sets up ssl certificates for me (with wildcard sub domains) and let’s me create sub domains on the fly so I can run multiple apps on my server quickly.

I don’t like nginx because managing the config files is a hassle, it’s complex, and there’s a lot of copying redundant config files over and over again.

https://github.com/garagescript/myproxy

I wrote a tool that parses a single user’s Cloudwatch metric logs as a .har file. Har files are used in the network developer pane in browsers. Doing this allowed us to visualize a customer’s journey and discover areas in the code with high latency!
I wrote a cooperative multitasking system for Turbo Pascal because I needed to be able to stream data through a series of text pipes and it was easier than faking it with a bunch of temporary files. I also wrote a series of data entry routines to allow the quick creation of database editing programs.

I haven't open sourced them because I doubt anyone would care about those old MS-DOS/TP7 tools now, and I'm not up for maintaining them.

> I haven't open sourced them because I doubt anyone would care about those old MS-DOS/TP7

Maybe not a huge audience but I bet the people needing them would be hugely grateful. I use a couple old tools that I gladly contribute ( money ) to because despite being old they work great and there is no alternative.

I built a really weirdly specific caching helper for Rails: https://gitlab.com/robotmay/chunky_cache

By default Rails issues a cache network request per call with the built-in view fragment cache helpers. It does allow multi-fetching in some situations but not when you have multiple cache calls in the same view.

I had the idea bumping around for years but finally mangled it into working and I'm oddly proud of how weird the solution is.

I built a framework for making static and semi-static websites, because I was dissatisfied with the state of accessibility and compatibility on the Web.

Out of the box, it builds something compatible with most browsers made since 1995, with and without JS, CSS, Unicode, and optionally supporting dynamic elements like commenting and private-key-backed user accounts.

It's nowhere near 0.1 yet, but I'm enjoying building it publicly.

I also built a browser plugin to remove twitter nanny nags and a censorship-resistant link sharing service for myself.
Most recently, I wrote this Docker deployment tool because I was tired of having to either set up a Kubernetes cluster or manually log into my home server to update stuff:

https://gitlab.com/stavros/harbormaster

It allows you to run Docker Compose containers from a simple git repo (no Docker registry), and to specify all the Compose apps you want to run in a straightforward YAML file. It'll take care of automatically pulling/restarting/deleting everything for you.

I love it.

This seems cool! I recently moved my homeserver and dev environment to a big docker-compose file and am loving it. Just curious, what prompted you to use multiple docker compose files over a single one?
The big advantage is that you can reuse the Compose file for other computers. For example, in my model, Home Assistant would come with a Harbormaster-compatible Compose file, so to run it you'd only need to include the Home Assistant repository URL in your config file, and you'd get automatic updates and everything without having to do anything else.
I wrote arXivist, a CLI for searching, storing, and recommending papers from preprint sites like arXiv. Haven't worked on it in a while due to school, but it fills a previously unmet space.

https://github.com/njhofmann/arXivist

I found myself frustrated with Node’s child_process library. Some great primitives but a lot of boilerplate to do simple tasks like return a Promise when a subprocess is known to have started. So I built teen_process as a wrapper with a nicer async interface, helpful defaults, and all around cleaner ways of doing subprocess management with Node. https://github.com/appium/node-teen_process

(NB: once the project was determined to be useful we moved it to the Appium org and it has seen lots of contributions from others, not just me)

A forked version of browserify-middleware which uses more than one thread: https://www.npmjs.com/package/browserify-middleware-concurre...

Solves a common problem in the enterprise world - you have a big monolith UI stack with multiple apps, and starting it in dev mode gets slower over time. Lots of companies just allow you to start one of the individual apps, or they start splitting up the stack. This speeds up starting the stack for a developer, since all the apps get "browserified" on their own thread.

all the other stuff is in my bio :)