Show HN: Tech jobs on the command line (github.com)
Command Jobs, a terminal-based job finder and application tracker designed specifically for me, and maybe also software engineers
The app scrapes web listings, saves them, processes them with GPT[1], then based on your resume and job preferences, it gives you the best matches
This is my first open source project, and also my first public project using LLMs
I'm really excited to share this with the HN community and would love to hear your feedback, suggestions, and any features you'd like to see in the future. Please check it out and let me know what you think!
https://github.com/nicobrenner/commandjobs
I'm currently working on adding more sources for job listings, periodic scraping via cronjobs, and alerts for new matches
Looking forward to your questions, feedback and requests. Thank you
[1]: Bring Your Own (OpenAPI) Key
60 comments
[ 4.0 ms ] story [ 132 ms ] threadI’m going one step at a time, the first tedious task I wanted to automate was searching and second-pass filtering
Most job listing sites have tools to filter by many different factors, including role, location, remote, salary range and seniority level, but usually the filters are either too strict or too permissive and I still end up having to read through tens or hundreds of listings to select just a few
Command Jobs gives me better matches faster by using GPT to match my resume and job preferences with the listings
Not defending the exclusion of salary, but I think it would be pretty naive to dismiss the fact that a profound amount of people have been forced into unemployment and can negotiate if it gets there.
I think folks know there's downward pressure, but I'd rather not anyone have to waste time engaging with employers offering minimum wage for software engineers.
I know of two options:
1) the simplest, is to ask GPT to tell me an estimate for a certain role, at a certain company, at a certain location/area - with the right prompt, and if it can find additional info online, it gives pretty decent guesses
2) if you are willing to code it in multiple checks and steps, you can have GPT help you navigate and scrape listings. For example, the listings in Ask HN usually have very little detail, and no ranges, but a lot of them have links to somewhere else with detailed info, including ranges - or they don't have links, but you can still find the info and ranges their careers page or some other platform
In the second case there's a lot of trial and error, for which GPT could be helpful in navigating with less supervision
Would love to get some tips or ideas on how to allow the user to manage the prompt, to make changes and test it (without braking it), from the command line
You can also go further by writing code that prompts the user for these values, one by one, and stores them in a config file. But for a CLI application, you might reasonably wonder if the juice is worth the squeeze there. CLI users tend to be comfortable editing a config file to hold a secret.
Edit: or if some of those values are actually optional, then you would maybe exclude them from this check.
Also pending is getting some tests in there, especially to test consistent LLM/GPT-dependent behavior
Totally right about the difference in expectations from a cli user pov. I just want to make sure this tool is helpful to more people than just myself
Let me know if you have any issues getting it up and running
I built out pretty much the exact same thing for JSON Resume (far uglier) but uses vector embeddings to rank relevant jobs -> https://registry.jsonresume.org/thomasdavis/jobs
My prompt when running it through GPT-3.5 sucks, doesn't always get location and salary. GPT-4 is way better but $$$$.
I've seen about 3-4 projects doing this with HN WhoIsHiring projects now, I wonder if we should all just share an API (of GPT parsed HN job posts) and pool some $$$. (In my version, I also created a JSON Schema that represents a Job Description)
scripts: https://github.com/jsonresume/jsonresume.org/tree/master/app... api code: https://github.com/jsonresume/jsonresume.org/blob/master/app...
Did you do any sort of benchmark between embeddings and direct inference?
How did you tune the distance for relevance, how did you decide on chunking and how many results would you include?
I want to try filtering using the distance of embeddings RAG style, but I wonder about how that changes the results
The current method the app uses feels pretty good, it does get you the best matches, and sometimes it also includes a few “extra” suggestions that typical filters would have left out, but that end up being potentially interesting matches
Do you mean that you pass the resume and job posting to the initial GPT prompt? and rely on the returned "fit_for_resume" property? (I'm curious if you tried using OpenAI function outputs too)
If so I didn't try that. And thinking about it, I like your style. (There's no reason why you couldn't do both anyway)
Another question; Did you try asking for a "fit_rating"?
Here is the prompt/call I used for GPT'ing the HN posts -> https://github.com/jsonresume/jsonresume.org/blob/master/app...
As for the distance of the vector similarity search. I am just using Supabase's pg_vector plugin support, with 3704 dimensions using the default search (cosine).
Thank you so much for the insightful comments and pointers
If you were expanding the functionality/features of LLM-assisted job searching, what do you think are the most useful challenges to tackle?
I have some wild ideas about distributed job searching systems running on people’s command lines hahaha
I've got a few that I will chime in later with. A simple one for now, is put a couple buttons next to the job listings in your UI.
Generate cover letter - Make a cover letter based off your resume and the job description
Generate tailored resume - Create a permutation of your resume to promote things relevant to that job description
Haven’t been able to get it down though. I’ve tried a few prompts in those lines, but I never really like the outputs
The iteration is still very helpful in fleshing out a good personalized resume/cover letter, but I feel the app needs something a bit more automated/straightforward for the users
Any ideas on how to structure good prompts or a workflow to make it work well in terms of user experience?
I definitely want to add scrapers for more sources
Haven't tried workday, is it popular for tech jobs (ie. with people that use the command line)?
Do you have a sample url with listings you could share?
https://workday.wd5.myworkdayjobs.com/Workday https://shipt.wd1.myworkdayjobs.com/en-US/Shipt_External
It doesn't seem like there's a global listing across myworkdayjobs.com sites - each company using it has its own sub-domain. Which sort of makes sense - Workday is an internal HR/employee portal tool licensed to companies.
It should be pretty straightforward to create a scraper for one of those portals
If you feel like taking a crack at it, checkout the code of the Ask HN who's hiring scraper here: https://github.com/nicobrenner/commandjobs/blob/47b5c89402a3...
I added a comment under the Add Sources issue, to support Workday's career portals: https://github.com/nicobrenner/commandjobs/issues/23#issueco...
Might try scraping it too ;)
Btw, random observation that surprised me when scraping Ask HN, most jobs (100+) are posted the first day of the month, but there are about 15-25 additional jobs posted everyday. This means it’s worth it as a user to get those extra daily listings, but it’s a pain to search through them
Feel free to scrape 200kjobs: https://200kjobs.com/feed
As an aggregator it’s true that finding salary ranges is quite the challenge
As an applicant, I’ve settled for just only applying to the positions that have a range
If Im really curious about a company without ranges in their listings, GPT/Google sometimes give decent estimates
But also, after seeing 100+ listings, I’ve noticed that for the positions I’m looking for, there’s a pretty standard range, which doesn’t vary too much, unless it’s a FAANG or super hot company
In any case, using Command Jobs I only get 5-20 job matches, which are also pretty easy to check manually, and it’s not too annoying, given that usually the matches are very interesting and also it’s not 100s of jobs to r
First search for the "Who is hiring post?": https://hn.algolia.com/api/v1/search_by_date?query=hiring&re...
Note the ID of the first (latest) post and then get the comments: https://hn.algolia.com/api/v1/items/39562986
hnhiring.com looks super cool, I'm curious if you have any stats or infographics about the data
have you played with the data and LLMs to try to get some insights?
I only use regexes to extract info. Haven't tried LLMs but that could be useful
What open source LLM did you use? Anything that would run on a CPU of a 5 year old MacBook Air (which I’m using as my home server)
Do you have the code for your project? Or maybe some features/ideas that you could share?
Thank you
At the moment I don't have it open sourced due to it being part of a larger project that I'm working on that contains tailwindui licensed components.
A cool feature that I'm working on is creating a firefox plugin so you can save/index job postings from other sites and extract out meta information via an LLM. Very similar to this chrome plugin. https://chromewebstore.google.com/detail/huntr-job-search-tr...
Thank you for the perspective on running local LLMs
Do you think that saas or local LLMs will be the most popular in the next 5 years?
Are you using the API on some project?
My first year of college, the school had a Telnet-based application to request classes, it would be amazing to build something like that
Any pointers on how to build the npx installer?
So essentially you should be able to create an ssh application with just a shell script
I recently wrote a minimal wrapper for that: https://github.com/mfiro/hnconnector
Super cool wrapper, love the methods that make it super easy to get the content
What would be the best way to for example, get the latest Ask HN: who's hiring page and all it's top level comments?
Thank you
ask_stories = client.get_askstories()
then you have the ids and you can sort them reversed to catch the latest stories.
client.get_item(<id>) will give you the story's details, in which you'll find item['kids'], which are the IDs for the top-level comments. You can then loop through these comment IDs to get more info on each comment.
Another option is to use Algolia's API for HN. But that's not yet supported in my wrapper.
It would be easier to use the JSON feed for a couple of reasons: - the data is segregated into types of things that you otherwise have to back out from the HTML/CSS - you don't need logic for parsing multiple pages
Have a look at code I wrote on this: https://github.com/hughdbrown/who-is-hiring
The code to get a month of Who Is Hiring posts just hits `https://hn.algolia.com/api/v1/items/{article_id}` with a GET request for a particular article_id. Finding the monthly article_ids is the only manual maintenance.
The code is split into multiple commandline tools that extract JSON from HN, preprocess the JSON, and filter on what I was interested in (rust job postings). You should be able to replace downstream parts like preprocessing and filtering.
See if any of this is helpful.
Yes, scraping the html directly was relatively easy
Will try Algolia and the api when doing a cron job for it