HN Explorer

Open Source LinikedIn Scraper

13 points by devildani ↗ HN
Hi All,

I am planning to build a LinkedIn scraper, which would scrape LinkedIn profiles, company pages, groups, searches (sales nav and normal searches), likes, comments, etc, for free. I already a set piece of codebase that I have built out. I wanted to know if anyone would be interested in using an open-source LinkedIn scraper? Will this be good option?

21 comments

[ 3.7 ms ] story [ 67.7 ms ] thread
greenpinia
As if there was not enough recruiter spam mail.
altdataseller
Ironically, sometimes I wish that LinkedIn vs HiQ verdict went the other way and made scraping LinkedIn illegal
satvikpendem
The judge ruled towards LinkedIn in the latest ruling of that case, which I suppose most don't know about due to it being an appeal.
devildani
I did read about this. Not most know about this.
altdataseller
I’m not a lawyer but does that mean scraping LinkedIn is indeed illegal? Or is it still yet to be determined?
devildani
TLDR: The short answer is: it is legal to scrape LinkedIn properly. However, scraping LinkedIn a certain way could be illegal.

Scraping data from public LinkedIn profiles? Ok. Scraping data from LinkedIn profiles with fake logged-in accounts? Not ok.

ldjkfkdsjnv
Use an LLM to do it
devildani
I am trying this out, I haven't really written any code to build the application. Generating codes via LLMs.
noashavit
There are services that do that, might not be open source but with free tiers like common room.

LinkedIn might not like you scraping the site as well, esp if you use sale navigator to get access to deeper info. Keep that in mind

devildani
Yes, I am aware about this and I won't be using my account for extraction. Users will put their session cookies for data extraction.
paulcole
Great idea. I love a tool that unexpectedly gets my account banned.
xorl
Facts, they will ban your LinkedIn account for scraping. Good luck!
paulcole
I work with someone in Spain and asked her to log into my Recruiter Lite account to review candidates. LinkedIn gave a pretty stern warning about sharing the account with anyone and that I’d be banned if it happened again.

I’m guessing that the same thing would happen if you started letting a scraper act on your behalf.

mdbmdb
I would be interested. I think LinkedIn aggressively tries to shut down things like you are trying to build though
devildani
Good to know. Can you share your email id? I will ping you as soon as I have the first version of application ready.
whodev
This is why I private my social media accounts. There is already enough spam and I don't feel it needs to be any easier.
vrepo54
I would be interested. How do you plan on dealing with IP address bans?