HN Explorer

gerenuk

↗ HN profile [ 81.1 ms ] full profile
Karma
0
Created
()
Submissions
0
  1. Ask HN: Mercury.com Alternatives?

    Hi there, We just received an email from Mercury that they are closing down operations from August 22, 2024 because we are resident of the country which is blacklisted. We've been using Mercury from last 4 years and…

    4 points 1
  2. Ask HN: How do you plan to work on the feature/code?

    Hi, While building the application feature, how do you plan to work on the specific feature? Like creating the user-stories, gathering requirements? I am looking to improve my thought-process and cover most of the…

    1 points 0
  3. Ask HN: How does your QA process look like?

    Hi, We're currently trying to build a proper testing process. Just having some unit tests and running the app a couple of times before shipping to production doesn't cut it anymore :) How do you organize the testing…

    1 points 0
  4. Ask HN: Identifying duplicate data from a large dataset?

    Hi, We have a dataset around 150 million URLs and other meta data in ElasticSearch and looking for an efficient way to identify the duplicate URLs/titles from our dataset. Used ElasticSearch term aggregation but it…

    2 points 3
  5. Ask HN: Storage DB for backlinks?

    Hey everyone, Looking for suggestion and recommendations, what would be your ideal choice if you are about to build a backlinks database and running later on PageRank algorithm on it. Our use case is to improve the…

    2 points 0
  6. Ask HN: Unsupervised text classification?

    Hi everyone, We are currently using text classification model trained on a labelled data set consisting of almost 500 hierarchical categories , For example a news article is labelled as News/Business News , now what we…

    1 points 0
  7. Ask HN: Unsupervised text classification?

    Hi everyone, We are currently using text classification model trained on a labelled data set consisting of almost 500 hierarchical categories , For example a news article is labelled as News/Business News , now what we…

    1 points 0
  8. Ask HN: What do you use for monitoring supervisord processes?

    A common scenario is workers/crawlers are running on more than 100+ machines with supervisord with different configuration? What would you suggest for monitoring and knowing what kind of workers are running on each…

    1 points 0
  9. Ask HN: What approach would you suggest for Text classification?

    Hey everyone! We are trying to solve a problem where we need to classify the articles into the right categories. Currently, using a FastText to train a model with 100,000 articles categorized into 600 categories. The…

    1 points 1
  10. Ask HN: Storing millions and billions of URLs?

    Hello Everyone! Currently, using ElasticSearch for storing the meta data and other raw data information but it is a very small scale around 500,000 domains. I have been tasked to scale it to 20-40 million domains and…

    12 points 10
  11. Ask HN: How do you manage to keep improving your ML models accuracy?

    Hello everyone, What do you guys do to improve training data set labels to get better success rate? Are you using some framework or doing it manually after some time? We have been working on topic modeling stuff, and…

    3 points 1
  12. Ask HN: Storage for Raw HTML? ElasticSearch or Something Else?

    Hi, I am working on a project where we need to store the response of a web page and have later on do the data processing such as NLP tasks, topic modeling, sentiment analysis etc. The average content/documents we are…

    2 points 1
  13. Golang or Python for the large scale distributed crawler?

    To deal better with concurrency which one would you opt for, as for the task queues, I have checked golang has "Machinery" library and for python its celery (we are currently using this). The bottleneck we are seeing is…

    1 points 1