I'm James, past data scientist, ML engineer, and now dev advocate at Pinecone.
I've been dedicating much of the past few months to building a course on what I've found to be one of the most fascinating emerging technologies in AI and ML, Semantic Search:
The course is completely free, it includes written and visual explanations, code implementations, and video walkthroughs. It focuses on the NLP side of semantic search and the first seven chapters cover:
1. Dense Vectors
2. Sentence Embeddings and Transformers
3. Training Sentence Transformers with Softmax Loss
4. Training Sentence Transformers with MNR Loss
5. Multilingual Sentence Transformers
6. Question Answering
7. Unsupervised Training for Sentence Transformers
There is more coming soon and I'll be around to answer any questions, let me know what you think.
What skills would you say are nessecary prerequisites for understanding this material?
I'm a writer, and in the past I have had success making poems and shit via messing around with various NLP tools like speech to text software and Google Books ngram viewer, but I've run into situations where I wish I could do something that isn't supported by the application.
Could this course be a good place to start? If not do you have any recommendations? I don't really know how to code but am comfortable using a command line interface.
Ideally I think it's useful to be familiar with Python and some NLP. Parts of the course can get reasonably in-depth, but we have tried to simplify every as much as possible and I think much of it can be followed without too much prerequisite knowledge.
That being said, as a writer it sounds like you'd be most interested in generative models that generate new text, we do cover that a little in the Question Answering chapter, but otherwise, not so much. Although I don't know your ideas or plans so I can't say for sure what is good/bad for you to focus on.
I do think you should read into transformer models like BERT and GPT, there is a very good free course from HuggingFace who are the 'de-facto framework' for NLP at the moment, that could be useful for you too:
https://huggingface.co/course/chapter1/1
(chapter one will introduce everything too, definitely go through that)
With HuggingFace, you can put together a simple T5 or GPT-2 text generation script with a few lines of code
If you want to just try something, you can look at the latest tranformer models like GPT and try fine-tuning them for poems.
You can see a tutorial for content creation here https://youtu.be/d_xRYyy2LFM
I am browsing the web in search for NLP resources as I consider a pivot into this new area (I already have a few years in data science and image processing) and finding this post on HN is a bless! I will be going through the course for sure!
But I want to mention for everyone taking it: please also keep in mind "dumb" logical operators like OR, AND, NOT, and quotes that skip the NLP and use the exact text.
What frustrates me with Google Search is that it won't show me what I know precisely how to find. This is why I use Google less and go to specialized sites directly (Stack Exchange, Wikipedia, PubMed...).
You can split the document by paragraph or sentence. Search results will then match a sentence in a document. You display with a link to the document with a reference to the span.
This course teaches how to use NLP to turn text data into dense vectors while capturing their meaning. Then you take those vectors and put them in a vector database — like Pinecone.io or Google Matching Engine — so you could do on-demand vector search. And voilà, you have a semantic search application!
30 comments
[ 2.4 ms ] story [ 86.3 ms ] threadI'm James, past data scientist, ML engineer, and now dev advocate at Pinecone.
I've been dedicating much of the past few months to building a course on what I've found to be one of the most fascinating emerging technologies in AI and ML, Semantic Search:
https://www.pinecone.io/learn/nlp
The course is completely free, it includes written and visual explanations, code implementations, and video walkthroughs. It focuses on the NLP side of semantic search and the first seven chapters cover:
1. Dense Vectors
2. Sentence Embeddings and Transformers
3. Training Sentence Transformers with Softmax Loss
4. Training Sentence Transformers with MNR Loss
5. Multilingual Sentence Transformers
6. Question Answering
7. Unsupervised Training for Sentence Transformers
There is more coming soon and I'll be around to answer any questions, let me know what you think.
Thanks all!
I'm a writer, and in the past I have had success making poems and shit via messing around with various NLP tools like speech to text software and Google Books ngram viewer, but I've run into situations where I wish I could do something that isn't supported by the application.
Could this course be a good place to start? If not do you have any recommendations? I don't really know how to code but am comfortable using a command line interface.
That being said, as a writer it sounds like you'd be most interested in generative models that generate new text, we do cover that a little in the Question Answering chapter, but otherwise, not so much. Although I don't know your ideas or plans so I can't say for sure what is good/bad for you to focus on.
I do think you should read into transformer models like BERT and GPT, there is a very good free course from HuggingFace who are the 'de-facto framework' for NLP at the moment, that could be useful for you too: https://huggingface.co/course/chapter1/1 (chapter one will introduce everything too, definitely go through that)
With HuggingFace, you can put together a simple T5 or GPT-2 text generation script with a few lines of code
What tools did you use to build it?
But I want to mention for everyone taking it: please also keep in mind "dumb" logical operators like OR, AND, NOT, and quotes that skip the NLP and use the exact text.
What frustrates me with Google Search is that it won't show me what I know precisely how to find. This is why I use Google less and go to specialized sites directly (Stack Exchange, Wikipedia, PubMed...).
You probably benefited from it several times already, but haven't realized, since it's subtle and works behind the scenes.
I have used search long enough to remember having only literal search, and I never had an issue with it.
[0]: https://www.gkogan.co/blog/pinecone/
How does your course relate to the vector search approach discussed here? https://cloud.google.com/blog/topics/developers-practitioner...
This course teaches how to use NLP to turn text data into dense vectors while capturing their meaning. Then you take those vectors and put them in a vector database — like Pinecone.io or Google Matching Engine — so you could do on-demand vector search. And voilà, you have a semantic search application!