Ask HN: How Can I Get into NLP (Natural Language Processing)?

I have no particular expertise on the topic, but just in case you missed it, there is this Quora question: https://www.quora.com/How-do-I-learn-Natural-Language-Proces...

It points to NLTK as the framework of choice, and has links to a couple MOOCs and tutorials.

There is a great set of lectures by Dan Jurafsky and Chris Manning: https://www.youtube.com/watch?v=nfoudtpBV68&list=PL6397E4B26...

It would be helpful to have some background in Machine Learning. For a good introductory course with a mix of mathematical background, see https://see.stanford.edu/Course/CS229

NLP in the more modern systems is backed by deep neural nets. Here's a course on NLP using deep learning: https://www.youtube.com/playlist?list=PLIiVRB6G_w0i-uOoS6cDh...

NLTK[0][1] (Natural Language Toolkit) was fantastic as an initial resource for me. Because it's a self contained book and library, I found it to have a very smooth learning curve. There is some introductory programming stuff that you can very easily just skip in the beginning so don't let that turn you off initially.

[0] http://nltk.org [1] http://nltk.org/book

If you're interested in deep learning for nlp... I suggest at least some familiarity with these papers. It sorta depends on what task you want to use it for.

https://github.com/andrewt3000/dl4nlp

We've just started adding lessons on this topic on egghead.io [0]

[0] https://egghead.io/lessons/node-js-break-up-language-strings...

I have found these sources useful for learning and prototyping NLP:

http://garysieling.com/blog/entity-recognition-with-scala-an...

http://tika.apache.org

NLTK is always a good starting point: http://www.nltk.org

I also wrote a 3-part article leveraging OpenNLP with Clojure:

http://edeferia.blogspot.com/2015/03/from-natural-language-t...

If you're interesting in applying NLP without necessarily having theoretical background, wit.ai offers some really impressive features.

Course also offers a good course:

https://www.coursera.org/learn/natural-language-processing

Take a look at Stanford CoreNLP: http://stanfordnlp.github.io/CoreNLP/

It's relatively fast (after model load time) and quite feature-rich.

You're probably looking for something a bit more sophisticated than what I'm about to mention, but if you don't need anything too sophisticated (that is, if you can significantly limit the domain of the speech you need to be able to understand), you could do something like what I did for "the computer" on my star trek-like space sim Space Nerds In Space: http://hackaday.com/2016/06/08/talking-star-trek/

I used pocketsphinx (trained with specially limited vocab) for speech to text, my own home grown Zork-esque parser for "understanding" the text and generating responses, and pico2wav for text to speech for the responses. That's described in a bit more detail here: https://scaryreasoner.wordpress.com/2016/05/14/speech-recogn...

For initial learning, I would second NLTK with: http://www.nltk.org

You can also checkout https://github.com/vseloved/cl-nlp. It is an NLP toolkit in Common Lisp. Vsevolod the project owner is a great guy to work with. I had contributed with some minor bug fixes, tests, documentation more than a year back, hence the mention of Vsevolod.

You could also think on the alternative lines of contributing to an open source project in NLP and building an application on top of it. Talking to any such project owner for expected sample apps might help, as they can go into that project gallery and you get to level up your skills. Hope this helps.

start a project with someone. write your own data scraper, and implement a model.

My recommendations, based on online courses and YouTube playlists I've taken:

- Coursera's old NLP course by Michael Collins, Columbia Univ. More of theory and concepts. It's discontinued now on coursera but the material is available at academictorrents. [1]

- NLP with Python and NLTK videos by sentdex [2]. Mostly programming, but with useful nuggets of concepts introduced here and there.

[1]: http://academictorrents.com/details/f99e7184fca947ee8f779016...

[2]: https://www.youtube.com/playlist?list=PLQVvvaa0QuDf2JswnfiGk...

Jurafaki and Martins Natural language processing is a great book covering a great deal pf topics in nlp.

https://spacy.io/ is one of the best library for NLP if you are using python

Check out Stanford's NLP libraries. We've been using those in production for years now. The documentation around it is not great, but the tools work well.

I did a one week ml stunt last year: https://medium.com/learning-new-stuff/machine-learning-in-a-...

I'd recommend starting with the Kaggle Bag of Words tutorial.

My suggestion is, in addition to using the videos and courses for background knowledge, to take up and work on a (non-homework) project, to truly explore the area.

For eg., Betty [1] is quite an interesting project with both real-life use and practical NLP considerations, and is looking for new maintainers. (I'm not affiliated, just interested in NLP myself and have been itching to get into betty for some time.)

If you like thinking about game design, there's also the option of Interactive Fiction [2], NLP-involving ones are called parser-based fictions I believe. A recent FLOSS podcast episode with folks from the IF Tech Foundation was pretty interesting and illuminating regarding this area.

[1] https://github.com/pickhardt/betty [2] http://iftechfoundation.org/frequently-asked-questions/

NLP is a huge topic, and the choice of materials pretty much depends on what you'd like to focus on. In my experience nothing beats a good textbook, especially if you do the exercises.

The classic NLP textbook is

* Jurafsky, Martin: "Speech and Language Processing" (https://web.stanford.edu/~jurafsky/slp3/) -- already mentioned here: a very solid overview textbook to give you an idea about the field;

Should you be interested in statistical NLP (even if it probably isn't as sexy as it used to be), the classic there is:

* Manning, Schütze: "Foundations of Statistical Natural Language Processing" (http://nlp.stanford.edu/fsnlp/).

Prolog and Natural-Language Analysis[1] is great from both theoretical and practical standpoints.

[1] http://www.mtome.com/Publications/PNLA/prolog-digital.pdf

The important advances in modern NLP are being made using neural networks. Specifically, recurrent neural networks (and sometimes convolutional neural networks) are consistently being demonstrated to replace decades of heuristics using representations learned entirely from data (identical advances are being made in computer vision). This is important because it's essentially impossible to enumerate all the subtleties inherent in language(s), and bespoke models will be naturally fragile as a result.

Some of the more interesting examples of neural network & NN-inspired approaches to NLP include: Question Answering [1, 2]

Sentiment Analysis [3]

Machine Translation [4]

Text Classification [5]

Natural Language Summarization [6]

etc. etc.

Some of the key players in the field include Richard Socher, Chris Manning, Quoc Le, Tomas Mikolov, Andrej Karpathy, David Blei, Jason Weston, Ronan Collobert, etc. This is a highly incomplete list, but if you use Google Scholar or visit their respective homepages you'll find loads of great material.

If (quite reasonably) you find the academic literature a bit imposing, I encourage you to explore the fundamentals by working through:

Andrew Ng's Coursera Machine Learning Course https://www.coursera.org/learn/machine-learning

Google's Deep Learning (Tensorflow) Course @ Udacity https://www.udacity.com/course/deep-learning--ud730

Chapters 1-5 of Introduction to Statistical Learning http://www.stat.berkeley.edu/~rabbee/s154/ISLR_First_Printin...

Part II of Yoshua Bengio's Deep Learning Book http://www.deeplearningbook.org/

& finally Richard Socher's excellent NLP course @ Stanford http://cs224d.stanford.edu/

https://www.youtube.com/watch?v=sU_Yu_USrNc&list=PL_6hBtWGKk... (all lectures)

It feels like a lot of material, and it is, but it's important to understand the foundation to do meaningful, flexible work. If you spend a week bringing a black box stack online and the machine does not produce a satisfyingly magical result you'll find yourself grasping at straws in the absence of strong fundamentals.

Some additional resources that are fantastic include Colah's blog (http://colah.github.io/), specifically the pieces on Representations & LSTMs (but all of it really), as well as Andrej Karpathy's piece on 'The Unreasonable Effectiveness of RNN's (http://karpathy.github.io/2015/05/21/rnn-effectiveness/).

Some other touchstones include:

http://www.wildml.com/2015/11/understanding-convolutional-ne...

http://www.wildml.com/2016/08/rnns-in-tensorflow-a-practical...

https://www.you...

Please read through the Handbook of NLP for a nice overview.

https://karczmarczuk.users.greyc.fr/TEACH/TAL/Doc/Handbook%2...

Hi, some tools seem work fine with English, so is there any good NLP tool for Chinese? Hope for some advice, thanks ahead.

I'm going through the stand ford cs224D videos, only done 3 videos and they are very theory focused, lots of math equations. Any one know other good materials on NLP using neural networks?

Watch the videos made by Jurafsky (Stanford) as a starting point.

They are quick. This will give you an overview of classical NLP.

From there, you can dig more where you want.

Check out this curated list of resources dedicated to Natural Language Processing on GitHub: https://github.com/keonkim/awesome-nlp. Also this is a great blog for understanding the business and high level aspects of the technology: https://lekta.ai/blog/