Show HN: Open-Source Performance Metrics and Benchmarks for Machine Learning/NLP (github.com)

Recently we had to evaluate if chatbot we built for an Austrian telecommunication provider would perform better on other NLP engines than the one we had in use (a cloud-based one). We took the training data and calculated common performance metrics, confusion matrices and accuracy scores for a bunch of the blockbuster providers (IBM Watson, Google Dialogflow, Amazon Lex, Microsoft LUIS, Rasa and some more).

We published the scripts in a Github repository and a blog article with instructions: https://medium.com/@floriantreml/tutorial-benchmark-your-cha...