Note that if you don't have access to a server with a GPU, don't bother with Haystack since most of their USP functionality requires a GPU -
and even that doesn't mean the queries will execute fast - expect double digit seconds per query. YMMV
Fair enough. That's the inherent 'feature' of the Transformer models, though. We have a series of articles on optimization here https://medium.com/deepset-ai/accelerate-your-qa-system-with... - but also, no, double digits seconds is not necessarily the best result :) Our users have done much better <g>
5 comments
[ 2.6 ms ] story [ 22.5 ms ] thread