How well do Large Language Models perform in arithmetic tasks? (arxiv.org)

I think if you combine Worlfram alpha with GPT-4, it could do a pretty good job! Have you tried anything around this?