Ask HN: CPU-based LLM for Markdown editor

I've developed a free, open-source, cross-platform desktop Markdown editor and would like to integrate an LLM into its ancillary podman container (that's used for typesetting). The editor can be used for writing novels and technical documents. A few areas where the integration may help:

* Grammar check

* Idea generation

* Rephrasing passages

* Translating short snippets

Requirements:

* Free, open source, MIT/Apache/BSD/etc. license

* Local only (no cloud)

* GPT-4 quality (or very close)

* Fallback to CPU, if GPU unavailable (or CPU-based)

* Trained on public works (no copyright material)

* Container-friendly (i.e., works with podman)

* No Python, if at all possible (not a deal-breaker)

Is there something that could be integrated at this time, or would you suggest waiting until the end of the year to see what shakes out?