For anyone interested in related research, I used https://mirrorthink.ai to find some background on the state-of-the-art.

(disclaimer: this is AI generated, but grounded on contents of papers, with real references, so I'd say it is still constructive)

The state-of-the-art in code generation has seen significant advancements with the deployment of large language models (LLMs) in various code authoring tools. One such example is the study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT [1], which evaluates the code quality of these AI-assisted code generation tools. The study reveals that ChatGPT generates correct code 65.2% of the time, while GitHub Copilot and Amazon CodeWhisperer achieve 46.3% and 31.1% correctness, respectively. These results indicate that LLMs have made substantial progress in generating high-quality code, but there is still room for improvement.

Other research in the field has explored various techniques to enhance code generation and assistance. For instance, RepoCoder [2] focuses on repository-level code completion by integrating code generation and retrieval models in an iterative paradigm. This approach considers the repository-level context, including customized information such as API definitions and identifier names, to improve code completion suggestions. Serenity [3] leverages library-based Python code analysis for code completion and automated machine learning. The authors explore the potential of data flow analysis produced by Serenity to improve code completion when combined with neural models.

In addition to these advancements, the field has seen progress in incorporating contextual information into code completion models. The paper on enriching source code with contextual data [4] investigates the impact of incorporating contextual information on the performance of code completion models. The authors conduct an empirical study to analyze the effectiveness of this approach. These achievements, along with the advancements in LLMs, contribute to the ongoing progress in code generation and assistance. As the field continues to evolve, it is expected that AI-assisted tools will become increasingly sophisticated and effective in assisting developers with various aspects of the software development process.

[1] Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT - 2023: https://arxiv.org/abs/2304.10778

[2] RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation - 2023: https://arxiv.org/abs/2303.12570

[3] Serenity: Library Based Python Code Analysis for Code Completion and Automated Machine Learning - 2023: https://arxiv.org/abs/2301.05108

[4] Enriching Source Code with Contextual Data for Code Completion Models: An Empirical Study - 2023: https://arxiv.org/abs/2304.12269