Mar 7, 2024

LLMs predict next edit operation instead of next token?

On Copilot in IntelliJ.

What if LLMs could be trained on predicting the next edit operation instead of the next token? By edit operation, I mean insert word, delete word or change order of words. This way, an LLM could create a text with its own "TODO" or "reference needed" markers, which then later get refined by itself.

Text and source code is best written top-down and extended and refined over and over again. No book or article on anything is written from start to finish and done. And yet, this is the behavior an LLM currently implies. LLMs don't see the intermediate human struggle and changes until the text is as it is.

Pair programming teaches you skills, which cannot be found in any book. Clearly, there are currently more books and web pages than recorded pair programming sessions. But maybe there are already enough of them or some could be donated from IntelliJ collaborative users working on open-source software?

I want a pair-programming-trained LLM, which suggests edit-operations.

No comments:

Post a Comment