It finally happened! An open sourced fine-tuned model becomes the best Coding LLM, beating GPT-4! Everything you need to know!
Press enter or click to view image in full size
In August 2023, I wrote a post about SQL Coder, highlighting its ability to beat GP3–3.5 at the time. Since then the open source community kept pushing forward with stronger models.
The conclusions of that blog post, on how these coding models impact our products and developers, still stand (see link below)!
For these benchmarks we as an industry need to build in real world examples. I have seen many sources claim to hit 90+ percent but in the real world you’re lucky to hit 50 percent.
