So I have been using cody for a while and the other day I noticed Deepseek in the options and decided to give it a go. For my use case, it turned out to be the best I have used so far and now… it’s gone! Is it coming back. If not anyone got any tips for which model they use for Unreal Engine C++ coding?
I really like the way it laid out its responses and it seemed to keep track of my context better. I am only using it to generate some boiler plate stuff to work from but it made that process so much quicker for me.
The reason behind dropping DeepSeek V3 is as follows:
These decisions are based on analysis by our engineering team, taking into account:
Usage patterns and adoption metrics within Cody: How the DeepSeek V3 model has been used and adopted by our Cody users, and expected adoption for R1.
Cost-Benefit Considerations for Cody: Evaluation of the costs associated with making these DeepSeek models available within Cody versus the benefits they provided to you as users.
Strategic Focus on Cody Model Enhancements: A strategic focus on optimizing the range of models available within Cody, including exploring and enhancing other model options such as o3-mini and potentially others that better align with Cody’s goals.
I would suggest Sonnet 3.5 new model and is still a very good model based on benchmarks.
Hi @PriNova I appreciate the response. I have been using Sonnet 3.5 as my default choice since I got a pro account a while ago. I understand your assessment process and what you ultimately decided with regards to Deepseek. I all honesty, it was running rings around Sonnet at least in my use case. Where sonnet would flip flop back and forth between two wrong suggestions, deepseek seemed to be able to move forward and try a new path. It’s responses broke down all the steps and really effectively separated each code block within its own step, etc. I’d pay double to have coding support as effective as I was getting from the short time I was using it over what Sonnet is giving me now.
Did you try other models too to evaluate which one is better especially for C++ programming?
At least, all boils down to the correct prompting and providing additional context.
Which IDE are you using? I guess Visual Studio/JB Rider for programming in Unreal Engine directly as recommended. Or do you use Cody for C++ Blueprints?
Yeah, I have also been trying with GPT-4o and o3-mini as well as sonnet (which was my main). This is in Rider, I use VSCode for my non-games work too. There is a frustrating issue with rider where I can’t copy out of the plugin window directly in rider or I would happily show you the massive difference as I still have my history and I worked on the same problem with both Sonnet, Deepseek and 4o. @PriNova sorry didn’t reply direct.
It certainly feels semantical or maybe just context, I actually started working on getting a basic multiplayer real-time session setup going as I wanted to test integrating external audio streams from various online sources into the voice chat rooms I built. To do that I wanted to have a basic model for testing I could modify outside of the standard approaches in Unreal so i figured I could quickly roll one with Cody as it is disposable. I started working on the problem with Sonnet and it was frustrating. Sonnet would give me a block of code using non existent methods and variables on the objects I had set up for it to use. I would then explain that the functionality it was trying to utilise didn’t exist and it was just hallucinating another non-existent approach.
This was frustrating obviously as I knew what was needed, I could have coded it myself from the opensource Library I was using but I just wanted to spit out a quick version to enable me to get on with my real task. After a while I asked deepseek (my first usage ever) to look at what had been generated and explained what I was trying to do and if it could assess and advise. It immediately spotted the issues and corrected them. it’s responses are much more verbose laying out what it was going to do in a step by step list and then going through each step one-by-one in detail with the code in a separate code block and finally summarising the whole thing, listing pros and cons and suggested next steps.
Getting the whole thing done was then a breeze, especially because the responses from deepseek continued to be laid out like this with full context for its solution and an accurate stepwise set of code blocks. I started a new chat with a different problem and it behaved the same way.
After it disappeared I went back to sonnet for some tweaks and realising my prompt might be making the difference I used the same opening prompt as I did with deepseek and it just started giving me the same runaround, “fixing” the issues I identified with other made up suggestions and when I told it those were wrong, it would go back to the first wrong suggestion ad infinitum. I switched to 4o and then 3o-mini and it was much the same. Sonnet would even re-add functionality that was already there with new method names etc and created a ton of redundant code and uncalled code etc.
Obviously all of the issues I was having with Sonnet and the others are expected of the models and perhaps it is a training set date issue with regards to unreal engine 5.4 which is what I was using but honestly, they were sometimes more of a hinderance than a help whereas deepseek was fantastic.
I have been using cody for a while but never for my unreal projects as I want to be explicit about those and the amount of interdependence with the editor makes it tricky. Sonnet has been great for all my little tasks outside of game development and little pet project but it just didn’t seem to cut it with Unreal. After using Deepseek with unreal I thought I was on to a game changer for speeding up my game dev work. Sorely disappointed it’s gone.
Running models on GPU is not for free even if the model is open source. Now scale that up to 100k users on a high traffic time. Depending on the LLM provider, this may cost a lot of money.