Overall Performance

Forum
,
1

Wonder if anyone else is having the following issues or has any thoughts on them.

Typically I have one chat going and it get’s quite long. Usually this is the case because I am doing something that it doesn’t make sense to start fresh again. I"m sure there is some impacts to performance because of this as well as impacts to context length being either full all the time or something else.

But this is what I’m seeing overall:

  • Copy code then click into chat box, freezes cody for sometimes minutes. Then cursor will appear and I can then paste. Overall the copy and paste into chat box is not performant at all.
  • The entire cody window goes grey and nothing shows. I’ve restarted VScode and everything else loads, just not the window with Cody. Looks like it does load minutes later.
  • Clicking to load a past chat takes a long time. Again, sometimes my chats get really long but if the chats are stored locally (I would assume), why does it take so long to load?
  • I’ve had several times in the past few days the response hang after submit. Sometimes it never responds. Other time it takes minutes. Obviously it depends on the model but I wouldn’t expect that using o4 through cody would be much less performant than using the chatgpt UI with o4.
  • Responses are generally slow enough that I will send a chat, then go to a different window, like Chrome, and do something. Then head back to VScode to see if it’s done responding. And it will have frozen sometimes. Other times it’s fine and the response is sitting there complete.

Overall, the experience would be much improved if overall performance and speed were improved. I may try to remove some other extensions like docker and see if helps.

Anyone else seeing similar issues?

Edit: Other VScode functionality and UI elements are not frozen or affected at all. Just the Cody window.

2

What version of Cody are you using? We have improved the chat history loading and other improvements with the latest update.

But I can say for sure, that a too long conversation will slow down the performance, because it will exceed the token limit and the LLM starts to hallucinate strongly.

3

v.1.84.0

Yep, totally aware of a long chat is likely contributing to this as well as the context length.

Does cody send the entire chat history, the history that will fill up the total context length, or error out when you reach the limit?

Just chatting with it I don’t get any context length warning. Including super long files or many files, it does give a warning that context length is exceeded and it won’t include the context. But for just regular chatting or pasting code into the chat box, I’ve not gotten that same error.

4

I’ve done a few things. I’ve disabled extensions that I’m not actively using. Didn’t seem to do much.

I’ve updated to the beta release. Seems to make some things snappier, however, still frozen.

I’ve had my last chat summarize what it’s doing and then used that to start a new chat. Which has significantly increased the speed of everything.

I’d attribute most all the issues to the super long chat. Could be good to put some warnings or graceful handling in place to reduce the affect of a long chat where possible. Not even in the LLM, most how the extension handles it.

If anything changes I’ll update this thread.

1 Like