Just started playing w. Cody and was asked by Sourcegraph to provide feedback here - hope this is the right forum/format.
In general I am impressed, but I have some (hopefully) constructive observations.
I have a huge codebase of many TypeScript projects, several of them re-usable libraries. I want to use Cody to analyze my code, provide suggestions, and particularly to add TSDoc comments documenting the code.
My first issue isn’t with Cody’s capability - but its own knowledge of its own abilities.
Using the Cody chat VSCode extension, I asked Cody if it could read and document the code in my workspace, and the response was it could. Then I asked if it could analyze and parse the code to create a JSON output file that could be used in RAG training for other LLMs, and Cody could do that as well. Asked if it could create a README.md project doc file - no problem.
I asked how long it would take, got an estimate of 2-3 hours. Over the next 3 days I kept checking on progress, and re-assured Cody was still working on it.
Eventually I asked it to just create an empty test file - cody-tst.txt in my workspace root, and it promised to do so immediately - with no result.
After a lot of prodding from me, the cody chat finally informed me it had no ability to write to any files in my workspace - it could only respond in the chat dialog.
Which is fine - except that Cody Chat spent days reassuring me it could create those files/docs, and I spent days waiting.
So it would be great if cody chat could have more insight into its own capabilities/limitations.
Is it necessary to create a context prompt instructing Cody to be realistic & honest?
Does that make sense?
I have lots more feedback - is this the best place for it?