O3 mini's source code output is not formatted as code

Forum
,
1

Code output from o3-mini often looks like regular text.

2

Yeah, o3-mini sends only raw text with its own formatting. You need to use the chat pre-instruction as a system prompt to instruct your own style of output you want.

3

This is an implementation detail that should not concern the user. Every o3-mini user faces this problem; I really think Sourcegraph should be taking care of this.

4

You mean that the Markdown formatting is sometimes interleaved with raw text and jumps back to Markdown even it should completely formatted as such?

5

For example:

For instance, if you already have external classes defined, you could do:

File: src/main/kotlin/com/foo/bar.kt

package com.foo.bar

sealed class PostIdOrUuid

// Wrapper for external PostId defined elsewhere data class PostIdWrapper(val postId: ExistingPostId) : PostIdOrUuid()

// Wrapper for external UUID (or a custom UUID type) defined elsewhere data class UUIDWrapper(val uuid: ExistingUUID) : PostIdOrUuid()

This way you can incorporate types defined elsewhere while keeping all direct subclasses in the same file.
…

I’ve changed the package and file names, but this is basically what I got.

6

And this only happen with o3-mini or all models? Because we have syntax highlighting for Kotlin code via the highlight.js activated.

7

o3-mini, I think. I will pay more attention to see if it happens with the others too.

8

If this is only o3-mini related, then it is not an implementation detail but indeed the output of o3-mini it how it was trained. I personally don’t like the format how o3-mini responds and haven’t such issue with other models.

1 Like
9

You could use a cheaper model to detect those code blocks and fix their formatting…

10

Are your engineers aware of o3’s Formatting re-enabled setting?

Markdown formatting: Starting with o1-2024-12-17, reasoning models in the API will avoid generating responses with markdown formatting. To signal to the model when you do want markdown formatting in the response, include the string Formatting re-enabled on the first line of your developer message.

https://platform.openai.com/docs/guides/reasoning-best-practices#how-to-prompt-reasoning-models-effectively

1 Like