MagneticOneOrchestrator faces error when using some models like claude 3-7 which doesn't return corrrect JSON type, JSON markdown instead

[perfogic]

Dear teams, i have some problem as belowed:

Here is example of code when I used claude 3-5:

    model_client = AnthropicChatCompletionClient(
        model="claude-3-5-sonnet-20240620", 
        api_key="",
    )
    result = await model_client.create(messages=[UserMessage(content="""
            I have data 'name: John, age: 30', please return json format
            Please output an answer in pure JSON format according to the following schema. The JSON object must be parsable as-is. DO NOT OUTPUT ANYTHING OTHER THAN JSON, AND DO NOT DEVIATE FROM THIS SCHEMA:                                                             
            {{
                "name": string,
                "age": int
            }},
            """, source="user")], json_output=True)
    print(result)

The response is correct json:

{
  name: "John",
  age: 30
}

Here is example of code when I used claude 3-7:

    model_client = AnthropicChatCompletionClient(
        model="claude-3-7-sonnet-20250219", 
        api_key="",
    )
    result = await model_client.create(messages=[UserMessage(content="""
            I have data 'name: John, age: 30', please return json format
            Please output an answer in pure JSON format according to the following schema. The JSON object must be parsable as-is. DO NOT OUTPUT ANYTHING OTHER THAN JSON, AND DO NOT DEVIATE FROM THIS SCHEMA:                                                             
            {{
                "name": string,
                "age": int
            }},
            """, source="user")], json_output=True)
    print(result)

The response is a json markdown:

\```json\n{\n    "name": "John",\n    "age": 30\n}\n\```

With claude 3-7, it will lead the magnetic one going into error since the result is not a json, you can refer on this:
https://github.com/microsoft/autogen/blob/main/python/packages/autogen-agentchat/src/autogen_agentchat/teams/_group_chat/_magentic_one/_magentic_one_orchestrator.py#L285-L418

I believe it is due to the prompt on ORCHESTRATOR_PROGRESS_LEDGER_PROMPT, and i suggest it can be changed to:

Request: {task}

Team: {team}

Answer these questions with reasoning:  
- Is the request fully satisfied? (True/False)  
- Are we in a loop, repeating actions or responses?  
- Are we making progress? (True if starting or adding value, False if stuck)  
- Who should speak next? (Select from: {names})  
- What instruction/question for this team member? (Address them directly)

Return a JSON object:  
{{  
  "is_request_satisfied": {{"reason": string, "answer": boolean}},  
  "is_in_loop": {{"reason": string, "answer": boolean}},  
  "is_progress_being_made": {{"reason": string, "answer": boolean}},  
  "next_speaker": {{"reason": string, "answer": string}},  
  "instruction_or_question": {{"reason": string, "answer": string}}
}}

I have tested and it works perfectly but we can't make sure that it will not ruin other code, so we just leave it as suggestion. What do you guys think ?

Thanks for your reading!

Coauthor with me on this: @tubackkhoa

[perfogic]

@ekzhu can you give me a check on this ? Thank you!

[ekzhu]

Can you create a PR that makes it possible to customize the prompts used in MagenticOneOrchestrator, while keeping the default prompts the same.

Another possible PR is to make use JSON mode or structured output within the orchestrator, as option.

[perfogic]

sure, let's me create it!