[BUG] Message Ordering Issue with Turbo Streams and Streaming Responses

[fluxsaas]

Basic checks

• I searched existing issues - this hasn't been reported
• I can reproduce this consistently
• This is a RubyLLM bug, not my application code

What's broken?

Problem

Assistant messages appear above user messages in real-time chat due to chat.ask() creating both messages within milliseconds, causing ActionCable broadcast race conditions.

Steps to Reproduce

1. Follow ruby_llm docs setup with Turbo Streams
2. Call chat.ask(user_content)
3. Observe message order in UI

Expected vs Actual Behavior

❌ Actual:   Assistant: [response]
             User: [question]

✅ Expected: User: [question] 
             Assistant: [response]

Evidence

Both messages created simultaneously:

# Log output
Message created: user - 7d50d730... - 2025-07-13 20:24:46 +0200
Message created: assistant - 0df903f1... - 2025-07-13 20:24:46 +0200

Workaround

Delay assistant message broadcasts:

after_create_commit do
  if role == 'assistant'
    broadcast_append_later_to(..., wait: 0.1.seconds)
  else
    broadcast_append_to(...)
  end
end

Environment

• ruby_llm: 1.3.1
• Rails: 8.0.1
• Frequency: Intermittent but common under load
• Note: Database order always correct (reloading the page corrects ordering), only affects real-time display

[tpaulshippy]

Are you using the rails integration? Maybe you could give us a minimal reproducible example?

[tpaulshippy]

On second look I may see how this could possibly happen. The ask method creates the user message and then immediately calls complete which calls on_new_message which is set up to create a blank assistant message.

[tpaulshippy]

Can you use https://rubyllm.com/guides/rails#alternative-validation-first-approach but persist when you get the first chunk?

[rclements]

I too am having this same issue.

[crmne]

Fixed in eef291e.

The root cause was creating the assistant message immediately after the user message causing ActionCable race conditions.

Now the assistant message is only created when the first content chunk arrives from the API. No race condition, no arbitrary delays, no workarounds needed in application code.

[crmne]

Turns out the fix above breaks with tool calls. I'm investigating a better one.

[crmne]

Reattempted in 701d5d0. Please try it in your app and see if it works better! I just released a 1.3.2beta1 today so you can test without installing a version from git.

I have to admit I'm not sure this is the best way to fix this as it doesn't fix the underlying problem with ActionCable. Perhaps some ActionCable experts can chime in on this? Perhaps @ioquatix or @palkan?

[ioquatix]

Sorry, it's already 1AM but I'll give you my brief thoughts.

• I was under the impression that there was no order guarantee with ActionCable?
• You may need to include a request/response ID which you can use to track the messages.
• Async::WebSocket does guarantee order so if you use that, it may prevent the issue.

[palkan]

tl;dr

I was under the impression that there was no order guarantee with ActionCable?

^ This.

Async::WebSocket does guarantee order so if you use that, it may prevent the issue.

But not `async-cable, right? (Because I don't see it doing anything about subscription adapters, and that where the cause of the problem is).

---

Action Cable uses a threaded executor to distribute broadcast messages, so if you send 10 messages they're picked up by 10 threads and delivered to connected clients concurrently.

The best way to fix this is to add the order information to the payload and re-order on the client. Yeah, write a bunch of boring code.

Or you can use AnyCable. It can provide order guarantees at the server level, so no code changes required.

[crmne]

Thanks for the insights @ioquatix and @palkan!

I'm working on adding documentation to help users handle this issue, and wanted to verify a few assumptions with you both:

1. ActionCable ordering: I understand that the default ActionCable implementation doesn't guarantee order, but does this vary by adapter? For instance, does async-cable behave differently?
2. Single-machine async stack: In my experience, using Falcon + async-cable on a single machine seems to maintain message order better. Is this because:
   - Fibers are cooperatively scheduled?
   - Single-machine deployment avoids distributed system challenges?
   - Or is this just coincidental/luck?
3. Best practices: For the docs, should I emphasize client-side reordering as the most universal solution, with AnyCable as the server-side alternative for those who need guarantees?

I've created a PR (#288) with documentation updates including a Stimulus controller for client-side reordering. Would love your technical review before merging!

The 1.3.2beta1 fix (delaying assistant message creation until first chunk) is working well for me, but I understand it doesn't fully solve the ordering issue - just reduces the race condition window. @rclements @fluxsaas would love to hear if that is working for you and if it resolved your issue.