Skip to content

[BUG] Issues ablout openai /v1/chat/completions interface in streaming mode #130

Open
@sunxichen

Description

@sunxichen

Issue description:
感觉目前的实现跟openai标准的输出的不太一样:

  1. finish_reason全都是null,即使生成到最后一个字符了也是null,正常应该是"stop"或"length"吧
  2. index全是0
  3. stop参数目前不支持:"The stop parameter is not currently supported"
  4. 在启动服务时,已经设置--eos_id 151645的情况下,生成的内容虽然在<|im_end|>之后终止了,但还是会返回<|im_end|>,正常情况下这个字符不应该返回的吧

Steps to reproduce:
请求示例:
{
"model": "Qwen",
"messages": [
{
"role": "user",
"content": "你好"
}
],
"stream":true,
"top_p":0.7,
"presence_penalty":1.2,
"max_tokens":512
}

返回示例:
data: {"id": "chatcmpl-e93674ffedbb465e9ac7d3b503e29f13", "object": "chat.completion.chunk", "created": 1694575540, "model": "Qwen", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "你好"}, "finish_reason": null}]}

data: {"id": "chatcmpl-e93674ffedbb465e9ac7d3b503e29f13", "object": "chat.completion.chunk", "created": 1694575540, "model": "Qwen", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "!"}, "finish_reason": null}]}

data: {"id": "chatcmpl-e93674ffedbb465e9ac7d3b503e29f13", "object": "chat.completion.chunk", "created": 1694575540, "model": "Qwen", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "很高兴"}, "finish_reason": null}]}

data: {"id": "chatcmpl-e93674ffedbb465e9ac7d3b503e29f13", "object": "chat.completion.chunk", "created": 1694575540, "model": "Qwen", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "为你"}, "finish_reason": null}]}

data: {"id": "chatcmpl-e93674ffedbb465e9ac7d3b503e29f13", "object": "chat.completion.chunk", "created": 1694575540, "model": "Qwen", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "提供"}, "finish_reason": null}]}

data: {"id": "chatcmpl-e93674ffedbb465e9ac7d3b503e29f13", "object": "chat.completion.chunk", "created": 1694575540, "model": "Qwen", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "帮助"}, "finish_reason": null}]}

data: {"id": "chatcmpl-e93674ffedbb465e9ac7d3b503e29f13", "object": "chat.completion.chunk", "created": 1694575540, "model": "Qwen", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "。"}, "finish_reason": null}]}

data: {"id": "chatcmpl-e93674ffedbb465e9ac7d3b503e29f13", "object": "chat.completion.chunk", "created": 1694575540, "model": "Qwen", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "<|im_end|>"}, "finish_reason": null}]}

Expected behavior:

感觉openai接口的功能还没有很完善,后续还会有继续更新的计划吗?

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions