Description
Issue description:
感觉目前的实现跟openai标准的输出的不太一样:
- finish_reason全都是null,即使生成到最后一个字符了也是null,正常应该是"stop"或"length"吧
- index全是0
- stop参数目前不支持:"The stop parameter is not currently supported"
- 在启动服务时,已经设置--eos_id 151645的情况下,生成的内容虽然在<|im_end|>之后终止了,但还是会返回<|im_end|>,正常情况下这个字符不应该返回的吧
Steps to reproduce:
请求示例:
{
"model": "Qwen",
"messages": [
{
"role": "user",
"content": "你好"
}
],
"stream":true,
"top_p":0.7,
"presence_penalty":1.2,
"max_tokens":512
}
返回示例:
data: {"id": "chatcmpl-e93674ffedbb465e9ac7d3b503e29f13", "object": "chat.completion.chunk", "created": 1694575540, "model": "Qwen", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "你好"}, "finish_reason": null}]}
data: {"id": "chatcmpl-e93674ffedbb465e9ac7d3b503e29f13", "object": "chat.completion.chunk", "created": 1694575540, "model": "Qwen", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "!"}, "finish_reason": null}]}
data: {"id": "chatcmpl-e93674ffedbb465e9ac7d3b503e29f13", "object": "chat.completion.chunk", "created": 1694575540, "model": "Qwen", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "很高兴"}, "finish_reason": null}]}
data: {"id": "chatcmpl-e93674ffedbb465e9ac7d3b503e29f13", "object": "chat.completion.chunk", "created": 1694575540, "model": "Qwen", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "为你"}, "finish_reason": null}]}
data: {"id": "chatcmpl-e93674ffedbb465e9ac7d3b503e29f13", "object": "chat.completion.chunk", "created": 1694575540, "model": "Qwen", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "提供"}, "finish_reason": null}]}
data: {"id": "chatcmpl-e93674ffedbb465e9ac7d3b503e29f13", "object": "chat.completion.chunk", "created": 1694575540, "model": "Qwen", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "帮助"}, "finish_reason": null}]}
data: {"id": "chatcmpl-e93674ffedbb465e9ac7d3b503e29f13", "object": "chat.completion.chunk", "created": 1694575540, "model": "Qwen", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "。"}, "finish_reason": null}]}
data: {"id": "chatcmpl-e93674ffedbb465e9ac7d3b503e29f13", "object": "chat.completion.chunk", "created": 1694575540, "model": "Qwen", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "<|im_end|>"}, "finish_reason": null}]}
Expected behavior:
感觉openai接口的功能还没有很完善,后续还会有继续更新的计划吗?