Qwen/Qwen3-Embedding-0.6B output error

[asdfgh19]

System Info

I run Qwen/Qwen3-Embedding-0.6B using the following command：
docker run -d --name text-embeddings-inference_server -p 8089:80 -v "$VOLUME":/data -e HF_ENDPOINT=https://hf-mirror.com --pull always ghcr.io/huggingface/text-embeddings-inference:cpu-1.7.4 --revision refs/pr/27 --pooling last-token --max-batch-tokens 512 --model-id Qwen/Qwen3-Embedding-0.6B

The following request returns different results when called multiple times, sometimes returning result 1 and sometimes returning result 2.

curl --location 'http://localhost:8089/embed' --header 'Content-Type: application/json' --data '{ "inputs": [ "{\"name\": \"风场\", \"type\": \"model\", \"id\": \"EnOS_Wind_Farm\", \"desc\": \"风场模型\"}", "{\"name\": \"发电量\", \"type\": \"metric\", \"id\": \"ActiveProduction\", \"model\": \"EnOS_Wind_Farm\", \"model_name\": \"风场\", \"data_type\": \"double\", \"unit\": \"s\"}", "{\"name\": \"发电量\", \"type\": \"metric\", \"id\": \"ActiveProduction\", \"model\": \"EnOS_Solar_Site\", \"model_name\": \"光伏场站\", \"data_type\": \"double\", \"unit\": \"s\"}" ] }'

result 1:
[[-0.03803645, 0.010309944, ...], [...], [...]]

result 2:
[[-0.02697843, -0.005487643, ...], [...], [...]]

Information

• Docker
• The CLI directly

Tasks

• An officially supported command
• My own modifications

Reproduction

1. docker run
2. call embed api

Expected behavior

Multiple requests with the same input return the same result

[asdfgh19]

In addition, concurrent calls from multiple threads will also affect each other, resulting in incorrect results

[alvarobartt]

Hey @asdfgh19 thanks for flagging! I'm already looking into it, but TL;DR the padding side for OrtBackend defaults to right, whereas the one used for Qwen3-Embedding models is left; meaning that when padding multiple sequences since the padding side is wrong, which shouldn't affect the pooling but might as well be the case for those diffs.

Thanks for reporting, and expect a patch fix later this week, sorry for the inconvenience! 🤗

[alvarobartt]

For context, see this PR description #700, specifically the NOTE that says the following:

Note

On a subsequent PR, both the input building and the pooling strategies for ONNX need to be updated to use the correct padding side, as it defaults to right padding which tends to be the norm, but there are models such as e.g. https://huggingface.co/onnx-community/Qwen3-Embedding-0.6B-ONNX, that use left padding instead.

[alvarobartt]

Hey again @asdfgh19, quick update from my side after reproducing your issue:

• The padding side seems to differ, but this shouldn’t affect pooling, so I don’t think that’s the real cause here.
• I confirmed that with both left and right padding, the issue persists: after some requests some embeddings seems to differ in the batch when using ONNX, even though the rest might be consistent i.e. not all the embeddings are different.
• Despite these differences, cosine similarity with the ground truth embedding is still 1.0 or close to 1.0 (~0.9999), so the embeddings are effectively the same.
• I was able to reproduce the issue using ONNX on CPU (with your shared command), but also on CPU, MPS and CUDA with Safetensors, meaning the issue is related to Qwen3 per se e.g. I couldn't reproduce with sentence-transformers/all-MiniLM-L6-v2 under the same scenario.

So, this seems tied to the Qwen3 model at first sight, but I’ll keep digging into this and try to reproduce with other architectures / ONNX exports.

Thanks again for reporting!