Add/vllm image #9
Conversation
| @@ -40,6 +45,7 @@ def __init__( | |||
| limit_worker_concurrency: int, | |||
| no_register: bool, | |||
| conv_template: str, | |||
| temp_img_dir: str, | |||
There was a problem hiding this comment.
Why do you pass the temp directory in? I notice down further you delete it which seems pretty drastic...so maybe this should be something owned by this library? Or maybe I misunderstood something?
There was a problem hiding this comment.
The reason is that we need a directory to save images, and we also need to give vllm access to that directory. so I wanted it to be defined per plugin instead of using a fixed directory. That’s why I’ll pass an image_dir to the OpenAI proxy from each plugin that calls it, so it can handle and save images within that plugin.
There was a problem hiding this comment.
Hmmm...this feels unusual, and I'm scared of deleting a directory on my hard drive accidentally. 😅
What if OpenAIProxyWorker made the directory and then there's a function to get the temp directory that plugins can call and pass to vllm/whatever? Would that work for the order of operations?
No description provided.