Conversation
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Quarol
left a comment
There was a problem hiding this comment.
Running on windows 11 failed (command used: llmsql --help.
The reason is: vLLM is ONLY supportend on Linux, so any import causes the crash.
We need to do a workaround (always making sure only transformers run).
My proposed changed are pushed onto the branch in question.
Another issue is that if we keep provider as a flag (--method) then we will have overload of disjoined flags for both options - vllm and transformers.
Hence my suggestions (in the latest commit):
vllmandtransormersbecame subcommands - they are not under--methodflag anymore- I did a workaround that does lazy import if linux is detected. It means if other OS is used then
vllm_interfacewill NOT be imported - so there is no crash.
Additional problem (commands ran on my own PC):
CLI is very slow - llmsql --help comm and takes about 6 seconds to output message.
…nd "vllm" became subcommands instead of being part of the "--method" flag.
|
Please, check the latest commit @DzmitryPihulski . If I got ur "ok", then I will approve the PR. |
cli updated