Replies: 1 comment
-
For what it's worth, this is how I built vllm for Neuron (but don't get your hopes up. It runs, and then crashes with some obscure error).
|
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi
I would like to run vllm using docker on Inferentia2.
I couldn't find a docker tag for neuron.
Tried building it myself, but it seems like Dockerfile.neuron is incomplete (no entrypoint).
Beta Was this translation helpful? Give feedback.
All reactions