Add the new Multi-Modal model of mistral AI: pixtral-12b

[SuperPat45]

Add the new Multi-Modal model of mistral AI: pixtral-12b:

https://huggingface.co/mistral-community/pixtral-12b-240910

It supports image encoder, can it also be added to the image generator API as an alternative to Stable Diffusion?

[AlexM4H]

Since yesterday vllm has internVL2 support. :-)

vllm-project/vllm/releases/tag/v0.6.1

[mudler]

I guess that would work already with llama.cpp GGUF models if/when is getting supported in there ( see also ggerganov/llama.cpp#9440 ).

I'd change the focus of this one to be more generic and add support for multimodal with vLLM, examples:

https://github.com/vllm-project/vllm/blob/main/examples/offline_inference_pixtral.py
https://github.com/vllm-project/vllm/blob/main/examples/offline_inference_vision_language_multi_image.py