Skip to content

Releases: modelscope/ms-swift

v2.4.2

18 Sep 16:56
@Jintao-Huang Jintao-Huang
v2.4.2
264b59f
Compare
Choose a tag to compare
v2.4.2 Latest
Latest

English Version

New Features:

  1. RLHF reconstruction, supporting all integrated multimodal models, compatible with DeepSpeed Zero2/Zero3, and supports lazy_tokenize.
  2. Using infer_backend vllm, inference deployment of multimodal large models supports multiple images.

New Models:

  1. Qwen2.5 series, Qwen2-vl-72b series (base/instruct/gptq-int4/gptq-int8/awq)
  2. Qwen2.5-math, Qwen2.5-coder series (base/instruct)
  3. Deepseek-v2.5

New Datasets:

  1. longwriter-6k-filtered

中文版

新特性：

  1. RLHF重构，支持所有已接入的多模态模型，兼容deepspeed zero2/zero3，支持lazy_tokenize
  2. 使用infer_backend vllm，推理部署多模态大模型支持多图.

新模型：

  1. qwen2.5系列、qwen2-vl-72b系列（base/instruct/gptq-int4/gptq-int8/awq）
  2. qwen2.5-math, qwen2.5-coder系列（base/instruct）
  3. deepseek-v2.5

新数据集：

  1. longwriter-6k-filtered

What's Changed

New Contributors

Full Changelog: v2.4.1...v2.4.2

Contributors

wangxingjun778, irexyc, and 4 other contributors
Assets 2
Loading

v2.4.1

13 Sep 05:03
@Jintao-Huang Jintao-Huang
v2.4.1
7545374
Compare
Choose a tag to compare
v2.4.1

English Version

New Features:

  1. Inference and deployment support for logprobs.
  2. RLHF support for lazy_tokenize.
  3. Multimodal model support for neftune.
  4. dynamic_eos compatibility with glm4 series and other models.

New Models:

  1. mplug-owl3, best practices can be found here.
  2. yi-coder 1.5b, base/chat model of 9b.
  3. minicpm3-4b.
  4. reflection-llama3.1-70b.

中文版

新功能：

  1. 推理和部署支持 logprobs。
  2. RLHF支持lazy_tokenize。
  3. 多模态模型支持neftune。
  4. dynamic_eos兼容glm4系列等模型。

新模型：

  1. mplug-owl3，最佳实践可以查看这里。
  2. yi-coder 1.5b、9b 的base/chat模型。
  3. minicpm3-4b。
  4. reflection-llama3.1-70b。

What's Changed

Full Changelog: v2.4.0...v2.4.1

Contributors

Jintao-Huang, Zhikaiiii, and 3 other contributors
Assets 2
Loading

v2.4.0

13 Sep 04:50
@Jintao-Huang Jintao-Huang
v2.4.0
0e4e9c3
Compare
Choose a tag to compare
v2.4.0

English Version

New Features:

  1. Support for Liger, which accommodates models like LLaMA, Qwen, Mistral, etc., and reduces memory usage by 10% to 60%.
  2. Support for custom loss function training using a registration mechanism.
  3. Training now supports pushing models to ModelScope and HuggingFace.
  4. Support for the freeze_vit parameter to control the behavior of full parameter training for multimodal models.

New Models:

  1. Qwen2-VL series includes GPTQ/AWQ quantized models. For best practices, see here.
  2. InternVL2 AWQ quantized models.

New Datasets:

  1. qwen2-pro series

中文版

新特性：

  1. 支持 Liger训练LLaMA、Qwen、Mistral 等模型，内存使用降低 10% 至 60%。
  2. 支持使用注册机制进行自定义损失函数的训练。
  3. 训练支持将模型推送至 ModelScope 和 HuggingFace。
  4. 支持 freeze_vit 参数，以控制多模态模型全参数训练的行为。

新模型：

  1. Qwen2-VL 系列包括 GPTQ/AWQ 量化模型，最佳实践可以查看这里。
  2. InternVL2 AWQ 量化模型。

新数据集：

  1. qwen2-pro 系列

What's Changed

Full Changelog: v2.3.2...v2.4.0

Contributors

Jintao-Huang, baoleai, and 2 other contributors
Assets 2
Loading

v2.3.2

24 Aug 04:42
@Jintao-Huang Jintao-Huang
v2.3.2
e2bba6e
Compare
Choose a tag to compare
v2.3.2

English Version

New Features:

  1. ReFT support: achieves parameter efficiency that is 15× to 65× greater than LoRA.
  2. Multimodal model supports zero3.
  3. Supports using environment variables to control parameters such as hd_num, max_num, and video_segments.

New Models:

  1. longwriter-glm4-9b, longwriter-llama3_1-8b
  2. phi3_5-mini-instruct, phi3_5-moe-instruct, phi3_5-vision-instruct
  3. llava-onevision-qwen2-0_5b-ov, llava-onevision-qwen2-7b-ov, llava-onevision-qwen2-72b-ov

New Datasets:

  1. longwriter-6k
  2. rlaif-v
  3. latex-ocr-print, latex-ocr-handwrite

中文版

新功能：

  1. 支持ReFT，实现了比 LoRA 高 15 倍到 65 倍的参数效率。
  2. 多模态模型支持 zero3。
  3. 支持使用环境变量控制模型特有的参数，如 hd_num、max_num 和 video_segments。

新模型：

  1. longwriter-glm4-9b, longwriter-llama3_1-8b
  2. phi3_5-mini-instruct, phi3_5-moe-instruct, phi3_5-vision-instruct
  3. llava-onevision-qwen2-0_5b-ov, llava-onevision-qwen2-7b-ov, llava-onevision-qwen2-72b-ov

新数据集：

  1. longwriter-6k
  2. rlaif-v
  3. latex-ocr-print, latex-ocr-handwrite

What's Changed

New Contributors

Full Changelog: v2.3.1...v2.3.2

Contributors

wning13, Jintao-Huang, and 4 other contributors
Assets 2
Loading

v2.3.1

19 Aug 03:11
@Jintao-Huang Jintao-Huang
v2.3.1
742d16b
Compare
Choose a tag to compare
v2.3.1

English Version

New Features:

  1. ms-swift paper published: https://arxiv.org/abs/2408.05517
  2. Web-UI supports audio and video.
  3. Support for deploying audio and video models using the OpenAI API.
  4. Utilizes a new multimodal training framework.
  5. supports inference acceleration for video models (lmdeploy & internvl2 series).

New Models:

  1. idefics3-8b-llama3
  2. llava-hf 72b, 110b, llama3-llava
  3. deepseek-coder-v2, deepseek-coder-lite-v2, deepseek-v2

中文版

新功能：

  1. 发布了 ms-swift 论文：https://arxiv.org/abs/2408.05517
  2. Web-UI 支持音频和视频。
  3. 支持使用 OpenAI API 部署音频和视频模型。
  4. 采用新的多模态训练框架。
  5. 支持视频模型的推理加速（lmdeploy 和 internvl2 系列）。

新模型：

  1. idefics3-8b-llama3
  2. llava-hf 72b、110b、llama3-llava
  3. deepseek-coder-v2、deepseek-coder-lite-v2、deepseek-v2

What's Changed

New Contributors

Full Changelog: v2.3.0...v2.3.1

Contributors

Yunnglin, Jintao-Huang, and 3 other contributors
Assets 2
Loading

v2.3.0

09 Aug 15:43
@tastelikefeet tastelikefeet
v2.3.0
451ef3d
Compare
Choose a tag to compare
v2.3.0

English Version

New Features

  1. Support for readthedocs documentation site at: https://swift.readthedocs.io/en/latest
  2. Support Megatron architecture training for QianWen series models, and added new pt command for pretraining. See docs: https://swift.readthedocs.io/en/latest/LLM/Megatron-training.html
  3. Support LMDeploy for inference and deployment, improving inference acceleration for multi-modal models. See: https://swift.readthedocs.io/en/latest/Multi-Modal/LmDeploy-inference-acceleration.html
  4. Support passing lora target modules via regular expressions
  5. Support configuring max_memory usage for each GPU in device_map
  6. export command supports BitsAndBytes quantization
  7. export command supports Ollama export: https://swift.readthedocs.io/en/latest/LLM/OLLaMA-Export.html
  8. Support Q-GaLore algorithm
  9. Support RLHF training for multi-modal models: https://swift.readthedocs.io/en/latest/Multi-Modal/human-preference-alignment-training-documentation.html
  10. Support evaluation on 100+ datasets for multi-modal models: https://swift.readthedocs.io/en/latest/LLM/LLM-eval.html
  11. Support resizing input images when memory usage is too high for multi-modal models
  12. Modified default lora injection for multi-modal model training. Now takes effect on LLM and projector, results are better without significantly increasing training memory.
  13. Support PEFT 0.12, and added new tuner: fourierft
  14. Support rope-scaling for multi-modal models
  15. Support streaming processing of datasets to reduce memory usage, enable with --streaming
  16. Support vLLM multi-modal inference and deployment
  17. Support grounding task for popular multi-modal models.

New Models

  1. qwen2-audio series
  2. qwen2-math
  3. codegeex4
  4. internvl2 series
  5. llava video
  6. xcomposer2.5
  7. cogvlm2-video
  8. numina-math
  9. mistral-nemo
  10. llama3.1 series
  11. mistral-large
  12. gemma-2-2b
  13. internlm2.5 1.8b 20b
  14. minicpm-v-v2_6-chat

Check: https://swift.readthedocs.io/en/latest/LLM/Supported-models-datasets.html

New Datasets

  1. zhihu-kol and zhihu-kol-filtered
  2. SA1B series multi-modal zh datasets

Check: https://swift.readthedocs.io/en/latest/LLM/Supported-models-datasets.html

中文版本

新功能

  1. 支持readthedocs文档库， 地址：https://swift.readthedocs.io/zh-cn/latest
  2. 支持千问系列模型的Megatron结构训练，并支持了新的pt命令用于预训练，详见文档：https://swift.readthedocs.io/zh-cn/latest/LLM/Megatron%E8%AE%AD%E7%BB%83%E6%96%87%E6%A1%A3.html
  3. 支持LMDeploy的推理和部署，更好地支持了多模态模型的推理加速，详见：https://swift.readthedocs.io/zh-cn/latest/Multi-Modal/LmDeploy%E6%8E%A8%E7%90%86%E5%8A%A0%E9%80%9F%E6%96%87%E6%A1%A3.html
  4. 支持以正则表达式方式传入lora target模块
  5. 支持配置device_map各GPU用量的max_memory
  6. export命令支持BitsAndBytes量化
  7. export命令支持Ollama导出：https://swift.readthedocs.io/zh-cn/latest/LLM/OLLAMA%E5%AF%BC%E5%87%BA%E6%96%87%E6%A1%A3.html
  8. 支持Q-GaLore算法
  9. 支持多模态模型的RLHF训练：https://swift.readthedocs.io/zh-cn/latest/Multi-Modal/%E4%BA%BA%E7%B1%BB%E5%81%8F%E5%A5%BD%E5%AF%B9%E9%BD%90%E8%AE%AD%E7%BB%83%E6%96%87%E6%A1%A3.html
  10. 支持多模态模型100+数据集的评测能力：https://swift.readthedocs.io/zh-cn/latest/LLM/LLM%E8%AF%84%E6%B5%8B%E6%96%87%E6%A1%A3.html
  11. 支持多模态模型显存占用过高时对输入图片进行缩放
  12. 修改了多模态模型训练的默认lora注入，目前对LLM和projector生效，不显著提高训练显存情况下效果更好
  13. 支持PEFT0.12，并支持了新的tuner：fourierft
  14. 支持多模态模型的rope-scaling
  15. 支持数据集的流式处理，降低显存消耗，使用--streaming开启
  16. 支持了vLLM的多模态推理部署能力
  17. 对部分多模态模型支持了grounding任务

新模型

  1. qwen2-audio系列模型
  2. qwen2-math
  3. codegeex4
  4. internvl2系列模型
  5. llava video
  6. xcomposer2.5
  7. cogvlm2-video
  8. numina-math
  9. mistral-nemo
  10. llama3.1系列
  11. mistral-large
  12. gemma-2-2b
  13. internlm2.5 1.8b 20b
  14. minicpm-v-v2_6-chat

参考：https://swift.readthedocs.io/zh-cn/latest/LLM/%E6%94%AF%E6%8C%81%E7%9A%84%E6%A8%A1%E5%9E%8B%E5%92%8C%E6%95%B0%E6%8D%AE%E9%9B%86.html

新数据集

  1. zhihu-kol和zhihu-kol-filtered数据集
  2. SA1B系列中文多模态数据集

参考：https://swift.readthedocs.io/zh-cn/latest/LLM/%E6%94%AF%E6%8C%81%E7%9A%84%E6%A8%A1%E5%9E%8B%E5%92%8C%E6%95%B0%E6%8D%AE%E9%9B%86.html

What's Changed

Read more

Contributors

Wondersui, starxhong, and 8 other contributors
Assets 2
Loading

v2.2.5

02 Aug 02:42
@Jintao-Huang Jintao-Huang
v2.2.5
73a1b44
Compare
Choose a tag to compare
v2.2.5

New Features:

  1. Support for RLHF with multimodal models, including DPO, ORPO, SIMPO, and CPO
  2. SWIFT infer, SWIFT deploy support using lmdeploy for inference acceleration.
  3. Support the use of Megatron for performing PT and SFT on the Qwen2 series models.
  4. Support the grounding task for InternVL2/Qwen-VL-Chat models

New Models:

  1. mistral-nemo series, mistral-large
  2. llama3.1 series

New Datasets:

  1. sa1b-dense-caption, sa1b-paired-caption
  2. rlaif-v
  3. zhihu-kol, zhihu-kol-filtered

What's Changed

New Contributors

Full Changelog: v2.2.3...v2.2.5

Contributors

starxhong, ArtificialZeng, and 5 other contributors
Assets 2
Loading

v2.2.3

20 Jul 13:12
@Jintao-Huang Jintao-Huang
v2.2.3
a48d65c
Compare
Choose a tag to compare
v2.2.3

New Features:

  1. support bnb and ollama export
  2. suport Q-Galore

New Models:

  1. numina-math-7b

Bug Fix:

  1. fix vllm>=0.5.1, TP
  2. fix internvl2 template
  3. fix glm4v merge-lora

What's Changed

Full Changelog: v2.2.2...v2.2.3

Contributors

Jintao-Huang, baoleai, and 2 other contributors
Assets 2
Loading

v2.2.2

13 Jul 15:12
@tastelikefeet tastelikefeet
v2.2.2
7163356
Compare
Choose a tag to compare
v2.2.2

English Version

Features

  1. Support lmdeploy for inference
  2. Support training for Internvl2 Video
  3. Support argument for LoRA target module in regex format
  4. Support RTD website
  5. Support argument of device_max_memory to config device_map memory usage.

New Models

  1. Support CogVLM2-Video

中文

新功能

  1. 支持lmdeploy框架的推理能力
  2. 支持InternVL2模型的视频训练能力
  3. 支持lora_target_regex参数，用来以正则表达式配置lora模块
  4. 支持RTD文档网站
  5. 支持device_max_memory参数来配置device_map的显存使用

新模型

  1. CogVLM2-Video

What's Changed

Full Changelog: v2.2.1...v2.2.2

Contributors

Jintao-Huang, tastelikefeet, and hjh0119
Assets 2
Loading

v2.2.1

08 Jul 07:08
@tastelikefeet tastelikefeet
v2.2.1
546e6dc
Compare
Choose a tag to compare
v2.2.1

English Version

New Features

  1. Multimodal: Supported a large number of multimodal datasets and restructured the multimodal architecture. Some models now support grounding tasks.
  2. Web-ui: Added support for RLHF, evaluation, and quantization.
  3. Evaluation Functionality: Refactored the evaluation functionality, now using OpenCompass internally, supporting over 50 evaluation datasets.
  4. Deployment Functionality: VLLM infer_backend now supports multimodal models.
  5. Agent Training: Refactored the construction, training, and deployment of agent datasets, making the agent pipeline more complete.
  6. Human Alignment: Added alignment algorithms such as KTO and CPO, and refactored the human alignment code.

New Models

  1. openbuddy-llama3-70b
  2. Deepseek-coder-v2
  3. llava1.5, llava1.6, llava-next-video
  4. gemma2
  5. Florence
  6. phi3-4k
  7. internlm2.5, xcomposer2.5
  8. internvl2
  9. codegeex4
  10. mistral-7b-instruct-v0.3

New Datasets

  1. Over 30 foundational multimodal datasets, including GQA, RefCOCO, and Llava-pretrain.
  2. Swift-mix general mixed dataset.
  3. Video-chatgpt video dataset.

中文版本

新功能

  1. 多模态：支持了非常多的多模态数据集，并重构了多模态架构，部分模型开始支持grounding任务
  2. Web-ui：支持了RLHF、评测和量化
  3. 评测功能：进行了重构，内部使用了OpenCompass，支持50+评测集
  4. 部署功能：VLLM infer_backend支持多模态模型
  5. Agent训练：重构了Agent数据集构造、训练、部署，Agent链路更加完整
  6. 人类对齐：增加了KTO、CPO等对齐算法，并重构了人类对齐的代码

新模型

  1. openbuddy-llama3-70b
  2. Deepseek-coder-v2
  3. llava1.5, llava1.6, llava-next-video
  4. gemma2
  5. Florence
  6. phi3-4k
  7. internlm2.5, xcomposer2.5
  8. internvl2
  9. codegeex4
  10. mistral-7b-instruct-v0.3

新数据集

  1. GQA、RefCOCO、Llava-pretrain等30+多模态基础数据集
  2. swift-mix通用混合数据集
  3. video-chatgpt视频数据集

What's Changed

New Contributors

Full Changelog: v2.1.1...v2.2.1

Contributors

anw90, jiaozhentian, and 4 other contributors
Assets 2
Loading