v2.2.1
English Version
New Features
- Multimodal: Supported a large number of multimodal datasets and restructured the multimodal architecture. Some models now support grounding tasks.
- Web-ui: Added support for RLHF, evaluation, and quantization.
- Evaluation Functionality: Refactored the evaluation functionality, now using OpenCompass internally, supporting over 50 evaluation datasets.
- Deployment Functionality: VLLM infer_backend now supports multimodal models.
- Agent Training: Refactored the construction, training, and deployment of agent datasets, making the agent pipeline more complete.
- Human Alignment: Added alignment algorithms such as KTO and CPO, and refactored the human alignment code.
New Models
- openbuddy-llama3-70b
- Deepseek-coder-v2
- llava1.5, llava1.6, llava-next-video
- gemma2
- Florence
- phi3-4k
- internlm2.5, xcomposer2.5
- internvl2
- codegeex4
- mistral-7b-instruct-v0.3
New Datasets
- Over 30 foundational multimodal datasets, including GQA, RefCOCO, and Llava-pretrain.
- Swift-mix general mixed dataset.
- Video-chatgpt video dataset.
中文版本
新功能
- 多模态:支持了非常多的多模态数据集,并重构了多模态架构,部分模型开始支持grounding任务
- Web-ui:支持了RLHF、评测和量化
- 评测功能:进行了重构,内部使用了OpenCompass,支持50+评测集
- 部署功能:VLLM infer_backend支持多模态模型
- Agent训练:重构了Agent数据集构造、训练、部署,Agent链路更加完整
- 人类对齐:增加了KTO、CPO等对齐算法,并重构了人类对齐的代码
新模型
- openbuddy-llama3-70b
- Deepseek-coder-v2
- llava1.5, llava1.6, llava-next-video
- gemma2
- Florence
- phi3-4k
- internlm2.5, xcomposer2.5
- internvl2
- codegeex4
- mistral-7b-instruct-v0.3
新数据集
- GQA、RefCOCO、Llava-pretrain等30+多模态基础数据集
- swift-mix通用混合数据集
- video-chatgpt视频数据集
What's Changed
- fix py38 by @Jintao-Huang in #1152
- refactor rlhf by @hjh0119 in #1090
- Fix/web UI 0617 by @tastelikefeet in #1158
- Add more datasets by @tastelikefeet in #1065
- support openbuddy-llama3-70b by @Jintao-Huang in #1162
- Add Deepseek coder v2 model by @tastelikefeet in #1164
- Update web-ui labels by @tastelikefeet in #1168
- fix hf download by @tastelikefeet in #1169
- fix kto bugs by @hjh0119 in #1173
- Fix preprocessor by @tastelikefeet in #1174
- refactor ui by @tastelikefeet in #1178
- fix DPO by @hjh0119 in #1181
- fix bugs by @Jintao-Huang in #1171
- 在argument.py添加hccl参数,用于NPU环境下训练使用 by @jiaozhentian in #1165
- Custom agent dataset training support by @hjh0119 in #1184
- RLHF UI by @tastelikefeet in #1182
- fix kto by @hjh0119 in #1187
- update agent doc by @hjh0119 in #1189
- Fix device map in limited mem by @tastelikefeet in #1190
- Fix sk will be uploaded by @tastelikefeet in #1192
- Fix dataset concatenation by @tastelikefeet in #1193
- Fix glm4v dataloader by @CjangCjengh in #1183
- fix glm4v images by @Jintao-Huang in #1194
- [TorchAcc] Cache the compiled results and remove some xla flags by @anw90 in #1160
- Fix vqa dataset loading by @tastelikefeet in #1195
- [TorchAcc] Add acc_steps arguments to speed up torchacc training by @anw90 in #1196
- fix minicpm-v data_collator by @Jintao-Huang in #1197
- Add a tuner to train a part of the model by @tastelikefeet in #1200
- Fix ConversationPreprocessor loading by @tastelikefeet in #1201
- Fix map cache on multi node by @tastelikefeet in #1203
- refactor multi-modal by @Jintao-Huang in #1212
- support llava 1.5 by @Jintao-Huang in #1217
- add in_browswer by @tastelikefeet in #1220
- [TorchAcc] Add USE_TORCH_XLA=0 flag for native swift scripts by @anw90 in #1221
- Fix glm4v batch_size>1 by @Jintao-Huang in #1223
- Add debug log support by @tastelikefeet in #1226
- Fix qlora deploy by @Jintao-Huang in #1224
- Add new dataset by @tastelikefeet in #1227
- fix refcoco by @tastelikefeet in #1232
- Fix bugs by @tastelikefeet in #1241
- fix bugs by @Jintao-Huang in #1242
- refactor inference by @Jintao-Huang in #1245
- Add more datasets by @tastelikefeet in #1246
- Support gemma2 by @Jintao-Huang in #1247
- Support gemma2 by @Jintao-Huang in #1248
- add push to hub tracker by @tastelikefeet in #1249
- support Florence series model by @hjh0119 in #1244
- update eval by @Jintao-Huang in #1251
- reset snapshot_download to modelscope by @tastelikefeet in #1254
- Feat/open compass by @tastelikefeet in #1213
- fix output empty will be filtered by @tastelikefeet in #1256
- add phi3 models by @tastelikefeet in #1259
- update dataset preprocess by @Jintao-Huang in #1257
- fix rlhf by @tastelikefeet in #1261
- fix rlhf by @Jintao-Huang in #1260
- Support llava1.6 mistral by @Jintao-Huang in #1268
- Add new dataset by @tastelikefeet in #1270
- support vllm & multimodal by @Jintao-Huang in #1229
- Fix bugs by @Jintao-Huang in #1275
- fix llamapro by @tastelikefeet in #1276
- fix xcomposer deploy by @Jintao-Huang in #1280
- Fix/0703 2 by @tastelikefeet in #1284
- Fix llamapro merge and quantized training by @tastelikefeet in #1289
- update lr_scheduler by @Jintao-Huang in #1288
- support internlm2.5 by @Jintao-Huang in #1291
- fix vllm by @Jintao-Huang in #1292
- fix florence & deploy by @Jintao-Huang in #1293
- fix mini-internvl-phi3 template by @Jintao-Huang in #1298
- fix florence by @hjh0119 in #1301
- Refactor UI and fix some bugs by @tastelikefeet in #1300
- fix dependency by @tastelikefeet in #1306
- support codegeex4 by @Jintao-Huang in #1305
- support internvl2 by @hjh0119 in #1304
- support llava video by @Jintao-Huang in #1307
- fix docs by @Jintao-Huang in #1309
- support lr_scheduler_kwargs by @Jintao-Huang in #1310
- Fix internvl2 template by @hjh0119 in #1308
- Fix bugs by @tastelikefeet in #1311
- support warmup_stable_decay by @Jintao-Huang in #1312
- Support xcomposer2.5 by @Jintao-Huang in #1287
- Fix bugs by @tastelikefeet in #1319
New Contributors
- @jiaozhentian made their first contribution in #1165
- @CjangCjengh made their first contribution in #1183
- @anw90 made their first contribution in #1160
Full Changelog: v2.1.1...v2.2.1