| | |
|---|
| | | |
|---|
| | | <a name="whats-new"></a> |
|---|
| | | ## What's new: |
|---|
| | | - 2024/03/05：Added support for the Whisper-large-v3 model, a multitasking model that can perform multilingual speech recognition, speech translation, and language identification. It can be downloaded from the[modelscope](https://www.modelscope.cn/models/iic/Whisper-large-v3/summary), and [openai](https://github.com/alibaba-damo-academy/FunASR/tree/main/examples/industrial_data_pretraining/whisper). |
|---|
| | | - 2024/03/05：Added the Qwen-Audio and Qwen-Audio-Chat large-scale audio-text multimodal models, which have topped multiple audio domain leaderboards. These models support speech dialogue, [usage](examples/industrial_data_pretraining/qwen_audio). |
|---|
| | | - 2024/03/05：Added support for the Whisper-large-v3 model, a multitasking model that can perform multilingual speech recognition, speech translation, and language identification. It can be downloaded from the[modelscope](examples/industrial_data_pretraining/whisper/demo.py), and [openai](examples/industrial_data_pretraining/whisper/demo_from_openai.py). |
|---|
| | | - 2024/03/05: Offline File Transcription Service 4.4, Offline File Transcription Service of English 1.5，Real-time Transcription Service 1.9 released，docker image supports ARM64 platform, update modelscope；([docs](runtime/readme.md)) |
|---|
| | | - 2024/01/30：funasr-1.0 has been released ([docs](https://github.com/alibaba-damo-academy/FunASR/discussions/1319)) |
|---|
| | | - 2024/01/30：emotion recognition models are new supported. [model link](https://www.modelscope.cn/models/iic/emotion2vec_base_finetuned/summary), modified from [repo](https://github.com/ddlBoJack/emotion2vec). |
|---|
| | |
|---|
| | | | cam++ <br> ( [⭐](https://modelscope.cn/models/iic/speech_campplus_sv_zh-cn_16k-common/summary) [🤗](https://huggingface.co/funasr/campplus) ) | speaker verification/diarization | 5000 hours | 7.2M | |
|---|
| | | | Whisper-large-v2 <br> ([⭐](https://www.modelscope.cn/models/iic/speech_whisper-large_asr_multilingual/summary) [🍀](https://github.com/openai/whisper) ) | speech recognition, with timestamps, non-streaming | multilingual | 1.5G | |
|---|
| | | | Whisper-large-v3 <br> ([⭐](https://www.modelscope.cn/models/iic/Whisper-large-v3/summary) [🍀](https://github.com/openai/whisper) ) | speech recognition, with timestamps, non-streaming | multilingual | 1.5G | |
|---|
| | | | Qwen-Audio <br> ([⭐](examples/industrial_data_pretraining/qwen_audio/demo.py) [🤗](https://huggingface.co/Qwen/Qwen-Audio) ) | audio-text multimodal models (pretraining) | multilingual | 8B | |
|---|
| | | | Qwen-Audio-Chat <br> ([⭐](examples/industrial_data_pretraining/qwen_audio/demo_chat.py) [🤗](https://huggingface.co/Qwen/Qwen-Audio-Chat) ) | audio-text multimodal models (chat) | multilingual | 8B | |
|---|
| | | |
|---|
| | | |
|---|
| | | |
|---|
| | |
|---|
| | | |
|---|
| | | More examples ref to [docs](https://github.com/alibaba-damo-academy/FunASR/tree/main/examples/industrial_data_pretraining) |
|---|
| | | |
|---|
| | | [//]: # (FunASR supports inference and fine-tuning of models trained on industrial datasets of tens of thousands of hours. For more details, please refer to ([modelscope_egs](https://alibaba-damo-academy.github.io/FunASR/en/modelscope_pipeline/quick_start.html)). It also supports training and fine-tuning of models on academic standard datasets. For more details, please refer to([egs](https://alibaba-damo-academy.github.io/FunASR/en/academic_recipe/asr_recipe.html)). The models include speech recognition (ASR), speech activity detection (VAD), punctuation recovery, language model, speaker verification, speaker separation, and multi-party conversation speech recognition. For a detailed list of models, please refer to the [Model Zoo](https://github.com/alibaba-damo-academy/FunASR/blob/main/docs/model_zoo/modelscope_models.md):) |
|---|
| | | |
|---|
| | | ## Export ONNX |
|---|
| | | |
|---|
| | | ### Command-line usage |
|---|
| | | ```shell |
|---|
| | | funasr-export ++model=paraformer ++quantize=false ++device=cpu |
|---|
| | | ``` |
|---|
| | | |
|---|
| | | ### Python |
|---|
| | | ```python |
|---|
| | | from funasr import AutoModel |
|---|
| | | |
|---|
| | | model = AutoModel(model="paraformer", device="cpu") |
|---|
| | | |
|---|
| | | res = model.export(quantize=False) |
|---|
| | | ``` |
|---|
| | | |
|---|
| | | ### Text ONNX |
|---|
| | | ```python |
|---|
| | | # pip3 install -U funasr-onnx |
|---|
| | | from funasr_onnx import Paraformer |
|---|
| | | model_dir = "damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch" |
|---|
| | | model = Paraformer(model_dir, batch_size=1, quantize=True) |
|---|
| | | |
|---|
| | | wav_path = ['~/.cache/modelscope/hub/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/example/asr_example.wav'] |
|---|
| | | |
|---|
| | | result = model(wav_path) |
|---|
| | | print(result) |
|---|
| | | ``` |
|---|
| | | |
|---|
| | | More examples ref to [demo](runtime/python/onnxruntime) |
|---|
| | | |
|---|
| | | ## Deployment Service |
|---|
| | | FunASR supports deploying pre-trained or further fine-tuned models for service. Currently, it supports the following types of service deployment: |
|---|
| | |
|---|
| | | |
|---|
| | | You can also scan the following DingTalk group or WeChat group QR code to join the community group for communication and discussion. |
|---|
| | | |
|---|
| | | |DingTalk group | WeChat group | |
|---|
| | | |:---:|:-----------------------------------------------------:| |
|---|
| | | |<div align="left"><img src="docs/images/dingding.jpg" width="250"/> | <img src="docs/images/wechat.png" width="215"/></div> | |
|---|
| | | | DingTalk group | WeChat group | |
|---|
| | | |:-------------------------------------------------------------------:|:-----------------------------------------------------:| |
|---|
| | | | <div align="left"><img src="docs/images/dingding.png" width="250"/> | <img src="docs/images/wechat.png" width="215"/></div> | |
|---|
| | | |
|---|
| | | ## Contributors |
|---|
| | | |
|---|
