| | |
|---|
| | | | [**News**](https://github.com/alibaba-damo-academy/FunASR#whats-new) |
|---|
| | | | [**Installation**](#installation) |
|---|
| | | | [**Quick Start**](#quick-start) |
|---|
| | | | [**Tutorial**](https://github.com/alibaba-damo-academy/FunASR/blob/main/docs/tutorial/README.md) |
|---|
| | | | [**Runtime**](./runtime/readme.md) |
|---|
| | | | [**Model Zoo**](#model-zoo) |
|---|
| | | | [**Contact**](#contact) |
|---|
| | |
|---|
| | | ``` |
|---|
| | | |
|---|
| | | ## Model Zoo |
|---|
| | | FunASR has open-sourced a large number of pre-trained models on industrial data. You are free to use, copy, modify, and share FunASR models under the [Model License Agreement](./MODEL_LICENSE). Below are some representative models, for more models please refer to the [Model Zoo](). |
|---|
| | | FunASR has open-sourced a large number of pre-trained models on industrial data. You are free to use, copy, modify, and share FunASR models under the [Model License Agreement](./MODEL_LICENSE). Below are some representative models, for more models please refer to the [Model Zoo](./model_zoo). |
|---|
| | | |
|---|
| | | (Note: ⭐ represents the ModelScope model zoo, 🤗 represents the Huggingface model zoo, 🍀 represents the OpenAI model zoo) |
|---|
| | | |
|---|
| | |
|---|
| | | | fsmn-vad <br> ( [⭐](https://modelscope.cn/models/damo/speech_fsmn_vad_zh-cn-16k-common-pytorch/summary) [🤗](https://huggingface.co/funasr/fsmn-vad) ) | voice activity detection | 5000 hours, Mandarin and English | 0.4M | |
|---|
| | | | fa-zh <br> ( [⭐](https://modelscope.cn/models/damo/speech_timestamp_prediction-v1-16k-offline/summary) [🤗](https://huggingface.co/funasr/fa-zh) ) | timestamp prediction | 5000 hours, Mandarin | 38M | |
|---|
| | | | cam++ <br> ( [⭐](https://modelscope.cn/models/iic/speech_campplus_sv_zh-cn_16k-common/summary) [🤗](https://huggingface.co/funasr/campplus) ) | speaker verification/diarization | 5000 hours | 7.2M | |
|---|
| | | | Whisper-large-v2 <br> ([⭐](https://www.modelscope.cn/models/iic/speech_whisper-large_asr_multilingual/summary) [🍀](https://github.com/openai/whisper) ) | speech recognition, with timestamps, non-streaming | multilingual | 1.5G | |
|---|
| | | | Whisper-large-v3 <br> ([⭐](https://www.modelscope.cn/models/iic/Whisper-large-v3/summary) [🍀](https://github.com/openai/whisper) ) | speech recognition, with timestamps, non-streaming | multilingual | 1.5G | |
|---|
| | | | Whisper-large-v2 <br> ([⭐](https://www.modelscope.cn/models/iic/speech_whisper-large_asr_multilingual/summary) [🍀](https://github.com/openai/whisper) ) | speech recognition, with timestamps, non-streaming | multilingual | 1550 M | |
|---|
| | | | Whisper-large-v3 <br> ([⭐](https://www.modelscope.cn/models/iic/Whisper-large-v3/summary) [🍀](https://github.com/openai/whisper) ) | speech recognition, with timestamps, non-streaming | multilingual | 1550 M | |
|---|
| | | | Qwen-Audio <br> ([⭐](examples/industrial_data_pretraining/qwen_audio/demo.py) [🤗](https://huggingface.co/Qwen/Qwen-Audio) ) | audio-text multimodal models (pretraining) | multilingual | 8B | |
|---|
| | | | Qwen-Audio-Chat <br> ([⭐](examples/industrial_data_pretraining/qwen_audio/demo_chat.py) [🤗](https://huggingface.co/Qwen/Qwen-Audio-Chat) ) | audio-text multimodal models (chat) | multilingual | 8B | |
|---|
| | | |
|---|
| | |
|---|
| | | <a name="quick-start"></a> |
|---|
| | | ## Quick Start |
|---|
| | | |
|---|
| | | Below is a quick start tutorial. Test audio files ([Mandarin](https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/vad_example.wav), [English]()). |
|---|
| | | Below is a quick start tutorial. Test audio files ([Mandarin](https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/vad_example.wav), [English](https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_en.wav)). |
|---|
| | | |
|---|
| | | ### Command-line usage |
|---|
| | | |
|---|
| | |
|---|
| | | from funasr import AutoModel |
|---|
| | | # paraformer-zh is a multi-functional asr model |
|---|
| | | # use vad, punc, spk or not as you need |
|---|
| | | model = AutoModel(model="paraformer-zh", vad_model="fsmn-vad", punc_model="ct-punc-c", |
|---|
| | | model = AutoModel(model="paraformer-zh", vad_model="fsmn-vad", punc_model="ct-punc", |
|---|
| | | # spk_model="cam++", |
|---|
| | | ) |
|---|
| | | res = model.generate(input=f"{model.model_path}/example/asr_example.wav", |
|---|
| | |
|---|
| | | from funasr import AutoModel |
|---|
| | | |
|---|
| | | model = AutoModel(model="fsmn-vad") |
|---|
| | | wav_file = f"{model.model_path}/example/asr_example.wav" |
|---|
| | | wav_file = f"{model.model_path}/example/vad_example.wav" |
|---|
| | | res = model.generate(input=wav_file) |
|---|
| | | print(res) |
|---|
| | | ``` |
|---|
| | |
|---|
| | | res = model.generate(input=(wav_file, text_file), data_type=("sound", "text")) |
|---|
| | | print(res) |
|---|
| | | ``` |
|---|
| | | |
|---|
| | | More examples ref to [docs](https://github.com/alibaba-damo-academy/FunASR/tree/main/examples/industrial_data_pretraining) |
|---|
| | | More usages ref to [docs](docs/tutorial/README_zh.md), |
|---|
| | | more examples ref to [demo](https://github.com/alibaba-damo-academy/FunASR/tree/main/examples/industrial_data_pretraining) |
|---|
| | | |
|---|
| | | |
|---|
| | | ## Export ONNX |
|---|
| | |
|---|
| | | res = model.export(quantize=False) |
|---|
| | | ``` |
|---|
| | | |
|---|
| | | ### Text ONNX |
|---|
| | | ### Test ONNX |
|---|
| | | ```python |
|---|
| | | # pip3 install -U funasr-onnx |
|---|
| | | from funasr_onnx import Paraformer |
|---|
