From 651737380b2be42ae5182a777abb0938a36aedc1 Mon Sep 17 00:00:00 2001
From: jmwang66 <wangjiaming.wjm@alibaba-inc.com>
Date: 星期三, 09 八月 2023 16:48:02 +0800
Subject: [PATCH] Merge branch 'main' into dev_wjm_modelscope
---
egs_modelscope/vad/TEMPLATE/README_zh.md | 113 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 113 insertions(+), 0 deletions(-)
diff --git a/egs_modelscope/vad/TEMPLATE/README_zh.md b/egs_modelscope/vad/TEMPLATE/README_zh.md
new file mode 100644
index 0000000..38123a4
--- /dev/null
+++ b/egs_modelscope/vad/TEMPLATE/README_zh.md
@@ -0,0 +1,113 @@
+(绠�浣撲腑鏂噟[English](./README.md))
+
+# 璇煶绔偣妫�娴�
+
+> **娉ㄦ剰**:
+> Pipeline 鏀寔鍦╗modelscope妯″瀷浠撳簱](https://alibaba-damo-academy.github.io/FunASR/en/model_zoo/modelscope_models.html#pretrained-models-on-modelscope)涓殑鎵�鏈夋ā鍨嬭繘琛屾帹鐞嗗拰寰皟銆傚湪杩欓噷锛屾垜浠互 FSMN-VAD 妯″瀷涓轰緥鏉ユ紨绀轰娇鐢ㄦ柟娉曘��
+
+## 鎺ㄧ悊
+
+### 蹇�熶娇鐢�
+#### [FSMN-VAD 妯″瀷](https://modelscope.cn/models/damo/speech_fsmn_vad_zh-cn-16k-common-pytorch/summary)
+```python
+from modelscope.pipelines import pipeline
+from modelscope.utils.constant import Tasks
+
+inference_pipeline = pipeline(
+ task=Tasks.voice_activity_detection,
+ model='damo/speech_fsmn_vad_zh-cn-16k-common-pytorch',
+)
+
+segments_result = inference_pipeline(audio_in='https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/vad_example.wav')
+print(segments_result)
+```
+#### [FSMN-VAD-瀹炴椂妯″瀷](https://modelscope.cn/models/damo/speech_fsmn_vad_zh-cn-16k-common-pytorch/summary)
+```python
+inference_pipeline = pipeline(
+ task=Tasks.voice_activity_detection,
+ model='damo/speech_fsmn_vad_zh-cn-16k-common-pytorch',
+ )
+import soundfile
+speech, sample_rate = soundfile.read("example/asr_example.wav")
+
+param_dict = {"in_cache": dict(), "is_final": False}
+chunk_stride = 1600# 100ms
+# first chunk, 100ms
+speech_chunk = speech[0:chunk_stride]
+rec_result = inference_pipeline(audio_in=speech_chunk, param_dict=param_dict)
+print(rec_result)
+# next chunk, 480ms
+speech_chunk = speech[chunk_stride:chunk_stride+chunk_stride]
+rec_result = inference_pipeline(audio_in=speech_chunk, param_dict=param_dict)
+print(rec_result)
+```
+婕旂ず绀轰緥锛屽畬鏁翠唬鐮佽鍙傝�� [demo](https://github.com/alibaba-damo-academy/FunASR/discussions/236)
+
+
+
+### API鎺ュ彛璇存槑
+#### pipeline瀹氫箟
+- `task`: `Tasks.voice_activity_detection`
+- `model`: [妯″瀷浠撳簱](https://alibaba-damo-academy.github.io/FunASR/en/model_zoo/modelscope_models.html#pretrained-models-on-modelscope) 涓殑妯″瀷鍚嶇О锛屾垨鏈湴纾佺洏涓殑妯″瀷璺緞
+- `ngpu`: `1`锛堥粯璁わ級锛屼娇鐢� GPU 杩涜鎺ㄧ悊銆傚鏋� ngpu=0锛屽垯浣跨敤 CPU 杩涜鎺ㄧ悊
+- `ncpu`: `1` 锛堥粯璁わ級锛岃缃敤浜� CPU 鍐呴儴鎿嶄綔骞惰鎬х殑绾跨▼鏁�
+- `output_dir`: `None` 锛堥粯璁わ級锛屽鏋滆缃紝杈撳嚭缁撴灉鐨勮緭鍑鸿矾寰�
+- `batch_size`: `1` 锛堥粯璁わ級锛岃В鐮佹椂鐨勬壒澶勭悊澶у皬
+#### pipeline 鎺ㄧ悊
+- `audio_in`: 瑕佽В鐮佺殑杈撳叆锛屽彲浠ユ槸锛�
+ - wav鏂囦欢璺緞, 渚嬪: asr_example.wav,
+ - pcm鏂囦欢璺緞, 渚嬪: asr_example.pcm,
+ - 闊抽瀛楄妭鏁版祦锛屼緥濡傦細楹﹀厠椋庣殑瀛楄妭鏁版暟鎹�
+ - 闊抽閲囨牱鐐癸紝渚嬪锛歚audio, rate = soundfile.read("asr_example_zh.wav")`, 鏁版嵁绫诲瀷涓� numpy.ndarray 鎴栬�� torch.Tensor
+ - wav.scp锛宬aldi 鏍峰紡鐨� wav 鍒楄〃 (`wav_id \t wav_path`), 渚嬪:
+ ```text
+ asr_example1 ./audios/asr_example1.wav
+ asr_example2 ./audios/asr_example2.wav
+ ```
+ 鍦ㄨ繖绉嶈緭鍏� `wav.scp` 鐨勬儏鍐典笅锛屽繀椤昏缃� `output_dir` 浠ヤ繚瀛樿緭鍑虹粨鏋�
+- `audio_fs`: 闊抽閲囨牱鐜囷紝浠呭湪 audio_in 涓� pcm 闊抽鏃惰缃�
+- `output_dir`: None 锛堥粯璁わ級锛屽鏋滆缃紝杈撳嚭缁撴灉鐨勮緭鍑鸿矾寰�
+
+
+### 浣跨敤澶氱嚎绋� CPU 鎴栧涓� GPU 杩涜鎺ㄧ悊
+FunASR 杩樻彁渚涗簡 [egs_modelscope/vad/TEMPLATE/infer.sh](infer.sh) 鑴氭湰锛屼互浣跨敤澶氱嚎绋� CPU 鎴栧涓� GPU 杩涜瑙ｇ爜銆�
+
+#### `infer.sh` 璁剧疆
+- `model`: [modelscope妯″瀷浠撳簱](https://alibaba-damo-academy.github.io/FunASR/en/model_zoo/modelscope_models.html#pretrained-models-on-modelscope)涓殑妯″瀷鍚嶇О锛屾垨鏈湴纾佺洏涓殑妯″瀷璺緞
+- `data_dir`: 鏁版嵁闆嗙洰褰曢渶瑕佸寘鎷� `wav.scp` 鏂囦欢銆傚鏋� `${data_dir}/text` 涔熷瓨鍦紝鍒欏皢璁＄畻 CER
+- `output_dir`: 璇嗗埆缁撴灉鐨勮緭鍑虹洰褰�
+- `batch_size`: `1`锛堥粯璁わ級锛屽湪 GPU 涓婅繘琛屾帹鐞嗙殑鎵瑰鐞嗗ぇ灏�
+- `gpu_inference`: `true` 锛堥粯璁わ級锛屾槸鍚︽墽琛� GPU 瑙ｇ爜锛屽鏋滆繘琛� CPU 鎺ㄧ悊锛屽垯璁剧疆涓� `false`
+- `gpuid_list`: `0,1` 锛堥粯璁わ級锛岀敤浜庢帹鐞嗙殑 GPU ID
+- `njob`: 浠呯敤浜� CPU 鎺ㄧ悊锛坄gpu_inference=false`锛夛紝`64`锛堥粯璁わ級锛孋PU 瑙ｇ爜鐨勪綔涓氭暟
+
+#### 浣跨敤澶氫釜 GPU 杩涜瑙ｇ爜锛�
+```shell
+ bash infer.sh \
+ --model "damo/speech_fsmn_vad_zh-cn-16k-common-pytorch" \
+ --data_dir "./data/test" \
+ --output_dir "./results" \
+ --batch_size 1 \
+ --gpu_inference true \
+ --gpuid_list "0,1"
+```
+#### 浣跨敤澶氱嚎绋� CPU 杩涜瑙ｇ爜锛�
+```shell
+ bash infer.sh \
+ --model "damo/speech_fsmn_vad_zh-cn-16k-common-pytorch" \
+ --data_dir "./data/test" \
+ --output_dir "./results" \
+ --gpu_inference false \
+ --njob 64
+```
+
+
+
+## Finetune with pipeline
+
+### Quick start
+
+### Finetune with your data
+
+## Inference with your finetuned model
+
--
Gitblit v1.9.1