From 33d3d2084403fd34b79c835d2f2fe04f6cd8f738 Mon Sep 17 00:00:00 2001 From: 游雁 <zhifu.gzf@alibaba-inc.com> Date: 星期三, 13 九月 2023 09:33:54 +0800 Subject: [PATCH] Merge branch 'main' of github.com:alibaba-damo-academy/FunASR add --- funasr/runtime/docs/SDK_advanced_guide_online_zh.md | 173 ++++++++++----------------------------------------------- 1 files changed, 32 insertions(+), 141 deletions(-) diff --git a/funasr/runtime/docs/SDK_advanced_guide_online_zh.md b/funasr/runtime/docs/SDK_advanced_guide_online_zh.md index c7603c8..f5f6806 100644 --- a/funasr/runtime/docs/SDK_advanced_guide_online_zh.md +++ b/funasr/runtime/docs/SDK_advanced_guide_online_zh.md @@ -1,146 +1,74 @@ -# FunASR瀹炴椂璇煶杞啓鏈嶅姟寮�鍙戞寚鍗� +# FunASR瀹炴椂璇煶鍚啓鏈嶅姟寮�鍙戞寚鍗� -FunASR鎻愪緵鍙究鎹锋湰鍦版垨鑰呬簯绔湇鍔″櫒閮ㄧ讲鐨勫疄鏃惰闊宠浆鍐欐湇鍔★紝鍐呮牳涓篎unASR宸插紑婧恟untime-SDK銆� +FunASR鎻愪緵鍙究鎹锋湰鍦版垨鑰呬簯绔湇鍔″櫒閮ㄧ讲鐨勫疄鏃惰闊冲惉鍐欐湇鍔★紝鍐呮牳涓篎unASR宸插紑婧恟untime-SDK銆� 闆嗘垚浜嗚揪鎽╅櫌璇煶瀹為獙瀹ゅ湪Modelscope绀惧尯寮�婧愮殑璇煶绔偣妫�娴�(VAD)銆丳araformer-large闈炴祦寮忚闊宠瘑鍒�(ASR)銆丳araformer-large娴佸紡璇煶璇嗗埆(ASR)銆佹爣鐐�(PUNC) 绛夌浉鍏宠兘鍔涖�傝蒋浠跺寘鏃㈠彲浠ュ疄鏃跺湴杩涜璇煶杞枃瀛楋紝鑰屼笖鑳藉鍦ㄨ璇濆彞灏剧敤楂樼簿搴︾殑杞啓鏂囧瓧淇杈撳嚭锛岃緭鍑烘枃瀛楀甫鏈夋爣鐐癸紝鏀寔楂樺苟鍙戝璺姹� -鏈枃妗ｄ负FunASR瀹炴椂杞啓鏈嶅姟寮�鍙戞寚鍗椼�傚鏋滄偍鎯冲揩閫熶綋楠屽疄鏃惰闊宠浆鍐欐湇鍔★紝鍙弬鑰僛蹇�熶笂鎵媇(#蹇�熶笂鎵�)銆� +鏈枃妗ｄ负FunASR瀹炴椂杞啓鏈嶅姟寮�鍙戞寚鍗椼�傚鏋滄偍鎯冲揩閫熶綋楠屽疄鏃惰闊冲惉鍐欐湇鍔★紝鍙弬鑰僛蹇�熶笂鎵媇(#蹇�熶笂鎵�)銆� ## 蹇�熶笂鎵� ### 闀滃儚鍚姩 -閫氳繃涓嬭堪鍛戒护鎷夊彇骞跺惎鍔‵unASR runtime-SDK鐨刣ocker闀滃儚锛� +閫氳繃涓嬭堪鍛戒护鎷夊彇骞跺惎鍔‵unASR杞欢鍖呯殑docker闀滃儚锛� ```shell -sudo docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.0 - -sudo docker run -p 10095:10095 -it --privileged=true -v /root:/workspace/models registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.0 +sudo docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.1 +mkdir -p ./funasr-runtime-resources/models +sudo docker run -p 10095:10095 -it --privileged=true -v ./funasr-runtime-resources/models:/workspace/models registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.1 ``` -濡傛灉鎮ㄦ病鏈夊畨瑁卍ocker锛屽彲鍙傝�僛Docker瀹夎](#Docker瀹夎) +濡傛灉鎮ㄦ病鏈夊畨瑁卍ocker锛屽彲鍙傝�僛Docker瀹夎](https://alibaba-damo-academy.github.io/FunASR/en/installation/docker_zh.html) ### 鏈嶅姟绔惎鍔� docker鍚姩涔嬪悗锛屽惎鍔� funasr-wss-server-2pass鏈嶅姟绋嬪簭锛� ```shell cd FunASR/funasr/runtime -./run_server_2pass.sh \ +nohup bash run_server_2pass.sh \ --download-model-dir /workspace/models \ --vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx \ --model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx \ --online-model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx \ - --punc-dir damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx + --punc-dir damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx > log.out 2>&1 & + +# 濡傛灉鎮ㄦ兂鍏抽棴ssl锛屽鍔犲弬鏁帮細--certfile 0 ``` 鏈嶅姟绔缁嗗弬鏁颁粙缁嶅彲鍙傝�僛鏈嶅姟绔弬鏁颁粙缁峕(#鏈嶅姟绔弬鏁颁粙缁�) ### 瀹㈡埛绔祴璇曚笌浣跨敤 涓嬭浇瀹㈡埛绔祴璇曞伐鍏风洰褰晄amples ```shell -wget https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/sample/funasr_online_samples.tar.gz +wget https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/sample/funasr_samples.tar.gz ``` -鎴戜滑浠ython璇█瀹㈡埛绔负渚嬶紝杩涜璇存槑锛屾敮鎸佸绉嶉煶棰戞牸寮忚緭鍏ワ紙.wav, .pcm, .mp3绛夛級锛屼篃鏀寔瑙嗛杈撳叆(.mp4绛�)锛屼互鍙婂鏂囦欢鍒楄〃wav.scp杈撳叆锛屽叾浠栫増鏈鎴风璇峰弬鑰冩枃妗ｏ紙[鐐瑰嚮姝ゅ](#瀹㈡埛绔敤娉曡瑙�)锛夛紝瀹氬埗鏈嶅姟閮ㄧ讲璇峰弬鑰僛濡備綍瀹氬埗鏈嶅姟閮ㄧ讲](#濡備綍瀹氬埗鏈嶅姟閮ㄧ讲) +鎴戜滑浠ython璇█瀹㈡埛绔负渚嬶紝杩涜璇存槑锛屾敮鎸侀煶棰戞牸寮忥紙.wav, .pcm锛夛紝浠ュ強澶氭枃浠跺垪琛╳av.scp杈撳叆锛屽叾浠栫増鏈鎴风璇峰弬鑰冩枃妗ｏ紙[鐐瑰嚮姝ゅ](#瀹㈡埛绔敤娉曡瑙�)锛夈�� ```shell python3 wss_client_asr.py --host "127.0.0.1" --port 10095 --mode 2pass ``` ------------------ -## Docker瀹夎 - -涓嬭堪姝ラ涓烘墜鍔ㄥ畨瑁卍ocker鐜鐨勬楠わ細 - -### docker鐜瀹夎 -```shell -# Ubuntu锛� -curl -fsSL https://test.docker.com -o test-docker.sh -sudo sh test-docker.sh -# Debian锛� -curl -fsSL https://get.docker.com -o get-docker.sh -sudo sh get-docker.sh -# CentOS锛� -curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun -# MacOS锛� -brew install --cask --appdir=/Applications docker -``` - -瀹夎璇﹁锛歨ttps://alibaba-damo-academy.github.io/FunASR/en/installation/docker.html - -### docker鍚姩 - -```shell -sudo systemctl start docker -``` - ## 瀹㈡埛绔敤娉曡瑙� 鍦ㄦ湇鍔″櫒涓婂畬鎴怓unASR鏈嶅姟閮ㄧ讲浠ュ悗锛屽彲浠ラ�氳繃濡備笅鐨勬楠ゆ潵娴嬭瘯鍜屼娇鐢ㄧ绾挎枃浠惰浆鍐欐湇鍔°�� 鐩墠鍒嗗埆鏀寔浠ヤ笅鍑犵缂栫▼璇█瀹㈡埛绔� -- [Python](#python-client) -- [CPP](#cpp-client) -- [html缃戦〉鐗堟湰](#Html缃戦〉鐗�) -- [Java](#Java-client) +- [Python](./SDK_tutorial_online_zh.md#python-client) +- [CPP](./SDK_tutorial_online_zh.md#cpp-client) +- [html缃戦〉鐗堟湰](./SDK_tutorial_online_zh.md#html-client) +- [Java](./SDK_tutorial_online_zh.md#java-client) +- [c\#](./SDK_tutorial_online_zh.md#c\#) -### python-client -鑻ユ兂鐩存帴杩愯client杩涜娴嬭瘯锛屽彲鍙傝�冨涓嬬畝鏄撹鏄庯紝浠ython鐗堟湰涓轰緥锛� - -```shell -python3 wss_client_asr.py --host "127.0.0.1" --port 10095 --mode offline --audio_in "../audio/asr_example.wav" --output_dir "./results" -``` - -鍛戒护鍙傛暟璇存槑锛� -```text ---host 涓篎unASR runtime-SDK鏈嶅姟閮ㄧ讲鏈哄櫒ip锛岄粯璁や负鏈満ip锛�127.0.0.1锛夛紝濡傛灉client涓庢湇鍔′笉鍦ㄥ悓涓�鍙版湇鍔″櫒锛岄渶瑕佹敼涓洪儴缃叉満鍣╥p ---port 10095 閮ㄧ讲绔彛鍙� ---mode offline琛ㄧず绂荤嚎鏂囦欢杞啓 ---audio_in 闇�瑕佽繘琛岃浆鍐欑殑闊抽鏂囦欢锛屾敮鎸佹枃浠惰矾寰勶紝鏂囦欢鍒楄〃wav.scp ---output_dir 璇嗗埆缁撴灉淇濆瓨璺緞 -``` - -### cpp-client -杩涘叆samples/cpp鐩綍鍚庯紝鍙互鐢╟pp杩涜娴嬭瘯锛屾寚浠ゅ涓嬶細 -```shell -./funasr-wss-client-2pass --server-ip 127.0.0.1 --port 10095 --wav-path ../audio/asr_example.wav -``` - -鍛戒护鍙傛暟璇存槑锛� - -```text ---server-ip 涓篎unASR runtime-SDK鏈嶅姟閮ㄧ讲鏈哄櫒ip锛岄粯璁や负鏈満ip锛�127.0.0.1锛夛紝濡傛灉client涓庢湇鍔′笉鍦ㄥ悓涓�鍙版湇鍔″櫒锛岄渶瑕佹敼涓洪儴缃叉満鍣╥p ---port 10095 閮ㄧ讲绔彛鍙� ---wav-path 闇�瑕佽繘琛岃浆鍐欑殑闊抽鏂囦欢锛屾敮鎸佹枃浠惰矾寰� -``` - -### Html缃戦〉鐗� - -鍦ㄦ祻瑙堝櫒涓墦寮� html/static/index.html锛屽嵆鍙嚭鐜板涓嬮〉闈紝鏀寔楹﹀厠椋庤緭鍏ヤ笌鏂囦欢涓婁紶锛岀洿鎺ヨ繘琛屼綋楠� - -<img src="images/html.png" width="900"/> - -### Java-client - -```shell -FunasrWsClient --host localhost --port 10095 --audio_in ./asr_example.wav --mode offline -``` -璇︾粏鍙互鍙傝�冩枃妗ｏ紙[鐐瑰嚮姝ゅ](../java/readme.md)锛� - - +璇︾粏鐢ㄦ硶鍙互鐐瑰嚮杩涘叆鏌ョ湅銆傛洿澶氱増鏈鎴风鏀寔璇峰弬鑰僛websocket/grpc鍗忚](./websocket_protocol_zh.md) ## 鏈嶅姟绔弬鏁颁粙缁嶏細 -funasr-wss-server鏀寔浠嶮odelscope涓嬭浇妯″瀷锛岃缃ā鍨嬩笅杞藉湴鍧�锛�--download-model-dir锛岄粯璁や负/workspace/models锛夊強model ID锛�--model-dir銆�--vad-dir銆�--punc-dir锛�,绀轰緥濡備笅锛� +funasr-wss-server-2pass鏀寔浠嶮odelscope涓嬭浇妯″瀷锛屾垨鑰呬粠鏈湴鐩綍璺緞鍚姩锛岀ず渚嬪涓嬶細 ```shell cd /workspace/FunASR/funasr/runtime/websocket/build/bin -./funasr-wss-server \ - --download-model-dir /workspace/models \ - --model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx \ - --online-model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx \ - --vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx \ - --punc-dir damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx \ +./funasr-wss-server-2pass \ --decoder-thread-num 32 \ --io-thread-num 8 \ - --port 10095 \ - --certfile ../../../ssl_key/server.crt \ - --keyfile ../../../ssl_key/server.key + --port 10095 ``` + 鍛戒护鍙傛暟浠嬬粛锛� ```text --download-model-dir 妯″瀷涓嬭浇鍦板潃锛岄�氳繃璁剧疆model ID浠嶮odelscope涓嬭浇妯″瀷 @@ -154,52 +82,15 @@ --port 鏈嶅姟绔洃鍚殑绔彛鍙凤紝榛樿涓� 10095 --decoder-thread-num 鏈嶅姟绔惎鍔ㄧ殑鎺ㄧ悊绾跨▼鏁帮紝榛樿涓� 8 --io-thread-num 鏈嶅姟绔惎鍔ㄧ殑IO绾跨▼鏁帮紝榛樿涓� 1 ---certfile ssl鐨勮瘉涔︽枃浠讹紝榛樿涓猴細../../../ssl_key/server.crt ---keyfile ssl鐨勫瘑閽ユ枃浠讹紝榛樿涓猴細../../../ssl_key/server.key +--certfile ssl鐨勮瘉涔︽枃浠讹紝榛樿涓猴細../../../ssl_key/server.crt锛屽闇�鍏抽棴锛岃缃负"" +--keyfile ssl鐨勫瘑閽ユ枃浠讹紝榛樿涓猴細../../../ssl_key/server.key锛屽闇�鍏抽棴锛岃缃负"" ``` -## 妯″瀷璧勬簮鍑嗗 +鎵ц涓婅堪鎸囦护鍚庯紝鍚姩瀹炴椂璇煶鍚啓鏈嶅姟銆傚鏋滄ā鍨嬫寚瀹氫负ModelScope涓璵odel id锛屼細鑷姩浠嶮oldeScope涓笅杞藉涓嬫ā鍨嬶細 +[FSMN-VAD妯″瀷](https://www.modelscope.cn/models/damo/speech_fsmn_vad_zh-cn-16k-common-onnx/summary)锛� +[Paraformer-lagre瀹炴椂妯″瀷](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx/summary ) +[Paraformer-lagre闈炲疄鏃舵ā鍨媇(https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx/summary) +[CT-Transformer鏍囩偣棰勬祴妯″瀷](https://www.modelscope.cn/models/damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx/summary) -濡傛灉鎮ㄩ�夋嫨閫氳繃funasr-wss-server浠嶮odelscope涓嬭浇妯″瀷锛屽彲浠ヨ烦杩囨湰姝ラ銆� +濡傛灉锛屾偍甯屾湜閮ㄧ讲鎮╢inetune鍚庣殑妯″瀷锛堜緥濡�10epoch.pb锛夛紝闇�瑕佹墜鍔ㄥ皢妯″瀷閲嶅懡鍚嶄负model.pb锛屽苟灏嗗師modelscope涓ā鍨媘odel.pb鏇挎崲鎺夛紝灏嗚矾寰勬寚瀹氫负`model_dir`鍗冲彲銆� -FunASR绂荤嚎鏂囦欢杞啓鏈嶅姟涓殑vad銆乤sr鍜宲unc妯″瀷璧勬簮鍧囨潵鑷狹odelscope锛屾ā鍨嬪湴鍧�璇﹁涓嬭〃锛� - -| 妯″瀷 | Modelscope閾炬帴 | -|------|---------------------------------------------------------------------------------------------------------------| -| VAD | https://www.modelscope.cn/models/damo/speech_fsmn_vad_zh-cn-16k-common-onnx/summary | -| ASR | https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx/summary | -| ASR | https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx/summary | -| PUNC | https://www.modelscope.cn/models/damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx/summary | - -瀹炴椂杞啓鏈嶅姟涓儴缃茬殑鏄噺鍖栧悗鐨凮NNX妯″瀷锛屼笅闈粙缁嶄笅濡備綍瀵煎嚭ONNX妯″瀷鍙婂叾閲忓寲锛氭偍鍙互閫夋嫨浠嶮odelscope瀵煎嚭ONNX妯″瀷銆佷粠finetune鍚庣殑璧勬簮瀵煎嚭妯″瀷锛� - -### 浠嶮odelscope瀵煎嚭ONNX妯″瀷 - -浠嶮odelscope缃戠珯涓嬭浇瀵瑰簲model name鐨勬ā鍨嬶紝鐒跺悗瀵煎嚭閲忓寲鍚庣殑ONNX妯″瀷锛� - -```shell -python -m funasr.export.export_model \ ---export-dir ./export \ ---type onnx \ ---quantize True \ ---model-name damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch \ ---model-name damo/speech_fsmn_vad_zh-cn-16k-common-pytorch \ ---model-name damo/punc_ct-transformer_zh-cn-common-vocab272727-pytorch -``` - -鍛戒护鍙傛暟浠嬬粛锛� -```text ---model-name Modelscope涓婄殑妯″瀷鍚嶇О锛屼緥濡俤amo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch ---export-dir ONNX妯″瀷瀵煎嚭鍦板潃 ---type 妯″瀷绫诲瀷锛岀洰鍓嶆敮鎸� ONNX銆乼orch ---quantize int8妯″瀷閲忓寲 -``` -### 浠巉inetune鍚庣殑璧勬簮瀵煎嚭妯″瀷 - -鍋囧鎮ㄦ兂閮ㄧ讲finetune鍚庣殑妯″瀷锛屽彲浠ュ弬鑰冨涓嬫楠わ細 - -灏嗘偍finetune鍚庨渶瑕侀儴缃茬殑妯″瀷锛堜緥濡�10epoch.pb锛夛紝閲嶅懡鍚嶄负model.pb锛屽苟灏嗗師modelscope涓ā鍨媘odel.pb鏇挎崲鎺夛紝鍋囧鏇挎崲鍚庣殑妯″瀷璺緞涓�/path/to/finetune/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch锛岄�氳繃涓嬭堪鍛戒护鎶奻inetune鍚庣殑妯″瀷杞垚onnx妯″瀷锛� - -```shell -python -m funasr.export.export_model --model-name /path/to/finetune/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch --export-dir ./export --type onnx --quantize True -``` -- Gitblit v1.9.1