| | |
|---|
| | | # ONNXRuntime-cpp for Websocket Server
|
|---|
| | | # Service with websocket-cpp
|
|---|
| | |
|
|---|
| | | ## Export the model
|
|---|
| | | ### Install [modelscope and funasr](https://github.com/alibaba-damo-academy/FunASR#installation)
|
|---|
| | |
|
|---|
| | | ```shell
|
|---|
| | | # pip3 install torch torchaudio
|
|---|
| | | pip install -U modelscope funasr
|
|---|
| | | pip3 install -U modelscope funasr
|
|---|
| | | # For the users in China, you could install with the command:
|
|---|
| | | # pip install -U modelscope funasr -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html -i https://mirror.sjtu.edu.cn/pypi/web/simple
|
|---|
| | | # pip3 install -U modelscope funasr -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html -i https://mirror.sjtu.edu.cn/pypi/web/simple
|
|---|
| | | ```
|
|---|
| | |
|
|---|
| | | ### Export [onnx model](https://github.com/alibaba-damo-academy/FunASR/tree/main/funasr/export)
|
|---|
| | |
|
|---|
| | | ```shell
|
|---|
| | | python -m funasr.export.export_model --model-name damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch --export-dir ./export --type onnx --quantize True
|
|---|
| | | python -m funasr.export.export_model \
|
|---|
| | | --export-dir ./export \
|
|---|
| | | --type onnx \
|
|---|
| | | --quantize True \
|
|---|
| | | --model-name damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch \
|
|---|
| | | --model-name damo/speech_fsmn_vad_zh-cn-16k-common-pytorch \
|
|---|
| | | --model-name damo/punc_ct-transformer_zh-cn-common-vocab272727-pytorch
|
|---|
| | | ```
|
|---|
| | |
|
|---|
| | | ## Building for Linux/Unix
|
|---|
| | |
|---|
| | | ```
|
|---|
| | |
|
|---|
| | | ### Build runtime
|
|---|
| | | required openssl lib
|
|---|
| | |
|
|---|
| | | ```shell
|
|---|
| | | git clone https://github.com/alibaba-damo-academy/FunASR.git && cd funasr/runtime/websocket
|
|---|
| | | apt-get install libssl-dev #ubuntu |
|---|
| | | # yum install openssl-devel #centos
|
|---|
| | |
|
|---|
| | |
|
|---|
| | | git clone https://github.com/alibaba-damo-academy/FunASR.git && cd FunASR/funasr/runtime/websocket
|
|---|
| | | mkdir build && cd build
|
|---|
| | | cmake -DCMAKE_BUILD_TYPE=release .. -DONNXRUNTIME_DIR=/path/to/onnxruntime-linux-x64-1.14.0
|
|---|
| | | make
|
|---|
| | |
|---|
| | |
|
|---|
| | | ```shell
|
|---|
| | | cd bin
|
|---|
| | | websocketmain [--model_thread_num <int>] [--decoder_thread_num
|
|---|
| | | <int>] [--io_thread_num <int>] [--port <int>]
|
|---|
| | | [--listen_ip <string>] [--wav-scp <string>]
|
|---|
| | | [--wav-path <string>] [--punc-config <string>]
|
|---|
| | | [--punc-model <string>] --am-config <string>
|
|---|
| | | --am-cmvn <string> --am-model <string>
|
|---|
| | | [--vad-config <string>] [--vad-cmvn <string>]
|
|---|
| | | [--vad-model <string>] [--] [--version] [-h]
|
|---|
| | | ./funasr-wss-server [--download-model-dir <string>]
|
|---|
| | | [--model-thread-num <int>] [--decoder-thread-num <int>]
|
|---|
| | | [--io-thread-num <int>] [--port <int>] [--listen_ip
|
|---|
| | | <string>] [--punc-quant <string>] [--punc-dir <string>]
|
|---|
| | | [--vad-quant <string>] [--vad-dir <string>] [--quantize
|
|---|
| | | <string>] --model-dir <string> [--keyfile <string>]
|
|---|
| | | [--certfile <string>] [--] [--version] [-h]
|
|---|
| | | Where:
|
|---|
| | | --wav-scp <string>
|
|---|
| | | wave scp path
|
|---|
| | | --wav-path <string>
|
|---|
| | | wave file path
|
|---|
| | | --download-model-dir <string>
|
|---|
| | | Download model from Modelscope to download_model_dir
|
|---|
| | |
|
|---|
| | | --punc-config <string>
|
|---|
| | | punc config path
|
|---|
| | | --punc-model <string>
|
|---|
| | | punc model path
|
|---|
| | | --model-dir <string>
|
|---|
| | | default: /workspace/models/asr, the asr model path, which contains model_quant.onnx, config.yaml, am.mvn
|
|---|
| | | --quantize <string>
|
|---|
| | | true (Default), load the model of model_quant.onnx in model_dir. If set false, load the model of model.onnx in model_dir
|
|---|
| | |
|
|---|
| | | --am-config <string>
|
|---|
| | | (required) am config path
|
|---|
| | | --am-cmvn <string>
|
|---|
| | | (required) am cmvn path
|
|---|
| | | --am-model <string>
|
|---|
| | | (required) am model path
|
|---|
| | | --vad-dir <string>
|
|---|
| | | default: /workspace/models/vad, the vad model path, which contains model_quant.onnx, vad.yaml, vad.mvn
|
|---|
| | | --vad-quant <string>
|
|---|
| | | true (Default), load the model of model_quant.onnx in vad_dir. If set false, load the model of model.onnx in vad_dir
|
|---|
| | |
|
|---|
| | | --vad-config <string>
|
|---|
| | | vad config path
|
|---|
| | | --vad-cmvn <string>
|
|---|
| | | vad cmvn path
|
|---|
| | | --vad-model <string>
|
|---|
| | | vad model path
|
|---|
| | | --decoder_thread_num <int>
|
|---|
| | | number of threads for decoder
|
|---|
| | | --io_thread_num <int>
|
|---|
| | | number of threads for network io
|
|---|
| | | --punc-dir <string>
|
|---|
| | | default: /workspace/models/punc, the punc model path, which contains model_quant.onnx, punc.yaml
|
|---|
| | | --punc-quant <string>
|
|---|
| | | true (Default), load the model of model_quant.onnx in punc_dir. If set false, load the model of model.onnx in punc_dir
|
|---|
| | |
|
|---|
| | | --decoder-thread-num <int>
|
|---|
| | | number of threads for decoder, default:8
|
|---|
| | | --io-thread-num <int>
|
|---|
| | | number of threads for network io, default:8
|
|---|
| | | --port <int>
|
|---|
| | | listen port, default:10095
|
|---|
| | | --certfile <string>
|
|---|
| | | default: ../../../ssl_key/server.crt, path of certficate for WSS connection. if it is empty, it will be in WS mode.
|
|---|
| | | --keyfile <string>
|
|---|
| | | default: ../../../ssl_key/server.key, path of keyfile for WSS connection
|
|---|
| | |
|
|---|
| | | Required: --am-config <string> --am-cmvn <string> --am-model <string> |
|---|
| | | If use vad, please add: [--vad-config <string>] [--vad-cmvn <string>] [--vad-model <string>]
|
|---|
| | | If use punc, please add: [--punc-config <string>] [--punc-model <string>] |
|---|
| | | example:
|
|---|
| | | ./bin/websocketmain --am-config /FunASR/funasr/runtime/onnxruntime/export/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/config.yaml --am-model /FunASR/funasr/runtime/onnxruntime/export/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/model.onnx --am-cmvn /FunASR/funasr/runtime/onnxruntime/export/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/am.mvn
|
|---|
| | | # you can use models downloaded from modelscope or local models:
|
|---|
| | | # download models from modelscope
|
|---|
| | | ./funasr-wss-server \
|
|---|
| | | --download-model-dir /workspace/models \
|
|---|
| | | --model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx \
|
|---|
| | | --vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
|---|
| | | --punc-dir damo/punc_ct-transformer_zh-cn-common-vocab272727-onnx
|
|---|
| | |
|
|---|
| | | # load models from local paths
|
|---|
| | | ./funasr-wss-server \
|
|---|
| | | --model-dir /workspace/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx \
|
|---|
| | | --vad-dir /workspace/models/damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
|
|---|
| | | --punc-dir /workspace/models/damo/punc_ct-transformer_zh-cn-common-vocab272727-onnx
|
|---|
| | |
|
|---|
| | | ```
|
|---|
| | |
|
|---|
| | | ## Run websocket client test
|
|---|
| | | Usage: websocketclient server_ip port wav_path threads_num
|
|---|
| | |
|
|---|
| | | ```shell
|
|---|
| | | ./funasr-wss-client --server-ip <string>
|
|---|
| | | --port <string>
|
|---|
| | | --wav-path <string>
|
|---|
| | | [--thread-num <int>] |
|---|
| | | [--is-ssl <int>] [--]
|
|---|
| | | [--version] [-h]
|
|---|
| | |
|
|---|
| | | Where:
|
|---|
| | | --server-ip <string>
|
|---|
| | | (required) server-ip
|
|---|
| | |
|
|---|
| | | --port <string>
|
|---|
| | | (required) port
|
|---|
| | |
|
|---|
| | | --wav-path <string>
|
|---|
| | | (required) the input could be: wav_path, e.g.: asr_example.wav;
|
|---|
| | | pcm_path, e.g.: asr_example.pcm; wav.scp, kaldi style wav list (wav_id \t wav_path)
|
|---|
| | |
|
|---|
| | | --thread-num <int>
|
|---|
| | | thread-num
|
|---|
| | |
|
|---|
| | | --is-ssl <int>
|
|---|
| | | is-ssl is 1 means use wss connection, or use ws connection
|
|---|
| | |
|
|---|
| | | example:
|
|---|
| | | websocketclient 127.0.0.1 8889 funasr/runtime/websocket/test.pcm.wav 64
|
|---|
| | | ./funasr-wss-client --server-ip 127.0.0.1 --port 10095 --wav-path test.wav --thread-num 1 --is-ssl 1
|
|---|
| | |
|
|---|
| | | result json, example like:
|
|---|
| | | {"text":"一二三四五六七八九十一二三四五六七八九十"}
|
|---|
| | | {"mode":"offline","text":"欢迎大家来体验达摩院推出的语音识别模型","wav_name":"wav2"}
|
|---|
| | | ```
|
|---|
| | |
|
|---|
| | |
|
|---|
| | | ## Acknowledge
|
|---|
| | | 1. This project is maintained by [FunASR community](https://github.com/alibaba-damo-academy/FunASR).
|
|---|
| | | 2. We acknowledge [zhaoming](https://github.com/zhaomingwork/FunASR/tree/add-offline-websocket-srv/funasr/runtime/websocket) for contributing the websocket(cpp-api).
|
|---|
| | |
|
|---|
| | |
|
|---|
