Service with grpc-cpp

For the Server

Build onnxruntime as it's document

Compile and install grpc v1.52.0 in case of grpc bugs

export GRPC_INSTALL_DIR=/data/soft/grpc
export PKG_CONFIG_PATH=$GRPC_INSTALL_DIR/lib/pkgconfig

git clone -b v1.52.0 --depth=1  https://github.com/grpc/grpc.git
cd grpc
git submodule update --init --recursive

mkdir -p cmake/build
pushd cmake/build
cmake -DgRPC_INSTALL=ON \
      -DgRPC_BUILD_TESTS=OFF \
      -DCMAKE_INSTALL_PREFIX=$GRPC_INSTALL_DIR \
      ../..
make
make install
popd

echo "export GRPC_INSTALL_DIR=/data/soft/grpc" >> ~/.bashrc
echo "export PKG_CONFIG_PATH=\$GRPC_INSTALL_DIR/lib/pkgconfig" >> ~/.bashrc
echo "export PATH=\$GRPC_INSTALL_DIR/bin/:\$PKG_CONFIG_PATH:\$PATH" >> ~/.bashrc
source ~/.bashrc

Compile and start grpc onnx paraformer server

# set -DONNXRUNTIME_DIR=/path/to/asrmodel/onnxruntime-linux-x64-1.14.0
./rebuild.sh

Start grpc paraformer server


./cmake/build/paraformer-server   --port-id <string> [--punc-quant <string>]
                                  [--punc-dir <string>] [--vad-quant <string>]
                                  [--vad-dir <string>] [--quantize <string>]
                                  --model-dir <string> [--] [--version] [-h]
Where:
   --port-id <string>
     (required)  port id
   --model-dir <string>
     (required)  the asr model path, which contains model.onnx, config.yaml, am.mvn
   --quantize <string>
     false (Default), load the model of model.onnx in model_dir. If set true, load the model of model_quant.onnx in model_dir

   --vad-dir <string>
     the vad model path, which contains model.onnx, vad.yaml, vad.mvn
   --vad-quant <string>
     false (Default), load the model of model.onnx in vad_dir. If set true, load the model of model_quant.onnx in vad_dir

   --punc-dir <string>
     the punc model path, which contains model.onnx, punc.yaml
   --punc-quant <string>
     false (Default), load the model of model.onnx in punc_dir. If set true, load the model of model_quant.onnx in punc_dir
  
   Required: --port-id <string>  --model-dir <string>
   If use vad, please add: --vad-dir <string>
   If use punc, please add: --punc-dir <string>

For the client

Install the requirements as in grpc-python

git clone https://github.com/alibaba/FunASR.git && cd FunASR
cd funasr/runtime/python/grpc
pip install -r requirements_client.txt

Generate protobuf file

Run on server, the two generated pb files are both used for server and client

# paraformer_pb2.py and paraformer_pb2_grpc.py are already generated, 
# regenerate it only when you make changes to ./proto/paraformer.proto file.
python -m grpc_tools.protoc  --proto_path=./proto -I ./proto    --python_out=. --grpc_python_out=./ ./proto/paraformer.proto

Start grpc client

# Start client.
python grpc_main_client_mic.py --host 127.0.0.1 --port 10095

Acknowledge

This project is maintained by FunASR community.
We acknowledge DeepScience for contributing the grpc service.