| | |
| | | # ONNXRuntime-python |
| | | |
| | | |
| | | ## Install `funasr-onnx` |
| | | |
| | | install from pip |
| | | |
| | | ```shell |
| | | pip install -U funasr-onnx |
| | | # For the users in China, you could install with the command: |
| | |
| | | ## Inference with runtime |
| | | |
| | | ### Speech Recognition |
| | | |
| | | #### Paraformer |
| | | |
| | | ```python |
| | | from funasr_onnx import Paraformer |
| | | from pathlib import Path |
| | |
| | | result = model(wav_path) |
| | | print(result) |
| | | ``` |
| | | |
| | | - `model_dir`: model_name in modelscope or local path downloaded from modelscope. If the local path is set, it should contain `model.onnx`, `config.yaml`, `am.mvn` |
| | | - `batch_size`: `1` (Default), the batch size duration inference |
| | | - `device_id`: `-1` (Default), infer on CPU. If you want to infer with GPU, set it to gpu_id (Please make sure that you have install the onnxruntime-gpu) |
| | |
| | | #### Paraformer-online |
| | | |
| | | ### Voice Activity Detection |
| | | |
| | | #### FSMN-VAD |
| | | |
| | | ```python |
| | | from funasr_onnx import Fsmn_vad |
| | | from pathlib import Path |
| | |
| | | result = model(wav_path) |
| | | print(result) |
| | | ``` |
| | | |
| | | - `model_dir`: model_name in modelscope or local path downloaded from modelscope. If the local path is set, it should contain `model.onnx`, `config.yaml`, `am.mvn` |
| | | - `batch_size`: `1` (Default), the batch size duration inference |
| | | - `device_id`: `-1` (Default), infer on CPU. If you want to infer with GPU, set it to gpu_id (Please make sure that you have install the onnxruntime-gpu) |
| | |
| | | |
| | | Output: `List[str]`: recognition result |
| | | |
| | | |
| | | #### FSMN-VAD-online |
| | | |
| | | ```python |
| | | from funasr_onnx import Fsmn_vad_online |
| | | import soundfile |
| | |
| | | if segments_result: |
| | | print(segments_result) |
| | | ``` |
| | | |
| | | - `model_dir`: model_name in modelscope or local path downloaded from modelscope. If the local path is set, it should contain `model.onnx`, `config.yaml`, `am.mvn` |
| | | - `batch_size`: `1` (Default), the batch size duration inference |
| | | - `device_id`: `-1` (Default), infer on CPU. If you want to infer with GPU, set it to gpu_id (Please make sure that you have install the onnxruntime-gpu) |
| | |
| | | |
| | | Output: `List[str]`: recognition result |
| | | |
| | | |
| | | ### Punctuation Restoration |
| | | |
| | | #### CT-Transformer |
| | | |
| | | ```python |
| | | from funasr_onnx import CT_Transformer |
| | | |
| | |
| | | result = model(text_in) |
| | | print(result[0]) |
| | | ``` |
| | | |
| | | - `model_dir`: model_name in modelscope or local path downloaded from modelscope. If the local path is set, it should contain `model.onnx`, `config.yaml`, `am.mvn` |
| | | - `device_id`: `-1` (Default), infer on CPU. If you want to infer with GPU, set it to gpu_id (Please make sure that you have install the onnxruntime-gpu) |
| | | - `quantize`: `False` (Default), load the model of `model.onnx` in `model_dir`. If set `True`, load the model of `model_quant.onnx` in `model_dir` |
| | |
| | | |
| | | Output: `List[str]`: recognition result |
| | | |
| | | |
| | | #### CT-Transformer-online |
| | | |
| | | ```python |
| | | from funasr_onnx import CT_Transformer_VadRealtime |
| | | |
| | |
| | | |
| | | print(rec_result_all) |
| | | ``` |
| | | |
| | | - `model_dir`: model_name in modelscope or local path downloaded from modelscope. If the local path is set, it should contain `model.onnx`, `config.yaml`, `am.mvn` |
| | | - `device_id`: `-1` (Default), infer on CPU. If you want to infer with GPU, set it to gpu_id (Please make sure that you have install the onnxruntime-gpu) |
| | | - `quantize`: `False` (Default), load the model of `model.onnx` in `model_dir`. If set `True`, load the model of `model_quant.onnx` in `model_dir` |
| | |
| | | |
| | | ## Performance benchmark |
| | | |
| | | Please ref to [benchmark](https://github.com/alibaba-damo-academy/FunASR/blob/main/funasr/runtime/python/benchmark_onnx.md) |
| | | Please ref to [benchmark](https://github.com/alibaba-damo-academy/FunASR/blob/main/funasr/runtime/docs/benchmark_onnx.md) |
| | | |
| | | ## Acknowledge |
| | | |
| | | 1. This project is maintained by [FunASR community](https://github.com/alibaba-damo-academy/FunASR). |
| | | 2. We partially refer [SWHL](https://github.com/RapidAI/RapidASR) for onnxruntime (only for paraformer model). |