From cda930c07de2dba47536063e2d8edd29d0c32b66 Mon Sep 17 00:00:00 2001
From: zhifu gao <zhifu.gzf@alibaba-inc.com>
Date: 星期五, 21 四月 2023 20:45:53 +0800
Subject: [PATCH] Merge pull request #396 from alibaba-damo-academy/dev_lyb

---
 funasr/runtime/python/benchmark_onnx_cpp.md |  107 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 107 insertions(+), 0 deletions(-)

diff --git a/funasr/runtime/python/benchmark_onnx_cpp.md b/funasr/runtime/python/benchmark_onnx_cpp.md
new file mode 100644
index 0000000..bf64163
--- /dev/null
+++ b/funasr/runtime/python/benchmark_onnx_cpp.md
@@ -0,0 +1,107 @@
+# CPU Benchmark (ONNX)
+
+## Configuration
+### Data set:
+Aishell1 [test set](https://www.openslr.org/33/) , the total audio duration is 36108.919 seconds.
+
+### Tools
+#### Install [modelscope and funasr](https://github.com/alibaba-damo-academy/FunASR#installation)
+
+```shell
+pip3 install torch torchaudio
+pip install -U modelscope
+pip install -U funasr
+```
+
+#### Export [onnx model](https://github.com/alibaba-damo-academy/FunASR/tree/main/funasr/export)
+
+```shell
+python -m funasr.export.export_model --model-name damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch --export-dir ./export --type onnx --quantize True
+```
+
+#### Building for Linux/Unix
+
+Download onnxruntime
+```shell
+# download an appropriate onnxruntime from https://github.com/microsoft/onnxruntime/releases/tag/v1.14.0
+# here we get a copy of onnxruntime for linux 64
+wget https://github.com/microsoft/onnxruntime/releases/download/v1.14.0/onnxruntime-linux-x64-1.14.0.tgz
+tar -zxvf onnxruntime-linux-x64-1.14.0.tgz
+```
+
+Install openblas
+```shell
+sudo apt-get install libopenblas-dev #ubuntu
+# sudo yum -y install openblas-devel #centos
+```
+
+Build runtime
+```shell
+git clone https://github.com/alibaba-damo-academy/FunASR.git && cd funasr/runtime/onnxruntime
+mkdir build && cd build
+cmake  -DCMAKE_BUILD_TYPE=release .. -DONNXRUNTIME_DIR=/path/to/onnxruntime-linux-x64-1.14.0
+make
+```
+
+#### Recipe
+
+set the model, data path and output_dir
+
+```shell
+./bin/funasr-onnx-offline-rtf /path/to/model_dir /path/to/wav.scp quantize(true or false) thread_num
+```
+
+The structure of /path/to/models_dir
+```
+config.yaml, am.mvn, model.onnx(or model_quant.onnx)
+```
+
+## [Paraformer-large](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary) 
+
+Number of Parameter: 220M 
+
+Storage size: 880MB
+
+Storage size after int8-quant: 237MB
+
+CER: 1.95%
+
+CER after int8-quant: 1.95%
+
+ ### Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz   16core-32processor    with avx512_vnni
+
+| concurrent-tasks    | processing time(s) |   RTF    | Speedup Rate |
+|---------------------|:------------------:|:--------:|:------------:|
+|  1   (onnx fp32)    |       2129s        | 0.058974 |      17      |
+|  1   (onnx int8)    |       1020s        | 0.02826  |      35      |
+|  8   (onnx fp32)    |        273s        | 0.007553 |     132      |
+|  8   (onnx int8)    |        128s        | 0.003558 |     281      |
+|  16   (onnx fp32)   |        146s        | 0.00403  |     248      |
+|  16   (onnx int8)   |        67s         | 0.001868 |     535      |
+|  32   (onnx fp32)   |        133s        | 0.003672 |     272      |
+|  32   (onnx int8)   |        64s         | 0.001778 |     562      |
+|  64   (onnx fp32)   |        136s        | 0.003771 |     265      |
+|  64   (onnx int8)   |        67s         | 0.001846 |     541      |
+|  96   (onnx fp32)   |        137s        | 0.003788 |     264      |
+|  96   (onnx int8)   |        68s         | 0.001875 |     533      |
+
+
+
+### Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz    32core-64processor   without avx512_vnni
+
+| concurrent-tasks    | processing time(s) | RTF      | Speedup Rate |
+|---------------------|--------------------|----------|--------------|
+|  1   (onnx fp32)    | 2903s              | 0.080404 | 12           |
+|  1   (onnx int8)    | 2714s              | 0.075168 | 13           |
+|  8   (onnx fp32)    | 373s               | 0.010329 | 97           |
+|  8   (onnx int8)    | 340s               | 0.009428 | 106          |
+|  16   (onnx fp32)   | 189s               | 0.005252 | 190          |
+|  16   (onnx int8)   | 174s               | 0.004817 | 207          |
+|  32   (onnx fp32)   | 109s               | 0.00301  | 332          |
+|  32   (onnx int8)   | 88s                | 0.00245  | 408          |
+|  64   (onnx fp32)   | 113s               | 0.003129 | 320          |
+|  64   (onnx int8)   | 79s                | 0.002201 | 454          |
+|  96   (onnx fp32)   | 115s               | 0.003183 | 314          |
+|  96   (onnx int8)   | 80s                | 0.002222 | 450          |
+
+

--
Gitblit v1.9.1