| | |
|---|
| | | # Voice Activity Detection |
|---|
| | | |
|---|
| | | > **Note**: |
|---|
| | | > The modelscope pipeline supports all the models in [model zoo](https://alibaba-damo-academy.github.io/FunASR/en/modelscope_models.html#pretrained-models-on-modelscope) to inference and finetune. Here we take the model of FSMN-VAD as example to demonstrate the usage. |
|---|
| | | > The modelscope pipeline supports all the models in [model zoo](https://alibaba-damo-academy.github.io/FunASR/en/model_zoo/modelscope_models.html#pretrained-models-on-modelscope) to inference and finetune. Here we take the model of FSMN-VAD as example to demonstrate the usage. |
|---|
| | | |
|---|
| | | ## Inference |
|---|
| | | |
|---|
| | |
|---|
| | | |
|---|
| | | |
|---|
| | | |
|---|
| | | #### API-reference |
|---|
| | | ##### Define pipeline |
|---|
| | | ### API-reference |
|---|
| | | #### Define pipeline |
|---|
| | | - `task`: `Tasks.voice_activity_detection` |
|---|
| | | - `model`: model name in [model zoo](https://alibaba-damo-academy.github.io/FunASR/en/modelscope_models.html#pretrained-models-on-modelscope), or model path in local disk |
|---|
| | | - `model`: model name in [model zoo](https://alibaba-damo-academy.github.io/FunASR/en/model_zoo/modelscope_models.html#pretrained-models-on-modelscope), or model path in local disk |
|---|
| | | - `ngpu`: `1` (Default), decoding on GPU. If ngpu=0, decoding on CPU |
|---|
| | | - `ncpu`: `1` (Default), sets the number of threads used for intraop parallelism on CPU |
|---|
| | | - `output_dir`: `None` (Default), the output path of results if set |
|---|
| | | - `batch_size`: `1` (Default), batch size when decoding |
|---|
| | | ##### Infer pipeline |
|---|
| | | #### Infer pipeline |
|---|
| | | - `audio_in`: the input to decode, which could be: |
|---|
| | | - wav_path, `e.g.`: asr_example.wav, |
|---|
| | | - pcm_path, `e.g.`: asr_example.pcm, |
|---|
| | |
|---|
| | | ### Inference with multi-thread CPUs or multi GPUs |
|---|
| | | FunASR also offer recipes [egs_modelscope/vad/TEMPLATE/infer.sh](https://github.com/alibaba-damo-academy/FunASR/blob/main/egs_modelscope/vad/TEMPLATE/infer.sh) to decode with multi-thread CPUs, or multi GPUs. |
|---|
| | | |
|---|
| | | - Setting parameters in `infer.sh` |
|---|
| | | - `model`: model name in [model zoo](https://alibaba-damo-academy.github.io/FunASR/en/modelscope_models.html#pretrained-models-on-modelscope), or model path in local disk |
|---|
| | | - `data_dir`: the dataset dir needs to include `wav.scp` |
|---|
| | | - `output_dir`: output dir of the recognition results |
|---|
| | | - `batch_size`: `64` (Default), batch size of inference on gpu |
|---|
| | | - `gpu_inference`: `true` (Default), whether to perform gpu decoding, set false for CPU inference |
|---|
| | | - `gpuid_list`: `0,1` (Default), which gpu_ids are used to infer |
|---|
| | | - `njob`: only used for CPU inference (`gpu_inference`=`false`), `64` (Default), the number of jobs for CPU decoding |
|---|
| | | - `checkpoint_dir`: only used for infer finetuned models, the path dir of finetuned models |
|---|
| | | - `checkpoint_name`: only used for infer finetuned models, `valid.cer_ctc.ave.pb` (Default), which checkpoint is used to infer |
|---|
| | | #### Settings of `infer.sh` |
|---|
| | | - `model`: model name in [model zoo](https://alibaba-damo-academy.github.io/FunASR/en/model_zoo/modelscope_models.html#pretrained-models-on-modelscope), or model path in local disk |
|---|
| | | - `data_dir`: the dataset dir needs to include `wav.scp` |
|---|
| | | - `output_dir`: output dir of the recognition results |
|---|
| | | - `batch_size`: `64` (Default), batch size of inference on gpu |
|---|
| | | - `gpu_inference`: `true` (Default), whether to perform gpu decoding, set false for CPU inference |
|---|
| | | - `gpuid_list`: `0,1` (Default), which gpu_ids are used to infer |
|---|
| | | - `njob`: only used for CPU inference (`gpu_inference`=`false`), `64` (Default), the number of jobs for CPU decoding |
|---|
| | | - `checkpoint_dir`: only used for infer finetuned models, the path dir of finetuned models |
|---|
| | | - `checkpoint_name`: only used for infer finetuned models, `valid.cer_ctc.ave.pb` (Default), which checkpoint is used to infer |
|---|
| | | |
|---|
| | | - Decode with multi GPUs: |
|---|
| | | #### Decode with multi GPUs: |
|---|
| | | ```shell |
|---|
| | | bash infer.sh \ |
|---|
| | | --model "damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch" \ |
|---|
| | | --data_dir "./data/test" \ |
|---|
| | | --output_dir "./results" \ |
|---|
| | | --batch_size 64 \ |
|---|
| | | --batch_size 1 \ |
|---|
| | | --gpu_inference true \ |
|---|
| | | --gpuid_list "0,1" |
|---|
| | | ``` |
|---|
| | | - Decode with multi-thread CPUs: |
|---|
| | | #### Decode with multi-thread CPUs: |
|---|
| | | ```shell |
|---|
| | | bash infer.sh \ |
|---|
| | | --model "damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch" \ |
|---|
| | | --data_dir "./data/test" \ |
|---|
| | | --output_dir "./results" \ |
|---|
| | | --gpu_inference false \ |
|---|
| | | --njob 64 |
|---|
| | | --njob 1 |
|---|
| | | ``` |
|---|
| | | |
|---|
| | | ## Finetune with pipeline |
|---|
