| | |
|---|
| | | # Get Started |
|---|
| | | This is an easy example which introduces how to train a paraformer model on AISHELL-1 data from scratch. According to this example, you can train other models (conformer, paraformer, etc.) on other datasets (AISHELL-1, AISHELL-2, etc.) similarly. |
|---|
| | | Here we take "Training a paraformer model from scratch using the AISHELL-1 dataset" as an example to introduce how to use FunASR. According to this example, users can similarly employ other datasets (such as AISHELL-2 dataset, etc.) to train other models (such as conformer, transformer, etc.). |
|---|
| | | |
|---|
| | | ## Overall Introduction |
|---|
| | | We provide a recipe `egs/aishell/paraformer/run.sh` for training a paraformer model on AISHELL-1 data . This recipe consists of five stages and support training on multiple GPUs and decoding by CPU or GPU. Before introduce each stage in detail, we first explain several variables which should be set by users. |
|---|
| | | We provide a recipe `egs/aishell/paraformer/run.sh` for training a paraformer model on AISHELL-1 dataset. This recipe consists of five stages, supporting training on multiple GPUs and decoding by CPU or GPU. Before introducing each stage in detail, we first explain several parameters which should be set by users. |
|---|
| | | - `CUDA_VISIBLE_DEVICES`: visible gpu list |
|---|
| | | - `gpu_num`: the number of GPUs used for training |
|---|
| | | - `gpu_inference`: whether to use GPUs for decoding |
|---|
| | | - `njob`: for CPU decoding, indicating the total number of CPU jobs; for GPU decoding, indicating the number of jobs on each GPU. |
|---|
| | | - `feats_dir`: the path to save processed data |
|---|
| | | - `exp_dir`: the path to save experimental results |
|---|
| | | - `data_aishell`: the path of raw AISHELL-1 data |
|---|
| | | - `tag`: the suffix of experimental result directory |
|---|
| | | - `njob`: for CPU decoding, indicating the total number of CPU jobs; for GPU decoding, indicating the number of jobs on each GPU |
|---|
| | | - `data_aishell`: the raw path of AISHELL-1 dataset |
|---|
| | | - `feats_dir`: the path for saving processed data |
|---|
| | | - `nj`: the number of jobs for data preparation |
|---|
| | | - `speed_perturb`: the range of speech perturbed |
|---|
| | | - `exp_dir`: the path for saving experimental results |
|---|
| | | - `tag`: the suffix of experimental result directory |
|---|
| | | |
|---|
| | | ## Stage 0: Data preparation |
|---|
| | | This stage processes raw AISHELL-1 data `$data_aishell` and generates the corresponding `wav.scp` and `text` in `$feats_dir/data/xxx` and `xxx` means `train/dev/test`. Here we assume you have already downloaded AISHELL-1 data. If not, you can download data [here](https://www.openslr.org/33/) and set the path for `$data_aishell`. Here we show examples for `wav.scp` and `text`, separately. |
|---|
| | | This stage processes raw AISHELL-1 dataset `$data_aishell` and generates the corresponding `wav.scp` and `text` in `$feats_dir/data/xxx`. `xxx` means `train/dev/test`. Here we assume users have already downloaded AISHELL-1 dataset. If not, users can download data [here](https://www.openslr.org/33/) and set the path for `$data_aishell`. The examples of `wav.scp` and `text` are as follows: |
|---|
| | | * `wav.scp` |
|---|
| | | ``` |
|---|
| | | BAC009S0002W0122 /nfs/ASR_DATA/AISHELL-1/data_aishell/wav/train/S0002/BAC009S0002W0122.wav |
|---|
| | |
|---|
| | | BAC009S0002W0124 自 六 月 底 呼 和 浩 特 市 率 先 宣 布 取 消 限 购 后 |
|---|
| | | ... |
|---|
| | | ``` |
|---|
| | | We can see that these two files both have two columns while the first column is the wav-id and the second column is the corresponding wav-path/label tokens. |
|---|
| | | These two files both have two columns, while the first column is wav ids and the second column is the corresponding wav paths/label tokens. |
|---|
| | | |
|---|
| | | ## Stage 1: Feature Generation |
|---|
| | | This stage extracts FBank feature from raw wav `wav.scp` and apply speed perturbation as data augmentation according to `speed_perturb`. You can set `nj` to control the number of jobs for feature generation. The output features are saved in `$feats_dir/dump/xxx/ark` and the corresponding `feats.scp` files are saved as `$feats_dir/dump/xxx/feats.scp`. An example of `feats.scp` can be seen as follows: |
|---|
| | | This stage extracts FBank features from `wav.scp` and apply speed perturbation as data augmentation according to `speed_perturb`. Users can set `nj` to control the number of jobs for feature generation. The generated features are saved in `$feats_dir/dump/xxx/ark` and the corresponding `feats.scp` files are saved as `$feats_dir/dump/xxx/feats.scp`. An example of `feats.scp` can be seen as follows: |
|---|
| | | * `feats.scp` |
|---|
| | | ``` |
|---|
| | | ... |
|---|
| | | BAC009S0002W0122_sp0.9 /nfs/haoneng.lhn/funasr_data/aishell-1/dump/fbank/train/ark/feats.16.ark:592751055 |
|---|
| | | BAC009S0002W0122_sp0.9 /nfs/funasr_data/aishell-1/dump/fbank/train/ark/feats.16.ark:592751055 |
|---|
| | | ... |
|---|
| | | ``` |
|---|
| | | Note that samples in this file have already been shuffled. This file contains two columns. The first column is the wav-id while the second column is the kaldi-ark feature path. Besides, `speech_shape` and `text_shape` are also generated in this stage, denoting the speech feature shape and text length of each sample. The examples are shown as follows: |
|---|
| | | Note that samples in this file have already been shuffled randomly. This file contains two columns. The first column is wav ids while the second column is kaldi-ark feature paths. Besides, `speech_shape` and `text_shape` are also generated in this stage, denoting the speech feature shape and text length of each sample. The examples are shown as follows: |
|---|
| | | * `speech_shape` |
|---|
| | | ``` |
|---|
| | | ... |
|---|
| | |
|---|
| | | BAC009S0002W0122_sp0.9 15 |
|---|
| | | ... |
|---|
| | | ``` |
|---|
| | | These two files have two columns. The first column is the wav-id and the second column is the corresponding speech feature shape and text length. |
|---|
| | | These two files have two columns. The first column is wav ids and the second column is the corresponding speech feature shape and text length. |
|---|
| | | |
|---|
| | | ## Stage 2: Dictionary Preparation |
|---|
| | | This stage prepares a dictionary, which is used as a mapping between label characters and integer indices during ASR training. The output dictionary file is saved as `$feats_dir/data/$lang_toekn_list/$token_type/tokens.txt`. Here we show an example of `tokens.txt` as follows: |
|---|
| | |
|---|
| | | |
|---|
| | | * Configuration |
|---|
| | | |
|---|
| | | We support CTC decoding, attention decoding and hybrid CTC-attention decoding in FunASR, which can be specified by `ctc_weight` in a YAML file in `conf` directory. Specifically, `ctc_weight=1.0` indicates attention decoding, `ctc_weight=0.0` indicates CTC decoding, `0.0<ctc_weight<1.0` indicates hybrid CTC-attention decoding. |
|---|
| | | We support CTC decoding, attention decoding and hybrid CTC-attention decoding in FunASR, which can be specified by `ctc_weight` in a YAML file in `conf` directory. Specifically, `ctc_weight=1.0` indicates CTC decoding, `ctc_weight=0.0` indicates attention decoding, `0.0<ctc_weight<1.0` indicates hybrid CTC-attention decoding. |
|---|
| | | |
|---|
| | | * CPU/GPU Decoding |
|---|
| | | |
|---|
