| | |
|---|
| | | conda activate funasr |
|---|
| | | . ./run.sh |
|---|
| | | ``` |
|---|
| | | The training log files are saved in `exp/*_train_*/log/train.log.*` and the inference results are saved in `exp/*_train_*/decode_asr_*`. |
|---|
| | | The training log files are saved in `exp/*_train_*/log/train.log.*`， which can be viewed using the following command: |
|---|
| | | ```sh |
|---|
| | | vim exp/*_train_*/log/train.log.0 |
|---|
| | | ``` |
|---|
| | | Users can observe the training loss, prediction accuracy and other training information, like follows: |
|---|
| | | ```text |
|---|
| | | ... 1epoch:train:751-800batch:800num_updates: ... loss_ctc=106.703, loss_att=86.877, acc=0.029, loss_pre=1.552 ... |
|---|
| | | ... 1epoch:train:801-850batch:850num_updates: ... loss_ctc=107.890, loss_att=87.832, acc=0.029, loss_pre=1.702 ... |
|---|
| | | ``` |
|---|
| | | At the end of each epoch, the evaluation metrics are calculated on the validation set, like follows: |
|---|
| | | ```text |
|---|
| | | ... [valid] loss_ctc=99.914, cer_ctc=1.000, loss_att=80.512, acc=0.029, cer=0.971, wer=1.000, loss_pre=1.952, loss=88.285 ... |
|---|
| | | ``` |
|---|
| | | |
|---|
| | | The inference results are saved in `exp/*_train_*/decode_asr_*/$dset`. The main two files are `text.cer` and `text.cer.txt`. `text.cer` saves the comparison between the recognized text and the reference text, like follows: |
|---|
| | | ```text |
|---|
| | | ... |
|---|
| | | BAC009S0764W0213(nwords=11,cor=11,ins=0,del=0,sub=0) corr=100.00%,cer=0.00% |
|---|
| | | ref: 构 建 良 好 的 旅 游 市 场 环 境 |
|---|
| | | res: 构 建 良 好 的 旅 游 市 场 环 境 |
|---|
| | | ... |
|---|
| | | ``` |
|---|
| | | `text.cer.txt` saves the final results, like follows: |
|---|
| | | ```text |
|---|
| | | %WER ... |
|---|
| | | %SER ... |
|---|
| | | Scored ... sentences, ... |
|---|
| | | ``` |
|---|
| | | |
|---|
| | | ## Introduction |
|---|
| | | We provide a recipe `egs/aishell/paraformer/run.sh` for training a paraformer model on AISHELL-1 dataset. This recipe consists of five stages, supporting training on multiple GPUs and decoding by CPU or GPU. Before introducing each stage in detail, we first explain several parameters which should be set by users. |
|---|
| | | - `CUDA_VISIBLE_DEVICES`: visible gpu list |
|---|
| | | - `gpu_num`: the number of GPUs used for training |
|---|
| | | - `gpu_inference`: whether to use GPUs for decoding |
|---|
| | | - `njob`: for CPU decoding, indicating the total number of CPU jobs; for GPU decoding, indicating the number of jobs on each GPU |
|---|
| | | - `CUDA_VISIBLE_DEVICES`: `0,1` (Default), visible gpu list |
|---|
| | | - `gpu_num`: `2` (Default), the number of GPUs used for training |
|---|
| | | - `gpu_inference`: `true` (Default), whether to use GPUs for decoding |
|---|
| | | - `njob`: `1` (Default),for CPU decoding, indicating the total number of CPU jobs; for GPU decoding, indicating the number of jobs on each GPU |
|---|
| | | - `raw_data`: the raw path of AISHELL-1 dataset |
|---|
| | | - `feats_dir`: the path for saving processed data |
|---|
| | | - `nj`: the number of jobs for data preparation |
|---|
| | | - `speed_perturb`: the range of speech perturbed |
|---|
| | | - `token_type`: `char` (Default), indicate how to process text |
|---|
| | | - `type`: `sound` (Default), set the input type |
|---|
| | | - `scp`: `wav.scp` (Default), set the input file |
|---|
| | | - `nj`: `64` (Default), the number of jobs for data preparation |
|---|
| | | - `speed_perturb`: `"0.9, 1.0 ,1.1"` (Default), the range of speech perturbed |
|---|
| | | - `exp_dir`: the path for saving experimental results |
|---|
| | | - `tag`: the suffix of experimental result directory |
|---|
| | | - `tag`: `exp1` (Default), the suffix of experimental result directory |
|---|
| | | - `stage` `0` (Default), start the recipe from the specified stage |
|---|
| | | - `stop_stage` `5` (Default), stop the recipe from the specified stage |
|---|
| | | |
|---|
| | | ### Stage 0: Data preparation |
|---|
| | | This stage processes raw AISHELL-1 dataset `$raw_data` and generates the corresponding `wav.scp` and `text` in `$feats_dir/data/xxx`. `xxx` means `train/dev/test`. Here we assume users have already downloaded AISHELL-1 dataset. If not, users can download data [here](https://www.openslr.org/33/) and set the path for `$raw_data`. The examples of `wav.scp` and `text` are as follows: |
|---|
| | |
|---|
| | | These two files both have two columns, while the first column is wav ids and the second column is the corresponding wav paths/label tokens. |
|---|
| | | |
|---|
| | | ### Stage 1: Feature and CMVN Generation |
|---|
| | | This stage computes CMVN based on `train` dataset, which is used in the following stages. Users can set `nj` to control the number of jobs for computing CMVN. The generated CMVN file is saved as `$feats_dir/data/train/cmvn/cmvn.mvn`. |
|---|
| | | This stage computes CMVN based on `train` dataset, which is used in the following stages. Users can set `nj` to control the number of jobs for computing CMVN. The generated CMVN file is saved as `$feats_dir/data/train/cmvn/am.mvn`. |
|---|
| | | |
|---|
| | | ### Stage 2: Dictionary Preparation |
|---|
| | | This stage processes the dictionary, which is used as a mapping between label characters and integer indices during ASR training. The processed dictionary file is saved as `$feats_dir/data/$lang_toekn_list/$token_type/tokens.txt`. An example of `tokens.txt` is as follows: |
|---|
| | |
|---|
| | | |
|---|
| | | * Decoding by CPU or GPU |
|---|
| | | |
|---|
| | | We support CPU and GPU decoding. For CPU decoding, |
|---|
| | | We support CPU and GPU decoding. For CPU decoding, set `gpu_inference=false` and `njob` to specific the total number of CPU jobs. For GPU decoding, first set `gpu_inference=true`. Then set `gpuid_list` to specific which GPUs for decoding and `njob` to specific the number of decoding jobs on each GPU. |
|---|
