| | |
|---|
| | | Then you can directly start the recipe as follows: |
|---|
| | | ```sh |
|---|
| | | conda activate funasr |
|---|
| | | . ./run.sh |
|---|
| | | . ./run.sh --CUDA_VISIBLE_DEVICES="0,1" --gpu_num=2 |
|---|
| | | ``` |
|---|
| | | |
|---|
| | | The training log files are saved in `${exp_dir}/exp/${model_dir}/log/train.log.*`， which can be viewed using the following command: |
|---|
| | |
|---|
| | | ... 1epoch:train:801-850batch:850num_updates: ... loss_ctc=107.890, loss_att=87.832, acc=0.029, loss_pre=1.702 ... |
|---|
| | | ``` |
|---|
| | | |
|---|
| | | Also, users can use tensorboard to observe these training information by the following command: |
|---|
| | | ```sh |
|---|
| | | tensorboard --logdir ${exp_dir}/exp/${model_dir}/tensorboard/train |
|---|
| | | ``` |
|---|
| | | |
|---|
| | | At the end of each epoch, the evaluation metrics are calculated on the validation set, like follows: |
|---|
| | | ```text |
|---|
| | | ... [valid] loss_ctc=99.914, cer_ctc=1.000, loss_att=80.512, acc=0.029, cer=0.971, wer=1.000, loss_pre=1.952, loss=88.285 ... |
|---|
| | | ``` |
|---|
| | | |
|---|
| | | Also, users can use tensorboard to observe these training information by the following command: |
|---|
| | | ```sh |
|---|
| | | tensorboard --logdir ${exp_dir}/exp/${model_dir}/tensorboard/train |
|---|
| | | ``` |
|---|
| | | Here is an example of loss: |
|---|
| | | |
|---|
| | | <img src="images/loss.png" width="200"/> |
|---|
| | | |
|---|
| | | The inference results are saved in `${exp_dir}/exp/${model_dir}/decode_asr_*/$dset`. The main two files are `text.cer` and `text.cer.txt`. `text.cer` saves the comparison between the recognized text and the reference text, like follows: |
|---|
| | | ```text |
|---|
| | |
|---|
| | | * `gpu_num`: `2` (Default), specify the number of GPUs for training. When `gpu_num > 1`, DistributedDataParallel (DDP, the detail can be found [here](https://pytorch.org/tutorials/intermediate/ddp_tutorial.html)) training will be enabled. Correspondingly, `CUDA_VISIBLE_DEVICES` should be set to specify which ids of GPUs will be used. |
|---|
| | | * `use_preprocessor`: `true` (Default), specify whether to use pre-processing on each sample |
|---|
| | | * `token_list`: the path of token list for training |
|---|
| | | * `dataset_type`: |
|---|
| | | * `dataset_type`: `small` (Default). FunASR supports `small` dataset type for training small datasets. Besides, an optional iterable-style DataLoader based on [Pytorch Iterable-style DataPipes](https://pytorch.org/data/beta/torchdata.datapipes.iter.html) for large datasets is supported and users can specify `dataset_type=large` to enable it. |
|---|
| | | * `data_dir`: the path of data. Specifically, the data for training is saved in `$data_dir/data/$train_set` while the data for validation is saved in `$data_dir/data/$valid_set` |
|---|
| | | * `data_file_names`: `"wav.scp,text"` specify the speech and text file names for ASR |
|---|
| | | * `cmvn_file`: the path of cmvn file |
|---|
| | | * `resume`: `true`, whether to enable "checkpoint training" |
|---|
| | | * `config`: the path of configuration file, which is usually a YAML file in `conf` directory. In FunASR, the parameters of the training, including model, optimization, dataset, etc., can also be set in this file. Note that if the same parameters are specified in both recipe and config file, the parameters of recipe will be employed |
|---|
| | | |
|---|
| | | * DataLoader |
|---|
| | | |
|---|
| | | We support an optional iterable-style DataLoader based on [Pytorch Iterable-style DataPipes](https://pytorch.org/data/beta/torchdata.datapipes.iter.html) for large dataset and users can set `dataset_type=large` to enable it. |
|---|
| | | |
|---|
| | | ### Stage 5: Decoding |
|---|
| | | This stage generates the recognition results and calculates the `CER` to verify the performance of the trained model. |
|---|
| | |
|---|
| | | encoder: conformer |
|---|
| | | encoder_conf: |
|---|
| | | output_size: 256 # dimension of attention |
|---|
| | | attention_heads: 4 # number of heads in multi-head attention |
|---|
| | | attention_heads: 4 # the number of heads in multi-head attention |
|---|
| | | linear_units: 2048 # the number of units of position-wise feed forward |
|---|
| | | num_blocks: 12 # the number of encoder blocks |
|---|
| | | dropout_rate: 0.1 |
|---|
