From 70a23e14cd27b29dbdb1a22381473acb92519a12 Mon Sep 17 00:00:00 2001 From: onlybetheone <iriszhangchong@gmail.com> Date: 星期四, 22 十二月 2022 17:06:50 +0800 Subject: [PATCH] FunTextProcessing --- fun_text_processing/README.md | 21 ++++++++++++++++++--- 1 files changed, 18 insertions(+), 3 deletions(-) diff --git a/fun_text_processing/README.md b/fun_text_processing/README.md index 878601b..3982bed 100644 --- a/fun_text_processing/README.md +++ b/fun_text_processing/README.md @@ -10,17 +10,32 @@ - FunTextProcessing supports inverse text processing (ITN), text processing (TN). - FunTextProcessing supports multilingual, 10+ languages for ITN, 5 languages for TN. +#### Installation + +Fun Text Processing, specifically (Inverse) Text Normalization, requires Pynini to be installed. +``` +bash fun_text_processing/install_pynini.sh +``` ### Example #### Inverse Text Processing (ITN) Given text inputs, such as speech recognition results, use `fun_text_processing/inverse_text_normalization/inverse_normalize.py` to output ITN results. You may refer to the following example scripts. ``` -test_file=fun_text_processing/inverse_text_normalization/id/id_itn_test_input.txt - -python fun_text_processing/inverse_text_normalization/inverse_normalize.py --input_file $test_file --cache_dir ./itn_model/ --output_file output.txt --language=id +python fun_text_processing/inverse_text_normalization/inverse_normalize.py --text="ratus dua puluh tiga" --language=id ``` +Arguments: +- text - Input text. Should not exceed 500 words. +- input_file - Input file with lines of input text. Only one of text or input_file is accepted. +- output_file - Output file to save normalizations. Needed if input_file is specified. +- language - language id. +- input_case - Only for text normalization. lower_cased or cased. +- verbose - Outputs intermediate information. +- cache_dir - Specifies a cache directory for compiled grammars. If grammars exist, this significantly improves speed. +- overwrite_cache - Updates grammars in cache. +- whitelist - TSV file with custom mappings of written text to spoken form. + ### Acknowledge 1. We borrowed a lot of codes from [NeMo](https://github.com/NVIDIA/NeMo). -- Gitblit v1.9.1