From 28ccfbfc51068a663a80764e14074df5edf2b5ba Mon Sep 17 00:00:00 2001 From: kongdeqiang <kongdeqiang960204@163.com> Date: 星期五, 13 三月 2026 17:41:41 +0800 Subject: [PATCH] 提交 --- fun_text_processing/README.md | 33 +++++++-------------------------- 1 files changed, 7 insertions(+), 26 deletions(-) diff --git a/fun_text_processing/README.md b/fun_text_processing/README.md index 9244e20..9947fa4 100644 --- a/fun_text_processing/README.md +++ b/fun_text_processing/README.md @@ -3,49 +3,30 @@ ### Introduction -FunTextProcessing is a Python toolkit for fundamental text processing in ASR including text processing , inverse text processing, which is included in the `FunASR`. +FunTextProcessing is a Python toolkit for fundamental text processing in ASR including text processing , inverse text processing, num2words, which is included in the `FunASR`. ### Highlights -- FunTextProcessing supports inverse text processing (ITN), text processing (TN). -- FunTextProcessing supports multilingual, 10+ languages for ITN, 5 languages for TN. +- FunTextProcessing supports inverse text processing (ITN), text processing (TN), number to words (num2words). +- FunTextProcessing supports multilingual, 10+ languages for ITN, 5 languages for TN, 50+ languages for num2words. -### Installation - -Fun Text Processing, specifically (Inverse) Text Normalization, requires Pynini to be installed. -``` -bash fun_text_processing/install_pynini.sh -``` ### Example #### Inverse Text Processing (ITN) Given text inputs, such as speech recognition results, use `fun_text_processing/inverse_text_normalization/inverse_normalize.py` to output ITN results. You may refer to the following example scripts. ``` -python fun_text_processing/inverse_text_normalization/inverse_normalize.py --text="one hundred twenty three" --language=en -``` +test_file=fun_text_processing/inverse_text_normalization/id/id_itn_test_input.txt +python fun_text_processing/inverse_text_normalization/inverse_normalize.py --input_file $test_file --cache_dir ./itn_model/ --output_file output.txt --language=id ``` -python fun_text_processing/inverse_text_normalization/inverse_normalize.py --text="ratus dua puluh tiga" --language=id --cache_dir ./model/ --output_file output.txt --overwrite_cache -cat output.txt -``` - -Arguments: -- text - Input text. Should not exceed 500 words. -- input_file - Input file with lines of input text. Only one of text or input_file is accepted. -- output_file - Output file to save normalizations. Needed if input_file is specified. -- language - language id. -- input_case - Only for text normalization. lower_cased or cased. -- verbose - Outputs intermediate information. -- cache_dir - Specifies a cache directory for compiled grammars. If grammars exist, this significantly improves speed. -- overwrite_cache - Updates grammars in cache. -- whitelist - TSV file with custom mappings of written text to spoken form. ### Acknowledge 1. We borrowed a lot of codes from [NeMo](https://github.com/NVIDIA/NeMo). 2. We refered the codes from [WeTextProcessing](https://github.com/wenet-e2e/WeTextProcessing) for Chinese inverse text normalization. +3. We borrowed a lot of codes from [num2words](https://pypi.org/project/num2words/) library for convert the number to words function in some languages. ### License -This project is licensed under the Apache-2.0 license. FunTextProcessing also contains various third-party components and some code modified from other repos under other open source licenses. +This project is licensed under the [The MIT License](https://opensource.org/licenses/MIT). FunTextProcessing also contains various third-party components and some code modified from other repos under other open source licenses. -- Gitblit v1.9.1