FUNASR训练
parent: 362a5ad6 | 补丁 | 提交 | ignore whitespace
hnluo
2023-07-17 33f1a9c1d946dfe011e446551e978a0768a1e998 
Merge pull request #745 from alibaba-damo-academy/dev_lhn

fix hotword tokenize
1个文件已修改
2 ■■■■■ 已修改文件
funasr/datasets/large_datasets/utils/tokenize.py 2 ●●●●● 补丁 | 查看 | 原始文档 | blame | 历史 
 funasr/datasets/large_datasets/utils/tokenize.py


@@ -54,9 +54,9 @@




    length = len(text)


    if 'hw_tag' in data:


        pre_index = None


        if hw_config['pre_hwlist'] is not None and hw_config['pre_prob'] > 0:


            # enable preset hotword detect in sampling


            pre_index = None


            for hw in hw_config['pre_hwlist']:


                hw = " ".join(seg_tokenize(hw, seg_dict))


                _find = " ".join(text).find(hw)