bug fix for punc and umap

shixian.shi

2024-01-23 ae4dceecf03c643be770e3274a070078e45fc9c7

1
{"bos_token": "<|endoftext|>", "eos_token": "<|endoftext|>", "unk_token": "<|endoftext|>"}