Searched refs:split_on_script_change (Results 1 – 3 of 3) sorted by relevance
| /external/libtextclassifier/utils/ |
| D | tokenizer.h | 54 const bool split_on_script_change,
59 const bool split_on_script_change) in Tokenizer() argument
62 split_on_script_change,
|
| D | tokenizer_test.cc | 36 const bool split_on_script_change, in TestingTokenizer() argument
39 internal_tokenizer_codepoint_ranges, split_on_script_change, in TestingTokenizer()
51 const bool split_on_script_change, in TestingTokenizerProxy() argument
78 type, &unilib_, configs_fb, internal_configs_fb, split_on_script_change, in TestingTokenizerProxy()
|
| D | tokenizer.cc | 32 const bool split_on_script_change, in Tokenizer() argument
36 split_on_script_change_(split_on_script_change), in Tokenizer()
|