Searched refs:remove_extra_whitespaces (Results 1 – 8 of 8) sorted by relevance
| /external/libtextclassifier/native/utils/sentencepiece/ |
| D | test_utils.cc | 29 bool remove_extra_whitespaces, in NormalizerFromSpec() argument
39 add_dummy_prefix, remove_extra_whitespaces, escape_whitespaces); in NormalizerFromSpec()
|
| D | normalizer.h | 47 bool remove_extra_whitespaces = true,
52 remove_extra_whitespaces_(remove_extra_whitespaces), in charsmap_trie_()
|
| D | test_utils.h | 30 bool remove_extra_whitespaces,
|
| /external/tflite-support/tensorflow_lite_support/custom_ops/kernel/sentencepiece/ |
| D | optimized_encoder.cc | 64 std::tuple<int, utils::string_view> remove_extra_whitespaces(const char* data, in remove_extra_whitespaces() function
122 if (config.remove_extra_whitespaces()) { in NormalizeString()
124 process_string(result, output_offsets, remove_extra_whitespaces); in NormalizeString()
|
| D | encoder_config.fbs | 38 remove_extra_whitespaces: bool;
|
| D | model_converter.cc | 116 model_config.normalizer_spec().remove_extra_whitespaces()); in ConvertSentencepieceModelToFlatBuffer()
|
| /external/libtextclassifier/native/utils/tflite/ |
| D | text_encoder_config.fbs | 53 remove_extra_whitespaces:bool = true;
|
| D | text_encoder.cc | 100 config->add_dummy_prefix(), config->remove_extra_whitespaces(), in Initialize()
|