CLDR Segmentation data # Copyright © 1991-2020 Unicode, Inc. # For terms of use, see http://www.unicode.org/copyright.html # Unicode and the Unicode Logo are registered trademarks of Unicode, Inc. in the U.S. and other countries. # CLDR data files are interpreted according to the LDML specification (http://unicode.org/reports/tr35/) The segments directory contains files used to customize the default segmentation data in the UCD. Currently this just applies to the Grapheme Cluster Break (GCB) (https://unicode.org/reports/tr29/) algorithm, to add support for not splitting Indic aksaras. The modifications are: 1. Adding 3 new character categories to https://unicode.org/reports/tr29/#Grapheme_Cluster_Break_Property_Values Virama=[\p{Gujr}\p{sc=Telu}\p{sc=Mlym}\p{sc=Orya}\p{sc=Beng}\p{sc=Deva}&\p{Indic_Syllabic_Category=Virama}] LinkingConsonant=[\p{Gujr}\p{sc=Telu}\p{sc=Mlym}\p{sc=Orya}\p{sc=Beng}\p{sc=Deva}&\p{Indic_Syllabic_Category=Consonant}] ExtCccZwj=[\p{gcb=Extend}-\p{ccc=0}] \p{gcb=ZWJ}] Note that these categories are not GCB property values: In fact, they overlap the GCB property values. It is not necessary for the rules to have disjoint categories. The list of scripts can be added to over time, as test files for them become available. 2. Adding a rule to https://unicode.org/reports/tr29/#Grapheme_Cluster_Boundary_Rules 9.3) LinkingConsonant ExtCccZwj* Virama ExtCccZwj* × LinkingConsonant 3. Adding test files supplied by India to org.unicode.cldr.unittest.data.graphemeCluster/* TestSegmenter-Bengali.txt TestSegmenter-Devanagari.txt TestSegmenter-Gujarati.txt TestSegmenter-Malayalam.txt TestSegmenter-Odia.txt TestSegmenter-Telugu.txt 4. Adding modified files in this directory, which can be used in place of the default files from https://unicode.org/Public/12.0.0/ucd/auxiliary/ GraphemeBreakTest.html GraphemeBreakTest.txt Note: The GraphemeBreakProperty.txt file is unmodified, as those properties don't change.