1<html> 2<head> 3<title>pcre2test specification</title> 4</head> 5<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> 6<h1>pcre2test man page</h1> 7<p> 8Return to the <a href="index.html">PCRE2 index page</a>. 9</p> 10<p> 11This page is part of the PCRE2 HTML documentation. It was generated 12automatically from the original man page. If there is any nonsense in it, 13please consult the man page, in case the conversion went wrong. 14<br> 15<ul> 16<li><a name="TOC1" href="#SEC1">SYNOPSIS</a> 17<li><a name="TOC2" href="#SEC2">PCRE2's 8-BIT, 16-BIT AND 32-BIT LIBRARIES</a> 18<li><a name="TOC3" href="#SEC3">INPUT ENCODING</a> 19<li><a name="TOC4" href="#SEC4">COMMAND LINE OPTIONS</a> 20<li><a name="TOC5" href="#SEC5">DESCRIPTION</a> 21<li><a name="TOC6" href="#SEC6">COMMAND LINES</a> 22<li><a name="TOC7" href="#SEC7">MODIFIER SYNTAX</a> 23<li><a name="TOC8" href="#SEC8">PATTERN SYNTAX</a> 24<li><a name="TOC9" href="#SEC9">SUBJECT LINE SYNTAX</a> 25<li><a name="TOC10" href="#SEC10">PATTERN MODIFIERS</a> 26<li><a name="TOC11" href="#SEC11">SUBJECT MODIFIERS</a> 27<li><a name="TOC12" href="#SEC12">THE ALTERNATIVE MATCHING FUNCTION</a> 28<li><a name="TOC13" href="#SEC13">DEFAULT OUTPUT FROM pcre2test</a> 29<li><a name="TOC14" href="#SEC14">OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION</a> 30<li><a name="TOC15" href="#SEC15">RESTARTING AFTER A PARTIAL MATCH</a> 31<li><a name="TOC16" href="#SEC16">CALLOUTS</a> 32<li><a name="TOC17" href="#SEC17">NON-PRINTING CHARACTERS</a> 33<li><a name="TOC18" href="#SEC18">SAVING AND RESTORING COMPILED PATTERNS</a> 34<li><a name="TOC19" href="#SEC19">SEE ALSO</a> 35<li><a name="TOC20" href="#SEC20">AUTHOR</a> 36<li><a name="TOC21" href="#SEC21">REVISION</a> 37</ul> 38<br><a name="SEC1" href="#TOC1">SYNOPSIS</a><br> 39<P> 40<b>pcre2test [options] [input file [output file]]</b> 41<br> 42<br> 43<b>pcre2test</b> is a test program for the PCRE2 regular expression libraries, 44but it can also be used for experimenting with regular expressions. This 45document describes the features of the test program; for details of the regular 46expressions themselves, see the 47<a href="pcre2pattern.html"><b>pcre2pattern</b></a> 48documentation. For details of the PCRE2 library function calls and their 49options, see the 50<a href="pcre2api.html"><b>pcre2api</b></a> 51documentation. 52</P> 53<P> 54The input for <b>pcre2test</b> is a sequence of regular expression patterns and 55subject strings to be matched. There are also command lines for setting 56defaults and controlling some special actions. The output shows the result of 57each match attempt. Modifiers on external or internal command lines, the 58patterns, and the subject lines specify PCRE2 function options, control how the 59subject is processed, and what output is produced. 60</P> 61<P> 62As the original fairly simple PCRE library evolved, it acquired many different 63features, and as a result, the original <b>pcretest</b> program ended up with a 64lot of options in a messy, arcane syntax for testing all the features. The 65move to the new PCRE2 API provided an opportunity to re-implement the test 66program as <b>pcre2test</b>, with a cleaner modifier syntax. Nevertheless, there 67are still many obscure modifiers, some of which are specifically designed for 68use in conjunction with the test script and data files that are distributed as 69part of PCRE2. All the modifiers are documented here, some without much 70justification, but many of them are unlikely to be of use except when testing 71the libraries. 72</P> 73<br><a name="SEC2" href="#TOC1">PCRE2's 8-BIT, 16-BIT AND 32-BIT LIBRARIES</a><br> 74<P> 75Different versions of the PCRE2 library can be built to support character 76strings that are encoded in 8-bit, 16-bit, or 32-bit code units. One, two, or 77all three of these libraries may be simultaneously installed. The 78<b>pcre2test</b> program can be used to test all the libraries. However, its own 79input and output are always in 8-bit format. When testing the 16-bit or 32-bit 80libraries, patterns and subject strings are converted to 16-bit or 32-bit 81format before being passed to the library functions. Results are converted back 82to 8-bit code units for output. 83</P> 84<P> 85In the rest of this document, the names of library functions and structures 86are given in generic form, for example, <b>pcre_compile()</b>. The actual 87names used in the libraries have a suffix _8, _16, or _32, as appropriate. 88<a name="inputencoding"></a></P> 89<br><a name="SEC3" href="#TOC1">INPUT ENCODING</a><br> 90<P> 91Input to <b>pcre2test</b> is processed line by line, either by calling the C 92library's <b>fgets()</b> function, or via the <b>libreadline</b> library. In some 93Windows environments character 26 (hex 1A) causes an immediate end of file, and 94no further data is read, so this character should be avoided unless you really 95want that action. 96</P> 97<P> 98The input is processed using using C's string functions, so must not 99contain binary zeros, even though in Unix-like environments, <b>fgets()</b> 100treats any bytes other than newline as data characters. An error is generated 101if a binary zero is encountered. By default subject lines are processed for 102backslash escapes, which makes it possible to include any data value in strings 103that are passed to the library for matching. For patterns, there is a facility 104for specifying some or all of the 8-bit input characters as hexadecimal pairs, 105which makes it possible to include binary zeros. 106</P> 107<br><b> 108Input for the 16-bit and 32-bit libraries 109</b><br> 110<P> 111When testing the 16-bit or 32-bit libraries, there is a need to be able to 112generate character code points greater than 255 in the strings that are passed 113to the library. For subject lines, backslash escapes can be used. In addition, 114when the <b>utf</b> modifier (see 115<a href="#optionmodifiers">"Setting compilation options"</a> 116below) is set, the pattern and any following subject lines are interpreted as 117UTF-8 strings and translated to UTF-16 or UTF-32 as appropriate. 118</P> 119<P> 120For non-UTF testing of wide characters, the <b>utf8_input</b> modifier can be 121used. This is mutually exclusive with <b>utf</b>, and is allowed only in 16-bit 122or 32-bit mode. It causes the pattern and following subject lines to be treated 123as UTF-8 according to the original definition (RFC 2279), which allows for 124character values up to 0x7fffffff. Each character is placed in one 16-bit or 12532-bit code unit (in the 16-bit case, values greater than 0xffff cause an error 126to occur). 127</P> 128<P> 129UTF-8 (in its original definition) is not capable of encoding values greater 130than 0x7fffffff, but such values can be handled by the 32-bit library. When 131testing this library in non-UTF mode with <b>utf8_input</b> set, if any 132character is preceded by the byte 0xff (which is an invalid byte in UTF-8) 1330x80000000 is added to the character's value. This is the only way of passing 134such code points in a pattern string. For subject strings, using an escape 135sequence is preferable. 136</P> 137<br><a name="SEC4" href="#TOC1">COMMAND LINE OPTIONS</a><br> 138<P> 139<b>-8</b> 140If the 8-bit library has been built, this option causes it to be used (this is 141the default). If the 8-bit library has not been built, this option causes an 142error. 143</P> 144<P> 145<b>-16</b> 146If the 16-bit library has been built, this option causes it to be used. If only 147the 16-bit library has been built, this is the default. If the 16-bit library 148has not been built, this option causes an error. 149</P> 150<P> 151<b>-32</b> 152If the 32-bit library has been built, this option causes it to be used. If only 153the 32-bit library has been built, this is the default. If the 32-bit library 154has not been built, this option causes an error. 155</P> 156<P> 157<b>-ac</b> 158Behave as if each pattern has the <b>auto_callout</b> modifier, that is, insert 159automatic callouts into every pattern that is compiled. 160</P> 161<P> 162<b>-AC</b> 163As for <b>-ac</b>, but in addition behave as if each subject line has the 164<b>callout_extra</b> modifier, that is, show additional information from 165callouts. 166</P> 167<P> 168<b>-b</b> 169Behave as if each pattern has the <b>fullbincode</b> modifier; the full 170internal binary form of the pattern is output after compilation. 171</P> 172<P> 173<b>-C</b> 174Output the version number of the PCRE2 library, and all available information 175about the optional features that are included, and then exit with zero exit 176code. All other options are ignored. If both -C and -LM are present, whichever 177is first is recognized. 178</P> 179<P> 180<b>-C</b> <i>option</i> 181Output information about a specific build-time option, then exit. This 182functionality is intended for use in scripts such as <b>RunTest</b>. The 183following options output the value and set the exit code as indicated: 184<pre> 185 ebcdic-nl the code for LF (= NL) in an EBCDIC environment: 186 0x15 or 0x25 187 0 if used in an ASCII environment 188 exit code is always 0 189 linksize the configured internal link size (2, 3, or 4) 190 exit code is set to the link size 191 newline the default newline setting: 192 CR, LF, CRLF, ANYCRLF, ANY, or NUL 193 exit code is always 0 194 bsr the default setting for what \R matches: 195 ANYCRLF or ANY 196 exit code is always 0 197</pre> 198The following options output 1 for true or 0 for false, and set the exit code 199to the same value: 200<pre> 201 backslash-C \C is supported (not locked out) 202 ebcdic compiled for an EBCDIC environment 203 jit just-in-time support is available 204 pcre2-16 the 16-bit library was built 205 pcre2-32 the 32-bit library was built 206 pcre2-8 the 8-bit library was built 207 unicode Unicode support is available 208</pre> 209If an unknown option is given, an error message is output; the exit code is 0. 210</P> 211<P> 212<b>-d</b> 213Behave as if each pattern has the <b>debug</b> modifier; the internal 214form and information about the compiled pattern is output after compilation; 215<b>-d</b> is equivalent to <b>-b -i</b>. 216</P> 217<P> 218<b>-dfa</b> 219Behave as if each subject line has the <b>dfa</b> modifier; matching is done 220using the <b>pcre2_dfa_match()</b> function instead of the default 221<b>pcre2_match()</b>. 222</P> 223<P> 224<b>-error</b> <i>number[,number,...]</i> 225Call <b>pcre2_get_error_message()</b> for each of the error numbers in the 226comma-separated list, display the resulting messages on the standard output, 227then exit with zero exit code. The numbers may be positive or negative. This is 228a convenience facility for PCRE2 maintainers. 229</P> 230<P> 231<b>-help</b> 232Output a brief summary these options and then exit. 233</P> 234<P> 235<b>-i</b> 236Behave as if each pattern has the <b>info</b> modifier; information about the 237compiled pattern is given after compilation. 238</P> 239<P> 240<b>-jit</b> 241Behave as if each pattern line has the <b>jit</b> modifier; after successful 242compilation, each pattern is passed to the just-in-time compiler, if available. 243</P> 244<P> 245<b>-jitfast</b> 246Behave as if each pattern line has the <b>jitfast</b> modifier; after 247successful compilation, each pattern is passed to the just-in-time compiler, if 248available, and each subject line is passed directly to the JIT matcher via its 249"fast path". 250</P> 251<P> 252<b>-jitverify</b> 253Behave as if each pattern line has the <b>jitverify</b> modifier; after 254successful compilation, each pattern is passed to the just-in-time compiler, if 255available, and the use of JIT for matching is verified. 256</P> 257<P> 258<b>-LM</b> 259List modifiers: write a list of available pattern and subject modifiers to the 260standard output, then exit with zero exit code. All other options are ignored. 261If both -C and -LM are present, whichever is first is recognized. 262</P> 263<P> 264<b>-pattern</b> <i>modifier-list</i> 265Behave as if each pattern line contains the given modifiers. 266</P> 267<P> 268<b>-q</b> 269Do not output the version number of <b>pcre2test</b> at the start of execution. 270</P> 271<P> 272<b>-S</b> <i>size</i> 273On Unix-like systems, set the size of the run-time stack to <i>size</i> 274mebibytes (units of 1024*1024 bytes). 275</P> 276<P> 277<b>-subject</b> <i>modifier-list</i> 278Behave as if each subject line contains the given modifiers. 279</P> 280<P> 281<b>-t</b> 282Run each compile and match many times with a timer, and output the resulting 283times per compile or match. When JIT is used, separate times are given for the 284initial compile and the JIT compile. You can control the number of iterations 285that are used for timing by following <b>-t</b> with a number (as a separate 286item on the command line). For example, "-t 1000" iterates 1000 times. The 287default is to iterate 500,000 times. 288</P> 289<P> 290<b>-tm</b> 291This is like <b>-t</b> except that it times only the matching phase, not the 292compile phase. 293</P> 294<P> 295<b>-T</b> <b>-TM</b> 296These behave like <b>-t</b> and <b>-tm</b>, but in addition, at the end of a run, 297the total times for all compiles and matches are output. 298</P> 299<P> 300<b>-version</b> 301Output the PCRE2 version number and then exit. 302</P> 303<br><a name="SEC5" href="#TOC1">DESCRIPTION</a><br> 304<P> 305If <b>pcre2test</b> is given two filename arguments, it reads from the first and 306writes to the second. If the first name is "-", input is taken from the 307standard input. If <b>pcre2test</b> is given only one argument, it reads from 308that file and writes to stdout. Otherwise, it reads from stdin and writes to 309stdout. 310</P> 311<P> 312When <b>pcre2test</b> is built, a configuration option can specify that it 313should be linked with the <b>libreadline</b> or <b>libedit</b> library. When this 314is done, if the input is from a terminal, it is read using the <b>readline()</b> 315function. This provides line-editing and history facilities. The output from 316the <b>-help</b> option states whether or not <b>readline()</b> will be used. 317</P> 318<P> 319The program handles any number of tests, each of which consists of a set of 320input lines. Each set starts with a regular expression pattern, followed by any 321number of subject lines to be matched against that pattern. In between sets of 322test data, command lines that begin with # may appear. This file format, with 323some restrictions, can also be processed by the <b>perltest.sh</b> script that 324is distributed with PCRE2 as a means of checking that the behaviour of PCRE2 325and Perl is the same. For a specification of <b>perltest.sh</b>, see the 326comments near its beginning. See also the #perltest command below. 327</P> 328<P> 329When the input is a terminal, <b>pcre2test</b> prompts for each line of input, 330using "re>" to prompt for regular expression patterns, and "data>" to prompt 331for subject lines. Command lines starting with # can be entered only in 332response to the "re>" prompt. 333</P> 334<P> 335Each subject line is matched separately and independently. If you want to do 336multi-line matches, you have to use the \n escape sequence (or \r or \r\n, 337etc., depending on the newline setting) in a single line of input to encode the 338newline sequences. There is no limit on the length of subject lines; the input 339buffer is automatically extended if it is too small. There are replication 340features that makes it possible to generate long repetitive pattern or subject 341lines without having to supply them explicitly. 342</P> 343<P> 344An empty line or the end of the file signals the end of the subject lines for a 345test, at which point a new pattern or command line is expected if there is 346still input to be read. 347</P> 348<br><a name="SEC6" href="#TOC1">COMMAND LINES</a><br> 349<P> 350In between sets of test data, a line that begins with # is interpreted as a 351command line. If the first character is followed by white space or an 352exclamation mark, the line is treated as a comment, and ignored. Otherwise, the 353following commands are recognized: 354<pre> 355 #forbid_utf 356</pre> 357Subsequent patterns automatically have the PCRE2_NEVER_UTF and PCRE2_NEVER_UCP 358options set, which locks out the use of the PCRE2_UTF and PCRE2_UCP options and 359the use of (*UTF) and (*UCP) at the start of patterns. This command also forces 360an error if a subsequent pattern contains any occurrences of \P, \p, or \X, 361which are still supported when PCRE2_UTF is not set, but which require Unicode 362property support to be included in the library. 363</P> 364<P> 365This is a trigger guard that is used in test files to ensure that UTF or 366Unicode property tests are not accidentally added to files that are used when 367Unicode support is not included in the library. Setting PCRE2_NEVER_UTF and 368PCRE2_NEVER_UCP as a default can also be obtained by the use of <b>#pattern</b>; 369the difference is that <b>#forbid_utf</b> cannot be unset, and the automatic 370options are not displayed in pattern information, to avoid cluttering up test 371output. 372<pre> 373 #load <filename> 374</pre> 375This command is used to load a set of precompiled patterns from a file, as 376described in the section entitled "Saving and restoring compiled patterns" 377<a href="#saverestore">below.</a> 378<pre> 379 #loadtables <filename> 380</pre> 381This command is used to load a set of binary character tables that can be 382accessed by the tables=3 qualifier. Such tables can be created by the 383<b>pcre2_dftables</b> program with the -b option. 384<pre> 385 #newline_default [<newline-list>] 386</pre> 387When PCRE2 is built, a default newline convention can be specified. This 388determines which characters and/or character pairs are recognized as indicating 389a newline in a pattern or subject string. The default can be overridden when a 390pattern is compiled. The standard test files contain tests of various newline 391conventions, but the majority of the tests expect a single linefeed to be 392recognized as a newline by default. Without special action the tests would fail 393when PCRE2 is compiled with either CR or CRLF as the default newline. 394</P> 395<P> 396The #newline_default command specifies a list of newline types that are 397acceptable as the default. The types must be one of CR, LF, CRLF, ANYCRLF, 398ANY, or NUL (in upper or lower case), for example: 399<pre> 400 #newline_default LF Any anyCRLF 401</pre> 402If the default newline is in the list, this command has no effect. Otherwise, 403except when testing the POSIX API, a <b>newline</b> modifier that specifies the 404first newline convention in the list (LF in the above example) is added to any 405pattern that does not already have a <b>newline</b> modifier. If the newline 406list is empty, the feature is turned off. This command is present in a number 407of the standard test input files. 408</P> 409<P> 410When the POSIX API is being tested there is no way to override the default 411newline convention, though it is possible to set the newline convention from 412within the pattern. A warning is given if the <b>posix</b> or <b>posix_nosub</b> 413modifier is used when <b>#newline_default</b> would set a default for the 414non-POSIX API. 415<pre> 416 #pattern <modifier-list> 417</pre> 418This command sets a default modifier list that applies to all subsequent 419patterns. Modifiers on a pattern can change these settings. 420<pre> 421 #perltest 422</pre> 423This line is used in test files that can also be processed by <b>perltest.sh</b> 424to confirm that Perl gives the same results as PCRE2. Subsequent tests are 425checked for the use of <b>pcre2test</b> features that are incompatible with the 426<b>perltest.sh</b> script. 427</P> 428<P> 429Patterns must use '/' as their delimiter, and only certain modifiers are 430supported. Comment lines, #pattern commands, and #subject commands that set or 431unset "mark" are recognized and acted on. The #perltest, #forbid_utf, and 432#newline_default commands, which are needed in the relevant pcre2test files, 433are silently ignored. All other command lines are ignored, but give a warning 434message. The <b>#perltest</b> command helps detect tests that are accidentally 435put in the wrong file or use the wrong delimiter. For more details of the 436<b>perltest.sh</b> script see the comments it contains. 437<pre> 438 #pop [<modifiers>] 439 #popcopy [<modifiers>] 440</pre> 441These commands are used to manipulate the stack of compiled patterns, as 442described in the section entitled "Saving and restoring compiled patterns" 443<a href="#saverestore">below.</a> 444<pre> 445 #save <filename> 446</pre> 447This command is used to save a set of compiled patterns to a file, as described 448in the section entitled "Saving and restoring compiled patterns" 449<a href="#saverestore">below.</a> 450<pre> 451 #subject <modifier-list> 452</pre> 453This command sets a default modifier list that applies to all subsequent 454subject lines. Modifiers on a subject line can change these settings. 455</P> 456<br><a name="SEC7" href="#TOC1">MODIFIER SYNTAX</a><br> 457<P> 458Modifier lists are used with both pattern and subject lines. Items in a list 459are separated by commas followed by optional white space. Trailing whitespace 460in a modifier list is ignored. Some modifiers may be given for both patterns 461and subject lines, whereas others are valid only for one or the other. Each 462modifier has a long name, for example "anchored", and some of them must be 463followed by an equals sign and a value, for example, "offset=12". Values cannot 464contain comma characters, but may contain spaces. Modifiers that do not take 465values may be preceded by a minus sign to turn off a previous setting. 466</P> 467<P> 468A few of the more common modifiers can also be specified as single letters, for 469example "i" for "caseless". In documentation, following the Perl convention, 470these are written with a slash ("the /i modifier") for clarity. Abbreviated 471modifiers must all be concatenated in the first item of a modifier list. If the 472first item is not recognized as a long modifier name, it is interpreted as a 473sequence of these abbreviations. For example: 474<pre> 475 /abc/ig,newline=cr,jit=3 476</pre> 477This is a pattern line whose modifier list starts with two one-letter modifiers 478(/i and /g). The lower-case abbreviated modifiers are the same as used in Perl. 479</P> 480<br><a name="SEC8" href="#TOC1">PATTERN SYNTAX</a><br> 481<P> 482A pattern line must start with one of the following characters (common symbols, 483excluding pattern meta-characters): 484<pre> 485 / ! " ' ` - = _ : ; , % & @ ~ 486</pre> 487This is interpreted as the pattern's delimiter. A regular expression may be 488continued over several input lines, in which case the newline characters are 489included within it. It is possible to include the delimiter within the pattern 490by escaping it with a backslash, for example 491<pre> 492 /abc\/def/ 493</pre> 494If you do this, the escape and the delimiter form part of the pattern, but 495since the delimiters are all non-alphanumeric, this does not affect its 496interpretation. If the terminating delimiter is immediately followed by a 497backslash, for example, 498<pre> 499 /abc/\ 500</pre> 501then a backslash is added to the end of the pattern. This is done to provide a 502way of testing the error condition that arises if a pattern finishes with a 503backslash, because 504<pre> 505 /abc\/ 506</pre> 507is interpreted as the first line of a pattern that starts with "abc/", causing 508pcre2test to read the next line as a continuation of the regular expression. 509</P> 510<P> 511A pattern can be followed by a modifier list (details below). 512</P> 513<br><a name="SEC9" href="#TOC1">SUBJECT LINE SYNTAX</a><br> 514<P> 515Before each subject line is passed to <b>pcre2_match()</b> or 516<b>pcre2_dfa_match()</b>, leading and trailing white space is removed, and the 517line is scanned for backslash escapes, unless the <b>subject_literal</b> 518modifier was set for the pattern. The following provide a means of encoding 519non-printing characters in a visible way: 520<pre> 521 \a alarm (BEL, \x07) 522 \b backspace (\x08) 523 \e escape (\x27) 524 \f form feed (\x0c) 525 \n newline (\x0a) 526 \r carriage return (\x0d) 527 \t tab (\x09) 528 \v vertical tab (\x0b) 529 \nnn octal character (up to 3 octal digits); always 530 a byte unless > 255 in UTF-8 or 16-bit or 32-bit mode 531 \o{dd...} octal character (any number of octal digits} 532 \xhh hexadecimal byte (up to 2 hex digits) 533 \x{hh...} hexadecimal character (any number of hex digits) 534</pre> 535The use of \x{hh...} is not dependent on the use of the <b>utf</b> modifier on 536the pattern. It is recognized always. There may be any number of hexadecimal 537digits inside the braces; invalid values provoke error messages. 538</P> 539<P> 540Note that \xhh specifies one byte rather than one character in UTF-8 mode; 541this makes it possible to construct invalid UTF-8 sequences for testing 542purposes. On the other hand, \x{hh} is interpreted as a UTF-8 character in 543UTF-8 mode, generating more than one byte if the value is greater than 127. 544When testing the 8-bit library not in UTF-8 mode, \x{hh} generates one byte 545for values less than 256, and causes an error for greater values. 546</P> 547<P> 548In UTF-16 mode, all 4-digit \x{hhhh} values are accepted. This makes it 549possible to construct invalid UTF-16 sequences for testing purposes. 550</P> 551<P> 552In UTF-32 mode, all 4- to 8-digit \x{...} values are accepted. This makes it 553possible to construct invalid UTF-32 sequences for testing purposes. 554</P> 555<P> 556There is a special backslash sequence that specifies replication of one or more 557characters: 558<pre> 559 \[<characters>]{<count>} 560</pre> 561This makes it possible to test long strings without having to provide them as 562part of the file. For example: 563<pre> 564 \[abc]{4} 565</pre> 566is converted to "abcabcabcabc". This feature does not support nesting. To 567include a closing square bracket in the characters, code it as \x5D. 568</P> 569<P> 570A backslash followed by an equals sign marks the end of the subject string and 571the start of a modifier list. For example: 572<pre> 573 abc\=notbol,notempty 574</pre> 575If the subject string is empty and \= is followed by whitespace, the line is 576treated as a comment line, and is not used for matching. For example: 577<pre> 578 \= This is a comment. 579 abc\= This is an invalid modifier list. 580</pre> 581A backslash followed by any other non-alphanumeric character just escapes that 582character. A backslash followed by anything else causes an error. However, if 583the very last character in the line is a backslash (and there is no modifier 584list), it is ignored. This gives a way of passing an empty line as data, since 585a real empty line terminates the data input. 586</P> 587<P> 588If the <b>subject_literal</b> modifier is set for a pattern, all subject lines 589that follow are treated as literals, with no special treatment of backslashes. 590No replication is possible, and any subject modifiers must be set as defaults 591by a <b>#subject</b> command. 592</P> 593<br><a name="SEC10" href="#TOC1">PATTERN MODIFIERS</a><br> 594<P> 595There are several types of modifier that can appear in pattern lines. Except 596where noted below, they may also be used in <b>#pattern</b> commands. A 597pattern's modifier list can add to or override default modifiers that were set 598by a previous <b>#pattern</b> command. 599<a name="optionmodifiers"></a></P> 600<br><b> 601Setting compilation options 602</b><br> 603<P> 604The following modifiers set options for <b>pcre2_compile()</b>. Most of them set 605bits in the options argument of that function, but those whose names start with 606PCRE2_EXTRA are additional options that are set in the compile context. For the 607main options, there are some single-letter abbreviations that are the same as 608Perl options. There is special handling for /x: if a second x is present, 609PCRE2_EXTENDED is converted into PCRE2_EXTENDED_MORE as in Perl. A third 610appearance adds PCRE2_EXTENDED as well, though this makes no difference to the 611way <b>pcre2_compile()</b> behaves. See 612<a href="pcre2api.html"><b>pcre2api</b></a> 613for a description of the effects of these options. 614<pre> 615 allow_empty_class set PCRE2_ALLOW_EMPTY_CLASS 616 allow_surrogate_escapes set PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES 617 alt_bsux set PCRE2_ALT_BSUX 618 alt_circumflex set PCRE2_ALT_CIRCUMFLEX 619 alt_verbnames set PCRE2_ALT_VERBNAMES 620 anchored set PCRE2_ANCHORED 621 auto_callout set PCRE2_AUTO_CALLOUT 622 bad_escape_is_literal set PCRE2_EXTRA_BAD_ESCAPE_IS_LITERAL 623 /i caseless set PCRE2_CASELESS 624 dollar_endonly set PCRE2_DOLLAR_ENDONLY 625 /s dotall set PCRE2_DOTALL 626 dupnames set PCRE2_DUPNAMES 627 endanchored set PCRE2_ENDANCHORED 628 escaped_cr_is_lf set PCRE2_EXTRA_ESCAPED_CR_IS_LF 629 /x extended set PCRE2_EXTENDED 630 /xx extended_more set PCRE2_EXTENDED_MORE 631 extra_alt_bsux set PCRE2_EXTRA_ALT_BSUX 632 firstline set PCRE2_FIRSTLINE 633 literal set PCRE2_LITERAL 634 match_line set PCRE2_EXTRA_MATCH_LINE 635 match_invalid_utf set PCRE2_MATCH_INVALID_UTF 636 match_unset_backref set PCRE2_MATCH_UNSET_BACKREF 637 match_word set PCRE2_EXTRA_MATCH_WORD 638 /m multiline set PCRE2_MULTILINE 639 never_backslash_c set PCRE2_NEVER_BACKSLASH_C 640 never_ucp set PCRE2_NEVER_UCP 641 never_utf set PCRE2_NEVER_UTF 642 /n no_auto_capture set PCRE2_NO_AUTO_CAPTURE 643 no_auto_possess set PCRE2_NO_AUTO_POSSESS 644 no_dotstar_anchor set PCRE2_NO_DOTSTAR_ANCHOR 645 no_start_optimize set PCRE2_NO_START_OPTIMIZE 646 no_utf_check set PCRE2_NO_UTF_CHECK 647 ucp set PCRE2_UCP 648 ungreedy set PCRE2_UNGREEDY 649 use_offset_limit set PCRE2_USE_OFFSET_LIMIT 650 utf set PCRE2_UTF 651</pre> 652As well as turning on the PCRE2_UTF option, the <b>utf</b> modifier causes all 653non-printing characters in output strings to be printed using the \x{hh...} 654notation. Otherwise, those less than 0x100 are output in hex without the curly 655brackets. Setting <b>utf</b> in 16-bit or 32-bit mode also causes pattern and 656subject strings to be translated to UTF-16 or UTF-32, respectively, before 657being passed to library functions. 658<a name="controlmodifiers"></a></P> 659<br><b> 660Setting compilation controls 661</b><br> 662<P> 663The following modifiers affect the compilation process or request information 664about the pattern. There are single-letter abbreviations for some that are 665heavily used in the test files. 666<pre> 667 bsr=[anycrlf|unicode] specify \R handling 668 /B bincode show binary code without lengths 669 callout_info show callout information 670 convert=<options> request foreign pattern conversion 671 convert_glob_escape=c set glob escape character 672 convert_glob_separator=c set glob separator character 673 convert_length set convert buffer length 674 debug same as info,fullbincode 675 framesize show matching frame size 676 fullbincode show binary code with lengths 677 /I info show info about compiled pattern 678 hex unquoted characters are hexadecimal 679 jit[=<number>] use JIT 680 jitfast use JIT fast path 681 jitverify verify JIT use 682 locale=<name> use this locale 683 max_pattern_length=<n> set the maximum pattern length 684 memory show memory used 685 newline=<type> set newline type 686 null_context compile with a NULL context 687 parens_nest_limit=<n> set maximum parentheses depth 688 posix use the POSIX API 689 posix_nosub use the POSIX API with REG_NOSUB 690 push push compiled pattern onto the stack 691 pushcopy push a copy onto the stack 692 stackguard=<number> test the stackguard feature 693 subject_literal treat all subject lines as literal 694 tables=[0|1|2|3] select internal tables 695 use_length do not zero-terminate the pattern 696 utf8_input treat input as UTF-8 697</pre> 698The effects of these modifiers are described in the following sections. 699</P> 700<br><b> 701Newline and \R handling 702</b><br> 703<P> 704The <b>bsr</b> modifier specifies what \R in a pattern should match. If it is 705set to "anycrlf", \R matches CR, LF, or CRLF only. If it is set to "unicode", 706\R matches any Unicode newline sequence. The default can be specified when 707PCRE2 is built; if it is not, the default is set to Unicode. 708</P> 709<P> 710The <b>newline</b> modifier specifies which characters are to be interpreted as 711newlines, both in the pattern and in subject lines. The type must be one of CR, 712LF, CRLF, ANYCRLF, ANY, or NUL (in upper or lower case). 713</P> 714<br><b> 715Information about a pattern 716</b><br> 717<P> 718The <b>debug</b> modifier is a shorthand for <b>info,fullbincode</b>, requesting 719all available information. 720</P> 721<P> 722The <b>bincode</b> modifier causes a representation of the compiled code to be 723output after compilation. This information does not contain length and offset 724values, which ensures that the same output is generated for different internal 725link sizes and different code unit widths. By using <b>bincode</b>, the same 726regression tests can be used in different environments. 727</P> 728<P> 729The <b>fullbincode</b> modifier, by contrast, <i>does</i> include length and 730offset values. This is used in a few special tests that run only for specific 731code unit widths and link sizes, and is also useful for one-off tests. 732</P> 733<P> 734The <b>info</b> modifier requests information about the compiled pattern 735(whether it is anchored, has a fixed first character, and so on). The 736information is obtained from the <b>pcre2_pattern_info()</b> function. Here are 737some typical examples: 738<pre> 739 re> /(?i)(^a|^b)/m,info 740 Capture group count = 1 741 Compile options: multiline 742 Overall options: caseless multiline 743 First code unit at start or follows newline 744 Subject length lower bound = 1 745 746 re> /(?i)abc/info 747 Capture group count = 0 748 Compile options: <none> 749 Overall options: caseless 750 First code unit = 'a' (caseless) 751 Last code unit = 'c' (caseless) 752 Subject length lower bound = 3 753</pre> 754"Compile options" are those specified by modifiers; "overall options" have 755added options that are taken or deduced from the pattern. If both sets of 756options are the same, just a single "options" line is output; if there are no 757options, the line is omitted. "First code unit" is where any match must start; 758if there is more than one they are listed as "starting code units". "Last code 759unit" is the last literal code unit that must be present in any match. This is 760not necessarily the last character. These lines are omitted if no starting or 761ending code units are recorded. The subject length line is omitted when 762<b>no_start_optimize</b> is set because the minimum length is not calculated 763when it can never be used. 764</P> 765<P> 766The <b>framesize</b> modifier shows the size, in bytes, of the storage frames 767used by <b>pcre2_match()</b> for handling backtracking. The size depends on the 768number of capturing parentheses in the pattern. 769</P> 770<P> 771The <b>callout_info</b> modifier requests information about all the callouts in 772the pattern. A list of them is output at the end of any other information that 773is requested. For each callout, either its number or string is given, followed 774by the item that follows it in the pattern. 775</P> 776<br><b> 777Passing a NULL context 778</b><br> 779<P> 780Normally, <b>pcre2test</b> passes a context block to <b>pcre2_compile()</b>. If 781the <b>null_context</b> modifier is set, however, NULL is passed. This is for 782testing that <b>pcre2_compile()</b> behaves correctly in this case (it uses 783default values). 784</P> 785<br><b> 786Specifying pattern characters in hexadecimal 787</b><br> 788<P> 789The <b>hex</b> modifier specifies that the characters of the pattern, except for 790substrings enclosed in single or double quotes, are to be interpreted as pairs 791of hexadecimal digits. This feature is provided as a way of creating patterns 792that contain binary zeros and other non-printing characters. White space is 793permitted between pairs of digits. For example, this pattern contains three 794characters: 795<pre> 796 /ab 32 59/hex 797</pre> 798Parts of such a pattern are taken literally if quoted. This pattern contains 799nine characters, only two of which are specified in hexadecimal: 800<pre> 801 /ab "literal" 32/hex 802</pre> 803Either single or double quotes may be used. There is no way of including 804the delimiter within a substring. The <b>hex</b> and <b>expand</b> modifiers are 805mutually exclusive. 806</P> 807<br><b> 808Specifying the pattern's length 809</b><br> 810<P> 811By default, patterns are passed to the compiling functions as zero-terminated 812strings but can be passed by length instead of being zero-terminated. The 813<b>use_length</b> modifier causes this to happen. Using a length happens 814automatically (whether or not <b>use_length</b> is set) when <b>hex</b> is set, 815because patterns specified in hexadecimal may contain binary zeros. 816</P> 817<P> 818If <b>hex</b> or <b>use_length</b> is used with the POSIX wrapper API (see 819<a href="#posixwrapper">"Using the POSIX wrapper API"</a> 820below), the REG_PEND extension is used to pass the pattern's length. 821</P> 822<br><b> 823Specifying wide characters in 16-bit and 32-bit modes 824</b><br> 825<P> 826In 16-bit and 32-bit modes, all input is automatically treated as UTF-8 and 827translated to UTF-16 or UTF-32 when the <b>utf</b> modifier is set. For testing 828the 16-bit and 32-bit libraries in non-UTF mode, the <b>utf8_input</b> modifier 829can be used. It is mutually exclusive with <b>utf</b>. Input lines are 830interpreted as UTF-8 as a means of specifying wide characters. More details are 831given in 832<a href="#inputencoding">"Input encoding"</a> 833above. 834</P> 835<br><b> 836Generating long repetitive patterns 837</b><br> 838<P> 839Some tests use long patterns that are very repetitive. Instead of creating a 840very long input line for such a pattern, you can use a special repetition 841feature, similar to the one described for subject lines above. If the 842<b>expand</b> modifier is present on a pattern, parts of the pattern that have 843the form 844<pre> 845 \[<characters>]{<count>} 846</pre> 847are expanded before the pattern is passed to <b>pcre2_compile()</b>. For 848example, \[AB]{6000} is expanded to "ABAB..." 6000 times. This construction 849cannot be nested. An initial "\[" sequence is recognized only if "]{" followed 850by decimal digits and "}" is found later in the pattern. If not, the characters 851remain in the pattern unaltered. The <b>expand</b> and <b>hex</b> modifiers are 852mutually exclusive. 853</P> 854<P> 855If part of an expanded pattern looks like an expansion, but is really part of 856the actual pattern, unwanted expansion can be avoided by giving two values in 857the quantifier. For example, \[AB]{6000,6000} is not recognized as an 858expansion item. 859</P> 860<P> 861If the <b>info</b> modifier is set on an expanded pattern, the result of the 862expansion is included in the information that is output. 863</P> 864<br><b> 865JIT compilation 866</b><br> 867<P> 868Just-in-time (JIT) compiling is a heavyweight optimization that can greatly 869speed up pattern matching. See the 870<a href="pcre2jit.html"><b>pcre2jit</b></a> 871documentation for details. JIT compiling happens, optionally, after a pattern 872has been successfully compiled into an internal form. The JIT compiler converts 873this to optimized machine code. It needs to know whether the match-time options 874PCRE2_PARTIAL_HARD and PCRE2_PARTIAL_SOFT are going to be used, because 875different code is generated for the different cases. See the <b>partial</b> 876modifier in "Subject Modifiers" 877<a href="#subjectmodifiers">below</a> 878for details of how these options are specified for each match attempt. 879</P> 880<P> 881JIT compilation is requested by the <b>jit</b> pattern modifier, which may 882optionally be followed by an equals sign and a number in the range 0 to 7. 883The three bits that make up the number specify which of the three JIT operating 884modes are to be compiled: 885<pre> 886 1 compile JIT code for non-partial matching 887 2 compile JIT code for soft partial matching 888 4 compile JIT code for hard partial matching 889</pre> 890The possible values for the <b>jit</b> modifier are therefore: 891<pre> 892 0 disable JIT 893 1 normal matching only 894 2 soft partial matching only 895 3 normal and soft partial matching 896 4 hard partial matching only 897 6 soft and hard partial matching only 898 7 all three modes 899</pre> 900If no number is given, 7 is assumed. The phrase "partial matching" means a call 901to <b>pcre2_match()</b> with either the PCRE2_PARTIAL_SOFT or the 902PCRE2_PARTIAL_HARD option set. Note that such a call may return a complete 903match; the options enable the possibility of a partial match, but do not 904require it. Note also that if you request JIT compilation only for partial 905matching (for example, jit=2) but do not set the <b>partial</b> modifier on a 906subject line, that match will not use JIT code because none was compiled for 907non-partial matching. 908</P> 909<P> 910If JIT compilation is successful, the compiled JIT code will automatically be 911used when an appropriate type of match is run, except when incompatible 912run-time options are specified. For more details, see the 913<a href="pcre2jit.html"><b>pcre2jit</b></a> 914documentation. See also the <b>jitstack</b> modifier below for a way of 915setting the size of the JIT stack. 916</P> 917<P> 918If the <b>jitfast</b> modifier is specified, matching is done using the JIT 919"fast path" interface, <b>pcre2_jit_match()</b>, which skips some of the sanity 920checks that are done by <b>pcre2_match()</b>, and of course does not work when 921JIT is not supported. If <b>jitfast</b> is specified without <b>jit</b>, jit=7 is 922assumed. 923</P> 924<P> 925If the <b>jitverify</b> modifier is specified, information about the compiled 926pattern shows whether JIT compilation was or was not successful. If 927<b>jitverify</b> is specified without <b>jit</b>, jit=7 is assumed. If JIT 928compilation is successful when <b>jitverify</b> is set, the text "(JIT)" is 929added to the first output line after a match or non match when JIT-compiled 930code was actually used in the match. 931</P> 932<br><b> 933Setting a locale 934</b><br> 935<P> 936The <b>locale</b> modifier must specify the name of a locale, for example: 937<pre> 938 /pattern/locale=fr_FR 939</pre> 940The given locale is set, <b>pcre2_maketables()</b> is called to build a set of 941character tables for the locale, and this is then passed to 942<b>pcre2_compile()</b> when compiling the regular expression. The same tables 943are used when matching the following subject lines. The <b>locale</b> modifier 944applies only to the pattern on which it appears, but can be given in a 945<b>#pattern</b> command if a default is needed. Setting a locale and alternate 946character tables are mutually exclusive. 947</P> 948<br><b> 949Showing pattern memory 950</b><br> 951<P> 952The <b>memory</b> modifier causes the size in bytes of the memory used to hold 953the compiled pattern to be output. This does not include the size of the 954<b>pcre2_code</b> block; it is just the actual compiled data. If the pattern is 955subsequently passed to the JIT compiler, the size of the JIT compiled code is 956also output. Here is an example: 957<pre> 958 re> /a(b)c/jit,memory 959 Memory allocation (code space): 21 960 Memory allocation (JIT code): 1910 961 962</PRE> 963</P> 964<br><b> 965Limiting nested parentheses 966</b><br> 967<P> 968The <b>parens_nest_limit</b> modifier sets a limit on the depth of nested 969parentheses in a pattern. Breaching the limit causes a compilation error. 970The default for the library is set when PCRE2 is built, but <b>pcre2test</b> 971sets its own default of 220, which is required for running the standard test 972suite. 973</P> 974<br><b> 975Limiting the pattern length 976</b><br> 977<P> 978The <b>max_pattern_length</b> modifier sets a limit, in code units, to the 979length of pattern that <b>pcre2_compile()</b> will accept. Breaching the limit 980causes a compilation error. The default is the largest number a PCRE2_SIZE 981variable can hold (essentially unlimited). 982<a name="posixwrapper"></a></P> 983<br><b> 984Using the POSIX wrapper API 985</b><br> 986<P> 987The <b>posix</b> and <b>posix_nosub</b> modifiers cause <b>pcre2test</b> to call 988PCRE2 via the POSIX wrapper API rather than its native API. When 989<b>posix_nosub</b> is used, the POSIX option REG_NOSUB is passed to 990<b>regcomp()</b>. The POSIX wrapper supports only the 8-bit library. Note that 991it does not imply POSIX matching semantics; for more detail see the 992<a href="pcre2posix.html"><b>pcre2posix</b></a> 993documentation. The following pattern modifiers set options for the 994<b>regcomp()</b> function: 995<pre> 996 caseless REG_ICASE 997 multiline REG_NEWLINE 998 dotall REG_DOTALL ) 999 ungreedy REG_UNGREEDY ) These options are not part of 1000 ucp REG_UCP ) the POSIX standard 1001 utf REG_UTF8 ) 1002</pre> 1003The <b>regerror_buffsize</b> modifier specifies a size for the error buffer that 1004is passed to <b>regerror()</b> in the event of a compilation error. For example: 1005<pre> 1006 /abc/posix,regerror_buffsize=20 1007</pre> 1008This provides a means of testing the behaviour of <b>regerror()</b> when the 1009buffer is too small for the error message. If this modifier has not been set, a 1010large buffer is used. 1011</P> 1012<P> 1013The <b>aftertext</b> and <b>allaftertext</b> subject modifiers work as described 1014below. All other modifiers are either ignored, with a warning message, or cause 1015an error. 1016</P> 1017<P> 1018The pattern is passed to <b>regcomp()</b> as a zero-terminated string by 1019default, but if the <b>use_length</b> or <b>hex</b> modifiers are set, the 1020REG_PEND extension is used to pass it by length. 1021</P> 1022<br><b> 1023Testing the stack guard feature 1024</b><br> 1025<P> 1026The <b>stackguard</b> modifier is used to test the use of 1027<b>pcre2_set_compile_recursion_guard()</b>, a function that is provided to 1028enable stack availability to be checked during compilation (see the 1029<a href="pcre2api.html"><b>pcre2api</b></a> 1030documentation for details). If the number specified by the modifier is greater 1031than zero, <b>pcre2_set_compile_recursion_guard()</b> is called to set up 1032callback from <b>pcre2_compile()</b> to a local function. The argument it 1033receives is the current nesting parenthesis depth; if this is greater than the 1034value given by the modifier, non-zero is returned, causing the compilation to 1035be aborted. 1036</P> 1037<br><b> 1038Using alternative character tables 1039</b><br> 1040<P> 1041The value specified for the <b>tables</b> modifier must be one of the digits 0, 10421, 2, or 3. It causes a specific set of built-in character tables to be passed 1043to <b>pcre2_compile()</b>. This is used in the PCRE2 tests to check behaviour 1044with different character tables. The digit specifies the tables as follows: 1045<pre> 1046 0 do not pass any special character tables 1047 1 the default ASCII tables, as distributed in 1048 pcre2_chartables.c.dist 1049 2 a set of tables defining ISO 8859 characters 1050 3 a set of tables loaded by the #loadtables command 1051</pre> 1052In tables 2, some characters whose codes are greater than 128 are identified as 1053letters, digits, spaces, etc. Tables 3 can be used only after a 1054<b>#loadtables</b> command has loaded them from a binary file. Setting alternate 1055character tables and a locale are mutually exclusive. 1056</P> 1057<br><b> 1058Setting certain match controls 1059</b><br> 1060<P> 1061The following modifiers are really subject modifiers, and are described under 1062"Subject Modifiers" below. However, they may be included in a pattern's 1063modifier list, in which case they are applied to every subject line that is 1064processed with that pattern. These modifiers do not affect the compilation 1065process. 1066<pre> 1067 aftertext show text after match 1068 allaftertext show text after captures 1069 allcaptures show all captures 1070 allvector show the entire ovector 1071 allusedtext show all consulted text 1072 altglobal alternative global matching 1073 /g global global matching 1074 jitstack=<n> set size of JIT stack 1075 mark show mark values 1076 replace=<string> specify a replacement string 1077 startchar show starting character when relevant 1078 substitute_callout use substitution callouts 1079 substitute_extended use PCRE2_SUBSTITUTE_EXTENDED 1080 substitute_literal use PCRE2_SUBSTITUTE_LITERAL 1081 substitute_matched use PCRE2_SUBSTITUTE_MATCHED 1082 substitute_overflow_length use PCRE2_SUBSTITUTE_OVERFLOW_LENGTH 1083 substitute_replacement_only use PCRE2_SUBSTITUTE_REPLACEMENT_ONLY 1084 substitute_skip=<n> skip substitution <n> 1085 substitute_stop=<n> skip substitution <n> and following 1086 substitute_unknown_unset use PCRE2_SUBSTITUTE_UNKNOWN_UNSET 1087 substitute_unset_empty use PCRE2_SUBSTITUTE_UNSET_EMPTY 1088</pre> 1089These modifiers may not appear in a <b>#pattern</b> command. If you want them as 1090defaults, set them in a <b>#subject</b> command. 1091</P> 1092<br><b> 1093Specifying literal subject lines 1094</b><br> 1095<P> 1096If the <b>subject_literal</b> modifier is present on a pattern, all the subject 1097lines that it matches are taken as literal strings, with no interpretation of 1098backslashes. It is not possible to set subject modifiers on such lines, but any 1099that are set as defaults by a <b>#subject</b> command are recognized. 1100</P> 1101<br><b> 1102Saving a compiled pattern 1103</b><br> 1104<P> 1105When a pattern with the <b>push</b> modifier is successfully compiled, it is 1106pushed onto a stack of compiled patterns, and <b>pcre2test</b> expects the next 1107line to contain a new pattern (or a command) instead of a subject line. This 1108facility is used when saving compiled patterns to a file, as described in the 1109section entitled "Saving and restoring compiled patterns" 1110<a href="#saverestore">below.</a> 1111If <b>pushcopy</b> is used instead of <b>push</b>, a copy of the compiled 1112pattern is stacked, leaving the original as current, ready to match the 1113following input lines. This provides a way of testing the 1114<b>pcre2_code_copy()</b> function. 1115The <b>push</b> and <b>pushcopy </b> modifiers are incompatible with compilation 1116modifiers such as <b>global</b> that act at match time. Any that are specified 1117are ignored (for the stacked copy), with a warning message, except for 1118<b>replace</b>, which causes an error. Note that <b>jitverify</b>, which is 1119allowed, does not carry through to any subsequent matching that uses a stacked 1120pattern. 1121</P> 1122<br><b> 1123Testing foreign pattern conversion 1124</b><br> 1125<P> 1126The experimental foreign pattern conversion functions in PCRE2 can be tested by 1127setting the <b>convert</b> modifier. Its argument is a colon-separated list of 1128options, which set the equivalent option for the <b>pcre2_pattern_convert()</b> 1129function: 1130<pre> 1131 glob PCRE2_CONVERT_GLOB 1132 glob_no_starstar PCRE2_CONVERT_GLOB_NO_STARSTAR 1133 glob_no_wild_separator PCRE2_CONVERT_GLOB_NO_WILD_SEPARATOR 1134 posix_basic PCRE2_CONVERT_POSIX_BASIC 1135 posix_extended PCRE2_CONVERT_POSIX_EXTENDED 1136 unset Unset all options 1137</pre> 1138The "unset" value is useful for turning off a default that has been set by a 1139<b>#pattern</b> command. When one of these options is set, the input pattern is 1140passed to <b>pcre2_pattern_convert()</b>. If the conversion is successful, the 1141result is reflected in the output and then passed to <b>pcre2_compile()</b>. The 1142normal <b>utf</b> and <b>no_utf_check</b> options, if set, cause the 1143PCRE2_CONVERT_UTF and PCRE2_CONVERT_NO_UTF_CHECK options to be passed to 1144<b>pcre2_pattern_convert()</b>. 1145</P> 1146<P> 1147By default, the conversion function is allowed to allocate a buffer for its 1148output. However, if the <b>convert_length</b> modifier is set to a value greater 1149than zero, <b>pcre2test</b> passes a buffer of the given length. This makes it 1150possible to test the length check. 1151</P> 1152<P> 1153The <b>convert_glob_escape</b> and <b>convert_glob_separator</b> modifiers can be 1154used to specify the escape and separator characters for glob processing, 1155overriding the defaults, which are operating-system dependent. 1156<a name="subjectmodifiers"></a></P> 1157<br><a name="SEC11" href="#TOC1">SUBJECT MODIFIERS</a><br> 1158<P> 1159The modifiers that can appear in subject lines and the <b>#subject</b> 1160command are of two types. 1161</P> 1162<br><b> 1163Setting match options 1164</b><br> 1165<P> 1166The following modifiers set options for <b>pcre2_match()</b> or 1167<b>pcre2_dfa_match()</b>. See 1168<a href="pcreapi.html"><b>pcreapi</b></a> 1169for a description of their effects. 1170<pre> 1171 anchored set PCRE2_ANCHORED 1172 endanchored set PCRE2_ENDANCHORED 1173 dfa_restart set PCRE2_DFA_RESTART 1174 dfa_shortest set PCRE2_DFA_SHORTEST 1175 no_jit set PCRE2_NO_JIT 1176 no_utf_check set PCRE2_NO_UTF_CHECK 1177 notbol set PCRE2_NOTBOL 1178 notempty set PCRE2_NOTEMPTY 1179 notempty_atstart set PCRE2_NOTEMPTY_ATSTART 1180 noteol set PCRE2_NOTEOL 1181 partial_hard (or ph) set PCRE2_PARTIAL_HARD 1182 partial_soft (or ps) set PCRE2_PARTIAL_SOFT 1183</pre> 1184The partial matching modifiers are provided with abbreviations because they 1185appear frequently in tests. 1186</P> 1187<P> 1188If the <b>posix</b> or <b>posix_nosub</b> modifier was present on the pattern, 1189causing the POSIX wrapper API to be used, the only option-setting modifiers 1190that have any effect are <b>notbol</b>, <b>notempty</b>, and <b>noteol</b>, 1191causing REG_NOTBOL, REG_NOTEMPTY, and REG_NOTEOL, respectively, to be passed to 1192<b>regexec()</b>. The other modifiers are ignored, with a warning message. 1193</P> 1194<P> 1195There is one additional modifier that can be used with the POSIX wrapper. It is 1196ignored (with a warning) if used for non-POSIX matching. 1197<pre> 1198 posix_startend=<n>[:<m>] 1199</pre> 1200This causes the subject string to be passed to <b>regexec()</b> using the 1201REG_STARTEND option, which uses offsets to specify which part of the string is 1202searched. If only one number is given, the end offset is passed as the end of 1203the subject string. For more detail of REG_STARTEND, see the 1204<a href="pcre2posix.html"><b>pcre2posix</b></a> 1205documentation. If the subject string contains binary zeros (coded as escapes 1206such as \x{00} because <b>pcre2test</b> does not support actual binary zeros in 1207its input), you must use <b>posix_startend</b> to specify its length. 1208</P> 1209<br><b> 1210Setting match controls 1211</b><br> 1212<P> 1213The following modifiers affect the matching process or request additional 1214information. Some of them may also be specified on a pattern line (see above), 1215in which case they apply to every subject line that is matched against that 1216pattern. 1217<pre> 1218 aftertext show text after match 1219 allaftertext show text after captures 1220 allcaptures show all captures 1221 allvector show the entire ovector 1222 allusedtext show all consulted text (non-JIT only) 1223 altglobal alternative global matching 1224 callout_capture show captures at callout time 1225 callout_data=<n> set a value to pass via callouts 1226 callout_error=<n>[:<m>] control callout error 1227 callout_extra show extra callout information 1228 callout_fail=<n>[:<m>] control callout failure 1229 callout_no_where do not show position of a callout 1230 callout_none do not supply a callout function 1231 copy=<number or name> copy captured substring 1232 depth_limit=<n> set a depth limit 1233 dfa use <b>pcre2_dfa_match()</b> 1234 find_limits find match and depth limits 1235 get=<number or name> extract captured substring 1236 getall extract all captured substrings 1237 /g global global matching 1238 heap_limit=<n> set a limit on heap memory (Kbytes) 1239 jitstack=<n> set size of JIT stack 1240 mark show mark values 1241 match_limit=<n> set a match limit 1242 memory show heap memory usage 1243 null_context match with a NULL context 1244 offset=<n> set starting offset 1245 offset_limit=<n> set offset limit 1246 ovector=<n> set size of output vector 1247 recursion_limit=<n> obsolete synonym for depth_limit 1248 replace=<string> specify a replacement string 1249 startchar show startchar when relevant 1250 startoffset=<n> same as offset=<n> 1251 substitute_callout use substitution callouts 1252 substitute_extedded use PCRE2_SUBSTITUTE_EXTENDED 1253 substitute_literal use PCRE2_SUBSTITUTE_LITERAL 1254 substitute_matched use PCRE2_SUBSTITUTE_MATCHED 1255 substitute_overflow_length use PCRE2_SUBSTITUTE_OVERFLOW_LENGTH 1256 substitute_replacement_only use PCRE2_SUBSTITUTE_REPLACEMENT_ONLY 1257 substitute_skip=<n> skip substitution number n 1258 substitute_stop=<n> skip substitution number n and greater 1259 substitute_unknown_unset use PCRE2_SUBSTITUTE_UNKNOWN_UNSET 1260 substitute_unset_empty use PCRE2_SUBSTITUTE_UNSET_EMPTY 1261 zero_terminate pass the subject as zero-terminated 1262</pre> 1263The effects of these modifiers are described in the following sections. When 1264matching via the POSIX wrapper API, the <b>aftertext</b>, <b>allaftertext</b>, 1265and <b>ovector</b> subject modifiers work as described below. All other 1266modifiers are either ignored, with a warning message, or cause an error. 1267</P> 1268<br><b> 1269Showing more text 1270</b><br> 1271<P> 1272The <b>aftertext</b> modifier requests that as well as outputting the part of 1273the subject string that matched the entire pattern, <b>pcre2test</b> should in 1274addition output the remainder of the subject string. This is useful for tests 1275where the subject contains multiple copies of the same substring. The 1276<b>allaftertext</b> modifier requests the same action for captured substrings as 1277well as the main matched substring. In each case the remainder is output on the 1278following line with a plus character following the capture number. 1279</P> 1280<P> 1281The <b>allusedtext</b> modifier requests that all the text that was consulted 1282during a successful pattern match by the interpreter should be shown, for both 1283full and partial matches. This feature is not supported for JIT matching, and 1284if requested with JIT it is ignored (with a warning message). Setting this 1285modifier affects the output if there is a lookbehind at the start of a match, 1286or, for a complete match, a lookahead at the end, or if \K is used in the 1287pattern. Characters that precede or follow the start and end of the actual 1288match are indicated in the output by '<' or '>' characters underneath them. 1289Here is an example: 1290<pre> 1291 re> /(?<=pqr)abc(?=xyz)/ 1292 data> 123pqrabcxyz456\=allusedtext 1293 0: pqrabcxyz 1294 <<< >>> 1295 data> 123pqrabcxy\=ph,allusedtext 1296 Partial match: pqrabcxy 1297 <<< 1298</pre> 1299The first, complete match shows that the matched string is "abc", with the 1300preceding and following strings "pqr" and "xyz" having been consulted during 1301the match (when processing the assertions). The partial match can indicate only 1302the preceding string. 1303</P> 1304<P> 1305The <b>startchar</b> modifier requests that the starting character for the match 1306be indicated, if it is different to the start of the matched string. The only 1307time when this occurs is when \K has been processed as part of the match. In 1308this situation, the output for the matched string is displayed from the 1309starting character instead of from the match point, with circumflex characters 1310under the earlier characters. For example: 1311<pre> 1312 re> /abc\Kxyz/ 1313 data> abcxyz\=startchar 1314 0: abcxyz 1315 ^^^ 1316</pre> 1317Unlike <b>allusedtext</b>, the <b>startchar</b> modifier can be used with JIT. 1318However, these two modifiers are mutually exclusive. 1319</P> 1320<br><b> 1321Showing the value of all capture groups 1322</b><br> 1323<P> 1324The <b>allcaptures</b> modifier requests that the values of all potential 1325captured parentheses be output after a match. By default, only those up to the 1326highest one actually used in the match are output (corresponding to the return 1327code from <b>pcre2_match()</b>). Groups that did not take part in the match 1328are output as "<unset>". This modifier is not relevant for DFA matching (which 1329does no capturing) and does not apply when <b>replace</b> is specified; it is 1330ignored, with a warning message, if present. 1331</P> 1332<br><b> 1333Showing the entire ovector, for all outcomes 1334</b><br> 1335<P> 1336The <b>allvector</b> modifier requests that the entire ovector be shown, 1337whatever the outcome of the match. Compare <b>allcaptures</b>, which shows only 1338up to the maximum number of capture groups for the pattern, and then only for a 1339successful complete non-DFA match. This modifier, which acts after any match 1340result, and also for DFA matching, provides a means of checking that there are 1341no unexpected modifications to ovector fields. Before each match attempt, the 1342ovector is filled with a special value, and if this is found in both elements 1343of a capturing pair, "<unchanged>" is output. After a successful match, this 1344applies to all groups after the maximum capture group for the pattern. In other 1345cases it applies to the entire ovector. After a partial match, the first two 1346elements are the only ones that should be set. After a DFA match, the amount of 1347ovector that is used depends on the number of matches that were found. 1348</P> 1349<br><b> 1350Testing pattern callouts 1351</b><br> 1352<P> 1353A callout function is supplied when <b>pcre2test</b> calls the library matching 1354functions, unless <b>callout_none</b> is specified. Its behaviour can be 1355controlled by various modifiers listed above whose names begin with 1356<b>callout_</b>. Details are given in the section entitled "Callouts" 1357<a href="#callouts">below.</a> 1358Testing callouts from <b>pcre2_substitute()</b> is decribed separately in 1359"Testing the substitution function" 1360<a href="#substitution">below.</a> 1361</P> 1362<br><b> 1363Finding all matches in a string 1364</b><br> 1365<P> 1366Searching for all possible matches within a subject can be requested by the 1367<b>global</b> or <b>altglobal</b> modifier. After finding a match, the matching 1368function is called again to search the remainder of the subject. The difference 1369between <b>global</b> and <b>altglobal</b> is that the former uses the 1370<i>start_offset</i> argument to <b>pcre2_match()</b> or <b>pcre2_dfa_match()</b> 1371to start searching at a new point within the entire string (which is what Perl 1372does), whereas the latter passes over a shortened subject. This makes a 1373difference to the matching process if the pattern begins with a lookbehind 1374assertion (including \b or \B). 1375</P> 1376<P> 1377If an empty string is matched, the next match is done with the 1378PCRE2_NOTEMPTY_ATSTART and PCRE2_ANCHORED flags set, in order to search for 1379another, non-empty, match at the same point in the subject. If this match 1380fails, the start offset is advanced, and the normal match is retried. This 1381imitates the way Perl handles such cases when using the <b>/g</b> modifier or 1382the <b>split()</b> function. Normally, the start offset is advanced by one 1383character, but if the newline convention recognizes CRLF as a newline, and the 1384current character is CR followed by LF, an advance of two characters occurs. 1385</P> 1386<br><b> 1387Testing substring extraction functions 1388</b><br> 1389<P> 1390The <b>copy</b> and <b>get</b> modifiers can be used to test the 1391<b>pcre2_substring_copy_xxx()</b> and <b>pcre2_substring_get_xxx()</b> functions. 1392They can be given more than once, and each can specify a capture group name or 1393number, for example: 1394<pre> 1395 abcd\=copy=1,copy=3,get=G1 1396</pre> 1397If the <b>#subject</b> command is used to set default copy and/or get lists, 1398these can be unset by specifying a negative number to cancel all numbered 1399groups and an empty name to cancel all named groups. 1400</P> 1401<P> 1402The <b>getall</b> modifier tests <b>pcre2_substring_list_get()</b>, which 1403extracts all captured substrings. 1404</P> 1405<P> 1406If the subject line is successfully matched, the substrings extracted by the 1407convenience functions are output with C, G, or L after the string number 1408instead of a colon. This is in addition to the normal full list. The string 1409length (that is, the return from the extraction function) is given in 1410parentheses after each substring, followed by the name when the extraction was 1411by name. 1412<a name="substitution"></a></P> 1413<br><b> 1414Testing the substitution function 1415</b><br> 1416<P> 1417If the <b>replace</b> modifier is set, the <b>pcre2_substitute()</b> function is 1418called instead of one of the matching functions (or after one call of 1419<b>pcre2_match()</b> in the case of PCRE2_SUBSTITUTE_MATCHED). Note that 1420replacement strings cannot contain commas, because a comma signifies the end of 1421a modifier. This is not thought to be an issue in a test program. 1422</P> 1423<P> 1424Unlike subject strings, <b>pcre2test</b> does not process replacement strings 1425for escape sequences. In UTF mode, a replacement string is checked to see if it 1426is a valid UTF-8 string. If so, it is correctly converted to a UTF string of 1427the appropriate code unit width. If it is not a valid UTF-8 string, the 1428individual code units are copied directly. This provides a means of passing an 1429invalid UTF-8 string for testing purposes. 1430</P> 1431<P> 1432The following modifiers set options (in additional to the normal match options) 1433for <b>pcre2_substitute()</b>: 1434<pre> 1435 global PCRE2_SUBSTITUTE_GLOBAL 1436 substitute_extended PCRE2_SUBSTITUTE_EXTENDED 1437 substitute_literal PCRE2_SUBSTITUTE_LITERAL 1438 substitute_matched PCRE2_SUBSTITUTE_MATCHED 1439 substitute_overflow_length PCRE2_SUBSTITUTE_OVERFLOW_LENGTH 1440 substitute_replacement_only PCRE2_SUBSTITUTE_REPLACEMENT_ONLY 1441 substitute_unknown_unset PCRE2_SUBSTITUTE_UNKNOWN_UNSET 1442 substitute_unset_empty PCRE2_SUBSTITUTE_UNSET_EMPTY 1443</pre> 1444See the 1445<a href="pcre2api.html"><b>pcre2api</b></a> 1446documentation for details of these options. 1447</P> 1448<P> 1449After a successful substitution, the modified string is output, preceded by the 1450number of replacements. This may be zero if there were no matches. Here is a 1451simple example of a substitution test: 1452<pre> 1453 /abc/replace=xxx 1454 =abc=abc= 1455 1: =xxx=abc= 1456 =abc=abc=\=global 1457 2: =xxx=xxx= 1458</pre> 1459Subject and replacement strings should be kept relatively short (fewer than 256 1460characters) for substitution tests, as fixed-size buffers are used. To make it 1461easy to test for buffer overflow, if the replacement string starts with a 1462number in square brackets, that number is passed to <b>pcre2_substitute()</b> as 1463the size of the output buffer, with the replacement string starting at the next 1464character. Here is an example that tests the edge case: 1465<pre> 1466 /abc/ 1467 123abc123\=replace=[10]XYZ 1468 1: 123XYZ123 1469 123abc123\=replace=[9]XYZ 1470 Failed: error -47: no more memory 1471</pre> 1472The default action of <b>pcre2_substitute()</b> is to return 1473PCRE2_ERROR_NOMEMORY when the output buffer is too small. However, if the 1474PCRE2_SUBSTITUTE_OVERFLOW_LENGTH option is set (by using the 1475<b>substitute_overflow_length</b> modifier), <b>pcre2_substitute()</b> continues 1476to go through the motions of matching and substituting (but not doing any 1477callouts), in order to compute the size of buffer that is required. When this 1478happens, <b>pcre2test</b> shows the required buffer length (which includes space 1479for the trailing zero) as part of the error message. For example: 1480<pre> 1481 /abc/substitute_overflow_length 1482 123abc123\=replace=[9]XYZ 1483 Failed: error -47: no more memory: 10 code units are needed 1484</pre> 1485A replacement string is ignored with POSIX and DFA matching. Specifying partial 1486matching provokes an error return ("bad option value") from 1487<b>pcre2_substitute()</b>. 1488</P> 1489<br><b> 1490Testing substitute callouts 1491</b><br> 1492<P> 1493If the <b>substitute_callout</b> modifier is set, a substitution callout 1494function is set up. The <b>null_context</b> modifier must not be set, because 1495the address of the callout function is passed in a match context. When the 1496callout function is called (after each substitution), details of the the input 1497and output strings are output. For example: 1498<pre> 1499 /abc/g,replace=<$0>,substitute_callout 1500 abcdefabcpqr 1501 1(1) Old 0 3 "abc" New 0 5 "<abc>" 1502 2(1) Old 6 9 "abc" New 8 13 "<abc>" 1503 2: <abc>def<abc>pqr 1504</pre> 1505The first number on each callout line is the count of matches. The 1506parenthesized number is the number of pairs that are set in the ovector (that 1507is, one more than the number of capturing groups that were set). Then are 1508listed the offsets of the old substring, its contents, and the same for the 1509replacement. 1510</P> 1511<P> 1512By default, the substitution callout function returns zero, which accepts the 1513replacement and causes matching to continue if /g was used. Two further 1514modifiers can be used to test other return values. If <b>substitute_skip</b> is 1515set to a value greater than zero the callout function returns +1 for the match 1516of that number, and similarly <b>substitute_stop</b> returns -1. These cause the 1517replacement to be rejected, and -1 causes no further matching to take place. If 1518either of them are set, <b>substitute_callout</b> is assumed. For example: 1519<pre> 1520 /abc/g,replace=<$0>,substitute_skip=1 1521 abcdefabcpqr 1522 1(1) Old 0 3 "abc" New 0 5 "<abc> SKIPPED" 1523 2(1) Old 6 9 "abc" New 6 11 "<abc>" 1524 2: abcdef<abc>pqr 1525 abcdefabcpqr\=substitute_stop=1 1526 1(1) Old 0 3 "abc" New 0 5 "<abc> STOPPED" 1527 1: abcdefabcpqr 1528</pre> 1529If both are set for the same number, stop takes precedence. Only a single skip 1530or stop is supported, which is sufficient for testing that the feature works. 1531</P> 1532<br><b> 1533Setting the JIT stack size 1534</b><br> 1535<P> 1536The <b>jitstack</b> modifier provides a way of setting the maximum stack size 1537that is used by the just-in-time optimization code. It is ignored if JIT 1538optimization is not being used. The value is a number of kibibytes (units of 15391024 bytes). Setting zero reverts to the default of 32KiB. Providing a stack 1540that is larger than the default is necessary only for very complicated 1541patterns. If <b>jitstack</b> is set non-zero on a subject line it overrides any 1542value that was set on the pattern. 1543</P> 1544<br><b> 1545Setting heap, match, and depth limits 1546</b><br> 1547<P> 1548The <b>heap_limit</b>, <b>match_limit</b>, and <b>depth_limit</b> modifiers set 1549the appropriate limits in the match context. These values are ignored when the 1550<b>find_limits</b> modifier is specified. 1551</P> 1552<br><b> 1553Finding minimum limits 1554</b><br> 1555<P> 1556If the <b>find_limits</b> modifier is present on a subject line, <b>pcre2test</b> 1557calls the relevant matching function several times, setting different values in 1558the match context via <b>pcre2_set_heap_limit()</b>, 1559<b>pcre2_set_match_limit()</b>, or <b>pcre2_set_depth_limit()</b> until it finds 1560the minimum values for each parameter that allows the match to complete without 1561error. If JIT is being used, only the match limit is relevant. 1562</P> 1563<P> 1564When using this modifier, the pattern should not contain any limit settings 1565such as (*LIMIT_MATCH=...) within it. If such a setting is present and is 1566lower than the minimum matching value, the minimum value cannot be found 1567because <b>pcre2_set_match_limit()</b> etc. are only able to reduce the value of 1568an in-pattern limit; they cannot increase it. 1569</P> 1570<P> 1571For non-DFA matching, the minimum <i>depth_limit</i> number is a measure of how 1572much nested backtracking happens (that is, how deeply the pattern's tree is 1573searched). In the case of DFA matching, <i>depth_limit</i> controls the depth of 1574recursive calls of the internal function that is used for handling pattern 1575recursion, lookaround assertions, and atomic groups. 1576</P> 1577<P> 1578For non-DFA matching, the <i>match_limit</i> number is a measure of the amount 1579of backtracking that takes place, and learning the minimum value can be 1580instructive. For most simple matches, the number is quite small, but for 1581patterns with very large numbers of matching possibilities, it can become large 1582very quickly with increasing length of subject string. In the case of DFA 1583matching, <i>match_limit</i> controls the total number of calls, both recursive 1584and non-recursive, to the internal matching function, thus controlling the 1585overall amount of computing resource that is used. 1586</P> 1587<P> 1588For both kinds of matching, the <i>heap_limit</i> number, which is in kibibytes 1589(units of 1024 bytes), limits the amount of heap memory used for matching. A 1590value of zero disables the use of any heap memory; many simple pattern matches 1591can be done without using the heap, so zero is not an unreasonable setting. 1592</P> 1593<br><b> 1594Showing MARK names 1595</b><br> 1596<P> 1597The <b>mark</b> modifier causes the names from backtracking control verbs that 1598are returned from calls to <b>pcre2_match()</b> to be displayed. If a mark is 1599returned for a match, non-match, or partial match, <b>pcre2test</b> shows it. 1600For a match, it is on a line by itself, tagged with "MK:". Otherwise, it 1601is added to the non-match message. 1602</P> 1603<br><b> 1604Showing memory usage 1605</b><br> 1606<P> 1607The <b>memory</b> modifier causes <b>pcre2test</b> to log the sizes of all heap 1608memory allocation and freeing calls that occur during a call to 1609<b>pcre2_match()</b> or <b>pcre2_dfa_match()</b>. These occur only when a match 1610requires a bigger vector than the default for remembering backtracking points 1611(<b>pcre2_match()</b>) or for internal workspace (<b>pcre2_dfa_match()</b>). In 1612many cases there will be no heap memory used and therefore no additional 1613output. No heap memory is allocated during matching with JIT, so in that case 1614the <b>memory</b> modifier never has any effect. For this modifier to work, the 1615<b>null_context</b> modifier must not be set on both the pattern and the 1616subject, though it can be set on one or the other. 1617</P> 1618<br><b> 1619Setting a starting offset 1620</b><br> 1621<P> 1622The <b>offset</b> modifier sets an offset in the subject string at which 1623matching starts. Its value is a number of code units, not characters. 1624</P> 1625<br><b> 1626Setting an offset limit 1627</b><br> 1628<P> 1629The <b>offset_limit</b> modifier sets a limit for unanchored matches. If a match 1630cannot be found starting at or before this offset in the subject, a "no match" 1631return is given. The data value is a number of code units, not characters. When 1632this modifier is used, the <b>use_offset_limit</b> modifier must have been set 1633for the pattern; if not, an error is generated. 1634</P> 1635<br><b> 1636Setting the size of the output vector 1637</b><br> 1638<P> 1639The <b>ovector</b> modifier applies only to the subject line in which it 1640appears, though of course it can also be used to set a default in a 1641<b>#subject</b> command. It specifies the number of pairs of offsets that are 1642available for storing matching information. The default is 15. 1643</P> 1644<P> 1645A value of zero is useful when testing the POSIX API because it causes 1646<b>regexec()</b> to be called with a NULL capture vector. When not testing the 1647POSIX API, a value of zero is used to cause 1648<b>pcre2_match_data_create_from_pattern()</b> to be called, in order to create a 1649match block of exactly the right size for the pattern. (It is not possible to 1650create a match block with a zero-length ovector; there is always at least one 1651pair of offsets.) 1652</P> 1653<br><b> 1654Passing the subject as zero-terminated 1655</b><br> 1656<P> 1657By default, the subject string is passed to a native API matching function with 1658its correct length. In order to test the facility for passing a zero-terminated 1659string, the <b>zero_terminate</b> modifier is provided. It causes the length to 1660be passed as PCRE2_ZERO_TERMINATED. When matching via the POSIX interface, 1661this modifier is ignored, with a warning. 1662</P> 1663<P> 1664When testing <b>pcre2_substitute()</b>, this modifier also has the effect of 1665passing the replacement string as zero-terminated. 1666</P> 1667<br><b> 1668Passing a NULL context 1669</b><br> 1670<P> 1671Normally, <b>pcre2test</b> passes a context block to <b>pcre2_match()</b>, 1672<b>pcre2_dfa_match()</b>, <b>pcre2_jit_match()</b> or <b>pcre2_substitute()</b>. 1673If the <b>null_context</b> modifier is set, however, NULL is passed. This is for 1674testing that the matching and substitution functions behave correctly in this 1675case (they use default values). This modifier cannot be used with the 1676<b>find_limits</b> or <b>substitute_callout</b> modifiers. 1677</P> 1678<br><a name="SEC12" href="#TOC1">THE ALTERNATIVE MATCHING FUNCTION</a><br> 1679<P> 1680By default, <b>pcre2test</b> uses the standard PCRE2 matching function, 1681<b>pcre2_match()</b> to match each subject line. PCRE2 also supports an 1682alternative matching function, <b>pcre2_dfa_match()</b>, which operates in a 1683different way, and has some restrictions. The differences between the two 1684functions are described in the 1685<a href="pcre2matching.html"><b>pcre2matching</b></a> 1686documentation. 1687</P> 1688<P> 1689If the <b>dfa</b> modifier is set, the alternative matching function is used. 1690This function finds all possible matches at a given point in the subject. If, 1691however, the <b>dfa_shortest</b> modifier is set, processing stops after the 1692first match is found. This is always the shortest possible match. 1693</P> 1694<br><a name="SEC13" href="#TOC1">DEFAULT OUTPUT FROM pcre2test</a><br> 1695<P> 1696This section describes the output when the normal matching function, 1697<b>pcre2_match()</b>, is being used. 1698</P> 1699<P> 1700When a match succeeds, <b>pcre2test</b> outputs the list of captured substrings, 1701starting with number 0 for the string that matched the whole pattern. 1702Otherwise, it outputs "No match" when the return is PCRE2_ERROR_NOMATCH, or 1703"Partial match:" followed by the partially matching substring when the 1704return is PCRE2_ERROR_PARTIAL. (Note that this is the 1705entire substring that was inspected during the partial match; it may include 1706characters before the actual match start if a lookbehind assertion, \K, \b, 1707or \B was involved.) 1708</P> 1709<P> 1710For any other return, <b>pcre2test</b> outputs the PCRE2 negative error number 1711and a short descriptive phrase. If the error is a failed UTF string check, the 1712code unit offset of the start of the failing character is also output. Here is 1713an example of an interactive <b>pcre2test</b> run. 1714<pre> 1715 $ pcre2test 1716 PCRE2 version 10.22 2016-07-29 1717 1718 re> /^abc(\d+)/ 1719 data> abc123 1720 0: abc123 1721 1: 123 1722 data> xyz 1723 No match 1724</pre> 1725Unset capturing substrings that are not followed by one that is set are not 1726shown by <b>pcre2test</b> unless the <b>allcaptures</b> modifier is specified. In 1727the following example, there are two capturing substrings, but when the first 1728data line is matched, the second, unset substring is not shown. An "internal" 1729unset substring is shown as "<unset>", as for the second data line. 1730<pre> 1731 re> /(a)|(b)/ 1732 data> a 1733 0: a 1734 1: a 1735 data> b 1736 0: b 1737 1: <unset> 1738 2: b 1739</pre> 1740If the strings contain any non-printing characters, they are output as \xhh 1741escapes if the value is less than 256 and UTF mode is not set. Otherwise they 1742are output as \x{hh...} escapes. See below for the definition of non-printing 1743characters. If the <b>aftertext</b> modifier is set, the output for substring 17440 is followed by the the rest of the subject string, identified by "0+" like 1745this: 1746<pre> 1747 re> /cat/aftertext 1748 data> cataract 1749 0: cat 1750 0+ aract 1751</pre> 1752If global matching is requested, the results of successive matching attempts 1753are output in sequence, like this: 1754<pre> 1755 re> /\Bi(\w\w)/g 1756 data> Mississippi 1757 0: iss 1758 1: ss 1759 0: iss 1760 1: ss 1761 0: ipp 1762 1: pp 1763</pre> 1764"No match" is output only if the first match attempt fails. Here is an example 1765of a failure message (the offset 4 that is specified by the <b>offset</b> 1766modifier is past the end of the subject string): 1767<pre> 1768 re> /xyz/ 1769 data> xyz\=offset=4 1770 Error -24 (bad offset value) 1771</PRE> 1772</P> 1773<P> 1774Note that whereas patterns can be continued over several lines (a plain ">" 1775prompt is used for continuations), subject lines may not. However newlines can 1776be included in a subject by means of the \n escape (or \r, \r\n, etc., 1777depending on the newline sequence setting). 1778</P> 1779<br><a name="SEC14" href="#TOC1">OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION</a><br> 1780<P> 1781When the alternative matching function, <b>pcre2_dfa_match()</b>, is used, the 1782output consists of a list of all the matches that start at the first point in 1783the subject where there is at least one match. For example: 1784<pre> 1785 re> /(tang|tangerine|tan)/ 1786 data> yellow tangerine\=dfa 1787 0: tangerine 1788 1: tang 1789 2: tan 1790</pre> 1791Using the normal matching function on this data finds only "tang". The 1792longest matching string is always given first (and numbered zero). After a 1793PCRE2_ERROR_PARTIAL return, the output is "Partial match:", followed by the 1794partially matching substring. Note that this is the entire substring that was 1795inspected during the partial match; it may include characters before the actual 1796match start if a lookbehind assertion, \b, or \B was involved. (\K is not 1797supported for DFA matching.) 1798</P> 1799<P> 1800If global matching is requested, the search for further matches resumes 1801at the end of the longest match. For example: 1802<pre> 1803 re> /(tang|tangerine|tan)/g 1804 data> yellow tangerine and tangy sultana\=dfa 1805 0: tangerine 1806 1: tang 1807 2: tan 1808 0: tang 1809 1: tan 1810 0: tan 1811</pre> 1812The alternative matching function does not support substring capture, so the 1813modifiers that are concerned with captured substrings are not relevant. 1814</P> 1815<br><a name="SEC15" href="#TOC1">RESTARTING AFTER A PARTIAL MATCH</a><br> 1816<P> 1817When the alternative matching function has given the PCRE2_ERROR_PARTIAL 1818return, indicating that the subject partially matched the pattern, you can 1819restart the match with additional subject data by means of the 1820<b>dfa_restart</b> modifier. For example: 1821<pre> 1822 re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ 1823 data> 23ja\=ps,dfa 1824 Partial match: 23ja 1825 data> n05\=dfa,dfa_restart 1826 0: n05 1827</pre> 1828For further information about partial matching, see the 1829<a href="pcre2partial.html"><b>pcre2partial</b></a> 1830documentation. 1831<a name="callouts"></a></P> 1832<br><a name="SEC16" href="#TOC1">CALLOUTS</a><br> 1833<P> 1834If the pattern contains any callout requests, <b>pcre2test</b>'s callout 1835function is called during matching unless <b>callout_none</b> is specified. This 1836works with both matching functions, and with JIT, though there are some 1837differences in behaviour. The output for callouts with numerical arguments and 1838those with string arguments is slightly different. 1839</P> 1840<br><b> 1841Callouts with numerical arguments 1842</b><br> 1843<P> 1844By default, the callout function displays the callout number, the start and 1845current positions in the subject text at the callout time, and the next pattern 1846item to be tested. For example: 1847<pre> 1848 --->pqrabcdef 1849 0 ^ ^ \d 1850</pre> 1851This output indicates that callout number 0 occurred for a match attempt 1852starting at the fourth character of the subject string, when the pointer was at 1853the seventh character, and when the next pattern item was \d. Just 1854one circumflex is output if the start and current positions are the same, or if 1855the current position precedes the start position, which can happen if the 1856callout is in a lookbehind assertion. 1857</P> 1858<P> 1859Callouts numbered 255 are assumed to be automatic callouts, inserted as a 1860result of the <b>auto_callout</b> pattern modifier. In this case, instead of 1861showing the callout number, the offset in the pattern, preceded by a plus, is 1862output. For example: 1863<pre> 1864 re> /\d?[A-E]\*/auto_callout 1865 data> E* 1866 --->E* 1867 +0 ^ \d? 1868 +3 ^ [A-E] 1869 +8 ^^ \* 1870 +10 ^ ^ 1871 0: E* 1872</pre> 1873If a pattern contains (*MARK) items, an additional line is output whenever 1874a change of latest mark is passed to the callout function. For example: 1875<pre> 1876 re> /a(*MARK:X)bc/auto_callout 1877 data> abc 1878 --->abc 1879 +0 ^ a 1880 +1 ^^ (*MARK:X) 1881 +10 ^^ b 1882 Latest Mark: X 1883 +11 ^ ^ c 1884 +12 ^ ^ 1885 0: abc 1886</pre> 1887The mark changes between matching "a" and "b", but stays the same for the rest 1888of the match, so nothing more is output. If, as a result of backtracking, the 1889mark reverts to being unset, the text "<unset>" is output. 1890</P> 1891<br><b> 1892Callouts with string arguments 1893</b><br> 1894<P> 1895The output for a callout with a string argument is similar, except that instead 1896of outputting a callout number before the position indicators, the callout 1897string and its offset in the pattern string are output before the reflection of 1898the subject string, and the subject string is reflected for each callout. For 1899example: 1900<pre> 1901 re> /^ab(?C'first')cd(?C"second")ef/ 1902 data> abcdefg 1903 Callout (7): 'first' 1904 --->abcdefg 1905 ^ ^ c 1906 Callout (20): "second" 1907 --->abcdefg 1908 ^ ^ e 1909 0: abcdef 1910 1911</PRE> 1912</P> 1913<br><b> 1914Callout modifiers 1915</b><br> 1916<P> 1917The callout function in <b>pcre2test</b> returns zero (carry on matching) by 1918default, but you can use a <b>callout_fail</b> modifier in a subject line to 1919change this and other parameters of the callout (see below). 1920</P> 1921<P> 1922If the <b>callout_capture</b> modifier is set, the current captured groups are 1923output when a callout occurs. This is useful only for non-DFA matching, as 1924<b>pcre2_dfa_match()</b> does not support capturing, so no captures are ever 1925shown. 1926</P> 1927<P> 1928The normal callout output, showing the callout number or pattern offset (as 1929described above) is suppressed if the <b>callout_no_where</b> modifier is set. 1930</P> 1931<P> 1932When using the interpretive matching function <b>pcre2_match()</b> without JIT, 1933setting the <b>callout_extra</b> modifier causes additional output from 1934<b>pcre2test</b>'s callout function to be generated. For the first callout in a 1935match attempt at a new starting position in the subject, "New match attempt" is 1936output. If there has been a backtrack since the last callout (or start of 1937matching if this is the first callout), "Backtrack" is output, followed by "No 1938other matching paths" if the backtrack ended the previous match attempt. For 1939example: 1940<pre> 1941 re> /(a+)b/auto_callout,no_start_optimize,no_auto_possess 1942 data> aac\=callout_extra 1943 New match attempt 1944 --->aac 1945 +0 ^ ( 1946 +1 ^ a+ 1947 +3 ^ ^ ) 1948 +4 ^ ^ b 1949 Backtrack 1950 --->aac 1951 +3 ^^ ) 1952 +4 ^^ b 1953 Backtrack 1954 No other matching paths 1955 New match attempt 1956 --->aac 1957 +0 ^ ( 1958 +1 ^ a+ 1959 +3 ^^ ) 1960 +4 ^^ b 1961 Backtrack 1962 No other matching paths 1963 New match attempt 1964 --->aac 1965 +0 ^ ( 1966 +1 ^ a+ 1967 Backtrack 1968 No other matching paths 1969 New match attempt 1970 --->aac 1971 +0 ^ ( 1972 +1 ^ a+ 1973 No match 1974</pre> 1975Notice that various optimizations must be turned off if you want all possible 1976matching paths to be scanned. If <b>no_start_optimize</b> is not used, there is 1977an immediate "no match", without any callouts, because the starting 1978optimization fails to find "b" in the subject, which it knows must be present 1979for any match. If <b>no_auto_possess</b> is not used, the "a+" item is turned 1980into "a++", which reduces the number of backtracks. 1981</P> 1982<P> 1983The <b>callout_extra</b> modifier has no effect if used with the DFA matching 1984function, or with JIT. 1985</P> 1986<br><b> 1987Return values from callouts 1988</b><br> 1989<P> 1990The default return from the callout function is zero, which allows matching to 1991continue. The <b>callout_fail</b> modifier can be given one or two numbers. If 1992there is only one number, 1 is returned instead of 0 (causing matching to 1993backtrack) when a callout of that number is reached. If two numbers (<n>:<m>) 1994are given, 1 is returned when callout <n> is reached and there have been at 1995least <m> callouts. The <b>callout_error</b> modifier is similar, except that 1996PCRE2_ERROR_CALLOUT is returned, causing the entire matching process to be 1997aborted. If both these modifiers are set for the same callout number, 1998<b>callout_error</b> takes precedence. Note that callouts with string arguments 1999are always given the number zero. 2000</P> 2001<P> 2002The <b>callout_data</b> modifier can be given an unsigned or a negative number. 2003This is set as the "user data" that is passed to the matching function, and 2004passed back when the callout function is invoked. Any value other than zero is 2005used as a return from <b>pcre2test</b>'s callout function. 2006</P> 2007<P> 2008Inserting callouts can be helpful when using <b>pcre2test</b> to check 2009complicated regular expressions. For further information about callouts, see 2010the 2011<a href="pcre2callout.html"><b>pcre2callout</b></a> 2012documentation. 2013</P> 2014<br><a name="SEC17" href="#TOC1">NON-PRINTING CHARACTERS</a><br> 2015<P> 2016When <b>pcre2test</b> is outputting text in the compiled version of a pattern, 2017bytes other than 32-126 are always treated as non-printing characters and are 2018therefore shown as hex escapes. 2019</P> 2020<P> 2021When <b>pcre2test</b> is outputting text that is a matched part of a subject 2022string, it behaves in the same way, unless a different locale has been set for 2023the pattern (using the <b>locale</b> modifier). In this case, the 2024<b>isprint()</b> function is used to distinguish printing and non-printing 2025characters. 2026<a name="saverestore"></a></P> 2027<br><a name="SEC18" href="#TOC1">SAVING AND RESTORING COMPILED PATTERNS</a><br> 2028<P> 2029It is possible to save compiled patterns on disc or elsewhere, and reload them 2030later, subject to a number of restrictions. JIT data cannot be saved. The host 2031on which the patterns are reloaded must be running the same version of PCRE2, 2032with the same code unit width, and must also have the same endianness, pointer 2033width and PCRE2_SIZE type. Before compiled patterns can be saved they must be 2034serialized, that is, converted to a stream of bytes. A single byte stream may 2035contain any number of compiled patterns, but they must all use the same 2036character tables. A single copy of the tables is included in the byte stream 2037(its size is 1088 bytes). 2038</P> 2039<P> 2040The functions whose names begin with <b>pcre2_serialize_</b> are used 2041for serializing and de-serializing. They are described in the 2042<a href="pcre2serialize.html"><b>pcre2serialize</b></a> 2043documentation. In this section we describe the features of <b>pcre2test</b> that 2044can be used to test these functions. 2045</P> 2046<P> 2047Note that "serialization" in PCRE2 does not convert compiled patterns to an 2048abstract format like Java or .NET. It just makes a reloadable byte code stream. 2049Hence the restrictions on reloading mentioned above. 2050</P> 2051<P> 2052In <b>pcre2test</b>, when a pattern with <b>push</b> modifier is successfully 2053compiled, it is pushed onto a stack of compiled patterns, and <b>pcre2test</b> 2054expects the next line to contain a new pattern (or command) instead of a 2055subject line. By contrast, the <b>pushcopy</b> modifier causes a copy of the 2056compiled pattern to be stacked, leaving the original available for immediate 2057matching. By using <b>push</b> and/or <b>pushcopy</b>, a number of patterns can 2058be compiled and retained. These modifiers are incompatible with <b>posix</b>, 2059and control modifiers that act at match time are ignored (with a message) for 2060the stacked patterns. The <b>jitverify</b> modifier applies only at compile 2061time. 2062</P> 2063<P> 2064The command 2065<pre> 2066 #save <filename> 2067</pre> 2068causes all the stacked patterns to be serialized and the result written to the 2069named file. Afterwards, all the stacked patterns are freed. The command 2070<pre> 2071 #load <filename> 2072</pre> 2073reads the data in the file, and then arranges for it to be de-serialized, with 2074the resulting compiled patterns added to the pattern stack. The pattern on the 2075top of the stack can be retrieved by the #pop command, which must be followed 2076by lines of subjects that are to be matched with the pattern, terminated as 2077usual by an empty line or end of file. This command may be followed by a 2078modifier list containing only 2079<a href="#controlmodifiers">control modifiers</a> 2080that act after a pattern has been compiled. In particular, <b>hex</b>, 2081<b>posix</b>, <b>posix_nosub</b>, <b>push</b>, and <b>pushcopy</b> are not allowed, 2082nor are any 2083<a href="#optionmodifiers">option-setting modifiers.</a> 2084The JIT modifiers are, however permitted. Here is an example that saves and 2085reloads two patterns. 2086<pre> 2087 /abc/push 2088 /xyz/push 2089 #save tempfile 2090 #load tempfile 2091 #pop info 2092 xyz 2093 2094 #pop jit,bincode 2095 abc 2096</pre> 2097If <b>jitverify</b> is used with #pop, it does not automatically imply 2098<b>jit</b>, which is different behaviour from when it is used on a pattern. 2099</P> 2100<P> 2101The #popcopy command is analagous to the <b>pushcopy</b> modifier in that it 2102makes current a copy of the topmost stack pattern, leaving the original still 2103on the stack. 2104</P> 2105<br><a name="SEC19" href="#TOC1">SEE ALSO</a><br> 2106<P> 2107<b>pcre2</b>(3), <b>pcre2api</b>(3), <b>pcre2callout</b>(3), 2108<b>pcre2jit</b>, <b>pcre2matching</b>(3), <b>pcre2partial</b>(d), 2109<b>pcre2pattern</b>(3), <b>pcre2serialize</b>(3). 2110</P> 2111<br><a name="SEC20" href="#TOC1">AUTHOR</a><br> 2112<P> 2113Philip Hazel 2114<br> 2115University Computing Service 2116<br> 2117Cambridge, England. 2118<br> 2119</P> 2120<br><a name="SEC21" href="#TOC1">REVISION</a><br> 2121<P> 2122Last updated: 14 September 2020 2123<br> 2124Copyright © 1997-2020 University of Cambridge. 2125<br> 2126<p> 2127Return to the <a href="index.html">PCRE2 index page</a>. 2128</p> 2129