1:mod:`token` --- Constants used with Python parse trees
2=======================================================
3
4.. module:: token
5   :synopsis: Constants representing terminal nodes of the parse tree.
6.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
7
8**Source code:** :source:`Lib/token.py`
9
10--------------
11
12This module provides constants which represent the numeric values of leaf nodes
13of the parse tree (terminal tokens).  Refer to the file :file:`Grammar/Grammar`
14in the Python distribution for the definitions of the names in the context of
15the language grammar.  The specific numeric values which the names map to may
16change between Python versions.
17
18The module also provides a mapping from numeric codes to names and some
19functions.  The functions mirror definitions in the Python C header files.
20
21
22.. data:: tok_name
23
24   Dictionary mapping the numeric values of the constants defined in this module
25   back to name strings, allowing more human-readable representation of parse trees
26   to be generated.
27
28
29.. function:: ISTERMINAL(x)
30
31   Return true for terminal token values.
32
33
34.. function:: ISNONTERMINAL(x)
35
36   Return true for non-terminal token values.
37
38
39.. function:: ISEOF(x)
40
41   Return true if *x* is the marker indicating the end of input.
42
43
44The token constants are:
45
46.. data:: ENDMARKER
47          NAME
48          NUMBER
49          STRING
50          NEWLINE
51          INDENT
52          DEDENT
53          LPAR
54          RPAR
55          LSQB
56          RSQB
57          COLON
58          COMMA
59          SEMI
60          PLUS
61          MINUS
62          STAR
63          SLASH
64          VBAR
65          AMPER
66          LESS
67          GREATER
68          EQUAL
69          DOT
70          PERCENT
71          BACKQUOTE
72          LBRACE
73          RBRACE
74          EQEQUAL
75          NOTEQUAL
76          LESSEQUAL
77          GREATEREQUAL
78          TILDE
79          CIRCUMFLEX
80          LEFTSHIFT
81          RIGHTSHIFT
82          DOUBLESTAR
83          PLUSEQUAL
84          MINEQUAL
85          STAREQUAL
86          SLASHEQUAL
87          PERCENTEQUAL
88          AMPEREQUAL
89          VBAREQUAL
90          CIRCUMFLEXEQUAL
91          LEFTSHIFTEQUAL
92          RIGHTSHIFTEQUAL
93          DOUBLESTAREQUAL
94          DOUBLESLASH
95          DOUBLESLASHEQUAL
96          AT
97          OP
98          ERRORTOKEN
99          N_TOKENS
100          NT_OFFSET
101
102
103.. seealso::
104
105   Module :mod:`parser`
106      The second example for the :mod:`parser` module shows how to use the
107      :mod:`symbol` module.
108
109