1:mod:`ast` --- Abstract Syntax Trees 2==================================== 3 4.. module:: ast 5 :synopsis: Abstract Syntax Tree classes and manipulation. 6 7.. sectionauthor:: Martin v. Löwis <martin@v.loewis.de> 8.. sectionauthor:: Georg Brandl <georg@python.org> 9 10**Source code:** :source:`Lib/ast.py` 11 12-------------- 13 14The :mod:`ast` module helps Python applications to process trees of the Python 15abstract syntax grammar. The abstract syntax itself might change with each 16Python release; this module helps to find out programmatically what the current 17grammar looks like. 18 19An abstract syntax tree can be generated by passing :data:`ast.PyCF_ONLY_AST` as 20a flag to the :func:`compile` built-in function, or using the :func:`parse` 21helper provided in this module. The result will be a tree of objects whose 22classes all inherit from :class:`ast.AST`. An abstract syntax tree can be 23compiled into a Python code object using the built-in :func:`compile` function. 24 25 26Node classes 27------------ 28 29.. class:: AST 30 31 This is the base of all AST node classes. The actual node classes are 32 derived from the :file:`Parser/Python.asdl` file, which is reproduced 33 :ref:`below <abstract-grammar>`. They are defined in the :mod:`_ast` C 34 module and re-exported in :mod:`ast`. 35 36 There is one class defined for each left-hand side symbol in the abstract 37 grammar (for example, :class:`ast.stmt` or :class:`ast.expr`). In addition, 38 there is one class defined for each constructor on the right-hand side; these 39 classes inherit from the classes for the left-hand side trees. For example, 40 :class:`ast.BinOp` inherits from :class:`ast.expr`. For production rules 41 with alternatives (aka "sums"), the left-hand side class is abstract: only 42 instances of specific constructor nodes are ever created. 43 44 .. attribute:: _fields 45 46 Each concrete class has an attribute :attr:`_fields` which gives the names 47 of all child nodes. 48 49 Each instance of a concrete class has one attribute for each child node, 50 of the type as defined in the grammar. For example, :class:`ast.BinOp` 51 instances have an attribute :attr:`left` of type :class:`ast.expr`. 52 53 If these attributes are marked as optional in the grammar (using a 54 question mark), the value might be ``None``. If the attributes can have 55 zero-or-more values (marked with an asterisk), the values are represented 56 as Python lists. All possible attributes must be present and have valid 57 values when compiling an AST with :func:`compile`. 58 59 .. attribute:: lineno 60 col_offset 61 62 Instances of :class:`ast.expr` and :class:`ast.stmt` subclasses have 63 :attr:`lineno` and :attr:`col_offset` attributes. The :attr:`lineno` is 64 the line number of source text (1-indexed so the first line is line 1) and 65 the :attr:`col_offset` is the UTF-8 byte offset of the first token that 66 generated the node. The UTF-8 offset is recorded because the parser uses 67 UTF-8 internally. 68 69 The constructor of a class :class:`ast.T` parses its arguments as follows: 70 71 * If there are positional arguments, there must be as many as there are items 72 in :attr:`T._fields`; they will be assigned as attributes of these names. 73 * If there are keyword arguments, they will set the attributes of the same 74 names to the given values. 75 76 For example, to create and populate an :class:`ast.UnaryOp` node, you could 77 use :: 78 79 node = ast.UnaryOp() 80 node.op = ast.USub() 81 node.operand = ast.Num() 82 node.operand.n = 5 83 node.operand.lineno = 0 84 node.operand.col_offset = 0 85 node.lineno = 0 86 node.col_offset = 0 87 88 or the more compact :: 89 90 node = ast.UnaryOp(ast.USub(), ast.Num(5, lineno=0, col_offset=0), 91 lineno=0, col_offset=0) 92 93 94.. _abstract-grammar: 95 96Abstract Grammar 97---------------- 98 99The abstract grammar is currently defined as follows: 100 101.. literalinclude:: ../../Parser/Python.asdl 102 :language: none 103 104 105:mod:`ast` Helpers 106------------------ 107 108Apart from the node classes, the :mod:`ast` module defines these utility functions 109and classes for traversing abstract syntax trees: 110 111.. function:: parse(source, filename='<unknown>', mode='exec') 112 113 Parse the source into an AST node. Equivalent to ``compile(source, 114 filename, mode, ast.PyCF_ONLY_AST)``. 115 116 117.. function:: literal_eval(node_or_string) 118 119 Safely evaluate an expression node or a string containing a Python literal or 120 container display. The string or node provided may only consist of the 121 following Python literal structures: strings, bytes, numbers, tuples, lists, 122 dicts, sets, booleans, and ``None``. 123 124 This can be used for safely evaluating strings containing Python values from 125 untrusted sources without the need to parse the values oneself. It is not 126 capable of evaluating arbitrarily complex expressions, for example involving 127 operators or indexing. 128 129 .. versionchanged:: 3.2 130 Now allows bytes and set literals. 131 132 133.. function:: get_docstring(node, clean=True) 134 135 Return the docstring of the given *node* (which must be a 136 :class:`FunctionDef`, :class:`ClassDef` or :class:`Module` node), or ``None`` 137 if it has no docstring. If *clean* is true, clean up the docstring's 138 indentation with :func:`inspect.cleandoc`. 139 140 141.. function:: fix_missing_locations(node) 142 143 When you compile a node tree with :func:`compile`, the compiler expects 144 :attr:`lineno` and :attr:`col_offset` attributes for every node that supports 145 them. This is rather tedious to fill in for generated nodes, so this helper 146 adds these attributes recursively where not already set, by setting them to 147 the values of the parent node. It works recursively starting at *node*. 148 149 150.. function:: increment_lineno(node, n=1) 151 152 Increment the line number of each node in the tree starting at *node* by *n*. 153 This is useful to "move code" to a different location in a file. 154 155 156.. function:: copy_location(new_node, old_node) 157 158 Copy source location (:attr:`lineno` and :attr:`col_offset`) from *old_node* 159 to *new_node* if possible, and return *new_node*. 160 161 162.. function:: iter_fields(node) 163 164 Yield a tuple of ``(fieldname, value)`` for each field in ``node._fields`` 165 that is present on *node*. 166 167 168.. function:: iter_child_nodes(node) 169 170 Yield all direct child nodes of *node*, that is, all fields that are nodes 171 and all items of fields that are lists of nodes. 172 173 174.. function:: walk(node) 175 176 Recursively yield all descendant nodes in the tree starting at *node* 177 (including *node* itself), in no specified order. This is useful if you only 178 want to modify nodes in place and don't care about the context. 179 180 181.. class:: NodeVisitor() 182 183 A node visitor base class that walks the abstract syntax tree and calls a 184 visitor function for every node found. This function may return a value 185 which is forwarded by the :meth:`visit` method. 186 187 This class is meant to be subclassed, with the subclass adding visitor 188 methods. 189 190 .. method:: visit(node) 191 192 Visit a node. The default implementation calls the method called 193 :samp:`self.visit_{classname}` where *classname* is the name of the node 194 class, or :meth:`generic_visit` if that method doesn't exist. 195 196 .. method:: generic_visit(node) 197 198 This visitor calls :meth:`visit` on all children of the node. 199 200 Note that child nodes of nodes that have a custom visitor method won't be 201 visited unless the visitor calls :meth:`generic_visit` or visits them 202 itself. 203 204 Don't use the :class:`NodeVisitor` if you want to apply changes to nodes 205 during traversal. For this a special visitor exists 206 (:class:`NodeTransformer`) that allows modifications. 207 208 209.. class:: NodeTransformer() 210 211 A :class:`NodeVisitor` subclass that walks the abstract syntax tree and 212 allows modification of nodes. 213 214 The :class:`NodeTransformer` will walk the AST and use the return value of 215 the visitor methods to replace or remove the old node. If the return value 216 of the visitor method is ``None``, the node will be removed from its 217 location, otherwise it is replaced with the return value. The return value 218 may be the original node in which case no replacement takes place. 219 220 Here is an example transformer that rewrites all occurrences of name lookups 221 (``foo``) to ``data['foo']``:: 222 223 class RewriteName(NodeTransformer): 224 225 def visit_Name(self, node): 226 return copy_location(Subscript( 227 value=Name(id='data', ctx=Load()), 228 slice=Index(value=Str(s=node.id)), 229 ctx=node.ctx 230 ), node) 231 232 Keep in mind that if the node you're operating on has child nodes you must 233 either transform the child nodes yourself or call the :meth:`generic_visit` 234 method for the node first. 235 236 For nodes that were part of a collection of statements (that applies to all 237 statement nodes), the visitor may also return a list of nodes rather than 238 just a single node. 239 240 Usually you use the transformer like this:: 241 242 node = YourTransformer().visit(node) 243 244 245.. function:: dump(node, annotate_fields=True, include_attributes=False) 246 247 Return a formatted dump of the tree in *node*. This is mainly useful for 248 debugging purposes. The returned string will show the names and the values 249 for fields. This makes the code impossible to evaluate, so if evaluation is 250 wanted *annotate_fields* must be set to ``False``. Attributes such as line 251 numbers and column offsets are not dumped by default. If this is wanted, 252 *include_attributes* can be set to ``True``. 253 254.. seealso:: 255 256 `Green Tree Snakes <https://greentreesnakes.readthedocs.org/>`_, an external documentation resource, has good 257 details on working with Python ASTs. 258