Searched refs:generate_tokens (Results 1 – 3 of 3) sorted by relevance
97 tokgen = generate_tokens(source)144 def generate_tokens(self, text): member in CachedTokenizer149 self.last_tokens = list(tokenize.generate_tokens(readline))153 generate_tokens = CachedTokenizer().generate_tokens variable
17 from coverage.phystokens import compile_unicode, generate_tokens110 tokgen = generate_tokens(self.text)
66 tokenize_tokens = tokenize.generate_tokens(f.readline)