Python - 下一个方法不能与生成器一起正常工作

Question

我在 python 中创建了一个类，它将代码流拆分为令牌并逐个令牌推进令牌以与它们一起使用

import re

class Tokenizer:

    def __init__(self, input_file):

        self.in_file = input_file

        self.tokens = []

        self.current_token = None

        self.next_token = None

        self.line = 1

    def split_tokens(self):

        ''' Create a list with all the tokens of the input file '''

        self.tokens = re.findall("\w+|[{}()\[\].;,+\-*/&|<>=~\n]", self.in_file)

    def __iter__(self):

        for token in self.tokens:

            if token != '\n':

                yield token 

            else:

                self.line += 1

    def advance(self):

        self.current_token = self.next_token

        self.next_token = next(self.__iter__())

初始化后：

text = 'constructor SquareGame03 new()\n\

       {let square=square;\n\

       let direction=direction;\n\

       return square;\n\

       }'

t = Tokenizer(text)

t.split_tokens()

t.advance()

如果我打印令牌，它似乎有效

print(t.current_token, t.next_token)

None constructor

但是 advance 方法的每一次调用都会给出这些结果：

t.advance()

print(t.current_token, t.next_token)

constructor constructor

t.advance()

print(t.current_token, t.next_token)

constructor constructor

所以它没有进步，我不明白为什么。

Answer 1

在这种情况下，.__iter__它被实现为生成器函数（而不是生成器迭代器），它返回一个生成器迭代器。每次Tokenizer.advance被调用时，都会创建一个新的 生成器迭代器并由 返回.__iter__。相反，迭代器应该Tokenizer在初始化阶段由对象存储以供所有后续使用。例如：import reclass Tokenizer:&nbsp; &nbsp; def __init__(self, input_file):&nbsp; &nbsp; &nbsp; &nbsp; self.in_file = input_file&nbsp; &nbsp; &nbsp; &nbsp; self.tokens = []&nbsp; &nbsp; &nbsp; &nbsp; self.current_token = None&nbsp; &nbsp; &nbsp; &nbsp; self.next_token = None&nbsp; &nbsp; &nbsp; &nbsp; self.line = 1&nbsp; &nbsp; def split_tokens(self):&nbsp; &nbsp; &nbsp; &nbsp; ''' Create a list with all the tokens of the input file '''&nbsp; &nbsp; &nbsp; &nbsp; self.tokens = re.findall("\w+|[{}()\[\].;,+\-*/&|<>=~\n]", self.in_file)&nbsp; &nbsp; &nbsp; &nbsp; self.iterator = self.__iter__()&nbsp; &nbsp; def __iter__(self):&nbsp; &nbsp; &nbsp; &nbsp; for token in self.tokens:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; if token != '\n':&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; yield token&nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; else:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; self.line += 1&nbsp; &nbsp; def advance(self):&nbsp; &nbsp; &nbsp; &nbsp; self.current_token = self.next_token&nbsp; &nbsp; &nbsp; &nbsp; self.next_token = next(self.iterator)另一个可以解释的最小例子：def fib():&nbsp; &nbsp; a = 0&nbsp; &nbsp; b = 1&nbsp; &nbsp; while True:&nbsp; &nbsp; &nbsp; &nbsp; yield b&nbsp; &nbsp; &nbsp; &nbsp; a, b = b, a + b# 1, 1, 2, ...fibs = fib()next(fibs)next(fibs)next(fibs)# 1, 1, 1, ...next(fib())next(fib())next(fib())顺便说一句，我看不出混合使用.__iter__魔术方法和单独.advance方法的原因。它可能会引起一些混乱。