为什么我收到 RuntimeError：生成器引发了 StopIteration？以及如何解决？

Question

我正在制作存储在列表 docToken 中的令牌的 Bigrams。

print(docToken[520])

输出：['sleepy', 'account', 'just', 'man', 'tired', 'twitter', 'case', 'romney', 'candidate', 'looks']

list(nltk.bigrams(docToken[520]))

输出：[('sleepy', 'account'), ('account', 'just'), ('just', 'man'), ('man', 'tired'), ('tired', 'twitter '), ('twitter', 'case'), ('case', 'romney'), ('romney', 'candidate'), ('candidate', 'looks')]

当我nltk.bigrams(docToken[i])在循环中使用时，我在范围> = 1000上遇到以下错误：

bigram=[]

for i in range(5000):

    ls=list(nltk.bigrams(docToken[i]))

    for j in ls:

        bigram.append(list(j))

当第一个循环中的 range(500) 时它工作得很好，但是当 Range 为 1000 或更大时，它给了我以下错误：

StopIteration                             Traceback (most recent call last) 

~\Anaconda3\lib\site-packages\nltk\util.py in ngrams(sequence, n, pad_left, 

  pad_right, left_pad_symbol, right_pad_symbol)

        467     while n > 1:

    --> 468         history.append(next(sequence))

        469         n -= 1

StopIteration: 

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)

<ipython-input-76-8982951528bd> in <module>()

      1 bigram=[]

      2 for i in range(5000):

----> 3     ls=list(nltk.bigrams(docToken[i]))

      4     for j in ls:

      5         bigram.append(list(j))

~\Anaconda3\lib\site-packages\nltk\util.py in bigrams(sequence, **kwargs)

    489     """

    490 

--> 491     for item in ngrams(sequence, 2, **kwargs):

    492         yield item

    493 

RuntimeError: generator raised StopIteration

Answer 1

我也面临同样的错误。一个可能的原因可能是其中一个元素docToken是一个空列表。例如，当i=2第二个元素是空列表时，以下代码会引发相同的错误。from nltk import bigramsdocToken= [['the', 'wildlings', 'are', 'dead'], [], ['do', 'the', 'dead', 'frighten', 'you', 'ser', 'waymar']]for i in range(3):&nbsp; &nbsp; print (i)&nbsp; &nbsp; print (list(nltk.bigrams(docToken[i])))输出：0[('the', 'wildlings'), ('wildlings', 'are'), ('are', 'dead')]1---------------------------------------------------------------------------StopIteration&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;Traceback (most recent call last)~\AppData\Local\Continuum\anaconda3\lib\site-packages\nltk\util.py in ngrams(sequence, n, pad_left, pad_right, left_pad_symbol, right_pad_symbol)&nbsp; &nbsp; 467&nbsp; &nbsp; &nbsp;while n > 1:--> 468&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;history.append(next(sequence))&nbsp; &nbsp; 469&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;n -= 1StopIteration:&nbsp;The above exception was the direct cause of the following exception:RuntimeError&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Traceback (most recent call last)<ipython-input-58-91f35cae32ed> in <module>&nbsp; &nbsp; &nbsp; 2 for i in range(3):&nbsp; &nbsp; &nbsp; 3&nbsp; &nbsp; &nbsp;print (i)----> 4&nbsp; &nbsp; &nbsp;list(nltk.bigrams(docToken[i]))~\AppData\Local\Continuum\anaconda3\lib\site-packages\nltk\util.py in bigrams(sequence, **kwargs)&nbsp; &nbsp; 489&nbsp; &nbsp; &nbsp;"""&nbsp; &nbsp; 490&nbsp;--> 491&nbsp; &nbsp; &nbsp;for item in ngrams(sequence, 2, **kwargs):&nbsp; &nbsp; 492&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;yield item&nbsp; &nbsp; 493&nbsp;RuntimeError: generator raised StopIteration您可以从中过滤出空列表docToken，然后创建二元组：docToken= [['the', 'wildlings', 'are', 'dead'], [], ['do', 'the', 'dead', 'frighten', 'you', 'ser', 'waymar']]docToken = [x for x in docToken if x]bigram = []for i in range(len(docToken)):&nbsp; &nbsp; bigram.append(["_".join(w) for w in&nbsp; bigrams(docToken[i])])bigram输出：[['the_wildlings', 'wildlings_are', 'are_dead'],&nbsp;['do_the',&nbsp; 'the_dead',&nbsp; 'dead_frighten',&nbsp; 'frighten_you',&nbsp; 'you_ser',&nbsp; 'ser_waymar']]另一个可能的原因可能是您nltk在 python 3.7 中使用了 3.3。请使用 nltk 3.4，它是第一个支持 Python 3.7 的版本，您的问题应该在这个版本中得到解决。请看这里。

Answer 2

我通过从 3.3 -> 3.4 升级 nltk 来解决这个问题做简单：pip&nbsp;install&nbsp;nltk==3.4

Answer 3

我无法解决此错误。不知道为什么nltk.bigrams(docToken[i])会生成这个，但我能够使用以下代码创建二元组。bigram={}for i in range(size):&nbsp; &nbsp; ls=[]&nbsp; &nbsp; for j in range(len(docToken[i])-1):&nbsp; &nbsp; &nbsp; &nbsp; for k in range(j,len(docToken[i])-1):&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ls.append([docToken[i][j],docToken[i][k+1]])&nbsp; &nbsp; bigram[i]=ls