加载 word2vec 模型时出现数组重塑错误

Question

我有以下代码：

from gensim.models import Word2Vec

model = Word2Vec.load('model2')

X = model[model.wv.vocab]

这段代码适用于我的一台机器，但不适用于另一台机器。模型文件是一样的。这是怎么回事？我收到的错误消息如下：

  File "/home/ec2-user/miniconda3/envs/word2vec/lib/python3.7/site-packages/gensim/models/word2vec.py", line 1330, in load

    model = super(Word2Vec, cls).load(*args, **kwargs)

  File "/home/ec2-user/miniconda3/envs/word2vec/lib/python3.7/site-packages/gensim/models/base_any2vec.py", line 1244, in load

    model = super(BaseWordEmbeddingsModel, cls).load(*args, **kwargs)

  File "/home/ec2-user/miniconda3/envs/word2vec/lib/python3.7/site-packages/gensim/models/base_any2vec.py", line 603, in load

    return super(BaseAny2VecModel, cls).load(fname_or_handle, **kwargs)

  File "/home/ec2-user/miniconda3/envs/word2vec/lib/python3.7/site-packages/gensim/utils.py", line 427, in load

    obj._load_specials(fname, mmap, compress, subname)

  File "/home/ec2-user/miniconda3/envs/word2vec/lib/python3.7/site-packages/gensim/utils.py", line 458, in _load_specials

    getattr(self, attrib)._load_specials(cfname, mmap, compress, subname)

  File "/home/ec2-user/miniconda3/envs/word2vec/lib/python3.7/site-packages/gensim/utils.py", line 469, in _load_specials

    val = np.load(subname(fname, attrib), mmap_mode=mmap)

  File "/home/ec2-user/miniconda3/envs/word2vec/lib/python3.7/site-packages/numpy/lib/npyio.py", line 440, in load

    pickle_kwargs=pickle_kwargs)

  File "/home/ec2-user/miniconda3/envs/word2vec/lib/python3.7/site-packages/numpy/lib/format.py", line 771, in read_array

    array.shape = shape

ValueError: cannot reshape array of size 16777184 into shape (134441,128)

为了安装 gensim，我使用了conda install -c anaconda gensim

Answer 1

文件大小是错误的。上传过程中一定发生了什么。对于大型模型，word2vec 将模型保存在 3 个文件中。假设您的模型名称是“model2”，您将拥有：模型2model2.trainables.syn1neg.npy模型2.wv.vectors.npy我的.wv.vectors.npy版本比我其他机器上的版本小了几千字节。