什么是 _md5.md5,为什么 hashlib.md5 这么慢?

当对缓慢的 stdlib实现_md5感到沮丧时,发现此未记录。hashlib.md5


在 Macbook 上:


>>> timeit hashlib.md5(b"hello world")

597 ns ± 17.2 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

>>> timeit _md5.md5(b"hello world")

224 ns ± 3.18 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

>>> _md5

<module '_md5' from '/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_md5.cpython-37m-darwin.so'>

在 Windows 机器上:


>>> timeit hashlib.md5(b"stonk overflow")

328 ns ± 21.8 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

>>> timeit _md5.md5(b"stonk overflow")

110 ns ± 12.5 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

>>> _md5

<module '_md5' (built-in)>

在 Linux 机器上:


>>> timeit hashlib.md5(b"https://adventofcode.com/2016/day/5")

259 ns ± 1.33 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

>>> timeit _md5.md5(b"https://adventofcode.com/2016/day/5")

102 ns ± 0.0576 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

>>> _md5

<module '_md5' from '/usr/local/lib/python3.8/lib-dynload/_md5.cpython-38-x86_64-linux-gnu.so'>

对于散列短消息,它的速度更快。对于长消息,类似的性能。


为什么它隐藏在下划线扩展模块中,为什么在 hashlib 中默认不使用这种更快的实现? 什么是_md5模块,为什么它没有公共 API?


炎炎设计
浏览 61回答 3
3回答

千巷猫影

在Python 2.&nbsp;5之前,哈希和摘要在它们自己的模块中实现(例如[Python&nbsp;2.Docs&nbsp;]: md5 - MD5 message digest algorithm)。从v2.5开始,[Python 2.6.Docs]: hashlib -添加了安全哈希和消息摘要。其目的是:提供对哈希/摘要的统一访问方法(通过它们的名称)切换(默认情况下)到外部密码学提供程序(这似乎是委托给该领域的某个专门实体的合乎逻辑的步骤,因为维护所有这些算法可能是一种矫枉过正)。那时OpenSSL是最好的选择:足够成熟、知名和兼容(有一堆类似的Java提供者,但那些都没什么用)作为#2的副作用。,Python实现对公共API隐藏(重命名为:_md5、_sha1、_sha256、_sha512,后添加:_blake2、_sha3),因为冗余通常会造成混淆。但是,另一个副作用是_hashlib.so对OpenSSL的libcrypto*.so的依赖(这是Nix(至少Lnx)特定的,在Win上,静态libeay32.lib链接在_hashlib.pyd,还有_ssl.pyd(我认为是蹩脚的),直到v3。7&nbsp;+,其中OpenSSL&nbsp;.dll是Python安装的一部分)。可能在90+%的机器上一切都很顺利,因为默认安装了OpenSSL,但是对于那些没有安装的机器,很多事情可能会被破坏,因为例如hashlib是由许多模块导入的(一个这样的例子是随机的它本身被许多其他人导入),因此与密码学完全无关的琐碎代码(至少不是第一眼看到的)将停止工作.&nbsp;这就是保留旧实现的原因(但同样,它们只是后备,因为OpenSSL版本/应该得到更好的维护)。[cfati@cfati-ubtu16x64-0:~/Work/Dev/StackOverflow/q059955854]> ~/sopr.sh*** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ***[064bit-prompt]> python3 -c "import sys, hashlib as hl, _md5, ssl;print(\"{0:}\n{1:}\n{2:}\n{3:}\".format(sys.version, _md5, hl._hashlib, ssl.OPENSSL_VERSION))"3.5.2 (default, Oct&nbsp; 8 2019, 13:06:37)[GCC 5.4.0 20160609]<module '_md5' (built-in)><module '_hashlib' from '/usr/lib/python3.5/lib-dynload/_hashlib.cpython-35m-x86_64-linux-gnu.so'>OpenSSL 1.0.2g&nbsp; 1 Mar 2016[064bit-prompt]>[064bit-prompt]> ldd /usr/lib/python3.5/lib-dynload/_hashlib.cpython-35m-x86_64-linux-gnu.so&nbsp; &nbsp; &nbsp; &nbsp; linux-vdso.so.1 =>&nbsp; (0x00007fffa7d0b000)&nbsp; &nbsp; &nbsp; &nbsp; libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f50d9e4d000)&nbsp; &nbsp; &nbsp; &nbsp; libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f50d9a83000)&nbsp; &nbsp; &nbsp; &nbsp; libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f50d963e000)&nbsp; &nbsp; &nbsp; &nbsp; /lib64/ld-linux-x86-64.so.2 (0x00007f50da271000)&nbsp; &nbsp; &nbsp; &nbsp; libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f50d943a000)[064bit-prompt]>[064bit-prompt]> openssl version -aOpenSSL 1.0.2g&nbsp; 1 Mar 2016built on: reproducible build, date unspecifiedplatform: debian-amd64options:&nbsp; bn(64,64) rc4(16x,int) des(idx,cisc,16,int) blowfish(idx)compiler: cc -I. -I.. -I../include&nbsp; -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wa,--noexecstack -Wall -DMD32_REG_T=int -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASMOPENSSLDIR: "/usr/lib/ssl"[064bit-prompt]>[064bit-prompt]> python3 -c "import _md5, hashlib as hl;print(_md5.md5(b\"A\").hexdigest(), hl.md5(b\"A\").hexdigest())"7fc56270e7a70fa81a5935b72eacbe29 7fc56270e7a70fa81a5935b72eacbe29根据[Python 3.Docs]:hashlib。算法保证:包含哈希算法名称的集合,保证在所有平台上都受此模块支持。请注意,尽管一些上游供应商提供了一个奇怪的“FIPS 兼容”Python 构建,但排除了它,但“md5”仍在此列表中。下面是一个自定义Python 2.7安装的示例(我很久以前构建的,值得一提的是它动态链接到OpenSSL .dll):e:\Work\Dev\StackOverflow\q059955854>sopr.bat*** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ***[prompt]> "F:\Install\pc064\HPE\OPSWpython\2.7.10__00\python.exe" -c "import sys, ssl;print(\"{0:}\n{1:}\".format(sys.version, ssl.OPENSSL_VERSION))"2.7.10 (default, Mar&nbsp; 8 2016, 15:02:46) [MSC v.1600 64 bit (AMD64)]OpenSSL 1.0.2j-fips&nbsp; 26 Sep 2016[prompt]> "F:\Install\pc064\HPE\OPSWpython\2.7.10__00\python.exe" -c "import hashlib as hl;print(hl.md5(\"A\").hexdigest())"7fc56270e7a70fa81a5935b72eacbe29[prompt]> "F:\Install\pc064\HPE\OPSWpython\2.7.10__00\python.exe" -c "import ssl;ssl.FIPS_mode_set(True);import hashlib as hl;print(hl.md5(\"A\").hexdigest())"Traceback (most recent call last):&nbsp; File "<string>", line 1, in <module>ValueError: error:060A80A3:digital envelope routines:FIPS_DIGESTINIT:disabled for fips至于速度问题我只能推测:Python实现(显然)是专门为Python编写的,这意味着它对于Python比通用版本“更优化”(是的,这在语法上是不正确的) ,并且还驻留在python*.so(或python可执行文件本身)中OpenSSL实现驻留在libcrypto*.so中,它被包装器_hashlib.so访问,它在Python类型 ( PyObject* ) 和OpenSSL类型( EVP_MD_CTX* )之间进行来回转换考虑到上述情况,前者(稍微)更快是有道理的(至少对于小消息,其中开销(函数调用和其他Python底层操作)与散列本身相比占总时间的很大一部分)。还有其他因素需要考虑(例如,是否使用了OpenSSL汇编器加速)。更新#0以下是我自己的一些基准。code00.py:#!/usr/bin/env pythonimport sysfrom hashlib import md5 as md5_opensslfrom _md5 import md5 as md5_builtinimport timeitdef main(*argv):&nbsp; &nbsp; base_text = b"A"&nbsp; &nbsp; number = 1000000&nbsp; &nbsp; print("timeit attempts number: {0:d}".format(number))&nbsp; &nbsp; #x = []&nbsp; &nbsp; #y = {}&nbsp; &nbsp; for count in range(0, 16):&nbsp; &nbsp; &nbsp; &nbsp; factor = 2 ** count&nbsp; &nbsp; &nbsp; &nbsp; text = base_text * factor&nbsp; &nbsp; &nbsp; &nbsp; globals_dict = {"text": text}&nbsp; &nbsp; &nbsp; &nbsp; #x.append(factor)&nbsp; &nbsp; &nbsp; &nbsp; print("\nUsing a {0:8d} (2 ** {1:2d}) bytes message".format(len(text), count))&nbsp; &nbsp; &nbsp; &nbsp; for func in [&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; md5_openssl,&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; md5_builtin,&nbsp; &nbsp; &nbsp; &nbsp; ]:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; globals_dict["md5"] = func&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; t = timeit.timeit(stmt="md5(text)", globals=globals_dict, number=number)&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; print("&nbsp; &nbsp; {0:12s} took: {1:11.6f} seconds".format(func.__name__, t))&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; #y.setdefault(func.__name__, []).append(t)&nbsp; &nbsp; #print(x, y)if __name__ == "__main__":&nbsp; &nbsp; print("Python {0:s} {1:d}bit on {2:s}\n".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))&nbsp; &nbsp; main(*sys.argv[1:])&nbsp; &nbsp; print("\nDone.")输出:Win 10 pc064(在戴尔 Precision 5510笔记本电脑上运行):[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.07.06_test0\Scripts\python.exe" code00.pyPython 3.7.6 (tags/v3.7.6:43364a7ae0, Dec 19 2019, 00:42:30) [MSC v.1916 64 bit (AMD64)] 64bit on win32timeit attempts number: 1000000Using a&nbsp; &nbsp; &nbsp; &nbsp; 1 (2 **&nbsp; 0) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 0.449134 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 0.120021 secondsUsing a&nbsp; &nbsp; &nbsp; &nbsp; 2 (2 **&nbsp; 1) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 0.460399 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 0.118555 secondsUsing a&nbsp; &nbsp; &nbsp; &nbsp; 4 (2 **&nbsp; 2) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 0.451850 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 0.121166 secondsUsing a&nbsp; &nbsp; &nbsp; &nbsp; 8 (2 **&nbsp; 3) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 0.438398 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 0.118127 secondsUsing a&nbsp; &nbsp; &nbsp; &nbsp;16 (2 **&nbsp; 4) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 0.454653 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 0.122818 secondsUsing a&nbsp; &nbsp; &nbsp; &nbsp;32 (2 **&nbsp; 5) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 0.450776 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 0.118594 secondsUsing a&nbsp; &nbsp; &nbsp; &nbsp;64 (2 **&nbsp; 6) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 0.555761 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 0.278812 secondsUsing a&nbsp; &nbsp; &nbsp; 128 (2 **&nbsp; 7) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 0.681296 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 0.455921 secondsUsing a&nbsp; &nbsp; &nbsp; 256 (2 **&nbsp; 8) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 0.895952 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 0.807457 secondsUsing a&nbsp; &nbsp; &nbsp; 512 (2 **&nbsp; 9) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 1.401584 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 1.499279 secondsUsing a&nbsp; &nbsp; &nbsp;1024 (2 ** 10) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 2.360966 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 2.878650 secondsUsing a&nbsp; &nbsp; &nbsp;2048 (2 ** 11) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 4.383245 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 5.655477 secondsUsing a&nbsp; &nbsp; &nbsp;4096 (2 ** 12) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 8.264774 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp;10.920909 secondsUsing a&nbsp; &nbsp; &nbsp;8192 (2 ** 13) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp;15.521947 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp;21.895179 secondsUsing a&nbsp; &nbsp; 16384 (2 ** 14) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp;29.947287 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp;43.198639 secondsUsing a&nbsp; &nbsp; 32768 (2 ** 15) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp;59.123447 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp;86.453821 secondsDone.Ubtu 16 pc064(在上述机器上的VirtualBox中运行的VM ):[064bit-prompt]> python3 code00.pyPython 3.5.2 (default, Oct&nbsp; 8 2019, 13:06:37) [GCC 5.4.0 20160609] 64bit on linuxtimeit attempts number: 1000000Using a&nbsp; &nbsp; &nbsp; &nbsp; 1 (2 **&nbsp; 0) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 0.246166 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 0.130589 secondsUsing a&nbsp; &nbsp; &nbsp; &nbsp; 2 (2 **&nbsp; 1) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 0.251019 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 0.127750 secondsUsing a&nbsp; &nbsp; &nbsp; &nbsp; 4 (2 **&nbsp; 2) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 0.257018 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 0.123116 secondsUsing a&nbsp; &nbsp; &nbsp; &nbsp; 8 (2 **&nbsp; 3) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 0.245399 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 0.128267 secondsUsing a&nbsp; &nbsp; &nbsp; &nbsp;16 (2 **&nbsp; 4) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 0.251832 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 0.136373 secondsUsing a&nbsp; &nbsp; &nbsp; &nbsp;32 (2 **&nbsp; 5) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 0.248410 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 0.140708 secondsUsing a&nbsp; &nbsp; &nbsp; &nbsp;64 (2 **&nbsp; 6) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 0.361016 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 0.267021 secondsUsing a&nbsp; &nbsp; &nbsp; 128 (2 **&nbsp; 7) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 0.478735 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 0.413986 secondsUsing a&nbsp; &nbsp; &nbsp; 256 (2 **&nbsp; 8) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 0.707602 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 0.695042 secondsUsing a&nbsp; &nbsp; &nbsp; 512 (2 **&nbsp; 9) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 1.216832 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 1.268570 secondsUsing a&nbsp; &nbsp; &nbsp;1024 (2 ** 10) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 2.122014 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 2.429623 secondsUsing a&nbsp; &nbsp; &nbsp;2048 (2 ** 11) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 4.158188 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 4.847686 secondsUsing a&nbsp; &nbsp; &nbsp;4096 (2 ** 12) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp; 7.839173 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp; 9.242224 secondsUsing a&nbsp; &nbsp; &nbsp;8192 (2 ** 13) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp;15.282232 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp;18.368874 secondsUsing a&nbsp; &nbsp; 16384 (2 ** 14) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp;30.681912 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp;36.755073 secondsUsing a&nbsp; &nbsp; 32768 (2 ** 15) bytes message&nbsp; &nbsp; openssl_md5&nbsp; took:&nbsp; &nbsp;60.230543 seconds&nbsp; &nbsp; md5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; took:&nbsp; &nbsp;73.237356 secondsDone.结果似乎与你的完全不同。就我而言:从 [~ 512B .. ~ 1KiB ] 大小的消息开始,OpenSSL实现似乎比内置的执行得更好我知道声明模式的结果太少,但似乎两种实现似乎与消息大小成线性比例(就时间而言)(但内置斜率似乎有点陡峭 - 这意味着它会表现更差从长远来看)总而言之,如果您的所有消息都很小,并且内置实现最适合您,那么请使用它。

慕斯王

Python 公共模块将方法委托给隐藏模块是很常见的。例如,该collections.abc模块的完整代码为:from _collections_abc import *from _collections_abc import __all__的函数hashlib是动态创建的:for __func_name in __always_supported:&nbsp; &nbsp; # try them all, some may not work due to the OpenSSL&nbsp; &nbsp; # version not supporting that algorithm.&nbsp; &nbsp; try:&nbsp; &nbsp; &nbsp; &nbsp; globals()[__func_name] = __get_hash(__func_name)的定义always_supported是:__always_supported = ('md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512',&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 'blake2b', 'blake2s',&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 'sha3_224', 'sha3_256', 'sha3_384', 'sha3_512',&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 'shake_128', 'shake_256')get_hash或者:___get_openssl_constructor ___get_builtin_constructortry:&nbsp; &nbsp; import _hashlib&nbsp; &nbsp; new = __hash_new&nbsp; &nbsp; __get_hash = __get_openssl_constructor&nbsp; &nbsp; algorithms_available = algorithms_available.union(&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; _hashlib.openssl_md_meth_names)except ImportError:&nbsp; &nbsp; new = __py_new&nbsp; &nbsp; __get_hash = __get_builtin_constructor__get_builtin_constructor是(再次)隐藏_hashlib模块的后备:def __get_openssl_constructor(name):&nbsp; &nbsp; if name in __block_openssl_constructor:&nbsp; &nbsp; &nbsp; &nbsp; # Prefer our blake2 and sha3 implementation.&nbsp; &nbsp; &nbsp; &nbsp; return __get_builtin_constructor(name)&nbsp; &nbsp; try:&nbsp; &nbsp; &nbsp; &nbsp; f = getattr(_hashlib, 'openssl_' + name)&nbsp; &nbsp; &nbsp; &nbsp; # Allow the C module to raise ValueError.&nbsp; The function will be&nbsp; &nbsp; &nbsp; &nbsp; # defined but the hash not actually available thanks to OpenSSL.&nbsp; &nbsp; &nbsp; &nbsp; f()&nbsp; &nbsp; &nbsp; &nbsp; # Use the C function directly (very fast)&nbsp; &nbsp; &nbsp; &nbsp; return f&nbsp; &nbsp; except (AttributeError, ValueError):&nbsp; &nbsp; &nbsp; &nbsp; return __get_builtin_constructor(name)在上面的hashlib代码中,你有这个:def __get_builtin_constructor(name):&nbsp; &nbsp; cache = __builtin_constructor_cache&nbsp; &nbsp; ...&nbsp; &nbsp; elif name in {'MD5', 'md5'}:&nbsp; &nbsp; &nbsp; &nbsp; import _md5&nbsp; &nbsp; &nbsp; &nbsp; cache['MD5'] = cache['md5'] = _md5.md5But md5is not in __block_openssl_constructor,因此该_hashlib/openssl版本优于该_md5/builtin版本:REPL 中的确认:>>> hashlib.md5<built-in function openssl_md5>>>> _md5.md5<built-in function md5>这些函数是 MD5 算法的不同实现,它们openssl_md5调用动态系统库。这就是为什么你有一些性能变化。第一个版本在https://github.com/python/cpython/blob/master/Modules/_hashopenssl.c中定义,另一个在https://github.com/python/cpython/blob/master/Modules/md5module .c,如果你想检查差异。那为什么_md5.md5定义了函数但从未使用过?我想这个想法是确保某些算法始终可用,即使openssl不存在:此模块中始终存在的哈希算法的构造函数是 sha1()、sha224()、sha256()、sha384()、sha512()、blake2b() 和 blake2s()。(https://docs.python.org/3/library/hashlib.html)

MMMHUHU

我在 bugs.python.org 上环顾四周并阅读 cpython git 提交历史的理论:cpython 在 2005 年切换到 openssl md5,因为它比内置实现更快。他们在 2007 年添加了一个新的内置实现,它比 openssl 更快,但从未切换回来。这两项更改均由 Gregory P. Smith 完成。这是我的证据。2005 年,Greg 创建了 bpo 问题“sha 和 md5 模块应尽可能使用 OpenSSL”。此提交中所做的更改。2007 年,Greg 在此提交中添加了新的快速 md5 模块。Python 3.8 中的_md5实现似乎基本相同(我正在查看提交 ea316fd21527)我认为 cpython 维护者可能会愿意切换回_md5它可用的时间,因为 openssl 实现更快不再是真的(过去 13 年可能是不真实的)。
打开App,查看更多内容
随时随地看视频慕课网APP

相关分类

Python