我有一个读取大型文本文件并处理Pools中每一行的代码。
对于elif,我需要将整个过程休眠120秒,换句话说,我希望所有创建的其他Pool都暂停。但是在120秒后,所有的Pool应该恢复工作。
该代码的功能与此类似:
from multiprocessing import Pool
import sys
sys.tracebacklimit = 0
def req(line):
if "@" not in line:
# (some function for processing here)
return line
elif "somestring" in line:
#HERE I NEED TO SLEEP ALL POOLS
else:
# (some function for processing)
return line
if __name__ == "__main__":
pool = Pool(20)
with open("list.txt") as source_file:
# chunk the work into batches of 20 lines at a time
pool.map(req, source_file, 35)
相关分类