在python中多线程时如何增加变量值

Question

我正在尝试制作一个具有多线程的网络爬虫以使其更快。我想让每次执行都增加价值。但有时该值会跳过或重复自身。

import threading

num = 0

def scan():

    while True:

        global num

        num += 1

        print(num)

        open('logs.txt','a').write(str(f'{num}\n'))

for x in range(500):

    threading.Thread(target=scan).start()

结果：

2

2

5

5

7

8

10

10

12

13

13

13

16

17

19

19

22

23

24

25

26

28

29

29

31

32

33

34

预期结果：

1

2

3

4

5

6

7

8

9

10

Answer 1

因此，由于该变量num是共享资源，因此您需要对其进行锁定。这是按如下方式完成的：num_lock = threading.Lock()每次要更新共享变量时，都需要线程先获取锁。一旦获得锁，只有该线程可以更新 num 的值，而在当前线程获得锁时，其他线程将无法这样做。确保在执行此操作时使用waitortry-finally块，以保证即使当前线程无法更新共享变量也将释放锁。是这样的：num_lock.acquire()try:        num+=1finally:   num_lock.release()使用with： with num_lock:   num+=1

Answer 2

一个重要的标注，除了threading.Lock：用于join使父线程等待分叉线程完成。没有这个，线程仍然会竞争。假设我正在使用num线程完成后：import threadinglock, num = threading.Lock(), 0def operation():    global num    print("Operation has started")    with lock:        num += 1threads = [threading.Thread(target=operation) for x in range(10)]for t in threads:    t.start()for t in threads:    t.join()print(num)没有连接，不一致（9 被打印一次，否则 10）：Operation has startedOperation has startedOperation has startedOperation has startedOperation has startedOperation has startedOperation has startedOperation has startedOperation has startedOperation has started9使用 join，它是一致的：Operation has startedOperation has startedOperation has startedOperation has startedOperation has startedOperation has startedOperation has startedOperation has startedOperation has startedOperation has started10

Answer 3

似乎是一种竞争条件。您可以使用锁，以便只有一个线程可以获得特定的数字。使用 lock 写入输出文件也是有意义的。这是一个带锁的例子。当然，您不能保证输出的写入顺序，但每个项目都应该恰好写入一次。在此示例中，我添加了 10000 的限制，以便您可以更轻松地检查最终是否将所有内容写入测试代码，因为否则无论您在什么时候中断它，都很难验证数字是被跳过还是只是等待锁定以写入输出。不是my_num共享的，因此您已经在该with num_lock部分内声明了它之后，您可以自由释放该锁（保护共享num）然后继续在my_num外部使用with，而其他线程可以访问该锁以声明自己的价值. 这最大限度地减少了持有锁的持续时间。import threadingnum = 0num_lock = threading.Lock()file_lock = threading.Lock()&nbsp; &nbsp;&nbsp;def scan():&nbsp; &nbsp; global num_lock, file_lock, num&nbsp; &nbsp;&nbsp;&nbsp; &nbsp; while num < 10000:&nbsp; &nbsp; &nbsp; &nbsp; with num_lock:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; num += 1&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; my_num = num&nbsp; &nbsp; &nbsp; &nbsp; # do whatever you want here using my_num&nbsp; &nbsp; &nbsp; &nbsp; # but do not touch num&nbsp; &nbsp; &nbsp; &nbsp; with file_lock:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; open('logs.txt','a').write(str(f'{my_num}\n'))&nbsp; &nbsp; &nbsp; &nbsp;&nbsp;threads = [threading.Thread(target=scan) for _ in range(500)]for thread in threads:&nbsp; &nbsp; thread.start()for thread in threads:&nbsp; &nbsp; thread.join()