Process to process communication and thread synchronization is an ever-changing topic. You should know a little about programming, but you can't say it in detail. On the one hand, in addition to the less possible use in the work, on the other hand, these concepts involve more and deeper things. Network programming, server-side programming, concurrent applications, etc. will be involved. Its development and debugging process are not intuitive. Because the principle of synchronous communication mechanism is the same, this paper hopes to use python instance to concretize the abstract concept.
Before reading, you can refer to the previous article: [python multithreading and multiprocessing and their regions Don't] learn about thread and process creation.
python multithreading synchronization
Two standard libraries, thread and threading, are provided in python to support threads. In python 3, support for the former has been given up, and the latter is a higher-level encapsulated thread library. The following are examples of the latter.
Synchronization and mutual exclusion
People who believe that there are too many mathematical operating systems are confused by these two concepts. What mutual exclusion is a special synchronization? Synchronization is a collaborative relationship among several key nodes in the process of multithreading or multi-process collaborative completion of a task.
In fact, these two concepts are all around a collaborative relationship, which can be clearly expressed through a specific example:
There are two threads, called thread A and thread B, where thread A writes A variable, and thread B reads the variable written by thread A, and thread A writes the variable first, and then thread B can read the variable, so there is A synchronization relationship between thread A and thread B;
===== Synchronization relation ===== Thread A: write(share_data) V(S) # Release resources Thread B: P(S) # Access to resources read(share_data)
If another thread C also needs to write this variable, then thread A and C are mutually exclusive, because only one thread can write this variable at the same time;
===== Mutually exclusive relationship ===== Thread A: Lock.acquire(); # Get lock write(share_data) Lock.release() # Release lock Thread C: Lock.acquire(); # Get lock write(share_data) Lock.release() # Release lock
Thread synchronization
The main thread and the thread it creates execute their own code until the end. Next, let's look at the synchronization between python threads
Is alternate execution thread safe?
Let's take a look at the following example:
share_data = 0
def tstart(arg):
time.sleep(0.1)
global share_data
for i in xrange(1000):
share_data += 1
if __name__ == '__main__':
t1 = threading.Thread(target = tstart, args = ('',))
t2 = threading.Thread(target = tstart, args = ('',))
t1.start()
t2.start()
t1.join()
t2.join()
print 'share_data result:', share_data
In most cases, the execution result of the above code is less than 2000. As mentioned in the previous article, a global interpreter lock (GIL) is introduced in CPython, the python interpreter, which means that only one thread is executing at any time. But there are still problems here. Why?
The root cause is that the write to share ﹣ u data is not an atomic operation. The thread is interrupted in the process of writing, and then the thread is switched to execute. When it comes back, it will continue to execute the interrupted write operation, but it may overwrite the result written by another thread in this period of time.
Here is a possible operation:
The actual calculation process may be more complex than that described above, which can be understood from the perspective of a single thread. Without synchronization measures, a single thread can not perceive the existence of other threads at all, read, calculate and write back data.
If the calculation process is interrupted after reading the data or before writing back the data, when it is executed again, even if the share_data in the memory has changed, the process will continue to execute from the interrupted place, and the calculation result will overwrite the current share_data value;
This is why there is only one thread executing at a time, but the result is still wrong. It can be imagined that if multiple threads execute in parallel without synchronization, the calculation process will be more chaotic.
If you are interested, you can use res of a global list to record the process of writing share ﹣ u data for each thread. You can see the writing process intuitively:
share_data = 0
res = []
def tstart(arg):
time.sleep(0.1)
global share_data
for i in xrange(1000):
res.append((arg, share_data))
share_data += 1
if __name__ == '__main__':
t1 = threading.Thread(target = tstart, args = ('1',))
t2 = threading.Thread(target = tstart, args = ('2',))
t1.start()
t2.start()
t1.join()
t2.join()
print res, len(res)
print 'share_data result:', share_data
The following is a possible result. It can be seen that the two threads did 2000 plus one operations on share data, but the result is not 2000
[('2', 0), ('2', 1), ('1', 2), ('1', 3), ('1', 4), ('1', 5), ('1', 6), ('1', 7), ('1', 8), ('1', 9), ('1', 10), ('1', 11), ('1', 12), ('2', 12), ('2', 13), ('2', 14), ('2', 15), ('2', 16), ('2', 17), ('2', 18), ('2', 19), ('2', 20), ('2', 21), ('2', 22), ('1', 13), ('1', 14), ('1', 15), ('1', 16), ('1', 17), ('1', 18), ('1', 19), ('1', 20), ('1', 21), ('1', 22), ('1', 23), ('2', 24), ('2', 25), ('2', 26), ('2', 27), ('2', 28), ('2', 29), ('2', 30), ('2', 31), ('2', 32), ('2', 33), ('2', 34), ('1', 35), ('1', 36), ('1', 37), ('1', 38), ('1', 39), ('1', 40), ('1', 41), ('1', 42), ('1', 43), ('1', 44), ('1', 45), ('2', 35), ('2', 47), ('2', 48), ('2', 49), ('2', 50), ('2', 51), ('2', 52), ('2', 53), ('2', 54), ('2', 55), ('2', 56), ('2', 57), ('1', 57), ('1', 58), ('1', 59), ('1', 60), ('1', 61), ('1', 62), ('1', 63), ('1', 64), ('1', 65), ('1', 66), ('1', 67), ('2', 58), ('2', 59), ('2', 60), ('2', 61), ('2', 62), ('2', 63), ('2', 64), ('2', 65), ('2', 66), ('2', 67), ('2', 68), ('1', 68), ('1', 70), ('1', 71), ('1', 72), ('1', 73), ('1', 74), ('1', 75), ('1', 76), ('1', 77), ('1', 78), ('1', 79), ('1', 80), ('2', 80), ('2', 81), ('2', 82), ('2', 83), ('2', 84), ('2', 85), ('2', 86), ('2', 87), ('2', 88), ('2', 89), ('2', 90), ('1', 81), ('1', 82), ('1', 83), ('1', 84), ('1', 85), ('1', 86), ('1', 87), ('1', 88), ('1', 89), ('1', 90), ('1', 91), ('2', 92), ('2', 93), ('2', 94), ('2', 95), ('2', 96), ('2', 97), ('2', 98), ('2', 99), ('2', 100), ('2', 101), ('2', 102), ('1', 103), ('1', 104), ('1', 105), ('1', 106), ('1', 107), ('1', 108), ('1', 109), ('1', 110), ('1', 111), ('1', 112), ('1', 113), ('2', 103), ('2', 115), ('2', 116), ('2', 117), ('2', 118), ('2', 119), ('2', 120), ('2', 121), ('2', 122), ('2', 123), ('2', 124), ('2', 125), ('1', 125), ('1', 126), ('1', 127), ('1', 128), ('1', 129), ('1', 130), ('1', 131), ('1', 132), ('1', 133), ('1', 134), ('1', 135), ('2', 137), ('2', 138), ('2', 139), ('2', 140), ('2', 141), ('2', 142), ('2', 143), ('2', 144), ('2', 145), ('2', 146), ('2', 147), ('1', 136), ('1', 148), ('1', 149), ('1', 150), ('1', 151), ('1', 152), ('1', 153), ('1', 154), ('1', 155), ('1', 156), ('1', 157), ('1', 158), ('2', 148), ('2', 149), ('2', 150), ('2', 151), ('2', 152), ('2', 153), ('2', 154), ('2', 155), ('2', 156), ('2', 157), ('2', 158), ('1', 160), ('1', 161), ('1', 162), ('1', 163), ('1', 164), ('1', 165), ('1', 166), ('1', 167), ('1', 168), ('1', 169), ('1', 170), ('2', 170), ('2', 171), ('2', 172), ('2', 173), ('2', 174), ('2', 175), ('2', 176), ('2', 177), ('2', 178), ('2', 179), ('2', 180), ('1', 171), ('1', 172), ('1', 173), ('1', 174), ('1', 175), ('1', 176), ('1', 177), ('1', 178), ('1', 179), ('1', 180), ('1', 181), ('2', 182), ('2', 183), ('2', 184), ('2', 185), ('2', 186), ('2', 187), ('2', 188), ('2', 189), ('2', 190), ('2', 191), ('2', 192), ('1', 193), ('1', 194), ('1', 195), ('1', 196), ('1', 197), ('1', 198), ('1', 199), ('1', 200), ('1', 201), ('1', 202), ('1', 203), ('2', 193), ('2', 205), ('2', 206), ('2', 207), ('2', 208), ('2', 209), ('2', 210), ('2', 211), ('2', 212), ('2', 213), ('2', 214), ('2', 215), ('1', 215), ('1', 216), ('1', 217), ('1', 218), ('1', 219), ('1', 220), ('1', 221), ('1', 222), ('1', 223), ('1', 224), ('1', 225), ('2', 216), ('2', 217), ('2', 218), ('2', 219), ('2', 220), ('2', 221), ('2', 222), ('2', 223), ('2', 224), ('2', 225), ('2', 226), ('1', 226), ('1', 228), ('1', 229), ('1', 230), ('1', 231), ('1', 232), ('1', 233), ('1', 234), ('1', 235), ('1', 236), ('1', 237), ('1', 238), ('2', 238), ('2', 239), ('2', 240), ('2', 241), ('2', 242), ('2', 243), ('2', 244), ('2', 245), ('2', 246), ('2', 247), ('2', 248), ('1', 239), ('1', 240), ('1', 241), ('1', 242), ('1', 243), ('1', 244), ('1', 245), ('1', 246), ('1', 247), ('1', 248), ('1', 249), ('2', 250), ('2', 251), ('2', 252), ('2', 253), ('2', 254), ('2', 255), ('2', 256), ('2', 257), ('2', 258), ('2', 259), ('2', 260), ('1', 261), ('1', 262), ('1', 263), ('1', 264), ('1', 265), ('1', 266), ('1', 267), ('1', 268), ('1', 269), ('1', 270), ('1', 271), ('2', 261), ('2', 273), ('2', 274), ('2', 275), ('2', 276), ('2', 277), ('2', 278), ('2', 279), ('2', 280), ('2', 281), ('2', 282), ('2', 283), ('1', 283), ('1', 284), ('1', 285), ('1', 286), ('1', 287), ('1', 288), ('1', 289), ('1', 290), ('1', 291), ('1', 292), ('1', 293), ('2', 295), ('2', 296), ('2', 297), ('2', 298), ('2', 299), ('2', 300), ('2', 301), ('2', 302), ('2', 303), ('2', 304), ('2', 305), ('1', 294), ('1', 306), ('1', 307), ('1', 308), ('1', 309), ('1', 310), ('1', 311), ('1', 312), ('1', 313), ('1', 314), ('1', 315), ('1', 316), ('2', 306), ('2', 307), ('2', 308), ('2', 309), ('2', 310), ('2', 311), ('2', 312), ('2', 313), ('2', 314), ('2', 315), ('2', 316), ('1', 318), ('1', 319), ('1', 320), ('1', 321), ('1', 322), ('1', 323), ('1', 324), ('1', 325), ('1', 326), ('1', 327), ('1', 328), ('2', 328), ('2', 329), ('2', 330), ('2', 331), ('2', 332), ('2', 333), ('2', 334), ('2', 335), ('2', 336), ('2', 337), ('2', 338), ('1', 329), ('1', 330), ('1', 331), ('1', 332), ('1', 333), ('1', 334), ('1', 335), ('1', 336), ('1', 337), ('1', 338), ('1', 339), ('2', 340), ('2', 341), ('2', 342), ('2', 343), ('2', 344), ('2', 345), ('2', 346), ('2', 347), ('2', 348), ('2', 349), ('2', 350), ('1', 351), ('1', 352), ('1', 353), ('1', 354), ('1', 355), ('1', 356), ('1', 357), ('1', 358), ('1', 359), ('1', 360), ('1', 361), ('2', 351), ('2', 363), ('2', 364), ('2', 365), ('2', 366), ('2', 367), ('2', 368), ('2', 369), ('2', 370), ('2', 371), ('2', 372), ('2', 373), ('1', 373), ('1', 374), ('1', 375), ('1', 376), ('1', 377), ('1', 378), ('1', 379), ('1', 380), ('1', 381), ('1', 382), ('1', 383), ('2', 374), ('2', 375), ('2', 376), ('2', 377), ('2', 378), ('2', 379), ('2', 380), ('2', 381), ('2', 382), ('2', 383), ('2', 384), ('1', 384), ('1', 386), ('1', 387), ('1', 388), ('1', 389), ('1', 390), ('1', 391), ('1', 392), ('1', 393), ('1', 394), ('1', 395), ('1', 396), ('2', 396), ('2', 397), ('2', 398), ('2', 399), ('2', 400), ('2', 401), ('2', 402), ('2', 403), ('2', 404), ('2', 405), ('2', 406), ('1', 397), ('1', 398), ('1', 399), ('1', 400), ('1', 401), ('1', 402), ('1', 403), ('1', 404), ('1', 405), ('1', 406), ('1', 407), ('2', 408), ('2', 409), ('2', 410), ('2', 411), ('2', 412), ('2', 413), ('2', 414), ('2', 415), ('2', 416), ('2', 417), ('2', 418), ('1', 419), ('1', 420), ('1', 421), ('1', 422), ('1', 423), ('1', 424), ('1', 425), ('1', 426), ('1', 427), ('1', 428), ('1', 429), ('2', 419), ('2', 431), ('2', 432), ('2', 433), ('2', 434), ('2', 435), ('2', 436), ('2', 437), ('2', 438), ('2', 439), ('2', 440), ('2', 441), ('1', 441), ('1', 442), ('1', 443), ('1', 444), ('1', 445), ('1', 446), ('1', 447), ('1', 448), ('1', 449), ('1', 450), ('1', 451), ('2', 453), ('2', 454), ('2', 455), ('2', 456), ('2', 457), ('2', 458), ('2', 459), ('2', 460), ('2', 461), ('2', 462), ('2', 463), ('1', 452), ('1', 464), ('1', 465), ('1', 466), ('1', 467), ('1', 468), ('1', 469), ('1', 470), ('1', 471), ('1', 472), ('1', 473), ('1', 474), ('2', 464), ('2', 465), ('2', 466), ('2', 467), ('2', 468), ('2', 469), ('2', 470), ('2', 471), ('2', 472), ('2', 473), ('2', 474), ('1', 476), ('1', 477), ('1', 478), ('1', 479), ('1', 480), ('1', 481), ('1', 482), ('1', 483), ('1', 484), ('1', 485), ('1', 486), ('2', 486), ('2', 487), ('2', 488), ('2', 489), ('2', 490), ('2', 491), ('2', 492), ('2', 493), ('2', 494), ('2', 495), ('2', 496), ('1', 487), ('1', 488), ('1', 489), ('1', 490), ('1', 491), ('1', 492), ('1', 493), ('1', 494), ('1', 495), ('1', 496), ('1', 497), ('2', 498), ('2', 499), ('2', 500), ('2', 501), ('2', 502), ('2', 503), ('2', 504), ('2', 505), ('2', 506), ('2', 507), ('2', 508), ('1', 509), ('1', 510), ('1', 511), ('1', 512), ('1', 513), ('1', 514), ('1', 515), ('1', 516), ('1', 517), ('1', 518), ('1', 519), ('2', 509), ('2', 521), ('2', 522), ('2', 523), ('2', 524), ('2', 525), ('2', 526), ('2', 527), ('2', 528), ('2', 529), ('2', 530), ('2', 531), ('1', 531), ('1', 532), ('1', 533), ('1', 534), ('1', 535), ('1', 536), ('1', 537), ('1', 538), ('1', 539), ('1', 540), ('1', 541), ('2', 532), ('2', 533), ('2', 534), ('2', 535), ('2', 536), ('2', 537), ('2', 538), ('2', 539), ('2', 540), ('2', 541), ('2', 542), ('1', 542), ('1', 544), ('1', 545), ('1', 546), ('1', 547), ('1', 548), ('1', 549), ('1', 550), ('1', 551), ('1', 552), ('1', 553), ('1', 554), ('2', 554), ('2', 555), ('2', 556), ('2', 557), ('2', 558), ('2', 559), ('2', 560), ('2', 561), ('2', 562), ('2', 563), ('2', 564), ('1', 555), ('1', 556), ('1', 557), ('1', 558), ('1', 559), ('1', 560), ('1', 561), ('1', 562), ('1', 563), ('1', 564), ('1', 565), ('2', 566), ('2', 567), ('2', 568), ('2', 569), ('2', 570), ('2', 571), ('2', 572), ('2', 573), ('2', 574), ('2', 575), ('2', 576), ('1', 577), ('1', 578), ('1', 579), ('1', 580), ('1', 581), ('1', 582), ('1', 583), ('1', 584), ('1', 585), ('1', 586), ('1', 587), ('2', 577), ('2', 589), ('2', 590), ('2', 591), ('2', 592), ('2', 593), ('2', 594), ('2', 595), ('2', 596), ('2', 597), ('2', 598), ('2', 599), ('1', 599), ('1', 600), ('1', 601), ('1', 602), ('1', 603), ('1', 604), ('1', 605), ('1', 606), ('1', 607), ('1', 608), ('1', 609), ('2', 611), ('2', 612), ('2', 613), ('2', 614), ('2', 615), ('2', 616), ('2', 617), ('2', 618), ('2', 619), ('2', 620), ('2', 621), ('1', 610), ('1', 622), ('1', 623), ('1', 624), ('1', 625), ('1', 626), ('1', 627), ('1', 628), ('1', 629), ('1', 630), ('1', 631), ('1', 632), ('2', 622), ('2', 623), ('2', 624), ('2', 625), ('2', 626), ('2', 627), ('2', 628), ('2', 629), ('2', 630), ('2', 631), ('2', 632), ('1', 634), ('1', 635), ('1', 636), ('1', 637), ('1', 638), ('1', 639), ('1', 640), ('1', 641), ('1', 642), ('1', 643), ('1', 644), ('2', 644), ('2', 645), ('2', 646), ('2', 647), ('2', 648), ('2', 649), ('2', 650), ('2', 651), ('2', 652), ('2', 653), ('2', 654), ('1', 645), ('1', 646), ('1', 647), ('1', 648), ('1', 649), ('1', 650), ('1', 651), ('1', 652), ('1', 653), ('1', 654), ('1', 655), ('2', 656), ('2', 657), ('2', 658), ('2', 659), ('2', 660), ('2', 661), ('2', 662), ('2', 663), ('2', 664), ('2', 665), ('2', 666), ('1', 667), ('1', 668), ('1', 669), ('1', 670), ('1', 671), ('1', 672), ('1', 673), ('1', 674), ('1', 675), ('1', 676), ('1', 677), ('2', 667), ('2', 679), ('2', 680), ('2', 681), ('2', 682), ('2', 683), ('2', 684), ('2', 685), ('2', 686), ('2', 687), ('2', 688), ('2', 689), ('1', 689), ('1', 690), ('1', 691), ('1', 692), ('1', 693), ('1', 694), ('1', 695), ('1', 696), ('1', 697), ('1', 698), ('1', 699), ('2', 690), ('2', 691), ('2', 692), ('2', 693), ('2', 694), ('2', 695), ('2', 696), ('2', 697), ('2', 698), ('2', 699), ('2', 700), ('1', 700), ('1', 702), ('1', 703), ('1', 704), ('1', 705), ('1', 706), ('1', 707), ('1', 708), ('1', 709), ('1', 710), ('1', 711), ('1', 712), ('2', 712), ('2', 713), ('2', 714), ('2', 715), ('2', 716), ('2', 717), ('2', 718), ('2', 719), ('2', 720), ('2', 721), ('2', 722), ('1', 713), ('1', 714), ('1', 715), ('1', 716), ('1', 717), ('1', 718), ('1', 719), ('1', 720), ('1', 721), ('1', 722), ('1', 723), ('2', 724), ('2', 725), ('2', 726), ('2', 727), ('2', 728), ('2', 729), ('2', 730), ('2', 731), ('2', 732), ('2', 733), ('2', 734), ('1', 735), ('1', 736), ('1', 737), ('1', 738), ('1', 739), ('1', 740), ('1', 741), ('1', 742), ('1', 743), ('1', 744), ('1', 745), ('2', 735), ('2', 747), ('2', 748), ('2', 749), ('2', 750), ('2', 751), ('2', 752), ('2', 753), ('2', 754), ('2', 755), ('2', 756), ('2', 757), ('1', 757), ('1', 758), ('1', 759), ('1', 760), ('1', 761), ('1', 762), ('1', 763), ('1', 764), ('1', 765), ('1', 766), ('1', 767), ('2', 769), ('2', 770), ('2', 771), ('2', 772), ('2', 773), ('2', 774), ('2', 775), ('2', 776), ('2', 777), ('2', 778), ('2', 779), ('1', 768), ('1', 780), ('1', 781), ('1', 782), ('1', 783), ('1', 784), ('1', 785), ('1', 786), ('1', 787), ('1', 788), ('1', 789), ('1', 790), ('2', 780), ('2', 781), ('2', 782), ('2', 783), ('2', 784), ('2', 785), ('2', 786), ('2', 787), ('2', 788), ('2', 789), ('2', 790), ('1', 792), ('1', 793), ('1', 794), ('1', 795), ('1', 796), ('1', 797), ('1', 798), ('1', 799), ('1', 800), ('1', 801), ('1', 802), ('2', 802), ('2', 803), ('2', 804), ('2', 805), ('2', 806), ('2', 807), ('2', 808), ('2', 809), ('2', 810), ('2', 811), ('2', 812), ('1', 803), ('1', 804), ('1', 805), ('1', 806), ('1', 807), ('1', 808), ('1', 809), ('1', 810), ('1', 811), ('1', 812), ('1', 813), ('2', 814), ('2', 815), ('2', 816), ('2', 817), ('2', 818), ('2', 819), ('2', 820), ('2', 821), ('2', 822), ('2', 823), ('2', 824), ('1', 825), ('1', 826), ('1', 827), ('1', 828), ('1', 829), ('1', 830), ('1', 831), ('1', 832), ('1', 833), ('1', 834), ('1', 835), ('2', 825), ('2', 837), ('2', 838), ('2', 839), ('2', 840), ('2', 841), ('2', 842), ('2', 843), ('2', 844), ('2', 845), ('2', 846), ('2', 847), ('1', 847), ('1', 848), ('1', 849), ('1', 850), ('1', 851), ('1', 852), ('1', 853), ('1', 854), ('1', 855), ('1', 856), ('1', 857), ('2', 848), ('2', 849), ('2', 850), ('2', 851), ('2', 852), ('2', 853), ('2', 854), ('2', 855), ('2', 856), ('2', 857), ('2', 858), ('1', 858), ('1', 860), ('1', 861), ('1', 862), ('1', 863), ('1', 864), ('1', 865), ('1', 866), ('1', 867), ('1', 868), ('1', 869), ('1', 870), ('2', 870), ('2', 871), ('2', 872), ('2', 873), ('2', 874), ('2', 875), ('2', 876), ('2', 877), ('2', 878), ('2', 879), ('2', 880), ('1', 871), ('1', 872), ('1', 873), ('1', 874), ('1', 875), ('1', 876), ('1', 877), ('1', 878), ('1', 879), ('1', 880), ('1', 881), ('2', 882), ('2', 883), ('2', 884), ('2', 885), ('2', 886), ('2', 887), ('2', 888), ('2', 889), ('2', 890), ('2', 891), ('2', 892), ('1', 893), ('1', 894), ('1', 895), ('1', 896), ('1', 897), ('1', 898), ('1', 899), ('1', 900), ('1', 901), ('1', 902), ('1', 903), ('2', 893), ('2', 905), ('2', 906), ('2', 907), ('2', 908), ('2', 909), ('2', 910), ('2', 911), ('2', 912), ('2', 913), ('2', 914), ('2', 915), ('1', 915), ('1', 916), ('1', 917), ('1', 918), ('1', 919), ('1', 920), ('1', 921), ('1', 922), ('1', 923), ('1', 924), ('1', 925), ('2', 927), ('2', 928), ('2', 929), ('2', 930), ('2', 931), ('2', 932), ('2', 933), ('2', 934), ('2', 935), ('2', 936), ('2', 937), ('1', 926), ('1', 938), ('1', 939), ('1', 940), ('1', 941), ('1', 942), ('1', 943), ('1', 944), ('1', 945), ('1', 946), ('1', 947), ('1', 948), ('2', 938), ('2', 939), ('2', 940), ('2', 941), ('2', 942), ('2', 943), ('2', 944), ('2', 945), ('2', 946), ('2', 947), ('2', 948), ('1', 950), ('1', 951), ('1', 952), ('1', 953), ('1', 954), ('1', 955), ('1', 956), ('1', 957), ('1', 958), ('1', 959), ('1', 960), ('2', 960), ('2', 961), ('2', 962), ('2', 963), ('2', 964), ('2', 965), ('2', 966), ('2', 967), ('2', 968), ('2', 969), ('2', 970), ('1', 961), ('1', 962), ('1', 963), ('1', 964), ('1', 965), ('1', 966), ('1', 967), ('1', 968), ('1', 969), ('1', 970), ('1', 971), ('2', 972), ('2', 973), ('2', 974), ('2', 975), ('2', 976), ('2', 977), ('2', 978), ('2', 979), ('2', 980), ('2', 981), ('2', 982), ('1', 983), ('1', 984), ('1', 985), ('1', 986), ('1', 987), ('1', 988), ('1', 989), ('1', 990), ('1', 991), ('1', 992), ('1', 993), ('2', 983), ('2', 995), ('2', 996), ('2', 997), ('2', 998), ('2', 999), ('2', 1000), ('2', 1001), ('2', 1002), ('2', 1003), ('2', 1004), ('2', 1005), ('1', 1005), ('1', 1006), ('1', 1007), ('1', 1008), ('1', 1009), ('1', 1010), ('1', 1011), ('1', 1012), ('1', 1013), ('1', 1014), ('1', 1015), ('2', 1006), ('2', 1007), ('2', 1008), ('2', 1009), ('2', 1010), ('2', 1011), ('2', 1012), ('2', 1013), ('2', 1014), ('2', 1015), ('2', 1016), ('1', 1016), ('1', 1018), ('1', 1019), ('1', 1020), ('1', 1021), ('1', 1022), ('1', 1023), ('1', 1024), ('1', 1025), ('1', 1026), ('1', 1027), ('1', 1028), ('2', 1028), ('2', 1029), ('2', 1030), ('2', 1031), ('2', 1032), ('2', 1033), ('2', 1034), ('2', 1035), ('2', 1036), ('2', 1037), ('2', 1038), ('1', 1029), ('1', 1030), ('1', 1031), ('1', 1032), ('1', 1033), ('1', 1034), ('1', 1035), ('1', 1036), ('1', 1037), ('1', 1038), ('1', 1039), ('2', 1040), ('2', 1041), ('2', 1042), ('2', 1043), ('2', 1044), ('2', 1045), ('2', 1046), ('2', 1047), ('2', 1048), ('2', 1049), ('2', 1050), ('1', 1051), ('1', 1052), ('1', 1053), ('1', 1054), ('1', 1055), ('1', 1056), ('1', 1057), ('1', 1058), ('1', 1059), ('1', 1060), ('1', 1061), ('2', 1051), ('2', 1063), ('2', 1064), ('2', 1065), ('2', 1066), ('2', 1067), ('2', 1068), ('2', 1069), ('2', 1070), ('2', 1071), ('2', 1072), ('2', 1073), ('1', 1073), ('1', 1074), ('1', 1075), ('1', 1076), ('1', 1077), ('1', 1078), ('1', 1079), ('1', 1080), ('1', 1081), ('1', 1082), ('1', 1083), ('2', 1085), ('2', 1086), ('2', 1087), ('2', 1088), ('2', 1089), ('2', 1090), ('2', 1091), ('2', 1092), ('2', 1093), ('2', 1094), ('2', 1095), ('1', 1084), ('1', 1096), ('1', 1097), ('1', 1098), ('1', 1099), ('1', 1100), ('1', 1101), ('1', 1102), ('1', 1103), ('1', 1104), ('1', 1105), ('1', 1106), ('2', 1096), ('2', 1097), ('2', 1098), ('2', 1099), ('2', 1100), ('2', 1101), ('2', 1102), ('2', 1103), ('2', 1104), ('2', 1105), ('2', 1106), ('1', 1108), ('1', 1109), ('1', 1110), ('1', 1111), ('1', 1112), ('1', 1113), ('1', 1114), ('1', 1115), ('1', 1116), ('1', 1117), ('1', 1118), ('2', 1118), ('2', 1119), ('2', 1120), ('2', 1121), ('2', 1122), ('2', 1123), ('2', 1124), ('2', 1125), ('2', 1126), ('2', 1127), ('2', 1128), ('1', 1119), ('1', 1120), ('1', 1121), ('1', 1122), ('1', 1123), ('1', 1124), ('1', 1125), ('1', 1126), ('1', 1127), ('1', 1128), ('1', 1129), ('2', 1130), ('2', 1131), ('2', 1132), ('2', 1133), ('2', 1134), ('2', 1135), ('2', 1136), ('2', 1137), ('2', 1138), ('2', 1139), ('2', 1140), ('1', 1141), ('1', 1142), ('1', 1143), ('1', 1144), ('1', 1145), ('1', 1146), ('1', 1147), ('1', 1148), ('1', 1149), ('1', 1150), ('1', 1151), ('2', 1141), ('2', 1153), ('2', 1154), ('2', 1155), ('2', 1156), ('2', 1157), ('2', 1158), ('2', 1159), ('2', 1160), ('2', 1161), ('2', 1162), ('2', 1163), ('1', 1163), ('1', 1164), ('1', 1165), ('1', 1166), ('1', 1167), ('1', 1168), ('1', 1169), ('1', 1170), ('1', 1171), ('1', 1172), ('1', 1173), ('2', 1164), ('2', 1165), ('2', 1166), ('2', 1167), ('2', 1168), ('2', 1169), ('2', 1170), ('2', 1171), ('2', 1172), ('2', 1173), ('2', 1174), ('1', 1174), ('1', 1176), ('1', 1177), ('1', 1178), ('1', 1179), ('1', 1180), ('1', 1181), ('1', 1182), ('1', 1183), ('1', 1184), ('1', 1185), ('1', 1186), ('2', 1186), ('2', 1187), ('2', 1188), ('2', 1189), ('2', 1190), ('2', 1191), ('2', 1192), ('2', 1193), ('2', 1194), ('2', 1195), ('2', 1196), ('1', 1187), ('1', 1188), ('1', 1189), ('1', 1190), ('1', 1191), ('1', 1192), ('1', 1193), ('1', 1194), ('1', 1195), ('1', 1196), ('1', 1197), ('2', 1198), ('2', 1199), ('2', 1200), ('2', 1201), ('2', 1202), ('2', 1203), ('2', 1204), ('2', 1205), ('2', 1206), ('2', 1207), ('2', 1208), ('1', 1209), ('1', 1210), ('1', 1211), ('1', 1212), ('1', 1213), ('1', 1214), ('1', 1215), ('1', 1216), ('1', 1217), ('1', 1218), ('1', 1219), ('2', 1209), ('2', 1221), ('2', 1222), ('2', 1223), ('2', 1224), ('2', 1225), ('2', 1226), ('2', 1227), ('2', 1228), ('2', 1229), ('2', 1230), ('2', 1231), ('1', 1231), ('1', 1232), ('1', 1233), ('1', 1234), ('1', 1235), ('1', 1236), ('1', 1237), ('1', 1238), ('1', 1239), ('1', 1240), ('1', 1241), ('2', 1243), ('2', 1244), ('2', 1245), ('2', 1246), ('2', 1247), ('2', 1248), ('2', 1249), ('2', 1250), ('2', 1251), ('2', 1252), ('2', 1253), ('1', 1242), ('1', 1254), ('1', 1255), ('1', 1256), ('1', 1257), ('1', 1258), ('1', 1259), ('1', 1260), ('1', 1261), ('1', 1262), ('1', 1263), ('1', 1264), ('2', 1254), ('2', 1255), ('2', 1256), ('2', 1257), ('2', 1258), ('2', 1259), ('2', 1260), ('2', 1261), ('2', 1262), ('2', 1263), ('2', 1264), ('1', 1266), ('1', 1267), ('1', 1268), ('1', 1269), ('1', 1270), ('1', 1271), ('1', 1272), ('1', 1273), ('1', 1274), ('1', 1275), ('1', 1276), ('2', 1276), ('2', 1277), ('2', 1278), ('2', 1279), ('2', 1280), ('2', 1281), ('2', 1282), ('2', 1283), ('2', 1284), ('2', 1285), ('2', 1286), ('1', 1277), ('1', 1278), ('1', 1279), ('1', 1280), ('1', 1281), ('1', 1282), ('1', 1283), ('1', 1284), ('1', 1285), ('1', 1286), ('1', 1287), ('2', 1288), ('2', 1289), ('2', 1290), ('2', 1291), ('2', 1292), ('2', 1293), ('2', 1294), ('2', 1295), ('2', 1296), ('2', 1297), ('2', 1298), ('1', 1299), ('1', 1300), ('1', 1301), ('1', 1302), ('1', 1303), ('1', 1304), ('1', 1305), ('1', 1306), ('1', 1307), ('1', 1308), ('1', 1309), ('2', 1299), ('2', 1311), ('2', 1312), ('2', 1313), ('2', 1314), ('2', 1315), ('2', 1316), ('2', 1317), ('2', 1318), ('2', 1319), ('2', 1320), ('2', 1321), ('1', 1321), ('1', 1322), ('1', 1323), ('1', 1324), ('1', 1325), ('1', 1326), ('1', 1327), ('1', 1328), ('1', 1329), ('1', 1330), ('1', 1331), ('2', 1322), ('2', 1323), ('2', 1324), ('2', 1325), ('2', 1326), ('2', 1327), ('2', 1328), ('2', 1329), ('2', 1330), ('2', 1331), ('2', 1332), ('1', 1332), ('1', 1334), ('1', 1335), ('1', 1336), ('1', 1337), ('1', 1338), ('1', 1339), ('1', 1340), ('1', 1341), ('1', 1342), ('1', 1343), ('1', 1344), ('2', 1344), ('2', 1345), ('2', 1346), ('2', 1347), ('2', 1348), ('2', 1349), ('2', 1350), ('2', 1351), ('2', 1352), ('2', 1353), ('2', 1354), ('1', 1345), ('1', 1346), ('1', 1347), ('1', 1348), ('1', 1349), ('1', 1350), ('1', 1351), ('1', 1352), ('1', 1353), ('1', 1354), ('1', 1355), ('2', 1356), ('2', 1357), ('2', 1358), ('2', 1359), ('2', 1360), ('2', 1361), ('2', 1362), ('2', 1363), ('2', 1364), ('2', 1365), ('2', 1366), ('1', 1367), ('1', 1368), ('1', 1369), ('1', 1370), ('1', 1371), ('1', 1372), ('1', 1373), ('1', 1374), ('1', 1375), ('1', 1376), ('1', 1377), ('2', 1367), ('2', 1379), ('2', 1380), ('2', 1381), ('2', 1382), ('2', 1383), ('2', 1384), ('2', 1385), ('2', 1386), ('2', 1387), ('2', 1388), ('2', 1389), ('1', 1389), ('1', 1390), ('1', 1391), ('1', 1392), ('1', 1393), ('1', 1394), ('1', 1395), ('1', 1396), ('1', 1397), ('1', 1398), ('1', 1399), ('2', 1401), ('2', 1402), ('2', 1403), ('2', 1404), ('2', 1405), ('2', 1406), ('2', 1407), ('2', 1408), ('2', 1409), ('2', 1410), ('2', 1411), ('1', 1400), ('1', 1412), ('1', 1413), ('1', 1414), ('1', 1415), ('1', 1416), ('1', 1417), ('1', 1418), ('1', 1419), ('1', 1420), ('1', 1421), ('1', 1422), ('2', 1412), ('2', 1413), ('2', 1414), ('2', 1415), ('2', 1416), ('2', 1417), ('2', 1418), ('2', 1419), ('2', 1420), ('2', 1421), ('2', 1422), ('1', 1424), ('1', 1425), ('1', 1426), ('1', 1427), ('1', 1428), ('1', 1429), ('1', 1430), ('1', 1431), ('1', 1432), ('1', 1433), ('1', 1434), ('2', 1434), ('2', 1435), ('2', 1436), ('2', 1437), ('2', 1438), ('2', 1439), ('2', 1440), ('2', 1441), ('2', 1442), ('2', 1443), ('2', 1444), ('1', 1435), ('1', 1436), ('1', 1437), ('1', 1438), ('1', 1439), ('1', 1440), ('1', 1441), ('1', 1442), ('1', 1443), ('1', 1444), ('1', 1445), ('2', 1446), ('2', 1447), ('2', 1448), ('2', 1449), ('2', 1450), ('2', 1451), ('2', 1452), ('2', 1453), ('2', 1454), ('2', 1455), ('2', 1456), ('1', 1457), ('1', 1458), ('1', 1459), ('1', 1460), ('1', 1461), ('1', 1462), ('1', 1463), ('1', 1464), ('1', 1465), ('1', 1466), ('1', 1467), ('2', 1457), ('2', 1469), ('2', 1470), ('2', 1471), ('2', 1472), ('2', 1473), ('2', 1474), ('2', 1475), ('2', 1476), ('2', 1477), ('2', 1478), ('2', 1479), ('1', 1479), ('1', 1480), ('1', 1481), ('1', 1482), ('1', 1483), ('1', 1484), ('1', 1485), ('1', 1486), ('1', 1487), ('1', 1488), ('1', 1489), ('2', 1480), ('2', 1481), ('2', 1482), ('2', 1483), ('2', 1484), ('2', 1485), ('2', 1486), ('2', 1487), ('2', 1488), ('2', 1489), ('2', 1490), ('1', 1490), ('1', 1492), ('1', 1493), ('1', 1494), ('1', 1495), ('1', 1496), ('1', 1497), ('1', 1498), ('1', 1499), ('1', 1500), ('1', 1501), ('1', 1502), ('2', 1502), ('2', 1503), ('2', 1504), ('2', 1505), ('2', 1506), ('2', 1507), ('2', 1508), ('2', 1509), ('2', 1510), ('2', 1511), ('2', 1512), ('1', 1503), ('1', 1504), ('1', 1505), ('1', 1506), ('1', 1507), ('1', 1508), ('1', 1509), ('1', 1510), ('1', 1511), ('1', 1512), ('1', 1513), ('2', 1514), ('2', 1515), ('2', 1516), ('2', 1517), ('2', 1518), ('2', 1519), ('2', 1520), ('2', 1521), ('2', 1522), ('2', 1523), ('2', 1524), ('1', 1525), ('1', 1526), ('1', 1527), ('1', 1528), ('1', 1529), ('1', 1530), ('1', 1531), ('1', 1532), ('1', 1533), ('1', 1534), ('1', 1535), ('2', 1525), ('2', 1537), ('2', 1538), ('2', 1539), ('2', 1540), ('2', 1541), ('2', 1542), ('2', 1543), ('2', 1544), ('2', 1545), ('2', 1546), ('2', 1547), ('1', 1547), ('1', 1548), ('1', 1549), ('1', 1550), ('1', 1551), ('1', 1552), ('1', 1553), ('1', 1554), ('1', 1555), ('1', 1556), ('1', 1557), ('2', 1559), ('2', 1560), ('2', 1561), ('2', 1562), ('2', 1563), ('2', 1564), ('2', 1565), ('2', 1566), ('2', 1567), ('2', 1568), ('1', 1558), ('1', 1570)] 2000
share_data result: 1571
This is the thread safety problem caused by multithreaded write operation. Specifically, this kind of thread synchronization belongs to mutual exclusion. Next, take a look at the multithreading synchronization provided by python.
The threading module provides some synchronization mechanisms for python threads. For specific usage, please refer to the documentation on the official website.
Lock: mutex lock. Only one thread can acquire the lock. The thread acquiring the lock can execute. Otherwise, it will block;
RLock: a recursive lock, also known as a reentrant lock. Threads that have acquired the lock can continue to acquire the lock multiple times without being blocked. The number of releases must be the same as the number of acquisitions before the lock can be released;
Condition: a condition variable that causes one thread to wait for another thread to meet certain conditions, such as changing state or a value. Then it will actively notify another thread and give up the lock;
Semaphore: signal lock. Provide a counter for the limited resources shared between threads. If there is no available resource, it will be blocked;
Event: event lock. Any number of threads wait for an event to occur. After the event occurs, all threads are activated;
Timer: a kind of timer (its usage is relatively simple, not including synchronization mechanism)
Mutex Lock
Its basic usage is very simple:
Create lock: Lock()
Acquire lock: acquire([blocking])
Release lock: release()
import threading
import time
lock = threading.Lock() # step 1: create mutex
share_data = 0
def tstart(arg):
time.sleep(0.1)
global share_data
if lock.acquire(): # step 2: get the mutex, otherwise block the current thread
share_data += 1
lock.release() # step 3: release the mutex
if __name__ == '__main__':
tlst = list()
for i in xrange(10):
t = threading.Thread(target=tstart, args=('',))
tlst.append(t)
for t in tlst:
t.start()
tlst[2].join()
print("This is main function at: %s" % time.time())
print 'share_data result:', share_data
Result:
This is main function at: 1564909315.86 share_data result: 7
The above result of share ﹣ u data is somewhat random, because we just wait for the second thread to finish execution, and then read the result directly and finish the main thread.
However, from the above results, we can infer that when the third thread ends and the main thread executes the result of outputting share_data, at least seven threads have completed the operation of adding 1 to share_data;
Recommend a python learning Qun: @ @ 913 × 266
Reclock RLock
After the current thread obtains the lock, it is possible to acquire the lock again before releasing the lock, resulting in deadlock. python introduces reentry locks.
import threading
import time
rlock = threading.RLock() # step 1: create a reentry lock
share_data = 0
def check_data():
global share_data
if rlock.acquire():
if share_data > 10:
share_data = 0
rlock.release()
def tstart(arg):
time.sleep(0.1)
global share_data
if rlock.acquire(): # step 2: get the reentry lock, otherwise block the current thread
check_data()
share_data += 1
rlock.release() # step 3: release the reentry lock
if __name__ == '__main__':
t1 = threading.Thread(target = tstart, args = ('',))
t1.start()
t1.join()
print("This is main function at: %s" % time.time())
print 'share_data result:', share_data
If this example uses a mutex, it will cause the current thread to block.
Semaphore
The semaphore has an initial value, which indicates the number of resources currently available. The semaphore will be added and subtracted dynamically during multithreading execution. The semaphore value at a certain time indicates the number of threads that can be added currently;
Semaphores operate in two ways:
Acquire (i.e. P operation)
Release (i.e. V operation)
The thread executing V operation is unlimited, and the thread executing P operation will be blocked when the resource is insufficient;
def get_wait_time():
return random.random()/5.0
# Resource number 0
S = threading.Semaphore(0)
def consumer(name):
S.acquire()
time.sleep(get_wait_time())
print name
def producer(name):
# time.sleep(0.1)
time.sleep(get_wait_time())
print name
S.release()
if __name__ == "__main__":
for i in xrange(5, 10):
c = threading.Thread(target=consumer, args=("consumer:%s"%i, ))
c.start()
for i in xrange(5):
p = threading.Thread(target=producer, args=("producer:%s"%i, ))
p.start()
time.sleep(2)
Here is a possible execution result:
producer:3 producer:0 consumer:6 producer:4 producer:1 consumer:5 producer:2 consumer:7 consumer:8 consumer:9
Condition
Next, let's look at another synchronization mechanism Condition, which is not very intuitive. To better view its working process, first define some functions:
def get_time():
return time.strftime("%Y-%m-%d %H:%M:%S")
def show_start_info(tname):
print '%s start at: %s' %(tname, get_time())
def show_acquire_info(tname):
print '%s acquire at: %s' % (tname, time.time())
def show_add_once_res(tname):
print '%s add: %s at: %s' % (tname, share_data, time.time())
def show_end_info(tname):
print 'End %s with: %s at: %s' % (tname, share_data, time.time())
def show_wait_info(tname):
print '%s wait at: %s' % (tname, time.time())
When the condition variable enables the thread to obtain the lock, it can actively give up the lock when the condition is not satisfied, and notify other blocked threads to wake up. The basic working process is as follows:
Create a global conditional variable object;
Each thread needs to acquire the condition variable before executing. If it gets, it will execute. Otherwise, it will block.
Some conditions will be judged during the current thread execution. If the conditions are not met, wait and release the lock actively. Calling wait will block the current thread;
After some processing changes are made, the current thread notifies and wakes up other threads through the notify method. Other threads in the wait state will judge the conditions again after receiving the notification. If the execution conditions are met, the thread will execute. Note that calling notify does not release the lock;
Repeat the process until the task is completed.
It's hard to understand here. After the current thread modifies the conditions, notify other threads through notify to check whether their respective execution conditions are met. However, the only lock held by the condition variable is owned by the current thread and has not been released. How can other threads execute?
The Python documentation provides the following instructions:
Note: an awakened thread does not actually return from its wait() call until it can reacquire the lock. Since notify() does not release the lock, its caller should.
That is to say, the notify wakeup thread will not return from its wait function and continue executing until it obtains the lock in the condition variable. The thread calling notify should actively release the lock because the notify function will not release it.
There will be a problem here. The current thread modifies the execution conditions of other threads, notifies other threads, and actively calls wait to release the lock to suspend itself. If the execution conditions of other threads are not met, all threads will block;
The following describes the use of conditional variables by printing the characters "A" and "B" alternately by two threads:
share_data, max_len = '#', 6
cond = threading.Condition()
def addA(tname):
show_start_info(tname)
cond.acquire()
time.sleep(1)
show_acquire_info(tname)
global share_data
while len(share_data) <= max_len:
if share_data[-1] != 'A':
share_data += 'A'
time.sleep(1)
cond.notify()
show_add_once_res(tname)
else:
# show_wait_info(tname)
cond.wait()
cond.release()
show_end_info(tname)
def addB(tname):
show_start_info(tname)
cond.acquire()
time.sleep(1)
show_acquire_info(tname)
global share_data
while len(share_data) <= max_len:
if share_data[-1] != 'B':
share_data += 'B'
time.sleep(1)
cond.notify()
show_add_once_res(tname)
else:
# show_wait_info(tname)
cond.wait()
cond.release()
show_end_info(tname)
if __name__ == "__main__":
t1 = threading.Thread(target=addA, args=("Thread 1", ))
t2 = threading.Thread(target=addB, args=("Thread 2", ))
t1.start()
t2.start()
t1.join()
t2.join()
print "share_data:", share_data
Result:
Thread 1 start at: 2019-08-10 17:47:54 Thread 2 start at: 2019-08-10 17:47:54 Thread 1 acquire at: 1565430475.68 Thread 1 add: #A at: 1565430476.68 Thread 2 acquire at: 1565430477.68 Thread 2 add: #AB at: 1565430478.68 Thread 1 add: #ABA at: 1565430479.68 Thread 2 add: #ABAB at: 1565430480.68 Thread 1 add: #ABABA at: 1565430481.68 Thread 2 add: #ABABAB at: 1565430482.68 End Thread 2 with: #ABABAB at: 1565430482.68 End Thread 1 with: #ABABAB at: 1565430482.68 share_data: #ABABAB //Copy code
In the result, we can see the process of double thread execution and the time node.
Event Event
Finally, let's look at a simple and crude way to synchronize threads: Event
The core of this approach is to use events to control the state of a global variable: True or False. In the process of thread execution, judge the value of variable first, and execute when it is True, otherwise call wait to block itself;
When the state of the global variable is set to True, all threads calling wait will wake up and enter the blocking state; when all threads need to be paused, use clear to set the global variable to False;
Next, use a three round game with two players rolling dice and a referee judging the outcome to demonstrate the use of event;
E = threading.Event()
E.clear()
res1, res2, cnt, lst = 0, 0, 3, ("player2", 'both', 'player1')
def show_round_res():
print ("Stop! judging... %s win!" % lst[cmp(res1, res2) + 1])
def judge():
global cnt
while cnt > 0:
print 'start game!'
E.set()
time.sleep(1)
E.clear()
show_round_res()
time.sleep(1)
cnt -= 1
print "game over by judge!"
def player1():
global res1
while cnt > 0:
if E.is_set():
res1 = random.randint(1, 6)
print "player1 get %d" % res1
time.sleep(1.5)
E.wait()
print "player1 quit!"
def player2():
global res2
while cnt > 0:
if E.is_set():
res2 = random.randint(1, 6)
print "player2 get %d" % res2
time.sleep(1.5)
E.wait()
print "player2 quit!"
if __name__ == "__main__":
t1 = threading.Thread(target=judge, args=( ))
t2 = threading.Thread(target=player1, args=( ))
t3 = threading.Thread(target=player2, args=( ))
t1.start()
t2.start()
t3.start()
t1.join()
E.set()
It should be noted that when a thread calls wait, it will block the current thread only when the variable value is False. If the global variable is True, it will return immediately;
Here is a possible result:
start game! player1 get 2 player2 get 2 Stop! judging... both win! start game! player2 get 3 player1 get 4 Stop! judging... player1 win! start game! player1 get 1 player2 get 3 Stop! judging... player2 win! game over by judge! player2 quit! player1 quit!
summary
The above are some synchronization measures taken to solve the thread safety problem. Python provides thread like synchronization measures for process synchronization, such as locks, semaphores, etc.
Different from sharing process resources among threads, processes have independent address space and different process memory space is isolated.
