Python GIL stands for global interpreter lock. As the name suggests, GIL is the lock to get control of the python interpreter.
GIL ensures that only one thread is in the execution state at any point of the time.
Memory management in Python is done using reference counting however, the real challenge is to protect the reference count variable against the race conditions so that no two threads increase or decrease reference counts value simultaneously.
Otherwise, there may be cases of memory leaks, or worst case the memory will be released for objects still having the reference alive.
As per the experiment carried out by https://www.dabeaz.com/python/UnderstandingGIL.pdf
def countdown(n): while n > 0: n -= 10 #Run it once with a lot of work
COUNT = 100000000 # 100 million countdown(COUNT) #Now, subdivide the work across two threads
t1 = Thread(target=countdown,args=(COUNT//2)) t2 = Thread(target=countdown,args=(COUNT//2)) t1.start() t2.start() t1.join() t2.join()
- Performance if work divided across 4 threads
- Performance if all but one CPU is disabled : 7.8s : 15.4s (2X slower!)
- Threaded (4 threads) : 15.7s (about the same) Threaded (2 threads)
- Threaded (4 threads) : 11.3s (~35% faster than running : 11.6s with all 4 cores)
The main reason behind above result is GIL.
GIL restricts interpreter to run just one thread at a time and doesn’t allow parallel execution.
GIL old implementation
- Thread needs to acquire GIL to start its execution.
- When the thread is waiting for any I/O event(read, write, etc.), it releases the GIL.
- The thread needs to wait for the GIL in the Ready state to execute again
- For CPU related tasks, context switch happens every “100 ticks”.
GIL new implementation (Python 3.2)
- Unlike old implementations, ticks are replaced with global variable.
- Thread executes until the value of variable is reduced to 1.
- If there is only one thread, then it will continue to execute.
- If there are more than one threads, they will wait for the other thread to release GIL
- As soon as the value of the variable is reduced to 1, a signal will be sent to the currently executing thread to release GIL.
- Other thread will acquire the GIL and will sent the Ack for the same.
However, the above implementation changes have not made any significant improvement in execution time and have also affected the I/O operation time.
Areas of improvement
- There should be some way to preempt the execution of the low priority threads in-case of high priority thread is waiting for the GIL in the ready state.
- There should be two separate queues maintained for threads based on the tasks they are executing, i.e CPU and I/O tasks.
That’s it. Thanks for reading!
About CauseCode: We are a technology company specializing in Healthtech related Web and Mobile application development. We collaborate with passionate companies looking to change health and wellness tech for good. If you are a startup, enterprise or generally interested in digital health, we would love to hear from you! Let's connect at email@example.com