Python Notes

Global Interpreter Lock (GIL)

The Global Interpreter Lock (GIL) is not what people think it is. I often see code like this:

# The following sleep is here only to allow other threads the
# opportunity to grab the Python GIL. (see pyepics/pyepics#171)
time.sleep(0)

The GIL doesn't work like that. CPU intensive Python threads do preempt each other. Here's the code:

import threading, time

def _loop(inc):
    global v

    while True:
        y = 1
        for x in range(100000):
            y += x
        v += inc

v = 1
threading.Thread(target=_loop, args=(1,)).start()
threading.Thread(target=_loop, args=(100000000,)).start()
while True:
    time.sleep(1)
    print(v)

The output looks like:

$ python cpu_bound.py
6100000069
12800000134
19700000197
25900000264

You can see both increments happening on the same variable.

The GIL does prevents Python from executing in parallel, but unlike (asyncio) coroutines, Python threads do preempt one another so any code that is compute intensive (aka high throughput or high performance computing) should run in a separate Python interpreter. Most applications are not compute intensive. Asynchronous applications, e.g. user interfaces or EPICS control systems, do require preemptable concurrency, which Python threading supports very well.

in consumes an iterable

For user-defined classes which do not define __contains__() but do define __iter__(), x in y is True if some value z, for which the expression x is z or x == z is true, is produced while iterating over y.

private note