? Python Concurrency with Threads

⚡ Quick Overview

Threads allow a Python program to perform multiple tasks seemingly at the same time within a single process. They are lightweight units of execution that share the same memory space. In CPython, the Global Interpreter Lock (GIL) means only one thread executes Python bytecode at a time, but threads are still very useful for I/O-bound workloads such as network calls, disk access, or waiting for user input.

In everyday Python, you will usually work with threads via the threading module or concurrent.futures.ThreadPoolExecutor.

? Key Concepts

• Process vs Thread – A process has its own memory; threads live inside a process and share memory.
• Concurrency vs Parallelism – Concurrency is dealing with many tasks at once; parallelism is actually doing many at the same instant.
• Global Interpreter Lock (GIL) – Only one thread can execute Python bytecode at a time in CPython.
• I/O-bound Tasks – Spend most of the time waiting (network, disk, APIs). Threads work great here.
• CPU-bound Tasks – Heavy computation (math, loops). Threads usually do not speed these up in CPython.
• Race Condition – Multiple threads updating shared data in a conflicting way.
• Lock / Mutex – An object used to guard critical sections and prevent race conditions.
• Thread Life Cycle – Create → Start → Run → (optionally) Join → Finish.

? Syntax and Theory

There are two common ways to create threads in Python:

1. Pass a target function to threading.Thread.
2. Subclass threading.Thread and override run().

Basic pattern using threading.Thread with a target function:

import threading

def worker(name):
    print(f"Starting task for {name}")
    # Do some work here...
    print(f"Finished task for {name}")

# Create thread objects
t1 = threading.Thread(target=worker, args=("Thread-1",))
t2 = threading.Thread(target=worker, args=("Thread-2",))

# Start threads (they run concurrently)
t1.start()
t2.start()

# Wait for both threads to finish
t1.join()
t2.join()

print("All work done!")

Thread-safe update using a lock:

import threading
import time

counter = 0
lock = threading.Lock()

def increment_many(times):
    global counter
    for _ in range(times):
        # Only one thread can enter this block at a time
        with lock:
            current = counter
            # Simulate some processing delay
            time.sleep(0.0001)
            counter = current + 1

threads = []
for _ in range(5):
    t = threading.Thread(target=increment_many, args=(100,))
    threads.append(t)
    t.start()

for t in threads:
    t.join()

print("Final counter value:", counter)  # Always 500 when protected by lock

? Code Examples

The following example simulates downloading multiple files using threads. It is I/O-bound (we use time.sleep to mimic network delay), so threads can overlap waiting time.

import threading
import time

def download_file(file_id, delay):
    print(f"[{file_id}] Starting download...")
    time.sleep(delay)  # Simulate network delay
    print(f"[{file_id}] Download complete!")

start = time.time()

threads = []
files = [("file-1", 2), ("file-2", 3), ("file-3", 1)]

for file_id, delay in files:
    t = threading.Thread(target=download_file, args=(file_id, delay))
    threads.append(t)
    t.start()

# Wait for all downloads to complete
for t in threads:
    t.join()

end = time.time()
print(f"All downloads finished in {end - start:.2f} seconds")

Using a thread pool with ThreadPoolExecutor is often cleaner:

from concurrent.futures import ThreadPoolExecutor, as_completed
import time

def fetch_page(url):
    print(f"Fetching {url}...")
    time.sleep(1)  # Simulate network delay
    return f"Content of {url}"

urls = [
    "https://example.com",
    "https://example.org",
    "https://example.net",
]

start = time.time()

with ThreadPoolExecutor(max_workers=3) as executor:
    futures = [executor.submit(fetch_page, url) for url in urls]

    for future in as_completed(futures):
        print("Result:", future.result())

end = time.time()
print(f"Fetched all pages in {end - start:.2f} seconds")

? Live Output and Explanation

? Practical Use Cases

• Making multiple HTTP API calls at once instead of one by one.
• Handling multiple client connections in a simple socket server.
• Background tasks such as logging, monitoring, or updating UI while main work continues.
• File uploads/downloads, or any work dominated by waiting for I/O.