Threading Basics

Your Python program runs line by line, top to bottom. That's great for simple tasks, but what happens when you need to download five files at once, or handle a hundred user requests without making each one wait in line? The program becomes slow and unresponsive. This lesson solves that problem by introducing threading — Python's built-in tool for handling concurrency. You'll learn to run multiple operations in parallel, making your programs faster and more efficient, all without leaving the comfort of the threading module.

The Problem This Lesson Solves

Imagine you're building a web scraper that fetches data from ten different APIs. Your current code does this:

import time

def fetch_data(url):
    print(f"Starting {url}")
    time.sleep(2)  # Simulating network delay
    print(f"Finished {url}")
    return f"Data from {url}"

start = time.time()
results = []
for url in ["api1", "api2", "api3"]:
    results.append(fetch_data(url))

print(f"Total time: {time.time() - start:.2f} seconds")

Output:

Starting api1
Finished api1
Starting api2
Finished api2
Starting api3
Finished api3
Total time: 6.01 seconds

Each API call waits for the previous one to finish, even though the computer could be working on multiple calls simultaneously. This is sequential execution, and it wastes resources. Your program is I/O-bound waiting for network responses, but it's doing no useful work during those waits.

Pro tip: Concurrency isn't just about speed. It's about responsiveness. A GUI app that uses threading won't freeze during a file save. A web server handling multiple requests stays snappy.

Core Concept / Mental Model

Think of a Python program as a single worker in a factory. That worker can only do one thing at a time. Threading adds more workers. Each worker (a thread) can run independently, sharing the same workspace (memory and resources).

• Sequential execution: You hire one worker; they do Task A, then Task B, then Task C.
• Threaded execution: You hire three workers; each takes one task, and they work simultaneously.

Key Definitions

• Thread: The smallest unit of execution within a process. A Python process starts with one main thread. You can spawn additional threads.
• Concurrency: Multiple threads making progress. They may not run at the exact same instant (due to the GIL), but appear to run simultaneously.
• Global Interpreter Lock (GIL): Python's internal lock that allows only one thread to execute Python bytecode at a time. This means threading is best for I/O-bound tasks (waiting for network, disk, user input), not CPU-bound tasks (like video encoding).

Mental model: Threading is like having multiple tabs open in a web browser. Each tab loads a page independently. Your computer's CPU switches between tabs so fast that they all seem to load at once.

How It Works Step by Step

Creating and using threads in Python involves a few straightforward steps:

Step 1: Import the threading module

import threading

Step 2: Define a function for each thread to run

This is the work each thread will perform. It must be callable (a function or a class with __call__).

def worker(name):
    print(f"Thread {name}: starting")
    # Simulate work
    import time
    time.sleep(1)
    print(f"Thread {name}: finished")

Step 3: Create thread objects

thread1 = threading.Thread(target=worker, args=("A",))
thread2 = threading.Thread(target=worker, args=("B",))

• target: The callable to run.
• args: A tuple of positional arguments.

Step 4: Start the threads

thread1.start()
thread2.start()

This schedules the threads to run. They won't block each other — execution starts immediately.

Step 5: Wait for threads to finish (join)

thread1.join()
thread2.join()

join() blocks the main thread until the specified thread completes. Without this, the main program could exit before threads finish.

Step 6: Run orchestrated code

print("All threads completed")

Pro tip: Always join() threads if you need to use their results later. If you just fire and forget (no join), main might exit prematurely, terminating orphaned threads.

Hands-On Walkthrough

Let's build a program that downloads multiple files concurrently. We'll simulate downloads with time.sleep.

Example 1: Basic threading with I/O-bound work

import threading
import time

def download_file(file_id):
    """Simulate downloading a file from a server."""
    print(f"Downloading file {file_id}...", end=" ")
    time.sleep(2)  # Simulate network delay
    print(f"File {file_id} complete.")

# Create threads for three file downloads
threads = []
for i in range(1, 4):
    t = threading.Thread(target=download_file, args=(i,))
    threads.append(t)
    t.start()

# Wait for all threads
for t in threads:
    t.join()

print("All downloads finished. Total time saved!")

Output:

Downloading file 1... Downloading file 2... Downloading file 3... File 1 complete.
File 2 complete.
File 3 complete.
All downloads finished. Total time saved!

Notice how the "Downloading" messages appear almost together, then the completions happen 2 seconds later. Total runtime ~2 seconds, not 6.

Example 2: Thread with return values using a list

Threads don't directly return values. Use a shared data structure like a list.

import threading
import time

results = []

def compute_square(n):
    time.sleep(1)
    results.append(n ** 2)

threads = []
for n in [2, 3, 5]:
    t = threading.Thread(target=compute_square, args=(n,))
    threads.append(t)
    t.start()

for t in threads:
    t.join()

print("Squares:", results)

Output:

Squares: [4, 9, 25]

Example 3: Threading with a class (alternate pattern)

import threading

class MyThread(threading.Thread):
    def __init__(self, name):
        super().__init__()
        self.name = name

    def run(self):
        print(f"Thread {self.name} is running")
        import time
        time.sleep(0.5)
        print(f"Thread {self.name} done")

thread = MyThread("Custom")
thread.start()
thread.join()

This is useful when you need to store thread-specific state beyond simple function arguments.

Compare Options / When to Choose What

For most beginners, ThreadPoolExecutor is simpler: no manual join(), built-in result handling. But raw Thread gives you fine-grained control.

Pro tip: If you're doing multiple HTTP requests, use requests with ThreadPoolExecutor. It's much cleaner than raw threading.

Troubleshooting & Edge Cases

Common Mistake: Not joining threads

t = threading.Thread(target=worker, args=(1,))
t.start()
# Missing t.join()
# Main thread might exit before worker finishes
print("Done")
# Output: Done (then worker may be cut off before completing)

Fix: Always call thread.join() before relying on thread work.

Edge Case: Race conditions

When multiple threads access shared variables without synchronization:

counter = 0

def increment():
    global counter
    for _ in range(100000):
        counter += 1

threads = [threading.Thread(target=increment) for _ in range(2)]
for t in threads:
    t.start()
for t in threads:
    t.join()
print(counter)  # Might be less than 200000!

Fix: Use a threading.Lock to protect access.

Error: Thread is daemonic

By default, threads are not daemons. If a thread is still running when main exits, Python waits for it. If you set daemon=True, threads are killed when main exits — useful but dangerous.

def loop_forever():
    while True:
        pass

t = threading.Thread(target=loop_forever, daemon=True)
t.start()  # Program exits immediately

Error: Trying to start a thread twice

t = threading.Thread(target=worker)
t.start()
t.start()  # RuntimeError: threads can only be started once

Fix: Create a new thread object for each run.

What You Learned & What's Next

You've now mastered the basics of handling concurrency with threading basics in Python. Specifically, you:

• Understood the problem threading solves: making I/O-bound programs faster.
• Learned the mental model: threads are workers sharing memory.
• Implemented threads with threading.Thread: create, start, join.
• Compared threading with alternatives like ThreadPoolExecutor and asyncio.
• Recognized common pitfalls: missing joins, race conditions, daemon threads.

These skills let you write Python programs that handle multiple tasks concurrently — scraping websites, processing user requests, or loading files without blocking the main flow.

Next up: You'll explore thread synchronization — using locks to prevent race conditions, and queues to safely pass data between threads. This deepens your ability to handle concurrency with threading basics in robust, production-level code.