Use with statement

Forgetting to close a file or release a database connection can lead to resource leaks, corrupted data, or even crashed applications. Python's with statement provides a clean, safe, and automatic way to manage resources — ensuring they are properly cleaned up even when errors occur.

The problem this lesson solves

When working with external resources like files, network sockets, or database connections, you must release them after use. Failing to do so can exhaust system limits (file descriptors, memory) or leave locked files. The traditional approach — manually calling close() — is error-prone: an exception after open() but before close() leaves the resource dangling.

# Without 'with' — error-prone
file = open('data.txt', 'w')
file.write('Hello')
# What if an exception happens here?
file.close()  # Never runs if exception!

The with statement solves this by guaranteeing cleanup, no matter how the block exits.

Core concept / mental model

Think of a context manager as a resource guard that you enter and automatically leave. The with statement:

1. Enters the context (acquires the resource).
2. Executes your code block.
3. Exits the context (releases the resource) — even if an exception occurs.

Pro tip: Any object that implements the context manager protocol (methods __enter__ and __exit__) can be used with with. Python's built-in open() function returns a file object that does exactly this.

Key vocabulary

• Context manager: Object that defines __enter__ and __exit__.
• Cleanup: The release of resources (close file, release lock, close socket).
• Exception safety: The guarantee that cleanup happens even on errors.

How it works step by step

When you write:

with expression as variable:
    # code block

Python does the following in order:

1. Evaluate the expression (e.g., open('file.txt')) to get a context manager.
2. Call the context manager's __enter__() method. Its return value is bound to variable (optional if you use as).
3. Execute the code block.
4. Call the context manager's __exit__() method — regardless of whether the block succeeded or raised an exception. If an exception occurred, __exit__ receives the exception type, value, and traceback.
5. Propagates the exception, unless __exit__ returns True (which suppresses it).

Visual example

class ManagedFile:
    def __init__(self, filename, mode):
        self.filename = filename
        self.mode = mode

    def __enter__(self):
        self.file = open(self.filename, self.mode)
        return self.file  # bound to 'as' variable

    def __exit__(self, exc_type, exc_val, exc_tb):
        if self.file:
            self.file.close()
        # Log error if needed
        print(f"Exiting. Exception? {exc_type is not None}")

with ManagedFile('test.txt', 'w') as f:
    f.write('Hello!')
    # raise Exception('Oops')  # Uncomment to test

print('File closed automatically.')

# Output (no exception):
# Exiting. Exception? False
# File closed automatically.

Hands-on walkthrough

Example 1: Working with files

# Write to a file safely
with open('log.txt', 'w') as f:
    f.write('2025-01-01: App started\n')
    f.write('2025-01-01: User logged in\n')
# At this point, file is automatically closed

# Read the file back
with open('log.txt', 'r') as f:
    content = f.read()

print(content)

Expected output:

2025-01-01: App started
2025-01-01: User logged in

Example 2: Multiple resources with nested with

with open('source.txt', 'r') as src:
    with open('dest.txt', 'w') as dest:
        for line in src:
            dest.write(line.upper())

Alternatively, combine them in one line (Python 3.10+):

with open('source.txt', 'r') as src, open('dest.txt', 'w') as dest:
    for line in src:
        dest.write(line.upper())

Example 3: Locking with threading.Lock

import threading

lock = threading.Lock()
shared_resource = []

def worker(item):
    with lock:  # Automatically acquire and release
        shared_resource.append(item)
        # Critical section

threads = [threading.Thread(target=worker, args=(i,)) for i in range(5)]
for t in threads:
    t.start()
for t in threads:
    t.join()

print(shared_resource)  # [0, 1, 2, 3, 4] (order depends)

Example 4: Custom context manager using contextlib

from contextlib import contextmanager

@contextmanager
def temp_file(filename):
    print('Creating temp file...')
    f = open(filename, 'w')
    try:
        yield f
    finally:
        f.close()
        print('Temp file closed.')

with temp_file('temp.txt') as f:
    f.write('Temporary data')

# Output:
# Creating temp file...
# Temp file closed.

Compare options / when to choose what

When to choose what: - Always prefer with for any resource that supports it (files, locks, database connections, subprocesses, sockets). - Use contextlib.contextmanager when you need a custom resource manager without writing a full class. - Avoid manual try/finally unless you're dealing with an object that doesn't implement the context manager protocol.

Troubleshooting & edge cases

1. Forgetting as when you need the resource

# Wrong — you can't use the file object
with open('data.txt', 'r'):  # No 'as'
    content = file.read()  # NameError: name 'file' is not defined

# Correct
with open('data.txt', 'r') as f:
    content = f.read()

2. Suppressing exceptions in custom __exit__

class SilentExit:
    def __enter__(self):
        return self
    def __exit__(self, exc_type, exc_val, exc_tb):
        return True  # Suppresses any exception

with SilentExit():
    raise ValueError("Oops")
print("Exception was swallowed!")  # Runs – exception silenced

Fix: Only suppress exceptions deliberately (e.g., when logging is sufficient).

3. File not closed when with block exits via break in a loop

for i in range(5):
    with open('data.txt', 'w') as f:
        f.write(str(i))
        if i == 2:
            break  # File still closed correctly

4. Resource not released if __enter__ raises

If open() itself fails (e.g., permission denied), __exit__ is not called because the context was never entered. This is correct — no resource was acquired.

What you learned & what's next

You now understand: - What the with statement is and the problem it solves. - How context managers work under the hood (__enter__ / __exit__). - How to use with for files, locks, and custom resources. - How to choose between with, try/finally, and contextlib. - How to troubleshoot common pitfalls.

Next step: In the upcoming lesson, you'll learn about exception handling with try/except — the mechanism Python uses to gracefully recover from runtime errors. Together with with, you'll be able to write robust, production-ready code.