Operate on files with os and pathlib

You've been writing Python for a few weeks now—you know how to create variables, write loops, and define functions. But when you try to read a file, rename a directory, or check if a path actually exists, you might find yourself writing clunky string-based path construction or hitting confusing FileNotFoundError exceptions. Python's os and pathlib modules were designed to solve exactly that pain, giving you a clean, cross-platform way to interact with the file system. By the end of this lesson, you'll confidently create, rename, delete, and query files and directories without fighting with paths again.

The problem this lesson solves

Many Python developers start by hacking together file paths as raw strings: 'data/../myfile.txt'. This approach breaks on Windows (backslashes vs. forward slashes), silently fails if a file doesn't exist, and makes code hard to read. Worse, manual string manipulation often leads to bugs like double slashes, missing parent directories, or accidental overwrites. The os module has been Python's workhorse for decades, but it returns raw strings. The newer pathlib (available since Python 3.4, fully recommended in 3.10+) provides an object-oriented, intuitive interface. Mastering both gives you the flexibility to work with legacy codebases (os) and write modern, clean scripts (pathlib).

Core concept / mental model

Think of a file path as an address, not just a string. Your house's address isn't a blob of text—it's structured: street, city, zip code. Similarly, pathlib.Path objects know their components (parent, stem, suffix) and methods (.exists(), .rename()). The os module treats files like a low-level system tool, while pathlib treats them like objects you can manipulate.

Pro Tip: If you're starting a new project, favor pathlib. It's more readable, easier to chain operations, and handles cross-platform differences automatically. Use os when you need to interact with legacy code or very low-level system calls (like process permissions).

Key definitions: - Absolute path: Full address from the root of the filesystem, e.g., /home/user/project/data.csv or C:\Users\user\project\data.csv. - Relative path: Address relative to the current working directory, e.g., data.csv or ../data.csv. - Parent directory: The folder containing the file/directory. - Suffix / extension: The part after the last dot, e.g., .txt, .py.

How it works step by step

1. Import the right module. pathlib is part of the standard library; just write from pathlib import Path. For os, write import os (or import os.path for common path functions).
2. Create a Path object. Use Path('some/path') or one of the class methods like Path.home() or Path.cwd().
3. Check existence and type. Call .exists() to see if the path points to something on disk; .is_file() or .is_dir() to distinguish.
4. Read, write, or delete. Use .read_text(), .write_text(), .unlink() for files; .mkdir(), .rmdir(), .rename() for directories.
5. Walk or glob. Use .iterdir() or .glob('*.txt') to list contents.

With os, the workflow is similar but function-based: os.path.exists(path), os.rename(src, dst), os.remove(path).

Hands-on walkthrough

Let's see both approaches in action. First, pathlib—modern and clean.

from pathlib import Path

# Create a path to a new file in the current directory
path = Path('examples/hello.txt')
print('Path object:', path)

# Create parent directories if they don't exist
path.parent.mkdir(parents=True, exist_ok=True)

# Write text to the file
path.write_text('Hello, PythonSkillset!')
print('File written successfully.')

# Check if it exists
print('Exists?', path.exists())  # True
print('Is a file?', path.is_file())  # True
print('File size:', path.stat().st_size, 'bytes')

# Rename the file
new_path = path.with_stem('greeting')  # changes stem but keeps suffix
path.rename(new_path)
print('Renamed to:', new_path)

# Read the content back
content = new_path.read_text()
print('Content:', content)

# Delete it
new_path.unlink()
print('Deleted.')

Expected output:

Path object: example.txt
File written successfully.
Exists? True
Is a file? True
File size: 22 bytes
Renamed to: greeting.txt
Content: Hello, PythonSkillset!
Deleted.

Now the equivalent with os and os.path:

import os

path = 'examples_os/hello.txt'
os.makedirs(os.path.dirname(path), exist_ok=True)

with open(path, 'w') as f:
    f.write('Hello, PythonSkillset!')

print('Exists?', os.path.exists(path))
print('Is file?', os.path.isfile(path))
print('Size:', os.path.getsize(path), 'bytes')

# Rename
new_path = 'examples_os/greeting.txt'
os.rename(path, new_path)
print('Renamed to:', new_path)

# Read
with open(new_path, 'r') as f:
    content = f.read()
print('Content:', content)

# Delete
os.remove(new_path)
print('Deleted.')

Expected output (similar):

Exists? True
Is file? True
Size: 22 bytes
Renamed to: examples_os/greeting.txt
Content: Hello, PythonSkillset!
Deleted.

Notice how pathlib makes chaining and object introspection more natural. Also, path.mkdir(parents=True) is cleaner than os.makedirs(os.path.dirname(path)).

Let's explore a directory walk with glob:

from pathlib import Path

# Recursively find all Python files in current directory
for py_file in Path.cwd().rglob('*.py'):
    print(py_file.name, '-', py_file.stat().st_size, 'bytes')

Expected output (example):

main.py - 1024 bytes
utils.py - 512 bytes

And with os:

import os

for root, dirs, files in os.walk('.'):
    for f in files:
        if f.endswith('.py'):
            full = os.path.join(root, f)
            size = os.path.getsize(full)
            print(f, '-', size, 'bytes')

Compare: pathlib.rglob('*.py') versus manual string matching—pathlib wins on readability.

Compare options / when to choose what

Quick rule: Use pathlib for 80% of your file operations (path construction, existence checks, reads/writes, listing). Fall back to os when you need os.chdir(), os.environ, or low-level file descriptors.

Troubleshooting & edge cases

• FileNotFoundError: Always check .exists() before .read_text() or .unlink(). Use .mkdir(exist_ok=True) to avoid FileExistsError.
• Permission denied (Windows): After .write_text(), the file may be locked. Close all editors pointing to it before renaming/deleting.
• Symbolic links: .is_file() returns True for symlinks that point to a file. Use .is_symlink() to detect them. os.path.islink() works similarly.
• Relative vs absolute: If you pass a relative path to Path(), it's relative to the current working directory, which might change if you call os.chdir(). Use Path.resolve() to get the absolute path.
• Empty directories: .rmdir() only works on empty directories. Use shutil.rmtree() for non-empty ones (or os.rmdir with os.scandir).

What you learned & what's next

You now understand that operate on files with os and pathlib is not just about random I/O—it's about choosing the right tool for the context. You practiced creating, reading, renaming, and deleting files with both modules, and learned to avoid common pitfalls like missing directories or permission errors. You can now explain the core idea: pathlib treats paths as objects, os treats them as strings, and both are cross-platform if used correctly.

Your next lesson builds on this foundation: Serializing data with json and pickle. You'll take the file operations you mastered here and apply them to read/write structured data, preparing you for configuration files, API responses, and data persistence in real Python projects.

Clap yourself on the back—you've just leveled up your Python filesystem skills! Now go practice: try writing a script that backs up all .txt files from one folder to another using pathlib.