Custom Iterators: __iter__ & __next__

Have you ever needed to iterate over something that isn't a simple list—like a sequence of numbers from a range you define at runtime, or a custom data structure—and found yourself writing clunky loops or building intermediate lists? Python's for loop works seamlessly with built-in types, but to make your own objects iterable you need to implement two special methods: __iter__ and __next__. By the end of this lesson you'll be able to design your own reusable iterators that integrate perfectly with Python's iteration protocol.

The problem this lesson solves

Imagine you're simulating a sensor that reads temperatures from a file, one reading every second. You could store all readings in a list and iterate over that—but that consumes memory for the whole dataset, and the sensor might produce readings indefinitely. Python's built-in list or range won't fit your custom logic. Without __iter__ and __next__ you'd have to manually track state with external variables and break conditions. This lesson replaces that fragile approach with a clean, reusable class that implements the iterator protocol.

Core concept / mental model

An iterator is an object that produces values one at a time on demand. Think of it like a bookshelf where you remove books one by one—you can't go back without starting over, and you don't load all books into your hands at once. In Python, the iteration protocol requires two methods:

• __iter__ – returns the iterator object itself (it's often just return self) and signals “I am ready to iterate.”
• __next__ – returns the next value in the sequence. When there are no more values, it must raise the built-in StopIteration exception to tell the for loop to stop.

This is Python's way of demanding a consistent contract: any object that implements these two methods can be used directly in a for loop.

How it works step by step

1. Define a class that has at least __iter__ and __next__ methods.
2. Initialize state in __init__ (e.g., a counter, a reader, or an index).
3. In __iter__, return self (or a separate iterator object if needed).
4. In __next__, update the state, check for exhaustion, and either return the next value or raise StopIteration.
5. Use the class in a for loop—Python calls __iter__ once, then repeatedly calls __next__ until StopIteration is raised.

Pro tip: If you forget to raise StopIteration, the loop will never end—a classic infinite-loop bug.

Hands-on walkthrough

Let's build a CountDown iterator that counts down from a given number.

class CountDown:
    def __init__(self, start):
        self.current = start

    def __iter__(self):
        return self

    def __next__(self):
        if self.current <= 0:
            raise StopIteration
        value = self.current
        self.current -= 1
        return value

# Usage
for num in CountDown(5):
    print(num)

Expected output:

5
4
3
2
1

Now a more practical Fibonacci iterator that generates an infinite sequence—memory efficient and lazy.

class Fibonacci:
    def __init__(self):
        self.a, self.b = 0, 1

    def __iter__(self):
        return self

    def __next__(self):
        self.a, self.b = self.b, self.a + self.b
        return self.a

# Print first 10 Fibonacci numbers
fib = Fibonacci()
for _ in range(10):
    print(next(fib))

Expected output:

1
1
2
3
5
8
13
21
34
55

Notice we used next(fib) manually here, which calls __next__ directly.

What if you need to iterate from the beginning again? The iterator protocol requires each __iter__ call to start fresh. We can create a separate iterable class that returns a new iterator each time:

class CountDownList:
    def __init__(self, start):
        self.start = start

    def __iter__(self):
        return CountDown(self.start)

# Now we can use it in multiple loops
cdl = CountDownList(3)
for num in cdl:
    print(num)
print("---")
for num in cdl:
    print(num)

Expected output:

3
2
1
---
3
2
1

Each for loop calls __iter__, which returns a fresh CountDown instance.

Compare options / when to choose what

• Generator functions (def gen(): yield ...) are often preferred for simple cases because Python automatically creates the __iter__ and __next__ machinery.
• Class-based iterators are better when you need to reset, implement bidirectional iteration, or wrap complex I/O (like a file reader that skips lines based on state).
• iter(callable, sentinel) is great for reading from a socket until a delimiter.

Troubleshooting & edge cases

1. StopIteration not raised – The loop runs forever. Always ensure your exit condition eventually becomes True and then raise StopIteration.
2. Iterator consumed once – A standard iterator is one-shot. If you need to iterate multiple times, either create a new iterator each time (by returning a fresh instance from __iter__) or use an iterable pattern as shown above.
3. Holding onto resources – If your iterator opens files or network connections, consider making it a context manager as well so it can be used with with.
4. Modifying the underlying data – If your iterator traverses a mutable collection, changes during iteration can cause StopIteration to be raised early or skip items. Either work on a copy or document this behavior.

What you learned & what's next

You now understand the iterator protocol and can create custom iterators with __iter__ and __next__. You've seen how to build stateful iterators (CountDown, Fibonacci) and reusable iterables (CountDownList). You can choose between class-based iterators and generators based on your needs.

Next up: you'll explore generators—Python's concise way to create iterators using yield—which often require even less code but are equally powerful.

Practice recap: Build a PowerTwo iterator that yields infinite powers of two (1, 2, 4, 8, ...). Add a max_limit parameter so it stops when the next value exceeds that limit. Use it in a for loop and verify the output.