Iterators and Generators

Iterators and generators provide a powerful way to work with sequences in Python. They allow programs to access elements one at a time without storing the entire sequence in memory. This makes them useful for handling large datasets, streaming data, or building custom sequence logic.

Iterators follow a standard protocol in Python, while generators offer a simpler and more efficient way to create iterators using the yield keyword. Together, they help improve performance and readability in many applications.

1. What Is an Iterator

An iterator is an object that allows sequential access to elements in a collection, one element at a time. Python uses iterators internally in loops such as for, list comprehensions, and many built-in functions.

An iterator must implement two methods:

• __iter__() — returns the iterator object
• __next__() — returns the next value and raises StopIteration when no items remain

Example: Basic Iterator

numbers = [10, 20, 30]
it = iter(numbers)

print(next(it))
print(next(it))
print(next(it))

Output:

10
20
30

If we call next(it) again, Python raises StopIteration.

2. Using Iterators with Built-in Objects

Many Python objects such as lists, tuples, dictionaries, sets, and strings are iterable. They can be passed to iter() to obtain an iterator.

Example: Iterator on a String

s = "Python"
it = iter(s)

print(next(it))
print(next(it))

Output:

P
y

The for loop internally uses this iterator mechanism.

3. Creating a Custom Iterator

A custom iterator is created by defining a class that implements the iterator protocol.

Example: Custom Iterator for Counting

class Counter:
   def __init__(self, limit):
       self.limit = limit
       self.current = 1

   def __iter__(self):
       return self

   def __next__(self):
       if self.current <= self.limit:
           value = self.current
           self.current += 1
           return value
       else:
           raise StopIteration

c = Counter(5)
for num in c:
   print(num)

Output:

1
2
3
4
5

The object becomes an iterator because it returns itself in __iter__() and generates values in __next__().

4. What Is a Generator

A generator is a simpler and more efficient way to create iterators. Instead of defining a class with __iter__() and __next__(), Python uses a function with the yield keyword.

Each time the generator function yields a value, its state is saved, allowing the next call to resume from the same point.

Generators provide:

• Cleaner syntax
• Automatic iterator creation
• Better memory usage

5. Generator Function Example

Example: A Simple Generator

def simple_gen():
   yield 1
   yield 2
   yield 3

g = simple_gen()

print(next(g))
print(next(g))
print(next(g))

Output:

1
2
3

6. Generators with Loops

Generators often use loops to produce a sequence of values.

Example: Number Generator

def count_up_to(n):
   for i in range(1, n + 1):
       yield i

for num in count_up_to(5):
   print(num)

Output:

1
2
3
4
5

7. Difference Between return and yield

• return ends the function completely.
• yield pauses the function and returns a value, resuming on the next call.

Example

def func():
   return 10

def gen():
   yield 10

print(func())
g = gen()
print(next(g))

Output:

10
10

But func() stops immediately, while gen() can return more values if needed.

8. Generator Expressions

Generator expressions are similar to list comprehensions, but they do not create the entire list in memory. Instead, values are produced one at a time.

Example: Generator Expression

g = (x*x for x in range(5))

print(next(g))
print(next(g))
print(list(g))

Output:

0
1
[4, 9, 16]

The remaining values are consumed by list(g).

9. Advantages of Generators

Generators provide several important benefits:

• Memory Efficiency – they do not store all values at once
• Lazy Evaluation – values are produced only when needed
• Readable Code – simpler than writing custom iterator classes
• Useful for Large Data Processing – logs, sensors, large files

10. Practical Examples

Example 1: Reading Large Files Line-by-Line

def read_file(path):
   with open(path) as f:
       for line in f:
           yield line

for line in read_file("sample.txt"):
   print(line.strip())

This approach avoids loading the entire file into memory.

Example 2: Infinite Sequence Generator

def infinite_counter():
   n = 1
   while True:
       yield n
       n += 1

c = infinite_counter()
print(next(c))
print(next(c))
print(next(c))

Output:

1
2
3

Example 3: Squares of Numbers

def squares(n):
   for i in range(n):
       yield i*i

print(list(squares(5)))

Output:

[0, 1, 4, 9, 16]