What Are Python Generators and Yield Keyword? See Examples

Generators in Python are special iterators that produce items lazily, meaning they generate values one at a time and do not store the entire sequence in memory.

The yield keyword is used in a function to return a value and pause the function’s state, allowing it to resume later.
Generators are memory-efficient and ideal for large datasets or streams.

Why Generators Are Important

Efficiently handle large or infinite sequences
Reduce memory usage compared to lists
Used in data pipelines, streaming, and lazy evaluation
Simplify iterator creation

Example 1: Basic Generator

def simple_gen():
    for i in range(5):
        yield i

gen = simple_gen()
for value in gen:
    print(value)

Output:

Example 2: Generator for Fibonacci Sequence

def fibonacci(n):
    a, b = 0, 1
    for _ in range(n):
        yield a
        a, b = b, a + b

for num in fibonacci(10):
    print(num)

Generates the sequence lazily without storing all numbers

Example 3: Real-World Scenario – Reading Large Files

def read_large_file(file_name):
    with open(file_name, "r") as file:
        for line in file:
            yield line.strip()

for line in read_large_file("large_log.txt"):
    print(line)

Memory-efficient way to process very large files line by line

Example 4: Pipeline Processing with Generators

def square_numbers(numbers):
    for n in numbers:
        yield n**2

numbers = range(1, 6)
squares = square_numbers(numbers)
for square in squares:
    print(square)

Useful in streaming data processing or real-time analytics

Best Practices

✔ Use generators for large datasets or infinite sequences
✔ Use yield instead of returning lists for memory efficiency
✔ Combine generators with map(), filter(), and comprehensions
✔ Avoid using them when all items need to be accessed repeatedly

Conclusion

Python generators and the yield keyword provide a powerful, memory-efficient way to iterate over sequences. Mastering them is essential for large-scale data processing, streaming applications, and real-world programming efficiency.

References

Internal Reference: https://savanka.com/category/learn/python/
External Reference: https://www.w3schools.com/python/