Working with Python Collections and Iterables

Python’s iterables and functional programming tools like map(), filter(), and reduce() enable efficient data processing. In this guide, you’ll learn to master iterators, lambda functions, and the itertools module for advanced workflows.

Iterables vs. Iterators

Definitions

• Iterable: An object that can return its elements one at a time (e.g., lists, tuples, strings).
• Iterator: An object representing a data stream. It implements __next__() to fetch items.

Using iter() and next()

my_list = [10, 20, 30]
iterator = iter(my_list)  # Convert iterable to iterator
print(next(iterator))     # 10
print(next(iterator))     # 20

Built-In Functions

map(): Apply a Function to Items

# Double each number in a list
numbers = [1, 2, 3]
doubled = map(lambda x: x * 2, numbers)  # Returns iterator
print(list(doubled))  # [2, 4, 6]

filter(): Select Items Conditionally

# Filter even numbers
numbers = [1, 2, 3, 4, 5]
evens = filter(lambda x: x % 2 == 0, numbers)
print(list(evens))  # [2, 4]

reduce(): Aggregate Values

from functools import reduce
# Sum all numbers
total = reduce(lambda x, y: x + y, [1, 2, 3, 4])
print(total)  # 10

Lambda Functions

Anonymous functions for concise one-liners:

# Syntax: lambda arguments: expression
add = lambda x, y: x + y
print(add(5, 3))  # 8

The itertools Module

Advanced tools for working with iterables:

1. chain(): Combine Iterables

from itertools import chain
list1 = [1, 2]
list2 = ['a', 'b']
all_items = chain(list1, list2)  # 1, 2, 'a', 'b'

2. cycle(): Repeat Indefinitely

from itertools import cycle
count = 0
for item in cycle(['a', 'b']):
    print(item)  # a, b, a, b, ...
    count += 1
    if count == 4: break

3. islice(): Slice Iterators

from itertools import islice
numbers = range(10)
sliced = islice(numbers, 2, 6)  # 2, 3, 4, 5

4. groupby(): Group by Key

from itertools import groupby
data = [("a", 1), ("a", 2), ("b", 3)]
for key, group in groupby(data, lambda x: x[0]):
    print(key, list(group))  # a [('a',1), ('a',2)], b [('b',3)]

Practice Work

Exercise 1: Convert Loop to map()

Convert this loop to use map():

numbers = [1, 2, 3]
squared = []
for num in numbers:
    squared.append(num ** 2)

squared = list(map(lambda x: x**2, numbers))

Exercise 2: Filter with Lambda

Use filter() to extract words longer than 3 characters:

words = ["apple", "cat", "dog", "elephant"]

long_words = list(filter(lambda x: len(x) > 3, words))

Exercise 3: Use groupby()

Group numbers by even/odd:

numbers = [1, 2, 3, 4, 5, 6]

from itertools import groupby
sorted_numbers = sorted(numbers, key=lambda x: x % 2)
for key, group in groupby(sorted_numbers, lambda x: x % 2):
    print("Even" if key == 0 else "Odd", list(group))

Workshop: Real-World Applications

Workshop 1: Data Pipeline

Process log files using itertools:

from itertools import chain
# Read multiple log files
logs = chain(open("log1.txt"), open("log2.txt"))
errors = [line for line in logs if "ERROR" in line]

Workshop 2: Aggregate Sales Data

Use reduce() to calculate total sales:

from functools import reduce
sales = [100, 200, 150]
total_sales = reduce(lambda x, y: x + y, sales)

Best Practices

• Prefer List Comprehensions for Simplicity: Use map()/filter() when readability improves.
• Avoid Complex Lambdas: Use named functions for multi-step logic.
• Use itertools for Memory Efficiency: Iterators save memory with large datasets.

Conclusion

Mastering iterables, functional tools, and itertools unlocks efficient data processing in Python. Practice with the exercises to streamline your workflows!

Next Steps: Explore Python Generators or Data Classes.