Python迭代器与生成器详解

发布时间: 2025-03-24 13:23

↑

# Python迭代器与生成器详解

本文将详细介绍Python中的迭代器(Iterator)和生成器(Generator)概念,帮助你深入理解这两个强大的特性及其在实际编程中的应用。

## 迭代器基础

### 什么是迭代器

迭代器是一个可以记住遍历位置的对象。迭代器对象从集合的第一个元素开始访问,直到所有的元素被访问完结束。迭代器只能往前不会后退。

### 可迭代对象和迭代器

```python
# 可迭代对象
my_list = [1, 2, 3, 4, 5]
my_iter = iter(my_list)  # 获取迭代器对象

# 使用next()函数
print(next(my_iter))  # 1
print(next(my_iter))  # 2

# 使用for循环
for item in my_list:
    print(item)
```

### 创建迭代器

```python
class Counter:
    def __init__(self, start, end):
        self.start = start
        self.end = end
    
    def __iter__(self):
        return self
    
    def __next__(self):
        if self.start > self.end:
            raise StopIteration
        current = self.start
        self.start += 1
        return current

# 使用自定义迭代器
counter = Counter(1, 3)
for num in counter:
    print(num)  # 1, 2, 3
```

## 生成器基础

### 什么是生成器

生成器是一种特殊的迭代器,它可以让我们更简单地创建迭代器。生成器使用yield语句,每次生成一个值。

### 生成器函数

```python
def countdown(n):
    while n > 0:
        yield n
        n -= 1

# 使用生成器
for i in countdown(5):
    print(i)  # 5, 4, 3, 2, 1

# 创建生成器对象
gen = countdown(3)
print(next(gen))  # 3
print(next(gen))  # 2
print(next(gen))  # 1
```

### 生成器表达式

```python
# 列表推导式
squares_list = [x**2 for x in range(5)]

# 生成器表达式
squares_gen = (x**2 for x in range(5))

print(squares_list)  # [0, 1, 4, 9, 16]
print(squares_gen)   # <generator object <genexpr> at ...>

# 使用生成器表达式
for square in squares_gen:
    print(square)
```

## 高级特性

### send()方法

```python
def counter():
    i = 0
    while True:
        val = (yield i)
        if val is not None:
            i = val
        else:
            i += 1

# 使用send()
c = counter()
print(next(c))      # 0
print(c.send(10))   # 10
print(next(c))      # 11
```

### yield from

```python
def sub_gen():
    yield 1
    yield 2
    yield 3

def main_gen():
    yield 'a'
    yield from sub_gen()
    yield 'b'

# 使用yield from
for item in main_gen():
    print(item)  # a, 1, 2, 3, b
```

## 实际应用示例

### 1. 内存效率的数据处理

```python
def read_large_file(file_path):
    with open(file_path, 'r') as file:
        for line in file:
            # 处理每一行之前先去除空白字符
            line = line.strip()
            if line:
                yield line

# 使用示例
def process_log_file(file_path):
    for line in read_large_file(file_path):
        # 处理每一行数据
        process_line(line)
```

### 2. 无限序列生成

```python
def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

# 使用生成器获取斐波那契数列的前10个数
fib = fibonacci()
for _ in range(10):
    print(next(fib))
```

### 3. 数据流处理管道

```python
def generate_data():
    # 模拟数据源
    for i in range(100):
        yield i

def filter_even(numbers):
    # 过滤出偶数
    for num in numbers:
        if num % 2 == 0:
            yield num

def multiply_by_three(numbers):
    # 将数字乘以3
    for num in numbers:
        yield num * 3

# 创建处理管道
data = generate_data()
even_numbers = filter_even(data)
result = multiply_by_three(even_numbers)

# 处理数据
for num in result:
    print(num)
```

### 4. 自定义范围生成器

```python
class NumberRange:
    def __init__(self, start, end, step=1):
        self.start = start
        self.end = end
        self.step = step
    
    def __iter__(self):
        return self.generator()
    
    def generator(self):
        num = self.start
        while num <= self.end:
            yield num
            num += self.step

# 使用示例
for num in NumberRange(1, 10, 2):
    print(num)  # 1, 3, 5, 7, 9
```

### 5. 协程实现状态机

```python
def state_machine():
    state = 'START'
    while True:
        if state == 'START':
            event = yield 'Ready for input'
            if event == 'advance':
                state = 'PROCESSING'
        elif state == 'PROCESSING':
            event = yield 'Processing data'
            if event == 'done':
                state = 'FINISHED'
        elif state == 'FINISHED':
            event = yield 'Finished processing'
            if event == 'reset':
                state = 'START'

# 使用状态机
sm = state_machine()
print(next(sm))         # Ready for input
print(sm.send('advance'))  # Processing data
print(sm.send('done'))     # Finished processing
print(sm.send('reset'))    # Ready for input
```

## 最佳实践

1. **使用生成器处理大数据**
```python
# 不好的做法
def get_all_lines(file_path):
    with open(file_path) as f:
        return f.readlines()  # 一次性读取所有行

# 好的做法
def get_lines(file_path):
    with open(file_path) as f:
        for line in f:  # 逐行读取
            yield line
```

2. **合理使用生成器表达式**
```python
# 不好的做法
sum([x*x for x in range(1000000)])  # 创建大列表

# 好的做法
sum(x*x for x in range(1000000))    # 使用生成器表达式
```

3. **迭代器链式处理**
```python
# 不好的做法
data = list(range(1000))
filtered = [x for x in data if x % 2 == 0]
result = [x * 2 for x in filtered]

# 好的做法
def process_data():
    for x in range(1000):
        if x % 2 == 0:
            yield x * 2
```

4. **使用itertools模块**
```python
from itertools import islice, chain

# 高效地处理迭代器
def process_in_batches(data, batch_size=100):
    iterator = iter(data)
    while True:
        batch = list(islice(iterator, batch_size))
        if not batch:
            break
        yield batch
```

通过本文的学习,你应该已经掌握了Python中迭代器和生成器的基本概念和使用方法。这些特性不仅可以帮助你编写更高效的代码,还能让你的程序更加优雅。继续练习和探索,你会发现更多迭代器和生成器的应用场景！

元素码农