Evaluation Timing of Set Comprehensions vs. Generator Expressions in Python

In Python, set comprehensions and generator expressions look almost identical:

{i for i in iterable}(i for i in iterable)

But they differ in when they are evaluated.

Set Comprehensions: Eager Evaluation

A set comprehension iterates over the iterable immediately upon creation and produces the final set object. For example:

array: list = [1, 2, 3]# When the next line runs, Python iterates over `array` and applies the filter condition.x: set = {i for i in array if array.count(i) > 1}# The append below does not affect the comprehension's resultarray.append(1)print(x)  # prints set()

Generator Expressions: Lazy Evaluation

A generator expression, by contrast, is lazily evaluated and does not iterate over any elements at creation time. PEP 289 gives its equivalent code directly:

array: list = [1, 2, 3]x: tuple = (i for i in array if array.count(i) > 1)# The x above is equivalent to:def __gen(bound_exp):    for i in bound_exp:        if array.count(i) > 1:            yield ix = __gen(iter(array))del __gen

Parameter binding follows the usual rules of Python function calls: the argument iter(array) is evaluated at the moment of the call, and bound_exp is bound to the resulting object from then on. Wherever the name array points afterwards has no effect on bound_exp.

We can observe the generator expression’s behavior by modifying the list:

array = [1, 2, 3]x = (i for i in array if array.count(i) > 1)# Mutating the list: the generator expression sees the modified list when iteratedarray.append(1)print(list(x))  # prints [1, 1]

If instead of mutating in place we rebind array to a new list:

array = [1, 2, 3]x = (i for i in array if array.count(i) > 1)array = [3, 3, 6]print(list(x))  # prints [3]

then the effect is equivalent to the following code, where the two occurrences of array are actually different objects:

(i for i in [1, 2, 3] if [3, 3, 6].count(i) > 1)

Summary