List Comprehensions & Generators

AI & Data Science 604

10 min read

·

Jul 11, 2026

--

Listen

Share

Make your code simpler and more efficient.

In the previous blog, about iterators, we have learnt about some handy functions that will allow you to effectively work with iterators e.g., enumerate() and zip().

In this this blog, you’ll build on your knowledge of iterators and be introduced to list comprehensions, which allow you to create complicated lists — and lists of lists — in one line of code! List comprehensions can dramatically simplify your code and make it more efficient. You’ll then learn about generators, which are extremely helpful when working with large sequences of data that you may not want to store in memory, but instead generate on the fly.

Press enter or click to view image in full size

List comprehensions.

List comprehensions are a powerful feature in Python that allows for cleaner and more efficient code when working with lists. They can be used for a variety of tasks, including transformations and filtering, making them an essential tool for Python developers.
It reduces the amount of code needed to generate lists. It’s often clearer and easier to understand than equivalent for loops. It’s generally faster than using a loop to build lists.

Let’s say that you have a list of numbers and you want to create a new list of numbers that’s the same as the old list, except that each number has 1 added to it.

nums = [12, 8, 21, 3, 16]
new_nums = []
for num in nums:    
    new_nums.append(num + 1)

print(new_nums)
# output: [13, 9, 22, 4, 17]

You could initialize a new empty list, loop through the old list, add 1 to each entry and append all new values to the new list, but for loops are inefficient, both computationally and in terms of coding time and space, particularly when you could do this in one line of code.

The list comprehension Syntax Anatomy

The basic syntax of a list comprehension is:
[output_expression for item in iterable].
To understand how this maps to a traditional for loop, look at this comparison:

• output_expression: The operation or value you want to output into the new list.
• item: The variable representing the current element in the loop.
• iterable: The collection you are looping through (e.g., a list, string, range, or iterator).

Within square brackets, you write the output expression which are values you wish to create, followed by the for clause referencing the original list.

So in our case, you open the square bracket, followed by num + 1 for num in nums and then you close the square bracket. This is a list comprehension and creates precisely the desired list!

nums = [12, 8, 21, 3, 16]
new_nums = [num + 1 for num in nums]
print(new_nums)

# Output: [13, 9, 22, 4, 17]

List comprehensions provide a concise, readable way to create lists in Python. Instead of initializing an empty list and using a for loop to append elements one by one, a list comprehension allows you to do it all in a single line of code.

See here the relationship between the for loop syntax and the list comprehension syntax. The power of list comprehensions is not merely relegated to the world of lists, however, you can write a list comprehension over any iterable.

Here’s an example of a list comprehension using a range object.

result = [num for num in range(11)]
print(result)

#Output: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

To summarize, list comprehensions collapse for loops for building lists into a single line and the required components are 1) an iterable, 2) an iterator variable that represents the members of the iterable and 3) an output expression.

Nested loops

That’s it. You can also use list comprehensions in place of nested for loops. lets say that we wanted to create a list of all pairs of integers where the 1st integer is between 0 and 1 and the 2nd between 6 and 7.

pairs_1 = []
for num1 in range(0, 2):
    for num2 inrange(6, 8):
        pairs_1.append((num1, num2))

print(pairs_1)

#Output: [(0, 6), (0, 7), (1, 6), (1, 7)]

This nested for loop would produce the required result. The question is, can we do the same with a list comprehension? And the answer is, yes, as follows.

pairs_2 = [(num1, num2) for num1 in range(0, 2) for num2 inrange(6, 8)]
print(pairs_2)
#Output: [(0, 6), (0, 7), (1, 6), (1, 7)]

Once again, within the square brackets, place the desired output expression followed by the two required for loop clauses.

When to Avoid List Comprehensions.

You may observe that while it keeps to a single line of code, we sacrifice some readability of the code as a tradeoff, so you’ll have to consider if you’d like to use list comprehensions in cases such as this. The more often you use this, the more you get used to reading list comprehensions, so readability may not be a problem for you later on. But do remember that others may have to read your code as well!

So, while powerful, list comprehensions can easily become unreadable if over-engineered. If your comprehension requires multiple nested loops, complex conditional logic, or spans across several lines, stick to a traditional for loop. Code clarity and maintainability should always come before saving lines of code.

Conditionals in comprehensions

Now that you know the basics of list comprehensions, lets check out some more advanced comprehension capabilities, such as conditionals!

You can extend list comprehensions by adding an if statement at the end to filter elements from the original iterable using this format: [output expression for item in iterable if condition].

print([num ** 2 for num in range(10) if num % 2 == 0])
# Output: [0, 4, 16, 36, 64]

Here we see that we can filter the output of a list comprehension using a conditional on the iterable: in this example, the resulting list is the square of the values in range(10) under the condition that the value itself is even.

If you have not seen it before, the percent operation (%) that you see being used in the comprehension is called the modulo operator. We can look at the Python documentation to see how the modulo operator is used and it shows that it produces the remainder from the division of the first argument by the second. Thus an integer modulo two is equal to zero if and only if the integer is even.

If you want to change the output based on a condition (rather than filtering elements out), place the if-else statement before the for loop. [expression_if_true if condition else expression_if_false for item in iterable].

We can also condition the list comprehension on the output expression. Here, for an even integer we output its square. In any other case, signified by the else clause, that is for odd integers, we output 0.

print([num ** 2 if num % 2 == 0 else 0 for num in range(10)])

#Output: [0, 0, 4, 0, 16, 0, 36, 0, 64, 0]

List comprehensions can be nested to flatten a matrix (a list of lists) or perform operations on multi-dimensional data. For Example: Flattening a Matrix.

matrix = [[1, 2], [3, 4], [5, 6]]
flattened = [num for row in matrix for num in row]
# Output: [1, 2, 3, 4, 5, 6]

💡 Tip: Reading nested comprehensions can be tricky. Read them in the exact order you would write the nested for loops: for row in matrix comes first, followed by for num in row.

Dict comprehensions

Now we can also write dictionary comprehensions to create new dictionaries from iterables. The syntax is almost the same as in list comprehensions and there are 2 differences:

1. We use curly braces instead of square brackets.
2. The key and value are separated by a colon in the output expression as we can see here.

pos_neg = {num: -num for num in range(9)}

print(pos_neg)
#Output: {0: 0, 1: -1, 2: -2, 3: -3, 4: -4, 5: -5, 6: -6, 7: -7, 8: -8}

print(type(pos_neg))
# Output: <class 'dict'>

In this example, we are creating a dictionary with keys positive integers and corresponding values the respective negative integers.

Introduction to generator expressions

Now we’re getting a bit more comfortable with comprehensions, we’re going to check out generators, which are related to comprehensions in way that will soon become evident.

If you understand list comprehensions, you are already 90% of the way to understanding generator expressions.

While a list comprehension builds and stores the entire list in memory all at once, a generator expression creates an iterator that yields elements one at a time, on demand (lazy evaluation). This makes them incredibly memory-efficient when dealing with large or infinite datasets.

A generator is like a list comprehension except that it does not store the list in memory and it does not construct the list but is an object. Also, we can iterate over to produce elements of the list as required.

The syntax is identical to a list comprehension, except you swap the square brackets [] for parentheses ().Here is how they contrast in action:

# List Comprehension (Creates the whole list immediately)
list_comp = [x**2 for x in range(1000000)] 

# Generator Expression (Creates a recipe to make numbers on the fly)
gen_exp = (x**2 for x in range(1000000))

If you print list_comp, you will see a massive list of numbers. If you print gen_exp, you will simply see: <generator object <genexpr> at 0x... >.

How to Get Values From a Generator

Because a generator expression produces an iterator, you retrieve its values using the exact same mechanisms you use for other iterators:

1. Using a for loop

The loop automatically requests the next item until the generator runs out of data. Here we can see that looping over a generator expression produces the elements of the analogous list. We can also pass a generator to the function list to create the list.

squares = (x**2 for x in range(3))
for num in squares:
    print(num)
# Output:
# 0
# 1
# 4

2. Using the next() function

You can manually pull values one by one, like any other iterator, we can pass a generator to the function next in order to iterate through its elements.

squares = (x**2 for x in range(3))

print(next(squares)) # Output: 0
print(next(squares)) # Output: 1
print(next(squares)) # Output: 4
# print(next(squares)) # Throws StopIteration exception

For the geeks like me, this is an example of something called lazy evaluation, whereby the evaluation of the expression is delayed until its value is needed.
This can help a great deal when working with extremely large sequences as you don’t want to store the entire list in memory, which is what comprehensions would do; you want to generate elements of the sequence on the fly.

The Massive Advantage: Memory Efficiency

Because generators don’t store their whole payload in memory, the difference in resource consumption can be staggering.

import sys

# A list of 10 million integers
big_list = [i for i in range(10000000)]
print(f"List size: {sys.getsizeof(big_list)} bytes") 
# Output: ~89 MB

# A generator for 10 million integers
big_gen = (i for i in range(10000000))
print(f"Generator size: {sys.getsizeof(big_gen)} bytes") 
# Output: ~200 bytes

Let’s say that we wanted to iterate over a very large sequence of numbers, such as from 0 up to 10 to the power of a million, or at least wanted to do so until another condition was satisfied. Look what happens when I try to build such an iterable list using a comprehension on servers.

My colleagues disconnect me because the list I’m trying to create can’t even be stored in memory! Be warned though, don’t try this at home, on our servers or yours!

Check this out, however: I can easily create the analogous generator object because it does not yet create the entire list.

Conditionals in generator expressions

What’s really cool is that anything we can do in a list comprehension such as filtering and applying conditionals, we can also do in a generator expression, such as you see here. You’ll get a whole bunch of practice with this in the upcoming exercises.

List Comprehension VS Generator.

When deciding whether to use a list comprehension or a generator expression in Python, the choice comes down to a tradeoff between memory efficiency and data reusability.

Here is a conceptual breakdown of how they compare across key dimensions.

1. Evaluation Strategy

• List Comprehension: Uses eager evaluation. It immediately computes every single element in the sequence the moment it is defined and builds the final collection right away.
• Generator Expression: Uses lazy evaluation. It does not compute anything upfront; instead, it remembers the instructions and evaluates each item one by one, only when specifically requested.

2. Memory Consumption

• List Comprehension: High memory footprint. Because it creates the entire collection at once, the memory it consumes scales linearly with the number of elements. If you generate millions of items, they all sit in your system’s RAM.
• Generator Expression: Extremely low memory footprint. It only holds a single item in memory at any given time. Because it processes items on the fly, a generator producing ten items takes up the exact same amount of memory as a generator producing ten billion items.

3. Reusability and Access

• List Comprehension: Generates a permanent list object. You can iterate through it as many times as you want, access elements out of order using indices, slice the data, or modify it later.
• Generator Expression: Generates a “one-time use” iterator. Once you read through the data sequence, the generator is exhausted. If you want to access the data a second time, you have to completely recreate the generator. Additionally, you cannot index or slice a generator.

Press enter or click to view image in full size

When to Choose Which?

• Choose List Comprehension when: You have a relatively small dataset, you need to keep the data around to look at it multiple times, or you require advanced operations like indexing, sorting, or slicing the resulting collection.
• Choose Generator Expression when: You are dealing with massive or infinite datasets (like reading large log files), or you are immediately feeding the data into a summary function that only needs to look at each item once (such as calculating a total sum, finding a maximum value, or looking for a specific match).