Iterators in Jinja

Functions like rejectattr, select, and selectattr return iterators instead of creating new lists immediately. This design helps save memory by deferring computation until the iterator is consumed in a loop or converted into a list.

Converting an Iterator to a List

To consume an iterator and create a new list, simply use the | list filter.

Example

{% set directory = context.directory | json | rejectattr("virtualOnNet") | list -%}

Explanation

• rejectattr("virtualOnNet"): Filters out items where the attribute virtualOnNet is True.
• | list: Converts the iterator returned by rejectattr into a new list.

WARNING

Memory Consumption If your list is very large, duplicating it (as | list does) can consume significant memory. For such cases, consider using iterators directly to save resources.

Using Iterators with Multiple Conditions

You can create separate iterators with different filtering conditions to process the same list in multiple ways, without duplicating the list in memory.

Example

{% set directory = context.directory | json -%}
{% set endusers = directory | rejectattr("virtualOnNet") | selectattr("userType", "eq", "EndUser") -%}
{% set virtualusers = directory | selectattr("virtualOnNet") %}

Explanation

• Original Data:

{% set directory = context.directory \| json -%}

The original list of dictionaries is converted from JSON.

• End Users Iterator:

{% set endusers = directory | rejectattr("virtualOnNet") | selectattr("userType", "eq", "EndUser") -%}

rejectattr("virtualOnNet"): filters out users where virtualOnNet is True.
selectattr("userType", "eq", "EndUser"): further filters the iterator to include only users with userType equal to EndUser.

• Virtual Users Iterator:

{% set virtualusers = directory | selectattr("virtualOnNet") %}

selectattr("virtualOnNet"): filters to include only users where virtualOnNet is True.

• Iterating Over Results You can consume the iterators in separate loops for End Users:

{% for user in endusers %}
  {{ user | pp_dict }}
{% endfor %}

And for Virtual Users:

{% for user in virtualusers %}
  {{ user | pp_dict }}
{% endfor %}

Combining Iterators and Lists

If you need to combine multiple iterators into one list, use | list to materialize them first:

{% set all_users = (endusers | list) + (virtualusers | list) %}

Best Practices for Iterators

• Use Iterators for Efficiency:
  • Iterators save memory and processing time when working with large datasets.
  • Avoid converting to a list unless necessary.
• Combine Filtering:
  • Use rejectattr and selectattr together for complex filtering.
• Debugging:
  • Use | list temporarily to inspect the contents of an iterator during development:

{{ endusers | list | pp_dict }}

Exercise: Practice Removing Attributes and Iterating

• Removing Attributes:
  • Use the pop method to remove a specific key from a dictionary.
  • Modify the input to test behavior when the key is absent.
• Using Iterators:
  • Use rejectattr and selectattr to filter items based on different conditions.
  • Experiment with combining filtered results into a single list.