Jinja Unique: filter function to remove duplicates

n

Understanding the unique Filter

nThe unique filter is a handy tool in Jinja for removing duplicate items from a list or any other iterable. It processes a sequence of items and returns a new sequence containing only the unique elements. A key feature is that it preserves the original order of the items, based on their first appearance. This is a crucial distinction from simply converting a list to a set, which would not guarantee the order.nnSince the filter returns an iterator, you often need to chain it with the list filter if you want to use the result as a regular list in your template. For instance, {{ items | unique | list }} is the common pattern.nn

---

nn

How to Use the Filter

nThe basic syntax for the unique filter is straightforward:n

{{ iterable | unique }}

nAs the example in the documentation shows, this will yield the unique items in the order they first appeared. For example, ['apple', 'banana', 'apple', 'orange'] | unique would result in an iterator for ['apple', 'banana', 'orange'].nnThe filter also includes two optional parameters that give you more control over how duplicates are identified:n

n
• case_sensitive (boolean, default False): By default, the unique filter is case-insensitive for strings. This means 'foo' and 'Foo' are considered the same. If you set case_sensitive=True, the filter will treat these as distinct values.

n

• attribute (string or integer, optional): This is a powerful feature for working with a list of objects or dictionaries. Instead of checking the uniqueness of the entire object, you can tell the filter to check for uniqueness based on a specific attribute or key. This is useful for removing duplicate user objects from a list based on a unique id or username attribute, for example.

n

nn

---

nn

Practical Examples

nLet’s walk through some practical scenarios to see the filter in action.n

Example 1: Basic Case-Insensitive Filtering

nThis is the default behavior and is perfect for cleaning up simple lists of strings.n

{% set tags = ['python', 'Jinja', 'Python', 'html'] %}n{% for tag in tags | unique | list %}n  <span>{{ tag }}</span>n{% endfor %}

nIn this case, the rendered output would be:n<span>python</span>n<span>Jinja</span>n<span>html</span>nnNotice that 'python' and 'Python' are treated as the same, with the first occurrence ('python') being kept.n

Example 2: Case-Sensitive Filtering

nIf you want 'python' and 'Python' to be considered different, you’d use case_sensitive=True.n

{% set tags = ['python', 'Jinja', 'Python', 'html'] %}n{% for tag in tags | unique(case_sensitive=True) | list %}n  <span>{{ tag }}</span>n{% endfor %}

nThe rendered output would now be:n<span>python</span>n<span>Jinja</span>n<span>Python</span>n<span>html</span>n

Example 3: Filtering a List of Objects by an Attribute

nThis is where the filter truly shines. Imagine you have a list of user objects, but some are duplicates. You can use the attribute parameter to remove them based on a unique identifier like their id.n

{% set users = [n  { 'id': 1, 'name': 'Alice' },n  { 'id': 2, 'name': 'Bob' },n  { 'id': 1, 'name': 'Alice' }n] %}n{% for user in users | unique(attribute='id') | list %}n  <p>{{ user.name }} (ID: {{ user.id }})</p>n{% endfor %}

nThe rendered output would be:n<p>Alice (ID: 1)</p>n<p>Bob (ID: 2)</p>nnThe second entry for Alice is correctly removed because her id attribute was a duplicate of the first entry. This pattern is very useful for displaying clean, deduplicated data from your backend.nn