🐥 Removing consecutive duplicates from a list of maps in Elixir

When working with telemetry or sensor data in Elixir, you might encounter a list where consecutive elements are identical. Often, you don’t want to remove all duplicates — only those that immediately follow each other.

For example, suppose you’re analyzing gear shifts from a cycling FIT file, and your data looks like this:

list = [
  %{rear: 21, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 15, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 19, front: 48}
]

In this sequence, %{rear: 19, front: 48} appears twice in a row multiple times. We want to remove only the immediate duplicates, keeping the overall order intact.

Here’s a simple and efficient way to do it:

filtered =
  Enum.reduce(list, [], fn
    current, [] ->
      [current]

    current, [prev | _] = acc ->
      if current == prev do
        acc
      else
        [current | acc]
      end
  end)
  |> Enum.reverse()

This results in:

[
  %{rear: 21, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48},
  %{rear: 17, front: 48},
  %{rear: 15, front: 48},
  %{rear: 17, front: 48},
  %{rear: 19, front: 48}
]

How it works:

• The reducer walks through the list while tracking the previous item.
• If the current element equals the previous one, it’s skipped.
• Otherwise, it’s prepended to the accumulator.
• Finally, we reverse the list to restore the original order (since prepending is more efficient than appending).

This approach is O(n) and works for any type of element (maps, structs, tuples, etc.), as long as equality comparison (==) makes sense for them.

For an even shorter version:

filtered =
  list
  |> Enum.reduce([], fn
    item, [] -> [item]
    item, acc when item == hd(acc) -> acc
    item, acc -> [item | acc]
  end)
  |> Enum.reverse()

A simple yet powerful trick to keep your Elixir data pipelines clean and efficient.