Is there a way to Memorize or Materialize an IEnumerable?

11
votes

When given an d you could be dealing with a fixed sequence like a list or array, an AST that will enumerate some external datasource, or even an AST on some existing collection. Is there a way to safely "materialize" the enumerable so that enumeration operations like foreach, count, etc. don't execute the AST each time?

I've often used .ToArray() to create this represenation but if the underlying storage is already a list or other fixed sequence, that seems like wasted copying. It would be nice if i could do

var enumerable = someEnumerable.Materialize();

if(enumberable.Any() {
  foreach(var item in enumerable) {
    ...
  }
} else {
  ...
}

Without having to worry that .Any() and foreach try to enumerate the sequence twice and without it unccessarily copying the enumerable.

3
c#linq
This is a nice idea, but I would point out that often, existingCollection.ToList is done to protect against mutations to the existing collection. – Ani
The issue with .ToList() is that it will create a list of enumerables that aren't lists (arrays, ICollections, etc.) and return a mutable collection. – Arne Claassen

3 Answers

10
votes

Easy enough:

public static IList<TSource> Materialize<TSource>(this IEnumerable<TSource> source)
{
    if (source is IList<TSource>)
    {
        // Already a list, use it as is
        return (IList<TSource>)source;
    }
    else
    {
        // Not a list, materialize it to a list
        return source.ToList();
    }
}
10
votes

Original answer:

Same as Thomas's answer, just a bit better according to me:

public static ICollection<T> Materialize<T>(this IEnumerable<T> source)
{
    // Null check...
    return source as ICollection<T> ?? source.ToList();
}

Please note that this tend to return the existing collection itself if its a valid collection type, or produces a new collection otherwise. While the two are subtly different, I don't think it could be an issue.

Edit:

Today this is a better solution:

public static IReadOnlyCollection<T> Materialize<T>(this IEnumerable<T> source)
{
    // Null check...
    switch (source)
    {
        case ICollection<T> collection:
            return new ReadOnlyCollectionAdapter<T>(collection);

        case IReadOnlyCollection<T> readOnlyCollection:
            return readOnlyCollection;

        default:
            return source.ToList();
    }
}

public class ReadOnlyCollectionAdapter<T> : IReadOnlyCollection<T>
{
    readonly ICollection<T> m_source;

    public ReadOnlyCollectionAdapter(ICollection<T> source) => m_source = source;

    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();

    public int Count => m_source.Count;

    public IEnumerator<T> GetEnumerator() => m_source.GetEnumerator();
}
3
votes

Check out this blog post I wrote a couple of years ago: http://www.fallingcanbedeadly.com/posts/crazy-extention-methods-tolazylist

In it, I define a method called ToLazyList that effectively does what you're looking for.

As written, it will eventually make a full copy of the input sequence, although you could tweak it so that instances of IList don't get wrapped in a LazyList, which would prevent this from happening (this action, however, would carry with it the assumption that any IList you get is already effectively memoized).