Why do iterators behave differently on exception than LINQ enumerables?

Question

I am studying the internal mechanics of the iterator methods, and I noticed a strange difference in behavior between the IEnumerator<T> obtained by an iterator and the IEnumerator<T> obtained by a LINQ method. If an exception happens during the enumeration, then:

The LINQ enumerator remains active. It skips an item but continues producing more.
The iterator enumerator becomes finished. It does not produce any more items.

Example. An IEnumerator<int> is enumerated stubbornly until it completes:

private static void StubbornEnumeration(IEnumerator<int> enumerator)
{
    using (enumerator)
    {
        while (true)
        {
            try
            {
                while (enumerator.MoveNext())
                {
                    Console.WriteLine(enumerator.Current);
                }
                Console.WriteLine("Finished");
                return;
            }
            catch (Exception ex)
            {
                Console.WriteLine($"Exception: {ex.Message}");
            }
        }
    }
}

Let's try enumerating a LINQ enumerator that throws on every 3rd item:

var linqEnumerable = Enumerable.Range(1, 10).Select(i =>
{
    if (i % 3 == 0) throw new Exception("Oops!");
    return i;
});
StubbornEnumeration(linqEnumerable.GetEnumerator());

Output:

1
2
Exception: Oops!
4
5
Exception: Oops!
7
8
Exception: Oops!
10
Finished

Now let's try the same with an iterator that throws on every 3rd item:

StubbornEnumeration(MyIterator().GetEnumerator());

static IEnumerable<int> MyIterator()
{
    for (int i = 1; i <= 10; i++)
    {
        if (i % 3 == 0) throw new Exception("Oops!");
        yield return i;
    }
}

Output:

1
2
Exception: Oops!
Finished

My question is: what is the reason for this inconsistency? And which behavior is more useful for practical applications?

Note: This observation was made following an answer by Dennis1679 in another iterator-related question.

Update: I made some more observations. Not all LINQ methods behave the same. For example the Take method is implemented internally as a TakeIterator on .NET Framework, so it behaves like an iterator (on exception completes immediately). But on .NET Core it's probably implemented differently because on exception it keeps going.

The first code example should be clear. Since you're catching every exception, the outer loop doesn't break. The example with yield return may look a little weird, but the compiler does a lot behind the scenes to make it work like that. — Dennis_E
@Dennis_E the same method StubbornEnumeration is used to enumerate both enumerators, the LINQ one and the iterator one. And the results are different. I didn't expect this difference to exist to be honest. — Theodor Zoulias

Rex M Rex M · Accepted Answer · 2019-10-06T12:18:25

The yield syntax in the second example makes the difference. When you use it, the compiler generates a state machine that manages a real enumerator under the hood. Throwing the exception exits the function and therefore terminates the state machine.

Why do iterators behave differently on exception than LINQ enumerables?

1 Answers