Why do Queryable.SelectMany(...) overloads accept Func<S, IEnumerable<R>> instead of Func<S, IQueryable<R>>?

Question

Consider the following function:

IQueryable<Bar> foo(IEnumerable<IQueryable<Bar>> sources)
{
    return
        from source in sources.AsQueryable()
        from bar in source
        where bar.Xzy == 123
        select bar;
}

Intuitively, I would expect this to execute the "from...where...select" expression in the context of each source. However, I believe it will instead only execute the "from bar in ..." portion against the source, and instead execute the "where...select" portion as LINQ-to-Objects query. The net result is that all the rows of SomeTable will be retrieved from each source, instead of only those matching the "where" condition.

At first glance, my guess is that this is because the call to SelectMany causes the "source" expression to be implicitly converted to an IEnumerable<Bar>. I'm not sure what the implementation would look like, but wouldn't it make sense for it to accept Func<S, IQueryable<R>> instead, so that the where...select expression is passed to the IQueryable provider?

What's the type of source.SomeTable? What happens if you change the return type of foo to IQueryable<Bar>? — Lee
@Lee, sorry, I think the SomeTable property doesn't make sense in the example. I'll "source.SomeTable" to "source". I think changing the return type to IQueryable<Bar> doesn't change the behavior of the query. — Aaron
The return type shoud make a different because the IEnumerable<T, TResult> overload returns IEnumerable<TResult> while the IQueryable overload returns an IQueryable<T>. So your function would fail to compile if the return type were changed to IQueryable<Bar> and an IEnumerable overload of SelectMany were being chosen. — Lee
@Lee, I don't quite follow you. What are you referring to by "the IEnumerable<T, TResult> overload"? I think the type of the query expression inside the method is IQueryable<Bar>, which also implements IEnumerable<Bar>, so I'm not sure why it matters unless the caller of foo is going to pass the returned value to other LINQ methods. — Aaron
It was a design decision to accept IEnumerable as selector. Are you merely asking why that decision was made? — alexm

lesscode lesscode · Accepted Answer · 2016-05-06T18:35:41

I think it comes down to how the compiler interprets your query. What it's actually turning your query into is something like this (a strict left-to-right parse):

sources.AsQueryable()
       .SelectMany(source => source)
       .Where(bar => bar.Xzy == 123)
       .Select(bar => bar)

Crucially, your filter operates on the result of enumerating each source first.

Note also that the AsQueryable() is actually redundant, since it isn't making your sources any more queryable than they already are, it's making the enumeration of your sources queryable, and you aren't querying that set anyway, you're querying the individual sources.

What you actually want is more like this, I think:

sources.SelectMany(source => source.Where(bar => bar.Xzy == 123))

This changes the interpretation of the relative "order-of-precedence" of the clauses.

~~I'm actually not sure how you would craft the latter using the LINQ syntax.~~

UPDATE: Actually, here's one way:

from source in sources
let qsource = (from bar in source
               where bar.Xzy == 123
               select bar)
from result in qsource
select result

I generally tend to prefer to craft non-trivial queries using the extension methods for this very reason.

Why do Queryable.SelectMany(...) overloads accept Func<S, IEnumerable<R>> instead of Func<S, IQueryable<R>>?

1 Answers