C++ overload function by return type

Question

If I thought I knew anything about C++ then it was that you can't overload functions by return type.

So can anyone explain what is going on here please?

class A { public: typedef int _foo; };
class B {};

template<class T>
typename T::_foo Foo(int)
{
    cout << "Foo(int)\n"; return typename T::_foo();
}

template<class T>
typename T Foo(char)
{
    cout << "Foo(char)\n"; return typename T();
}

int main()
{
    Foo<A>(0);      // Writes "Foo(int)", as expected.
    Foo<B>(0);      // Writes "Foo(char), expected error unable to compile template.
    return 0;
}

There are two classes A and B. A defines typedef _foo, B does not. There are two overloads of function template Foo, Foo(int) and Foo(char). Foo(int) returns T::_foo, Foo(char) returns T.

Foo(0) is then called twice. This is an exact match for Foo(int) so I would expect Foo<A>(0) to compile ok, and Foo<B>(0) to fail to compile since B does not define the type _foo used in the template.

What actually happens is that Foo<B>(0) completely ignores Foo(int) and instantiates Foo(char) instead. But by the normal rules of overload resolution Foo(0) is clearly an exact match for Foo(int), and the only thing that makes Foo(char) a more viable match is the return type which should not be considered.

To verify that it is the return value that is affecting the overload resolution just add this:

template<class T>
void Bar(int)  { typename T::_foo a; cout << "Bar(int)\n"; }

template<class T>
void Bar(char) { cout << "Bar(char)\n"; }

Bar<A>(0);      // Writes "Bar(int), as expected.
//Bar<B>(0);    // Error C2039: '_foo' : is not a member of 'B', as expected.

This makes it clear that in the absence of the return value Foo(int) is indeed the correct overload, and that if the template cannot resolve the types used from its template argument that failure to compile is the normal outcome.

Answer 1

You're not overloading on return type, you're specializing a function template and when the Foo<B>(int) specialization forms the invalid type B::_foo that specialization is removed from the overload set by SFINAE, leaving the Foo<B>(char) function as the only viable function.

In more detail, the call to Foo<A>(0) first performs name lookup to find all the Foo names in scope, then instantiates any function templates to find the overload candidates, then overload resolution chooses the best match.

The step of instantiating the function templates produces these two function declarations:

int Foo<A>(int);
A Foo<A>(char);

Overload resolution chooses the first one as the best match.

However when calling Foo<B>(0) the instantiations produce these declarations:

<invalid type>  Foo<B>(int);
B Foo<B>(char);

The first declaration is not valid, so there is only one candidate for overload resolution, so that is the one that gets called.

In your Bar example the invalid type that gets formed during instantiation is not in "the immediate context" of the function declaration (it's in the function definition i.e. body) so SFINAE does not apply.

Answer 2

template<class T>
typename T::_foo Foo(int);

template<class T>
typename T Foo(char);

So your code declares this overloaded function. That's nice.

Foo<A>(0);

In this case, the compiler tries to fill out the template for the prototypes declared above, which would be:

int Foo(int); 
A foo(char); 

And since you're passing an integer as a parameter, the first is a better match, so the compiler uses that one.

Foo<B>(0);

Again the compiler sees this line and tries to fill out the template for the prototypes, but...

WTFDOESNTMAKESENSE?!?!? Foo(int); 
A foo(char); 

So clearly, the first one doesn't even make sense, so it discards that and uses the second overload. This actually has nothing to do with return types, it has to do with how template prototypes are filled out before it decides which function you mean. Here's your example rearranged to clarify:

template<class T>
int foo(T::_foo) {}
template<class T>
int foo(char) {}

int main() {
    foo<A>(0); //uses the first, `int foo(int)` better than `int foo(char)`
    foo<B>(0); //uses the second, because the first doesn't work with B.

This is called SFINAE, and note that it only works in very particular circumstances in template parameters, return types, and function parameters, but not the function body itself. This is why your "verification" caused an error, because it can't tell that one of the functions is invalid from the prototype, and the prototype is the only thing considered when it's deciding between overloads.