This is due to the relative ordering of overload resolution, permission to overload a template, instantiate a template, and instantiate a template.
Let's look first at C ++ 11. When the compiler needs to evaluate decltype(iter(Int<0>{})) , it performs overload resolution on the iter name, called with the arguments prvalue Int<0> . Since the template is in the overload set, we use 14.8.3 [temp.over] :
1 - a function template can be overloaded with either (non-template) functions of its name, or (other) function templates with the same name. When a call to this name is recorded (explicitly or implicitly using operator notation), the output of the template argument (14.8.2) and verification of any explicit template arguments (14.3) are performed for each template function to search for the template argument values (if any) that may used with this function template to instantiate a function template that can be called using call arguments. [...]
As a result, the declaration template<int i> constexpr auto iter(...) -> ... is created (14.7.1p10 [temp.inst] ) with i = 0 , which forces decltype(iter(Int<-1>{})) and turn off the rabbit hole of negative integers.
It doesn't matter that constexpr auto iter(Int<0>) -> Int<0> will be the best overload (at 13.3.3p1 [over.match.best] ), because we never go that far; the compiler goes far to negative infinity.
In contrast, when using C ++ 14, the return types 7.1.6.4p12 [dcl.spec.auto] apply:
12 - The conclusion of the type of the return value for the function template with a placeholder in its declared type occurs when the definition is created by the instance [...]
Since instantiating the instance occurs after resolving the template overload (14.7.1p3), the failed iter<0> template is never created; 14.8.3p5:
5 - To include specialization in a set of candidate functions, only the signature of the specialization of the function template is required. Therefore, to resolve a call for which the candidate is the template specialist, only a function template declaration is required.
"Signature" iter<0> here (Int<0>) -> decltype(auto) , a signature containing a placeholder type (7.1.6.4).
Suggested workaround: Use SFINAE to prevent any attempt to call iter(Int<-1>{}) :
template<int i> constexpr auto iter(Int<i>) -> decltype(iter(typename std::enable_if<i != 0, Int<i-1>>::type{})); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^
Note that SFINAE must go inside decltype and indeed inside the iter call.