“Failed Map Tasks exceeded allowed limit” Error from Hadoop

Question

I Have an "Failed Map Tasks exceeded allowed limit" Error while I use C# to run the MapReduce example application as show on below. Can any one tell me why it keep showing me this error? Appreciate it.

            public override void Map(string inputLine, MapperContext context)
        {
            //Extract the namespace declarations in the Csharp files
            var reg = new Regex(@"(using)\s[A-za-z0-9_\.]*\;");
            var matches = reg.Matches(inputLine);

            foreach (Match match in matches)
            {
                //Just emit the namespaces.
                context.EmitKeyValue(match.Value, "1");
            }
        }
    }

    //Reducer
    public class NamespaceReducer : ReducerCombinerBase
    {
        //Accepts each key and count the occurrances
        public override void Reduce(string key, IEnumerable<string> values, ReducerCombinerContext context)
        {
            //Write back  
            context.EmitKeyValue(key, values.Count().ToString());
        }
    }

    //Our Namespace counter job
    public class NamespaceCounterJob : HadoopJob<NamespaceMapper, NamespaceReducer>
    {
        public override HadoopJobConfiguration Configure(ExecutorContext context)
        {
            var config = new HadoopJobConfiguration();
            config.InputPath = "Input/CodeFiles";
            config.OutputFolder = "Output/CodeFiles";
            return config;
        }
    }

    static void Main(string[] args)
    {
        var hadoop = Hadoop.Connect();
        var result = hadoop.MapReduceJob.ExecuteJob<NamespaceCounterJob>();

    }

==============================================================================

The Job tracker log for the error is show as below.

Thanks for all your help.

Unhandled Exception: Microsoft.Hadoop.MapReduce.StreamingException: The user type could not be loaded. DLL=c:\hadoop\HDFS\mapred\local\taskTracker\Administrator\jobcache\job_201309041952_0030\attempt_201309041952_0030_m_000000_0\work\MRRunner.exe, Type=MRRunner.Program+NamespaceMapper ---> System.IO.FileNotFoundException: Could not load file or assembly 'file:///c:\hadoop\HDFS\mapred\local\taskTracker\Administrator\jobcache\job_201309041952_0030\attempt_201309041952_0030_m_000000_0\work\MRRunner.exe' or one of its dependencies. The system cannot find the file specified.

Job Tracker Log

It would be helpful if you could show what the Failed Map Tasks output is. — xdumaine

Tariq Tariq · Accepted Answer · 2013-09-06T00:01:44

This error shows that too many map tasks have failed. There could be n number of reasons for this. Without any logs or trace it would be difficult to tell you something properly, but you could try to see the trace of failed mappers. It'll give you a better idea of the problem. Just pint your browser to JobTracker webui(JobTracker_Host:50030). There you can find all the failed jobs. Go to this particular job and click on it. This will show you all the maps(completed and failed). Click on the failed mapper and select the ALL option inside Task Logs column on the next page. You can find entire trace here.

HTH

“Failed Map Tasks exceeded allowed limit” Error from Hadoop

2 Answers