Yarn Scheduler Load Simulator - fair-scheduler

0
votes

I want to do a performance analysis of my fair scheduler configuration, and I found this tool - "YARN Scheduler Load Simulator (SLS)" which I believe can help me out. I've followed the document and setup YARN SLS to run with a sample job trace.

But I am not able to figure out how exactly to monitor scheduler behavior with this tool. It runs and shows scheduler behavior for the sample job traces I input, even without the fair-scheduler config.

If someone can point me out to how I can configure YARN SLS to use my fair-scheduler.xml, and insights on how to monitor the scheduler simulation (or better yet, what to look for), it'll be really helpful for me.

Thanks!

1
hadoophadoop-yarn
Did you look at the UI? It shows the utilization of the cluster pretty clearly. – tk421
Like I said, I was able to start YARN SLS. and yes, I'm able to view the UI dashboard too. The thing is I haven't placed any scheduler config in Hadoop conf directory for the simulation to pick up. But it still runs the job traces I've given with respective queues. My query is,How do I configure SLS to use my fair-scheduler config? How can I analyze my scheduler queue resource usage using YARN SLS? or is it just the info available in SLS dashboard and nothing else? :) – abisheksampath
You should just have to place it under your Hadoop conf directory. If you configure --output-dir you should get all the metrics generated from the SLS run. – tk421

1 Answers

0
votes

I am assuming that you have compiled your own version of Hadoop. Otherwise I don't think you can get your implementation of FAIR running.

If I am not mistaken SLS is simulating the scheduler regarding the configurations that yarn-site.xml is providing. So that is where you should define the type of Scheduler you want to use.