Yarn Fair Scheduler queue

Question

I am seeking your help please in Hadoop Yarn Fair Scheduler. My understanding is as follows:

The default YARN settings allow when queue is not explicit by application it will create a queue on the fly with the user name.

Q1) Is my understanding correct?

Q2) If yes, what is the weight for this on the fly queue will be?

yarn.scheduler.fair.allow-undeclared-pools When set to true, the Fair Scheduler uses the username as the default pool name, in the event that a pool name is not specified. When set to false, all applications are run in a shared pool, called default. Default: true.

yarn.scheduler.fair.user-as-default-queue When set to true, pools specified in applications but not explicitly configured, are created at runtime. When set to false, applications specifying pools not explicitly configured run in a pool named default. This setting applies when an application explicitly specifies a pool and when the application runs in a pool named with the username associated with the application. Default: true.

JaviOverflow JaviOverflow · Accepted Answer · 2018-03-06T13:54:51

You can create a new queue where all applications that didn't specify a queue can go and then give the weight you want.

<queuePlacementPolicy>
    <rule name="specified" create="false" />
    <rule name="primaryGroup" create="false" />
    <rule name="default" queue="yourDefaultQueueName" />
</queuePlacementPolicy>

Yarn Fair Scheduler queue

1 Answers