How to delete a queue in YARN?

Question

I see a queue in the yarn configuration file that I want to delete:

<property>
  <name>yarn.scheduler.capacity.root.queues</name>
  <value>a,b,c</value>
  <description>The queues at the this level (root is the root queue).
  </description>
</property>

<property>
  <name>yarn.scheduler.capacity.root.a.queues</name>
  <value>a1,a2</value>
  <description>The queues at the this level (root is the root queue).
  </description>
</property>

<property>
  <name>yarn.scheduler.capacity.root.b.queues</name>
  <value>b1,b2,b3</value>
  <description>The queues at the this level (root is the root queue).
  </description>
</property>

Say I want to remove queue c. I remove c from the list under the line <name>yarn.scheduler.capacity.root.queues</name> so that line looks like this:

<value>a,b,c</value>

Then I go to the command line and run yarn rmadmin -refreshQueues.

But I get the following error message:

Caused by java.io.IOException: c cannot be found during refresh!`

I'm trying to delete the queue c. How do I delete it?

I noticed here it says that

Note: Queues cannot be deleted, only addition of new queues is supported - the updated queue configuration should be a valid one i.e. queue-capacity at each level should be equal to 100%.

...so, how do I delete a queue if I don't need it anymore?

thanks.

From that page: "Administrators can add additional queues at runtime, but queues cannot be deleted at runtime." It sounds like you cannot delete at runtime, BUT you can stop/kill the YARN, update the config file (and remove c) and run YARN with the new configuration. — Arash
That is a known issue: issues.apache.org/jira/browse/YARN-4022 — Geethu Jose

Arash Arash · Accepted Answer · 2017-03-06T19:57:48

From that page:

Administrators can add additional queues at runtime, but queues cannot be deleted at runtime.

It sounds like you cannot delete at runtime, BUT you can stop/kill the YARN, update the config file (and remove c) and run YARN with the new configuration. So if you can afford to stop/start the YARN then here is the process:

Note: Before killing the YARN Read here on the effects on the system depending on your YARN version.

Here goes the process:

Stop the MapReduce JobHistory service, ResourceManager service, and NodeManager on all nodes where they are running, as follows:

 sudo service hadoop-mapreduce-historyserver stop
 sudo service hadoop-yarn-resourcemanager stop
 sudo service hadoop-yarn-nodemanager stop

Then Edit the config file and remove c

Start the MapReduce JobHistory server, ResourceManager, and NodeManager on all nodes where they were previously running, as follows:

 sudo service hadoop-mapreduce-historyserver start
 sudo service hadoop-yarn-resourcemanager start
 sudo service hadoop-yarn-nodemanager start

Commands are from here

How to delete a queue in YARN?

2 Answers