How can I enforce a specific queue when I execute a Hive job ?

[4a87b4a36dad41706305]

I am running Trifacta 4.1.1 on Cloudera CDH 5.12

For resource management purposes, I would like to assign all Trifacta jobs to a specific queue. Is this possible ?

[Trifacta_Alumni]

This is based on the existing solution. Type in "Enforce Yarn queue through Hive connection in Cloudera" in the search button to see the detailed solution with example.

1. Log into your Trifacta Server as an administrator

2. Go to URL http://<server>:3005/v2/connections and save the content under original_connection.txt

3. Create a params file, make sure that the params section of #2 is retained and the entry connectStrOpts is appended with “;sess_var_list?mapred.job.queue.name=<your_queue_name>”

Replace <your_queue_name> with the actual Yarn queue name you want the job executed in

[Trifacta_Alumni · Accepted solution]

The first answer is to route any Hadoop jobs sourced from Hive to a specific YARN job queue.

But if you need to route any Trifacta jobs to a specific Yarn queue, you need to specify the queue name in Trifacta config file (/opt/trifacta/conf/trifacta-conf.json) under "spark.props.spark.yarn.queue": "default" (replace default by your queue name).