topic How can I enforce a specific queue when I execute a Hive job ? in Alteryx Designer Cloud Discussions

How can I enforce a specific queue when I execute a Hive job ?

4a87b4a36dad41706305 — Tue, 05 Dec 2017 19:27:49 GMT

I am running Trifacta 4.1.1 on Cloudera CDH 5.12

For resource management purposes, I would like to assign all Trifacta jobs to a specific queue. Is this possible ?

Re: How can I enforce a specific queue when I execute a Hive job ?

Trifacta_Alumni — Tue, 02 Jan 2018 21:39:34 GMT

This is based on the existing solution. Type in "Enforce Yarn queue through Hive connection in Cloudera" in the search button to see the detailed solution with example.

1. Log into your Trifacta Server as an administrator

2. Go to URL http://<server>:3005/v2/connections and save the content under original_connection.txt

3. Create a params file, make sure that the params section of #2 is retained and the entry connectStrOpts is appended with “;sess_var_list?mapred.job.queue.name=<your_queue_name>”

Replace <your_queue_name> with the actual Yarn queue name you want the job executed in

Re: How can I enforce a specific queue when I execute a Hive job ?

Trifacta_Alumni — Wed, 03 Jan 2018 10:59:59 GMT

The first answer is to route any Hadoop jobs sourced from Hive to a specific YARN job queue.

But if you need to route any Trifacta jobs to a specific Yarn queue, you need to specify the queue name in Trifacta config file (/opt/trifacta/conf/trifacta-conf.json) under "spark.props.spark.yarn.queue": "default" (replace default by your queue name).