What is the difference between -hivevar and -hiveconf?

Question

From the hive -h:

--hiveconf <property=value> Use value for given property --hivevar <key=value> Variable subsitution to apply to hive commands. eg --hivevar A=B 

Answer 1

I did not quite understand that the examples from the documentation were adequate, so I tried to answer.

In the beginning there was only --hiveconf , and the replacement of variables did not exist.

The --hiveconf parameter allowed users to set hive configuration values from the command line and that’s it. All hive configuration values ​​are stored in the hiveconf namespace, i.e. hiveconf:mapred.reduce.tasks . These values ​​allowed you to control things like the number of cards and reducers if status messages should be displayed, and if the script should continue on error.

Later a replacement was added. This meant that you can now use variables in queries with the syntax ${...} . However, the only variables you could set from the command line were in the hiveconf namespace using --hiveconf so that users put their variables.

Putting your personal variables in the Hive configuration namespace will probably not break anything, but it is also not a very good form. It was later suggested that the hivevar namespace be added specifically for custom variables, which could also be defined on the command line with --hivevar . This meant a cleaner separation between hive configuration values ​​and user variables.

In short:
The hiveconf and --hiveconf should be used to set the Hive configuration values.
The hivevar and --hivevar should be used to define custom variables.
Setting custom variables in the hiveconf namespace hiveconf probably not break anything, but is not recommended.

Answer 2

@Llama explained this in detail, along with the fact that both types of variables are available in different ways.

The variables --hivevar accessed using ${var-name} , and --hiveconf is --hiveconf by ${hiveconf:var-name} .

eg. Below is the access variable and type its value in the bush.

hivevar:

 hive --hivevar a='this is a' -e '!echo ${a};' 

way out: this is a

hiveconf:

 hive --hiveconf a='this is a' -e '!echo ${hiveconf:a};' 

way out: this is a

Answer 3

No difference other than a namespace. hiveconf and hivevar are different namespaces. The hivevar namespace was added simply to separate the property namespace from the variable namespace. See https://issues.apache.org/jira/browse/HIVE-2020 for more details.

Answer 4

You can specify this for a difference.

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+VariableSubstitution

There are three namespaces for variables - hiveconf, system and env. ( Custom variables can also be created in a separate namespace with the define or hivevar option in Hive versions 0.8.0 and later.)

Answer 5

In addition to the above, we can also use them at the beginning of the script as:

hiveconf:

 SET this_dt = CURRENT_DATE; select ${hiveconf:this_dt}; 

hivevar:

 set hivevar:cur_dt=current_date; select ${hivevar:cur_dt};