Compression confirmation is allowed for specific Hive tables.

Question

I need to compare a number of tables, some are compressed and some are not. I compress by installing:

hive> SET hive.exec.compress.output=true;
hive> SET mapred.output.compression.codec=org.apache.hadoop.io.compress.SnappyCodec;

... and using INSERT OVERWRITE to populate the table. Is there a way to confirm through the command line (similar to DESCRIBE EXTENDED) that output compression is enabled for a particular table?

Answer 1

Below is the path to table files in hdfs

desc formatted <tablename>

hive> desc formatted cobtest; Ok col_name data_type comment

computer_name data_type comment

batch_sk int No String geo_cd No string env_cd No

Details of the table

: : steve CreateTime: 21 23:36:37 PST 2013 LastAccessTime: UNKNOWN : : 0 : hdfs://localhost: 9000/user/hive/warehouse/cobtest : MANAGED_TABLE :   numFiles 1   numPartitions 0   numRows 0   rawDataSize 0   totalSize 473   transient_lastDdlTime 1385105797

SerDe: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe InputFormat: org.apache.hadoop.mapred.TextInputFormat OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat : : -1 : [] : [] Desc Params:   field.delim\t   serialization.format\t , : 0.203 , : 34 ()

dfs -lsr, , .

hive> dfs -lsr hdfs://localhost:9000/user/hive/warehouse/cobtest ;
-rw-r--r--   1 steve supergroup        473 2013-11-21 23:36 /user/hive/warehouse/cobtest/UDFPafCobIndTest.input**.tsv**

SNAPPY - . :

SET hive.exec.compress.output=true;
SET mapred.output.compression.codec=org.apache.hadoop.io.compress.SnappyCodec;
SET mapred.output.compression.type=BLOCK;

Answer 2

describe formatted orc_with_compress_setting_table, - :

Compressed: No .

:

, . "", , .

from: https://www.cloudera.com/documentation/enterprise/5-5-x/topics/impala_describe.html