Geek Answers Handbook

Hive 0.13 msck repair table only lists sections not in the metastor

I am trying to use the Hive command (0.13) msck repair tableto restore partitions, and it lists only partitions not added to the metastor, instead of adding them to the metastor.

here is the output of the command

partitions not in metastore externalexample:CreatedAt=26 04%3A50%3A56 UTC 2014/profileLocation="Chicago"

here how i create an external table

CREATE EXTERNAL TABLE IF NOT EXISTS ExternalExample(
        tweetId BIGINT, username STRING,
        txt STRING, CreatedAt STRING,
        profileLocation STRING,
        favc BIGINT,retweet STRING,retcount BIGINT,followerscount BIGINT)
    COMMENT 'This is the Twitter streaming data'
    ROW FORMAT DELIMITED
    FIELDS TERMINATED BY '\t'
    STORED AS TEXTFILE
    location '/user/hue/exttable/';

Did I miss something?

+4
source share
6 answers

Please make sure that the name of the partitions defined in the table definition matches the name of the partition in hdfs.

For example, in the example of creating a table, I see that you did not define any parries at all.

I think you want to do something like this (pay attention to using PARTITIONED BY):

create external table ExternalExample(tweetId BIGINT, username STRING, txt STRING,favc BIGINT,retweet STRING,retcount BIGINT,followerscount BIGINT) PARTITIONED BY (CreatedAt STRING, profileLocation STRING) COMMENT 'This is the Twitter streaming data' ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE location '/user/hue/exttable/';

hdfs :

/user/hue/exttable/CreatedAt=<someString>/profileLocation=<someString>/your-data-file
+4

MSCK REPAIR TABLE, , , ( ).

ALTER TABLE ADD PARTITION, , , , HDFS, "" , ,

, MSCK REPAIR TABLE .

, , ALTER TABLE ADD PARTITION. , .

+3

Hive . , , HDFS ( hasoop fs -put), .

ALTER TABLE ExternalExample ADD PARTITION

, ,

MSCK REPAIR TABLE ExternalExample; , HDFS, .

Ref https://issues.apache.org/jira/browse/HIVE-874

+1

MSCK REPAIR TABLE ExternalTable , -, hive 1.2.1, ALTER TABLE ExternalTable RECOVER PARTITION, , , , .. /some_external_path/mypartion=01 , /some_external_path/myParition=01 ;

profileLocation profileLocation profile_location , .

Hive 1.2

+1

1) You need to specify the sections 2) The section names must have all lowercase letters. See This https://singhanuvrat.com/hive-partition-column-name-camelcase-bad-idea-b89796d4e741#.16d7uqfot

0
source

You just do not have enough PARTITIONED BY (CreatedAt STRING, profileLocation STRING).

-1
source

More articles:


All Articles