Could not find uri with dfs.encryption.key.provider.uri key to create a Provider key in HDFS encryption for CDH 5.4

Question

CDH Version: CDH5.4.5

Problem. When HDFS encryption is enabled using KMS, available in Hadoop CDH 5.4, receiving an error while entering a file into the encryption zone.

Steps:

The steps to encrypt Hadoop are as follows:

• Key Creation [SUCCESS]

  [tester@master ~]$ hadoop key create 'TDEHDP' -provider kms://https@10.1.118.1/key_generator/kms -size 128 tde group has been successfully created with options Options{cipher='AES/CTR/NoPadding', bitLength=128, description='null', attributes=null}. KMSClientProvider[https://10.1.118.1/key_generator/kms/v1/] has been updated. 

2. Creating a directory [SUCCESS]

 [tester@master ~]$ hdfs dfs -mkdir /user/tester/vs_key_testdir 

3. Adding an encryption zone [SUCCESS]

    [tester@master ~]$ hdfs crypto -createZone -keyName 'TDEHDP' -path /user/tester/vs_key_testdir Added encryption zone /user/tester/vs_key_testdir 

4. Copy file to encryption Zone [ERROR]

    [tdetester@master ~]$ hdfs dfs -copyFromLocal test.txt /user/tester/vs_key_testdir 

09/15/04 06:06:33 ERROR hdfs.KeyProviderCache: Could not find uri with key [dfs.encryption.key.provider.uri] to create keyProvider !! copyFromLocal: KeyProvider not configured, encrypted file unavailable 15/09/04 06:06:33 ERROR hdfs.DFSClient: unable to close index 20823 org.apache.hadoop.ipc.RemoteException (org.apache.hadoop.hdfs.server.namenode .LeaseExpiredException): No lease on / user / tester / vs _key_testdir / test.txt.COPYING (inode 20823): File does not exist. the holder DFSClient_NONMAPREDUCE_1061684229_1 does not have open files.

Any idea / suggestion would be helpful.

Answer 1

This issue was cross-cutting here: https://community.cloudera.com/t5/Storage-Random-Access-HDFS/Could-not-find-uri-with-key-dfs-encryption-key-provider-uri-to/ td-p / 31637

Main conclusion: this is not a problem

Here is the answer that was provided by the support staff:

The main versions of the CDH release are: base. The hotfix for a harmless log entry due to HDFS-7931 is present in all CDH5 releases since CDH 5.4.1.

If you see this error in the context of configuring KMS, then it is worth considering. If you do not use KMS or EZ, then the error can be ignored. Alternatively, you can upgrade to the latest CDH5 (5.4.x or 5.5.x) to get a bug fix that makes an error only when in the context of KMS it is configured on an encrypted path.

In your fragment of the log I do not see a problem (canary does not seem to work?) If you are trying to report an error, please send us more characteristics of the failure, since the HDFS-7931 is a minor to issue an unnecessary printout of the log.