How can I guarantee that magazines are kept forever in Kafka?

Question

I need to configure Kafka to never delete logs. Looking at their documentation, I see two kinds of parameters that control this:

• The "Journal Cleaner" described at http://kafka.apache.org/documentation.html#compaction . Setting log.cleaner.enable = false seems to prevent janitor from deleting or compressing logs.

• The logging mechanism described in http://kafka.apache.org/documentation.html#configuration in the log.retention.<time scales> section. Setting log.retention.ms = -1 seems to signal endless hold.

My question is:

What is the relationship between these two mechanisms, are they the only two, and are the above settings necessary and sufficient to prevent the deletion of logs?

Answer 1

Not sure which version you are using. For me, I use this:

 log.retention.hours=2147483647 

It is approximately 245,000 years old. I believe that at some point I tried -1 , but used the maximum value. Perhaps log.retention.ms takes different values, but independently - the second of your configuration options should be sufficient. My guess is that no option is needed; either will be enough. Of course, the second in itself is enough.