I have Apache Kafka in version 0.9.x with the built-in KafkaCSVReporter for metrics enabled.
kafka.metrics.polling.interval.secs=5 kafka.metrics.reporters=kafka.metrics.KafkaCSVMetricsReporter kafka.csv.metrics.dir=/tmp/kafka/metrics kafka.csv.metrics.reporter.enabled=true
I set the directory for the generated csv files and started processing the data (client-manufacturer). There are several files in my csv directory:
BytesRejectedPerSec.csv LeaderCount.csv PurgatorySize.csv RequestQueueTimeMs.csv ResponseQueueTimeMs.csv ResponseSendTimeMs.csv ThrottleTimeMs.csv TotalFetchRequestsPerSec.csv TotalTimeMs.csv
Unfortunately, there are none that are most needed: ByteInPerSec and ByteOutPerSec. Interestingly, the interesting fact is that when I connect JMX with the Kafka server using jconsole, I can see the MBeans ByteInPerSec and ByteOutPerSec tabs in the metrics, but csv files are not generated for these indicators. What for? Should I set any additional properties in the configuration file? Can I get these indicators in csv format? I want to say that the Kafka server metric (not the producer / consumer metrics) is interesting to me. Thank you in advance for any help. Regards Marcin
metrics apache-kafka
Marcin kaptacz
source share