{"id":2257,"date":"2022-04-22T09:09:17","date_gmt":"2022-04-22T09:09:17","guid":{"rendered":"https:\/\/blog.samarthya.me\/wps\/?p=2257"},"modified":"2022-04-22T09:12:29","modified_gmt":"2022-04-22T09:12:29","slug":"jmx-monitoring-and-managing-kafka","status":"publish","type":"post","link":"https:\/\/blog.samarthya.me\/wps\/2022\/04\/22\/jmx-monitoring-and-managing-kafka\/","title":{"rendered":"JMX: Monitoring and Managing Kafka"},"content":{"rendered":"\n
\"\"<\/div>
Oracle<\/div><\/div>

The Java virtual machine (Java VM) has built-in instrumentation that enables you to monitor and manage it using the Java Management Extensions (JMX) technology. <\/p><\/div><\/div>\n\n\n\n

To enable and configure monitoring of Java VM for any java-application using the ready-to-use JMX agent, you must set certain system properties when you start the Java VM.<\/p>\n\n\n\n

-Doptions1=value1<\/code><\/pre>\n\n\n\n
The default location for the configuration file is <\/p>\n\n\n\n
jre_home\/lib\/management\/management.properties<\/code><\/pre>\n\n\n\n
Or you can set using <\/p>\n\n\n\n
-Dcom.sun.management.config.file=<location><\/code><\/pre>\n\n\n\n
Example: On my OSX<\/h3>\n\n\n\n
\/usr\/local\/Cellar\/openjdk@11\/11.0.14.1\/libexec\/openjdk.jdk\/Contents\/Home\/conf\/management<\/code><\/pre>\n\n\n\n
You can set ready-to-use monitoring and management properties in a configuration file or on the command line. Properties are defined here<\/a>.<\/p>Oracle Documentation<\/cite><\/blockquote><\/figure>\n\n\n\n
Monitoring: Kafka<\/h2>\n\n\n\n
Now let’s configure JMX monitoring specifically for monitoring Kafka Broker. <\/p>\n\n\n\n
Configuration Assumptions<\/h3>\n\n\n\n
  • Kafka SSL configuration is enabled (keystore and truststore)<\/li><\/ul>\n\n\n\n
    Configured as a Service<\/h3>\n\n\n\n
    If your kafka broker is configured as service you need two basic things as environment<\/p>\n\n\n\n
    Environment=\"JMX_PORT=10167\"\nEnvironment=\"KAFKA_JMX_OPTS=-Dcom.sun.management.config.file=\/opt\/kafka\/config\/jmx.kafka\"<\/code><\/pre>\n\n\n\n
    You might recognize the com.sun.management.config.file<\/code>. This is the same parameter that defines the options for the JMX configuration file location.<\/p>\n\n\n\n
    jmx.kafka<\/code><\/h4>\n\n\n\n
    In this file the contents identify the specific settings enabled for the JMX monitoring (explained below)<\/p>\n\n\n\n
    com.sun.management.jmxremote=true\ncom.sun.management.jmxremote.port=10167\ncom.sun.management.jmxremote.rmi.port=10168\njava.rmi.server.hostname=broker1.test\ncom.sun.management.jmxremote.password.toHashes=true\njava.net.preferIPv4Stack=true\ncom.sun.management.jmxremote.ssl.config.file=\/opt\/kafka\/config\/jmxremote.ssl\ncom.sun.management.jmxremote.ssl=true\ncom.sun.management.jmxremote.registry.ssl=true\ncom.sun.management.jmxremote.authenticate=true\ncom.sun.management.jmxremote.ssl.need.client.auth=true\ncom.sun.management.jmxremote.password.file=\/opt\/kafka\/config\/jmxremote.password\ncom.sun.management.jmxremote.access.file=\/opt\/kafka\/config\/jmxremote.access<\/code><\/pre>\n\n\n\n
    Let’s look in some detail<\/p>\n\n\n\n
    \n
    \n
    \n
    \n
    com.sun.management.jmxremote<\/div><\/div>
    \n
    In the current Java SE platform, it is no longer necessary to set this system property.<\/p>\n<\/div><\/div><\/div><\/div>\n\n\n\n
    com.sun.management.jmxremote.port<\/div><\/div>
    \n
    • To enable monitoring and management from remote systems define the jmremote.port<\/code><\/li>
    • Remote monitoring and management requires security to ensure that unauthorized persons cannot control or monitor your application. Password authentication over the Secure Sockets Layer (SSL) and Transport Layer Security (TLS) is enabled by default. <\/li><\/ul>\n<\/div><\/div><\/div><\/div>\n\n\n\n
      com.sun.management.jmxremote.rmi.port<\/div><\/div>
      \n
      • Port number to enable JMX RMI connection.<\/li>
      • In addition to publishing an RMI connector for local access, setting this property publishes an additional RMI connector in a private read-only registry at the specified port using the name, jmxrmi<\/code>. <\/li><\/ul>\n<\/div><\/div><\/div><\/div>\n\n\n\n
        java.rmi.server.hostname<\/div><\/div>
        \n
        • For remote stubs to be associated with a specific interface address, the java.rmi.server.hostname<\/code> system property must be set.<\/li>
        • Our example uses a vagrant host broker1.test<\/code><\/li><\/ul>\n<\/div><\/div><\/div><\/div>\n\n\n\n
          com.sun.management.jmxremote.password.toHashes<\/div><\/div>
          \n
          If passwords are in clear, they will be overwritten by their hash <\/p>\n<\/div><\/div><\/div><\/div>\n\n\n\n
          com.sun.management.jmxremote.ssl.config.file<\/div><\/div>
          \n
          Specifies the location of the SSL configuration file.<\/p>\n\n\n\n
          Contents<\/h4>\n\n\n\n
          javax.net.ssl.keyStore=\/opt\/kafka\/ssl\/kafka.broker.keystore.jks\njavax.net.ssl.keyStorePassword=changeit\njavax.net.ssl.trustStore=\/opt\/kafka\/ssl\/kafka.broker.truststore.jks\njavax.net.ssl.trustStorePassword=changeit<\/code><\/pre>\n<\/div><\/div><\/div><\/div>\n\n\n\n
          com.sun.management.jmxremote.ssl<\/div><\/div>
          \n
          • Default value is true<\/code><\/li>
          • Enables secure monitoring using SSL.<\/li><\/ul>\n<\/div><\/div><\/div><\/div>\n\n\n\n
            com.sun.management.jmxremote.registry.ssl<\/div><\/div>
            \n
            • Binds the RMI connector stub to an RMI registry that is protected by SSL.<\/li><\/ul>\n<\/div><\/div><\/div><\/div>\n\n\n\n
              com.sun.management.jmxremote.authenticate<\/div><\/div>
              \n
              • Enables or disables password authentication for JMX.<\/li><\/ul>\n<\/div><\/div><\/div><\/div>\n\n\n\n
                com.sun.management.jmxremote.ssl.need.client.auth<\/div><\/div>
                \n
                • Performs client authentication if this property is true<\/code> and the property com.sun.management.jmxremote.ssl<\/code> is also true<\/code>.<\/li><\/ul>\n<\/div><\/div><\/div><\/div>\n\n\n\n
                  com.sun.management.jmxremote.password.file<\/div><\/div>
                  \n
                  • Specifies the location for the password file.<\/li>
                  • If com.sun.management.jmxremote.authenticate<\/code> is false<\/code>, then this property, and the password and access files are ignored. Otherwise, the password file must exist and be in the valid format. <\/li>
                  • If the password file is empty or nonexistent, then no access is allowed.<\/li><\/ul>\n<\/div><\/div><\/div><\/div>\n\n\n\n
                    com.sun.management.jmxremote.access.file<\/div><\/div>
                    \n
                    • Specifies the location for the access file. If com.sun.management.jmxremote.authenticate<\/code> is false<\/code>, then this property, and the password and access files, are ignored. Otherwise, the access file must exist and be in the valid format. <\/li>
                    • If the access file is empty or nonexistent, then no access is allowed.<\/li><\/ul>\n<\/div><\/div><\/div><\/div>\n<\/div><\/div>\n<\/div>\n<\/div>\n<\/div><\/div>\n\n\n\n
                      Once all the configuration is supplied and the service is running you can run JConsole and try and connect to the JMX port. <\/p>\n\n\n\n
                      jconsole<\/strong> -debug -J-Djavax.net.ssl.trustStore=kafka.broker.truststore.jks -J-Djavax.net.ssl.trustStorePassword=changeit -J-Djavax.net.ssl.keyStore=kafka.broker.keystore.jks -J-Djavax.net.ssl.keyStorePassword=changeit<\/code><\/pre>\n\n\n\n
                      \"\"
                      JConsole to connect and view<\/figcaption><\/figure><\/div>\n\n\n\n
                      \"\"<\/figure><\/div>\n","protected":false},"excerpt":{"rendered":"
                      Oracle The Java virtual machine (Java VM) has built-in instrumentation that enables you to monitor and manage it using the Java Management Extensions (JMX) technology.  To enable and configure monitoring of Java VM for any java-application using the ready-to-use JMX agent, you must set certain system properties when you start the Java VM. The default […]<\/p>\n","protected":false},"author":1,"featured_media":2261,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"image","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[34],"tags":[255,256],"class_list":["post-2257","post","type-post","status-publish","format-image","has-post-thumbnail","hentry","category-technical","tag-jmx","tag-monitoring","post_format-post-format-image"],"_links":{"self":[{"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/posts\/2257","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/comments?post=2257"}],"version-history":[{"count":7,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/posts\/2257\/revisions"}],"predecessor-version":[{"id":2267,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/posts\/2257\/revisions\/2267"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/media\/2261"}],"wp:attachment":[{"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/media?parent=2257"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/categories?post=2257"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/tags?post=2257"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}