{"id":709,"date":"2020-06-05T04:18:06","date_gmt":"2020-06-05T04:18:06","guid":{"rendered":"https:\/\/blog.samarthya.me\/wps\/?p=709"},"modified":"2020-06-05T04:33:38","modified_gmt":"2020-06-05T04:33:38","slug":"ldap-realm-in-elasticsearch","status":"publish","type":"post","link":"https:\/\/blog.samarthya.me\/wps\/2020\/06\/05\/ldap-realm-in-elasticsearch\/","title":{"rendered":"LDAP Realm in Elasticsearch"},"content":{"rendered":"<p>It is a short <strong>howto<\/strong> guide which shall talk about adding a LDAP realm to an Elasticsearch 7.6.2 cluster.<\/p>\n<h2>Realms<\/h2>\n<p>It is a concept not alien to IDAM professionals. In simple words it is a system that allows you to authenticate and authorize an incoming request. It will define the entity USER, that may include at a bare minimum (not limited to)<\/p>\n<ul>\n<li>A unique name, which can be queried upon<\/li>\n<li>Password, which can be validated<\/li>\n<li>Roles that defines its access privileges for resources mapped to.<\/li>\n<\/ul>\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>The official documentation is available <a rel=\"noreferrer noopener\" href=\"https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/reference\/7.x\/realms.html\" target=\"_blank\">here<\/a><\/p><\/blockquote>\n\n\n\n<h2 class=\"wp-block-heading\">Prerequisites<\/h2>\n\n\n\n<p>To being with, have a local ldap instance configured. In my case I am using Openldap configured on a Centos 7.0 machine ldap.myserver.com.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>GET _cat\/nodes?pretty\n\n100.9.48.79   60 47 2 0.14 0.23 0.19 dil  - node1.elasticsearch.node\n100.9.32.43   58 93 4 0.92 1.01 0.88 dilm * node2.elasticsearch.node\n100.8.122.121 68 72 2 0.27 0.23 0.18 dil  - node3.elasticsearch.node\n100.7.144.95  51 92 1 0.37 0.40 0.41 dilm - master.elasticsearch.node<\/code><\/pre>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>The master node is <code>node2.elasticsearch.node<\/code><\/p><\/blockquote>\n\n\n\n<p>For managing my ldap &#8211; <a href=\"https:\/\/ldap.myserver.com\">ldap.myserver.com<\/a>, I am using a free tool <strong>Jxplorer<\/strong> to connect. The connection settings are as under<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>myserver.ldap=ldap.myserver.com\nmyserver.ldap.11=\nmyserver.ldap.3=636\nmyserver.ldap.5=LDAP v3\nmyserver.ldap.7.1=\nmyserver.ldap.7.5=false\nmyserver.ldap.8.1=SSL + User + Password\nmyserver.ldap.8.3=cn\\=Manager,dc\\=saurabh,dc\\=com<\/code><\/pre>\n\n\n\n<p>I have pre-configured few users and groups in my ldap.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>version: 1\ndn: dc=saurabh,dc=com\nobjectClass: dcObject\nobjectClass: organization\nobjectClass: top\ndc: saurabh\ndescription: My Ldap Server\no: saurabh.com\n\ndn: cn=Manager,dc=saurabh,dc=com\nobjectClass: organizationalRole\nobjectClass: top\ncn: Manager\ndescription: Directory Manager\n\ndn: ou=People,dc=saurabh,dc=com\nobjectClass: organizationalUnit\nobjectClass: top\nou: People\n\ndn: uid=samarthya,ou=People,dc=saurabh,dc=com\nobjectClass: account\nobjectClass: posixAccount\nobjectClass: shadowAccount\nobjectClass: top\ncn: samarthya\ngidNumber: 7002\nhomeDirectory: \/home\/samarthya\nloginShell: \/bin\/bash\nou: cn=Manager,ou=Group,dc=saurabh,dc=com\nshadowLastChange: 0\nshadowMax: 0\nshadowWarning: 0\nuid: samarthya\nuidNumber: 8002\nuserPassword:: e1NTSEF9ZGc2VDgxbVZwaFJGcjZNd2lUWjVwQVVTbTZybjNPU2RIZ09FN2thQ\n VoyWTV6UU8xREFON3FRPT0=\n\ndn: uid=saurabh,ou=People,dc=saurabh,dc=com\nobjectClass: account\nobjectClass: posixAccount\nobjectClass: shadowAccount\nobjectClass: top\ncn: saurabh\ndescription: Saurabh Sharma\ngecos: administrators\ngidNumber: 7001\nhomeDirectory: \/home\/samarthya\nloginShell: \/bin\/bash\nou: cn=analysts,ou=Group,dc=saurabh,dc=com\nuid: saurabh\nuidNumber: 8001\nuserPassword:: e1NTSEF9ZGc2VDgxbVZwaFJGcjZNd2lUWjVwQVVTbTZybjNPU2RIZ09FN2thQ\n VoyWTV6UU8xREFON3FRPT0=\n\n\ndn: uid=user1,ou=People,dc=saurabh,dc=com\nobjectClass: account\nobjectClass: posixAccount\nobjectClass: shadowAccount\nobjectClass: top\ncn: user1\ndescription: user 1\ngidNumber: 7006\nhomeDirectory: \/home\/user1\nloginShell: \/bin\/bash\nou: cn=administrators,ou=Group,dc=saurabh,dc=com\nuid: user1\nuidNumber: 8006\nuserPassword:: e1NTSEF9ZGc2VDgxbVZwaFJGcjZNd2lUWjVwQVVTbTZybjNPU2RIZ09FN2thQ\n VoyWTV6UU8xREFON3FRPT0=\n\ndn: uid=user2,ou=People,dc=saurabh,dc=com\nobjectClass: account\nobjectClass: posixAccount\nobjectClass: shadowAccount\nobjectClass: top\ncn: user2\ndescription: User Two\ngidNumber: 7007\nhomeDirectory: \/home\/user2\nou: cn=readers,ou=Group,dc=saurabh,dc=com\nuid: user2\nuidNumber: 8007\nuserPassword:: e1NTSEF9S2FQZE4ybW56V3k4aHBDUVpURk9KdW5PMm9aZmdQNXpwb3pHNC81R\n XhtMk8rU0oxK01RUWFRPT0=\n\ndn: uid=user3,ou=People,dc=saurabh,dc=com\nobjectClass: account\nobjectClass: posixAccount\nobjectClass: shadowAccount\nobjectClass: top\ncn: user3\ngidNumber: 7008\nhomeDirectory: \/home\/user3\nloginShell: \/bin\/bash\nou: cn=readers,ou=Group,dc=saurabh,dc=com\nuid: user3\nuidNumber: 8008\nuserPassword:: e1NTSEF9eTdsZFZFaUlJUUVwYjF3R3VybG96dFhWSE9XV2ZsTVpHT1VPakxXR\n nZIdXg0YTg0dXFRdzJRPT0=\n\ndn: ou=Group,dc=saurabh,dc=com\nobjectClass: organizationalUnit\nobjectClass: top\nou:: R3JvdXAg\n\ndn: cn=Manager,ou=Group,dc=saurabh,dc=com\nobjectClass: posixGroup\nobjectClass: top\ncn: Manager\ngidNumber: 1005\nmemberUid: uid=samarthya,ou=People,dc=saurabh,dc=com\n\ndn: cn=administrators,ou=Group,dc=saurabh,dc=com\nobjectClass: posixGroup\nobjectClass: top\ncn: administrators\ndescription: Administrators\ngidNumber: 1009\nmemberUid: uid=saurabh,ou=People,dc=saurabh,dc=com\nmemberUid: uid=user1,ou=People,dc=saurabh,dc=com\n\ndn: cn=analysts,ou=Group,dc=saurabh,dc=com\nobjectClass: posixGroup\nobjectClass: top\ncn: analysts\ngidNumber: 1008\nmemberUid: uid=saurabh,ou=People,dc=saurabh,dc=com\n\n\ndn: cn=readers,ou=Group,dc=saurabh,dc=com\nobjectClass: posixGroup\nobjectClass: top\ncn: readers\ngidNumber: 1006\nmemberUid: uid=user2,ou=People,dc=saurabh,dc=com\nmemberUid: uid=user3,ou=People,dc=saurabh,dc=com<\/code><\/pre>\n\n\n\n<p>I have security enabled for my elasticsearch cluster and SSL communication as well (using self-signed certificates)<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Elasticsearch Configuration<\/h2>\n\n\n\n<pre class=\"wp-block-code\"><code>xpack.security.enabled: true\nxpack:\n  security:\n    authc:\n      realms:\n        native.secops:\n          order: 0\n        ldap.secldap:\n          order: 1\n          bind_dn: \"cn=Manager,dc=saurabh,dc=com\"\n          url: \"ldaps:\/\/ldap.myserver.com:636\"\n          user_search:\n            base_dn: \"ou=people,dc=saurabh,dc=com\"\n            filter: \"(uid={0})\"\n          group_search:\n            base_dn: \"ou=group,dc=saurabh,dc=com\"\n          ssl:\n            verification_mode: none\n            certificate_authorities: &#91;\"\/etc\/pki\/ldapserver.crt\"]<\/code><\/pre>\n\n\n\n<p>As you can see the order for secldap is <code>1<\/code> and as result if the same user exists in both realms (native &amp; ldap) it will be first checked against native.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ldap.secldap:\n  order: 1\n  bind_dn: \"cn=Manager,dc=saurabh,dc=com\"\n  url: \"ldaps:\/\/ldap.myserver.com:636\"\n  user_search:\n    base_dn: \"ou=people,dc=saurabh,dc=com\"\n    filter: \"(uid={0})\"\n  group_search:\n    base_dn: \"ou=group,dc=saurabh,dc=com\"\n  ssl:\n    verification_mode: none\n    certificate_authorities: &#91;\"\/etc\/pki\/ldapserver.crt\"]<\/code><\/pre>\n\n\n\n<p>I have disabled the SSL verification, but if you have the ca cert added you can simply enable it. Essentially I have integrated the LDAP with the elasticsearch, once restarted this realm is available to be evaluated against.<\/p>\n\n\n\n<p>At the moment there is no role mapping defined let me create a <strong>user<\/strong> role in Kibana.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/dzone.com\/storage\/temp\/13568995-screenshot-2020-06-04-at-94248-pm.png\" alt=\"\"\/><figcaption>Role creation from Kibana<\/figcaption><\/figure>\n\n\n\n<p>Let&#8217;s use the role_mapping API to define a mapping<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>POST _security\/role_mapping\/ldapuser\n{\n  \"enabled\" : true,\n    \"roles\" : &#91;\n      \"user\"\n    ],\n    \"rules\" : {\n      \"any\": &#91;\n        {\n          \"field\":\n            {\n              \"dn\": \"*,ou=People,dc=saurabh,dc=com\"\n            }\n          \n        },\n        {\n          \"field\" : {\n            \"groups\" : \"cn=readers,ou=Group,dc=saurabh,dc=com\"\n          }\n        }\n      ]\n    },\n    \"metadata\" : {\n      \"host\" : \"ldaps:\/\/ldap.myserver.com:636\"\n    }\n}<\/code><\/pre>\n\n\n\n<p>I have defined a mapping to add the role &#8211; `user` for any USER in ldap.<\/p>\n\n\n\n<p>If call is successful it should return<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>{\n  \"role_mapping\" : {\n    \"created\" : true\n  }\n}<\/code><\/pre>\n\n\n\n<p>Now let&#8217;s use the <a rel=\"noreferrer noopener\" href=\"https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/reference\/7.x\/security-api-authenticate.html\" target=\"_blank\">_authenticate<\/a> api for a non existent user &#8211; user4 which we will soon add to the ldap.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>curl https:\/\/node1.elasticsearch.node:9200\/_security\/_authenticate?pretty -u user4:admin\n\n{\n  \"error\" : {\n    \"root_cause\" : &#91;\n      {\n        \"type\" : \"security_exception\",\n        \"reason\" : \"unable to authenticate user &#91;user4] for REST request &#91;\/_security\/_authenticate?pretty]\",\n        \"header\" : {\n          \"WWW-Authenticate\" : &#91;\n            \"Bearer realm=\\\"security\\\"\",\n            \"ApiKey\",\n            \"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\"\n          ]\n        }\n      }\n    ],\n    \"type\" : \"security_exception\",\n    \"reason\" : \"unable to authenticate user &#91;user4] for REST request &#91;\/_security\/_authenticate?pretty]\",\n    \"header\" : {\n      \"WWW-Authenticate\" : &#91;\n        \"Bearer realm=\\\"security\\\"\",\n        \"ApiKey\",\n        \"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\"\n      ]\n    }\n  },\n  \"status\" : 401\n}<\/code><\/pre>\n\n\n\n<p>As expected the user is denied access.<\/p>\n\n\n\n<p>Let&#8217;s add the user to ldap but let&#8217;s not add as a member of any group.<\/p>\n\n\n\n<figure class=\"wp-block-image is-style-rounded\"><img decoding=\"async\" src=\"https:\/\/dzone.com\/storage\/temp\/13569009-screenshot-2020-06-04-at-95231-pm.png\" alt=\"\"\/><figcaption>Adding user4 with password admin<\/figcaption><\/figure>\n\n\n\n<p>I have added user4 with password admin. Let us try the access again.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>curl https:\/\/node1.elasticsearch.node:9200\/_security\/_authenticate?pretty -u user4:admin\n{\n  \"username\" : \"user4\",\n  \"roles\" : &#91;\n    \"user\"\n  ],\n  \"full_name\" : null,\n  \"email\" : null,\n  \"metadata\" : {\n    \"ldap_dn\" : \"uid=user4,ou=People,dc=saurabh,dc=com\",\n    \"ldap_groups\" : &#91; ]\n  },\n  \"enabled\" : true,\n  \"authentication_realm\" : {\n    \"name\" : \"secldap\",\n    \"type\" : \"ldap\"\n  },\n  \"lookup_realm\" : {\n    \"name\" : \"secldap\",\n    \"type\" : \"ldap\"\n  }\n}<\/code><\/pre>\n\n\n\n<p>You can see the added user successfully authenticates, and has no ldap group membership. How about adding the membership and role mapping in roles_mapping.yml.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># Role mapping configuration file which has elasticsearch roles as keys\n# that map to one or more user or group distinguished names\n\n#roleA:   this is an elasticsearch role\n#  - groupA-DN  this is a group distinguished name\n#  - groupB-DN\n#  - user1-DN   this is the full user distinguished name\n\n#power_user:\n#  - \"cn=admins,dc=example,dc=com\"\n#user:\n#  - \"cn=users,dc=example,dc=com\"\n#  - \"cn=admins,dc=example,dc=com\"\n#  - \"cn=John Doe,cn=other users,dc=example,dc=com\"\nkibana_user:\n  - \"cn=readers,ou=Group,dc=saurabh,dc=com\"<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image is-style-rounded\"><img decoding=\"async\" src=\"https:\/\/dzone.com\/storage\/temp\/13569018-screenshot-2020-06-04-at-95905-pm.png\" alt=\"\"\/><figcaption>Added membership<\/figcaption><\/figure>\n\n\n\n<p>Once added let&#8217;s execute the _authenticate api again.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>curl https:\/\/node1.elasticsearch.node:9200\/_security\/_authenticate?pretty -u user4:admin\n{\n  \"username\" : \"user4\",\n  \"roles\" : &#91;\n    \"kibana_user\",\n    \"user\"\n  ],\n  \"full_name\" : null,\n  \"email\" : null,\n  \"metadata\" : {\n    \"ldap_dn\" : \"uid=user4,ou=People,dc=saurabh,dc=com\",\n    \"ldap_groups\" : &#91;\n      \"cn=readers,ou=Group,dc=saurabh,dc=com\"\n    ]\n  },\n  \"enabled\" : true,\n  \"authentication_realm\" : {\n    \"name\" : \"secldap\",\n    \"type\" : \"ldap\"\n  },\n  \"lookup_realm\" : {\n    \"name\" : \"secldap\",\n    \"type\" : \"ldap\"\n  }\n}<\/code><\/pre>\n\n\n\n<p>In case you do not get the group membership, please clear the realm cache which is usually valid for 20 mins.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>POST _security\/realm\/secldap\/_clear_cache<\/code><\/pre>\n\n\n\n<p>This bring us to the end of the blog. In summary, I showed how easy it is to configure the realm and once restarted you can easily add role mappings using API or role_mapping.yml. In case of questions please feel free to drop comments.<\/p>\n\n\n\n<h4 class=\"has-text-align-center wp-block-heading\">&#8212; THE &#8211; END &#8212;<\/h4>\n","protected":false},"excerpt":{"rendered":"<p>It is a short howto guide which shall talk about adding a LDAP realm to an Elasticsearch 7.6.2 cluster. Realms It is a concept not alien to IDAM professionals. In simple words it is a system that allows you to authenticate and authorize an incoming request. It will define the entity USER, that may include [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":711,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"image","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[34],"tags":[53,77,76],"class_list":["post-709","post","type-post","status-publish","format-image","has-post-thumbnail","hentry","category-technical","tag-elasticsearch","tag-openldap","tag-realms","post_format-post-format-image"],"_links":{"self":[{"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/posts\/709","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/comments?post=709"}],"version-history":[{"count":0,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/posts\/709\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/media\/711"}],"wp:attachment":[{"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/media?parent=709"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/categories?post=709"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/tags?post=709"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}