{"id":490,"date":"2020-03-29T17:37:33","date_gmt":"2020-03-29T17:37:33","guid":{"rendered":"https:\/\/www.samarthya.me\/wps\/?p=490"},"modified":"2020-03-29T17:51:30","modified_gmt":"2020-03-29T17:51:30","slug":"text-elastic","status":"publish","type":"post","link":"https:\/\/blog.samarthya.me\/wps\/2020\/03\/29\/text-elastic\/","title":{"rendered":"Text – Elastic"},"content":{"rendered":"\n

If you are not aware of mapping, it might be a little confusing but I would try my best.<\/p>\n\n\n\n

Every document that gets indexed in elastic has fields, every field that is accessible has an associated type with it (Inverted Index) which helps elastic organize data for quick retrieval.<\/p>\n\n\n\n

I create a sample index my_index with a field country_name<\/code> that was auto-mapped as I added the value.<\/p>\n\n\n\n

\n
POST my_index\/_doc\/1\n {\n   \"country_name\": \"The United States of America\"\n }\n\n\nGET my_index\/_mapping\n\n{\n   \"my_index\" : {\n     \"mappings\" : {\n       \"properties\" : {\n         \"country_name\" : {\n           \"type\" : \"text\",\n           \"fields\" : {\n             \"keyword\" : {\n               \"type\" : \"keyword\",\n               \"ignore_above\" : 256\n             }\n           }\n         }\n       }\n     }\n   }\n }<\/pre>\n<\/div><\/div>\n\n\n\n
Text<\/h2>\n\n\n\n
As evident the default type provided to the country_name<\/code> is TEXT, and another something called fields<\/code> (Multi-fields which I will touch upon later).<\/p>\n\n\n\n
From the official documentation<\/p>\n\n\n\n
A field to index full-text values, such as the body of an email or the description of a product. These fields are analyzed<\/code>, that is they are passed through an analyzer<\/a> to convert the string into a list of individual terms before being indexed.<\/p>@elastic.co<\/cite><\/blockquote>\n\n\n\n
If you need to index structured content it is better to use Keyword (the one in the fields as shown above)<\/p>\n\n\n\n
Keyword is typically used for Filtering, like show all the movies of category comedy.<\/p>\n\n\n\n
What is the difference?<\/h3>\n\n\n\n
One is for full text search and the other for aggregations and sorting.<\/p>\n\n\n\n
\n
A simple search request<\/p>\n\n\n\n
GET my_index\/_search<\/pre>\n\n\n\n
Since there is only one document currently it should return something like<\/p>\n\n\n\n
{
   \"took\" : 1,
   \"timed_out\" : false,
   \"_shards\" : {
     \"total\" : 1,
     \"successful\" : 1,
     \"skipped\" : 0,
     \"failed\" : 0
   },
   \"hits\" : {
     \"total\" : {
       \"value\" : 1,
       \"relation\" : \"eq\"
     },
     \"max_score\" : 1.0,
     \"hits\" : [
       {
         \"_index\" : \"my_index\",
         \"_type\" : \"_doc\",
         \"_id\" : \"1\",
         \"_score\" : 1.0,
         \"_source\" : {
           \"country_name\" : \"The United States of America\"
         }
       }
     ]
   }
 }<\/pre>\n\n\n\n
Let us run an aggregation to highlight the difference<\/p>\n\n\n\n
GET my_index\/_search
 {
   \"aggs\": {
     \"Top Country\": {
       \"terms\": {
         \"field\": \"country_name\",
         \"size\": 10
       }
     }
   }
 }<\/pre>\n\n\n\n
What do you guess the response will be? I am running a terms<\/code> aggregation on country_name<\/code>.<\/p>\n\n\n\n
You should get an error like below<\/p>\n\n\n\n
\"error\": {
     \"root_cause\": [
       {
         \"type\": \"illegal_argument_exception\",
         \"reason\": \"Fielddata is disabled on text fields by default. Set fielddata=true on [country_name] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.\"
       }
     ],<\/pre>\n\n\n\n
If it is simple TEXT<\/code> it will be broken down to be indexed, to make the following query possible<\/p>\n\n\n\n
GET my_index\/_search
 {
   \"size\": 0, 
   \"query\": {
     \"match\": {
       \"country_name\": \"America\"
     }
   }
 }<\/pre>\n\n\n\n
It should respond as under<\/p>\n\n\n\n
{
   \"took\" : 1,
   \"timed_out\" : false,
   \"_shards\" : {
     \"total\" : 1,
     \"successful\" : 1,
     \"skipped\" : 0,
     \"failed\" : 0
   },
   \"hits\" : {
     \"total\" : {
       \"value\" : 1,
       \"relation\" : \"eq\"
     },
     \"max_score\" : null,
     \"hits\" : [ ]
   }
 }<\/pre>\n\n\n\n
But if I aggregate and since it is broken down it wont work, and hence the error as seen above. If I want that agg<\/code> to work, I will simply modify the query as under.<\/p>\n\n\n\n
GET my_index\/_search
 {
   \"aggs\": {
     \"Top Country\": {
       \"terms\": {
         \"field\": \"country_name.keyword\",
         \"size\": 10
       }
     }
   }
 }<\/pre>\n\n\n\n
and when executed it shall return something like<\/p>\n\n\n\n
{
\n  \"took\" : 1,
\n  \"timed_out\" : false,
\n  \"_shards\" : {
\n    \"total\" : 1,
\n    \"successful\" : 1,
\n    \"skipped\" : 0,
\n    \"failed\" : 0
\n  },
\n  \"hits\" : {
\n    \"total\" : {
\n      \"value\" : 1,
\n      \"relation\" : \"eq\"
\n    },
\n    \"max_score\" : 1.0,
\n    \"hits\" : [
\n      {
\n        \"_index\" : \"my_index\",
\n        \"_type\" : \"_doc\",
\n        \"_id\" : \"1\",
\n        \"_score\" : 1.0,
\n        \"_source\" : {
\n          \"country_name\" : \"The United States of America\"
\n        }
\n      }
\n    ]
\n  },
\n  \"aggregations\" : {
\n    \"Top Country\" : {
\n      \"doc_count_error_upper_bound\" : 0,
\n      \"sum_other_doc_count\" : 0,
\n      \"buckets\" : [
\n        {
\n          \"key\" : \"The United States of America\",
\n          \"doc_count\" : 1
\n        }
\n      ]
\n    }
\n  }
\n}<\/pre>\n<\/div><\/div>\n\n\n\n
Simple yet effective.<\/p>\n\n\n\n
Helpful links<\/p>\n\n\n\n