Terraform OpenSearch module Part 1

Hi, devops fans. Current lecture is the 1st part from series of articles devoted to the theme how to deploy a high available OpenSearch cluster at AWS. Let’s skip to our terraform module and investigate our physical file’s structure at first

Most of the logic is placed in the es.tf file. Current file is not so big from content side aspect. But it has a lot of details worth of discussion. That is why we will go step by step via every terrafrom section. I will provide explanations gradually and in the end the whole terrafrom code would be represented.

First of all we have to define an ES service linked role. That will allow ES to manage AWS resources for the opensearch domain

resource "aws_iam_service_linked_role" "es" {
  aws_service_name = "es.amazonaws.com"
}

OK, great Sergii, but what is domain? For sure such a question will appear in your mind if you are working with OpenSearch for the first time. OpenSearch domain is synonymous with an OpenSearch cluster. In case you are interested more in a service-linked role, here is the official documentation. The most essential part here is the 1st sentence:

A service-linked role is a unique type of IAM role that is linked directly to an AWS service. Service-linked roles are predefined by the service and include all the permissions that the service requires to call other AWS services on your behalf. 

So, in such a way we give all required permissions for OpenSearch cluster in order it could operate. But it is not the end. We also have to define who can access our cluster and what operations can he perform. It is done with resource-based policies. Here is the definition that we are going to use for learning purposes.

access_policies = <<CONFIG
  {
      "Version": "2012-10-17",
      "Statement": [
          {
              "Action": "es:*",
              "Principal": "*",
              "Effect": "Allow",
              "Resource": "arn:aws:es:${var.region}:${var.account_id}:domain/${var.domain_name}/*"
          }
      ]
  }
  CONFIG

As you see we allow too much here. At production you have to create it in a more strict way. Here is the long article around access management for OpenSearch cluster. But even that is not all you can do. As you can go even deeper and define so-called “fine-grained” access. Here is the ofiicila documentation for that. To understand the power of the current mechanism let’s read the next 2 sentences:

Fine-grained access control offers additional ways of controlling access to your data on Amazon OpenSearch Service. For example, depending on who makes the request, you might want a search to return results from only one index. 

As you see the access management configuration at AWS OpenSearch service is very built up and flexible. The minus of that – all it looks overwhelming and complicated when you start to work with OpenSearch. Security area is so huge that it is possible to write separate “book” about that. It is a little bit outside the limits of the current article. But if you are interested about it, please write to me at email – in case I see a big demand from the reader’s side – then I will write several more articles around security configuration for OpenSearch cluster. For now we will continue with the simple security configuration avoiding advanced security options.

advanced_security_options {
    enabled = false
}

OK, suppose we have finished with permissions and security, let’s return to our cluster or maybe better to say opensearch domain. 

resource "aws_opensearch_domain" "es" {
  domain_name    = var.domain_name
  engine_version = var.versionES
  ......
}

So we have to define a domain name – as it is a terraform module – we are going to set as variable. Then we have to set the engine version. And here it is not so easy to decide what to choose. Let’s visit terraform opensearch official documentation

Let’s visit the AWS link that I marked with red rectangle at screen. Scroll down to the 7th position

Here we can read that AWS for sure recommends to use OpenSearch instead of Elasticsearch 🙂 . But what we are mostly interested in is: supported versions. That is what we can use. Personally I used extensively 1.2 edition at commercial production environments, which appeared to be  fully compatible with ES 7th version.

One more interesting thing – let’s open Opensearch release page. And ups – we have even another situation. Here is some 2.6 version is already present.

You may ask – what is going on here? The deal is that OpenSearch product and releases at it appears faster then AWS is able to adjust it accordingly to OpenSearch cloud service. Simply be careful with that – and for terrafrom better use that one page to determine supported version which can be used in the cloud.

Huhh, suppose that version issue can be considered closed. Now we can finally move to cluster configuration. But let’s do it at another article. Please, check my blog regularly or simply  subscribe to my newsletter. Alternatively, you may pass all material at once in convenient and fast way at my on-line course at udemy. Below is the link to the course. As the reader of that blog you are also getting possibility to use coupon for the best possible low price.

---