ElasticSearch

• Postman Queries (pulled from reshard testing)
• snapshot s3 repository
• snapshot s3 repository restore
• other queries
• reindex
• Settings
• mapping
• new index
• mapping with parameters
• wait for completion
• nodes stats
• Kibana
• ElasticSearch to s3 snapshot
• Steps to link ES cluster to s3 and create/restore snapshots

Postman Queries (pulled from reshard testing)

snapshot s3 repository

Put: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/_snapshot/s3_repository?pretty&verify=false

Authorization:

snapshot s3 repository restore

Post: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/_snapshot/s3_repository/prod-snapshot-10-1-2020/_restore

Authorization:

other queries

Get: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/_snapshot/s3_repository

Get: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/_cat/indices/twi*?v&s=index

Get: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/_cat/indices

Delete: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/_snapshot/s3_repository/prod-snapshot-9-18-2020

Post: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/_snapshot/s3_repository/prod-snapshot-9-22-2020/_restore

Get: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/_tasks?actions=*search&detailed

Post: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/_tasks/cc2IKK9yQtKiGs1Zs46BzA:47566757/_cancel

Get: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/_tasks?actions=*search

Put: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/_snapshot/

Put: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/rubex-usprod-esclusteridx/

Post: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/_

Post: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/

reindex

Post: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/_reindex

Body:

Settings

Put: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/rubex_usprod_idx_012022/_settings/

Body:

mapping

Put: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/rubex_foo/_mapping

Body:

new index

Put: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/rubex_usprod_newindex2022

Body:

mapping with parameters

Put: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/rubex_usprod_idx_012022/_mapping/nodebaseindexitem

Body:

wait for completion

Post: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/_snapshot/s3_repository/prod-snapshot-10-14-2020?wait_for_completion=true

Authorization:

Body:

nodes stats

Get: https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/_nodes/stats?pretty

Kibana

https://vpc-rubex-elasticsearch-prod-8-6-soqlodwed3jcm7gcbrpwhguh7a.us-east-1.es.amazonaws.com/_plugin/kibana/app/kibana#/dev_tools/console?_g=()

Console (in Dev Tools):

GET _cat/indices?v
GET nodebaseindexitemindex/_settings
GET _snapshot?pretty

PUT rubex_usprod_indextesting

GET /_cluster/state

GET nodebaseindexitemindex/_settings
POST _reindex

GET _tasks?detailed=true&actions=*reindex

GET  rubex-usprod-esclusteridx?pretty

GET _tasks?detailed=true&actions=*reindex

GET rubex_usprod_newindex2022/_settings

GET rubex-usprod-esclusteridx/_mapping
GET rubex_usprod_newindex2022/_mapping?pretty=true

GET rubex_foo/_mapping/_nodebaseindexitem
GET _cat/indices?v

GET _cat/shards/nodebaseindexitemindex?v
GET _cat/pending_tasks
GET nodebaseindexitemindex

ElasticSearch to s3 snapshot

Steps to link ES cluster to s3 and create/restore snapshots

In the Account that the S3 bucket is going to live, create a new IAM policy with the following permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": [
                "arn:aws:s3:::S3-BUCKET-NAME/*",
                "arn:aws:iam::ACCOUNT-ID:role/*"
            ]
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::S3-BUCKET-NAME"
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": "arn:aws:s3:::S3-BUCKET-NAME/*"
        }
    ]
}

Create a new Role in that AWS account with the above policy, and also create a Trust permission with the following details:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Principal": {
                "Service": "es.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Create a new user in the account and give it the the policy created earlier

---------Create connection to repository
    
PUT: https://SOURCE-CLUSTER-ENDPOINT/_snapshot/snapshot-respository

Body: as JSON
    {   "type": "s3",
    "settings": {
        "bucket": "S3-BUCKET-NAME",
        "region": "us-east-1",
        "role_arn": "ARN FOR THE NEWLY CREATED ROLE"
    }
 }

--------Create backup

PUT: https://vpc-rbx-stg-es-6-8-f7kozzmqn3r2ygnqsqicjlrism.us-east-1.es.amazonaws.com/_snapshot/snapshot-respository/2023-01-05

--------Restore Status
GET: https://vpc-rbx-stg-es-6-8-f7kozzmqn3r2ygnqsqicjlrism.us-east-1.es.amazonaws.com/_snapshot/snapshot-respository/2023-01-05/_status

------Restore
POST: https://vpc-rbx-stg-es-6-8-shardings-ma7dgqmnnngrd6wthxwfgkot5u.us-east-1.es.amazonaws.com/_snapshot/snapshot-respository/2023-01-05/_restore

------Restore status
GET: https://vpc-rbx-stg-es-6-8-shardings-ma7dgqmnnngrd6wthxwfgkot5u.us-east-1.es.amazonaws.com/_cat/recovery