Sign Requests to AWS Elasticsearch for IAM using Python

From Knowledge Center
Jump to: navigation, search

If you allow IAM Users to access AWS Elasticsearch service, then any requests to the Elasticsearch endpoint should be signed.

If you are using a client like curl, the call will be un-authorized, as the verification service do not know where the call is made from.

The Workaround:

You can however allow this by allowing IP Based Access in you Elasticsearch Policy.

This will show you how to sign the requests, when making the request to Elasticsearch.

The Python SigV4 Script:

import sys, os, base64, datetime, hashlib, hmac, urllib
import requests # pip install requests

method = 'GET'
service = 'es'
host = sys.argv[1]
region = sys.argv[2]
endpoint = 'https://' + sys.argv[1]

def sign(key, msg):
    return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()

def getSignatureKey(key, dateStamp, regionName, serviceName):
    kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
    kRegion = sign(kDate, regionName)
    kService = sign(kRegion, serviceName)
    kSigning = sign(kService, 'aws4_request')
    return kSigning

access_key = os.environ.get('AWS_ACCESS_KEY')
secret_key = os.environ.get('AWS_SECRET_KEY')
if access_key is None or secret_key is None:
    print 'No access key is available.'
    sys.exit()

t = datetime.datetime.utcnow()
amz_date = t.strftime('%Y%m%dT%H%M%SZ') # Format date as YYYYMMDD'T'HHMMSS'Z'
datestamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope

canonical_uri ='/'

canonical_headers = 'host:' + host + '\n'
signed_headers = 'host'
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'

canonical_querystring = 'Action=CreateUser&UserName=NewUser&Version=2010-05-08'
canonical_querystring += '&X-Amz-Algorithm=AWS4-HMAC-SHA256'
canonical_querystring += '&X-Amz-Credential=' + urllib.quote_plus(access_key + '/' + credential_scope)
canonical_querystring += '&X-Amz-Date=' + amz_date
canonical_querystring += '&X-Amz-Expires=30'
canonical_querystring += '&X-Amz-SignedHeaders=' + signed_headers

payload_hash = hashlib.sha256('').hexdigest()
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash

string_to_sign = algorithm + '\n' +  amz_date + '\n' +  credential_scope + '\n' +  hashlib.sha256(canonical_request).hexdigest()
signing_key = getSignatureKey(secret_key, datestamp, region, service)
signature = hmac.new(signing_key, (string_to_sign).encode("utf-8"), hashlib.sha256).hexdigest()
canonical_querystring += '&X-Amz-Signature=' + signature

request_url = endpoint + "?" + canonical_querystring

print '\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++'
print 'Request URL = ' + request_url
r = requests.get(request_url)

print '\nRESPONSE++++++++++++++++++++++++++++++++++++'
print 'Response code: %d\n' % r.status_code
print r.text

Export your Access Key and Secret-Key:

$ export AWS_ACCESS_KEY=your-aws-access-key-id
$ export AWS_SECRET_KEY=your-aws-secret-key

Make the Request:

$ python sign-req-url-v4.py es-endpoint-test.es.amazonaws.com eu-west-1