如何将弹性搜索索引的响应保存到 s3 存储桶中

Question

如何在我的索引下面生成

1. 在你的本地创建一个环境

2. 使用 pip 安装必要的库（elasticsearch、requests、requests_aws4auth、boto3）

3. 使用 lambda_function.py在里面创建文件env\Lib\site-packages\并添加以下代码

4. 压缩上述文件夹并将其命名为 lambda_function.zip 并上传到 lambda 函数，您可以在其中创建具有必要 IAM 角色的函数

import boto3

 from requests_aws4auth import AWS4Auth

 from elasticsearch import Elasticsearch, RequestsHttpConnection

 session = boto3.session.Session()

 credentials = session.get_credentials()

 awsauth = AWS4Auth(credentials.access_key,

                    credentials.secret_key,

                    session.region_name, 'es',

                    session_token=credentials.token)

 es = Elasticsearch(

     ['https://search-testelastic-2276kyz2u4l3basec63onfq73a.us-east-1.es.amazonaws.com'],

     http_auth=awsauth,

     use_ssl=True,

     verify_certs=True,

     connection_class=RequestsHttpConnection

 )

 def lambda_handler(event, context):

     es.cluster.health()

     es.indices.create(index='my-index', ignore=400)

     r = [{'Name': 'Dr. Christopher DeSimone', 'Specialised and Location': 'Health'},

  {'Name': 'Dr. Tajwar Aamir (Aamir)', 'Specialised and Location': 'Health'}]

     for e in enumerate(r):

          es.index(index="my-index", body=e[1])

回复如下

{"took":2,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":3,"relation":"eq"},"max_score":1.0,"hits":[{"_index":"my-index_1","_type":"_doc","_id":"elqrJHMB10jKFvejVaNM","_score":1.0,"_source":{"Name":"Dr. Christopher DeSimone","Specialised and Location":"Health"}},{"_index":"my-index_1","_type":"_doc","_id":"e1qrJHMB10jKFvejVqMK","_score":1.0,"_source":{"Name":"Dr. Tajwar Aamir (Aamir)","Specialised and Location":"Health"}},{"_index":"my-index_1","_type":"_doc","_id":"fFqrJHMB10jKFvejVqMR","_score":1.0,"_source":{"Name":"Dr. Bernard M. Aaron","Specialised and Location":"Health"}}]}}

如何将上述响应保存为 s3 存储桶中文件夹中的 json

存储桶名称 = test20220elastic

Answer 1

您可以重用您的session对象来创建 S3 资源：es = ...s3 = session.resource('s3')bucket = s3.Bucket('test20220elastic')def lambda_handler(event, context):    ...    for e in enumerate(r):        result = es.index(index="my-index", body=e[1])        bucket.put_object(Body=json.dumps(result), Key="my_folder/my_result.json",                          ContentType="application/json")不过，您可能希望为每个结果建立一个不同的键名。