将数据从 AWS Lambda 发送到 SQS 队列时连接重置

Question

我正在使用适用于 Java 的 AWS 开发工具包，其中我将数据从 AWS Lambda 发送到 SQS。

我们得到了例外：

Caused by: java.net.SocketException: Connection reset

at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:115)

at java.net.SocketOutputStream.write(SocketOutputStream.java:155)

at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:431)

at sun.security.ssl.OutputRecord.write(OutputRecord.java:417)

at sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:886)

at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:857)

at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)

at org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:124)

at org.apache.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:160)

at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:113)

at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:120)

at org.apache.http.entity.StringEntity.writeTo(StringEntity.java:167)

at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156)

at org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:160)

at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238)

at com.amazonaws.http.protocol.SdkHttpRequestExecutor.doSendRequest(SdkHttpRequestExecutor.java:63)

at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)

at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)

at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)

at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)

背景我们正在尝试做的事情：

我们有一个主要的 Lambda 函数，它创建和初始化一个 SQS 队列，并包含应处理的每条记录的详细信息。现在需要设置 SQS 队列以从队列中创建 X 条消息的批次，并为每个批次自动调用另一个 SQS Lambda 函数。

Answer 1

每个批次的最大消息数为 10。您不能一次用 20k 填充 SQS 队列并发送该请求。试着把它分成10个。https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-limits.html#limits-queues

Answer 2

我们可以批量发送 10 个。工作代码：List<SendMessageBatchRequestEntry> sqsList= new LinkedList<SendMessageBatchRequestEntry>();&nbsp; &nbsp; int batchId = 1; //To send a unique batchId for each msg in a batch&nbsp; &nbsp; for (Metadata metadata: metadataList) {&nbsp; &nbsp; &nbsp; &nbsp; String jsonString = new Gson().toJson(metadata);&nbsp; &nbsp; &nbsp; &nbsp; if (sqsList.size() == 10) {&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; amazonSqs.sendMessageBatch(new SendMessageBatchRequest(queueUrl, sqsList));&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; sqsList.clear();&nbsp; &nbsp; &nbsp; &nbsp; }&nbsp;&nbsp; &nbsp; &nbsp; &nbsp; sqsList.add(new SendMessageBatchRequestEntry(batchId + "", jsonString));&nbsp;&nbsp; &nbsp; &nbsp; &nbsp; batchId++;&nbsp; &nbsp; }&nbsp; &nbsp; if(sqsList.size()>0) {&nbsp; &nbsp; &nbsp; &nbsp; amazonSqs.sendMessageBatch(new SendMessageBatchRequest(queueUrl, sqsList));&nbsp; &nbsp; }

Answer 3

似乎您的代码很好，据我记得（我自己已经多次看到此错误），由于 SDK 如何重用 HTTP 连接，在使用 SDK 时会不时发生这种情况。此错误仅告诉您您的 Lambda 重置了 HTTP 连接，但 SDK 具有内置功能以重试失败的请求，因此如果您在每个请求上都没有看到此错误，您应该没问题。