I am using elastical to connect to ElasticSearch via node.js.
In the process of profiling my application with Nodetime to improve performance, I noticed something strange. My ElasticSearch "PUT" queries on the _bulk index often lead to a "socket hang". In addition, these calls require a huge amount of processor time.
I close every _bulk @ 10 index request for indexing, and as you can see, the length of the content of the requests does not even reach 50 Kb, so it’s hard to imagine that size is a problem. However, the response time is> 60 seconds, and the CPU time> 10+ seconds! Clap !!
In debugging attempts, I started running ElasticSearch in the foreground. I noticed this strange error:
[2013-02-27 11:42:39,188][WARN ][index.gateway.s3 ] [Lady Mandarin] [network][1] failed to read commit point [commit-f34] java.io.IOException: Failed to get [commit-f34] at org.elasticsearch.common.blobstore.support.AbstractBlobContainer.readBlobFully(AbstractBlobContainer.java:83) at org.elasticsearch.index.gateway.blobstore.BlobStoreIndexShardGateway.buildCommitPoints(BlobStoreIndexShardGateway.java:847) at org.elasticsearch.index.gateway.blobstore.BlobStoreIndexShardGateway.doSnapshot(BlobStoreIndexShardGateway.java:188) at org.elasticsearch.index.gateway.blobstore.BlobStoreIndexShardGateway.snapshot(BlobStoreIndexShardGateway.java:160) at org.elasticsearch.index.gateway.IndexShardGatewayService$2.snapshot(IndexShardGatewayService.java:271) at org.elasticsearch.index.gateway.IndexShardGatewayService$2.snapshot(IndexShardGatewayService.java:265) at org.elasticsearch.index.engine.robin.RobinEngine.snapshot(RobinEngine.java:1090) at org.elasticsearch.index.shard.service.InternalIndexShard.snapshot(InternalIndexShard.java:496) at org.elasticsearch.index.gateway.IndexShardGatewayService.snapshot(IndexShardGatewayService.java:265) at org.elasticsearch.index.gateway.IndexShardGatewayService$SnapshotRunnable.run(IndexShardGatewayService.java:366) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) Caused by: Status Code: 404, AWS Service: Amazon S3, AWS Request ID: ..., AWS Error Code: NoSuchKey, AWS Error Message: The specified key does not exist., S3 Extended Request ID: .... at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:548) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:288) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:170) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:2632) at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:811) at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:717) at org.elasticsearch.cloud.aws.blobstore.AbstractS3BlobContainer$1.run(AbstractS3BlobContainer.java:73)
I know that I am using an outdated gateway (S3 gateway gateway). However, given that I have several servers running in the Amazon Cloud that need to exchange data (I use ElasticSearch for caching), I see no alternative until the ElasticSearch team issues a replacement for the S3 Bucket ...
Other than this problem with _bulk calls, I don't see any problems. Searches etc. Everyone returns quickly and efficiently.
source share