Geek Answers Handbook

How to load Java OutputStream in AWS S3

I create PDF documents in memory as OutputStream s. They must be uploaded to S3. My problem is that it is not possible to create a PutObjectRequest from an OutputStream directly (according to this thread in the AWS dev forum ), I am using aws-java-sdk-s3 v1.10.8 in Dropwizard .

The workarounds that I see so far are as follows:

  • Copy OutputStream to InputStream and accept that there is twice as much RAM.
  • Bring OutputStream to InputStream and accept the overhead of the extra stream (see this answer )

If I do not find a better solution, I will go with No. 1, because it looks like I could allow extra memory easier than threads / CPU in my setup.

Is there any other, possibly more effective way to achieve this that I have so far forgotten?

Edit: My OutputStream : ByteArrayOutputStream s

+6
source share
2 answers

I solved this by subclassing ConvertibleOutputStream :

 public class ConvertibleOutputStream extends ByteArrayOutputStream { //Craetes InputStream without actually copying the buffer and using up mem for that. public InputStream toInputStream(){ return new ByteArrayInputStream(buf, 0, count); } }
+2
source

What is the actual type of your OutputStream ? Since this is an abstract class, it does not say where the data actually goes (or if it even goes anywhere).

But suppose you're talking about ByteArrayOutputStream , as it at least stores data in memory (unlike many others).

If you create a ByteArrayInputStream from your buffer, there is no duplicated memory. This is the whole idea of ​​streaming.

+1
source

More articles:


All Articles