Saturday, 9 February 2019

AWS SDK JS S3 getObject Stream Metadata

I have code similar to the following to pipe an S3 object back to the client as the response using Express, which is working perfectly.

const s3 = new AWS.S3();
const params = {
    Bucket: 'myBucket',
    Key: 'myImageFile.jpg'
};
s3.getObject(params).createReadStream().pipe(res);

Problem is, I want to be able to access some of the properties in the response I get back from S3, such as LastModified, ContentLength, ETag, and more. I want to use those properties to send as headers in the response to the client, as well as for logging information.

Due to the fact that it is creating a stream I can't figure out how to get those properties.

I have thought about doing a separate s3.headObject call to get the data I need, but that seems really wasteful, and will end up adding a large amount of cost at scale.

I have also considered ditching the entire stream idea and just having it return a buffer, but again, that seems really wasteful to be using extra memory when I don't need it, and will probably add more cost at scale due to the extra servers needed to handle requests and the extra memory needed to process each request.


How can I get back a stream using s3.getObject along with all the metadata and other properties that s3.getObject normally gives you back?


Something like the following would be amazing:

const s3 = new AWS.S3();
const params = {
    Bucket: 'myBucket',
    Key: 'myImageFile.jpg',
    ReturnType: 'stream'
};
const s3Response = await s3.getObject(params).promise();
s3Response.Body.pipe(res); // where `s3Response.Body` is a stream
res.set("ETag", s3Response.ETag);
res.set("Content-Type", s3Response.ContentType);
console.log("Metadata: ", s3Response.Metadata);

But according to the S3 documentation it doesn't look like that is possible. Is there another way to achieve something like that?



from AWS SDK JS S3 getObject Stream Metadata

No comments:

Post a Comment