Monday, 13 May 2019

Accept gzip encoding using httr R

I subscribe to a financial data provider ORATS. The software engineer reached out to me to let me know my GET() requests are timing out. He said to allow gzip encoding in my GET() request header. The SWE does not code in R and has sent me some node.js code to lean on.

I thought the httr GET() request automatically compressed the files to gzip.

Below is the node.js code provided by the SWE followed by my current R code that worked until I increased the size of the file I am pulling from their API (starting to time out).

const request = require('request');

const options = {
  url: 'https://api.orats.io/data/cores/general?include=earn',
  headers: {
  'Authorization' : 'your authorization token',
  'Accept-Encoding' : 'gzip'
  },
  gzip : true
};

request(options, function(err, response, body){
// Body is already uncompressed b/c the request library uncompresses it for you.
console.log(JSON.parse(body));
});


R code:
library(httr)
x = GET(url, add_headers(Authorization = token))
y = rawToChar(x$content)

I would like this code to request the gziped file. Thank you.



from Accept gzip encoding using httr R

No comments:

Post a Comment