Friday, 20 July 2018

Use cURL in PHP to fetch url content without redirecting (rewriting) to that url

I want to call my Google App Script web service from PHP and I'm trying to use cURL for that, with GET method for starters. The service reponds with a simple text yes. I have found the PHP function (see below) to perform a GET request.
The first response from App Script service is always a redirect so I turned following redirects off in the configuration array in the fetching function, read the url and performed another request. However on the second request url I get redirected to the service url (with the text yes gets displayed as a result of the service) which is not desirable. I want to programatically process reponses, not having cURL influence the output or behavior in the browser.
Initial state and how it should work:
  1. I have a functioning App Script web service URL.
  2. The web service return the text yes.
  3. I have a PHP script which calls the web service.
  4. After the call the PHP script echoes the content returned by the service.
  5. After displaying the content returned by the service the PHP script echoes the word hello.
What actually happens when opened the PHP script in browser:
  1. I perform the first GET request and get response.
  2. I fetch the redirect_url from the response and perform a second GET request to that.
  3. The word hello get displayed on its for a second, wtill with the url of the PHP script.
  4. Url in the browser changes to that of the service, hello disappears and the service response yes appears (probably just got redirected).
PHP script I Use:
$response = get_web_page($url);

$url2 = $response["redirect_url"];

$response = get_web_page($url2);

echo $response["content"];

echo "hello";

PHP function I use to fetch the URL:
function get_web_page( $url, $cookiesIn = '' ){
        $options = array(
            CURLOPT_RETURNTRANSFER => true,     // return web page
            CURLOPT_HEADER         => true,     //return headers in addition to content
            CURLOPT_FOLLOWLOCATION => false,     // follow redirects
            CURLOPT_ENCODING       => "",       // handle all encodings
            CURLOPT_AUTOREFERER    => false,     // set referer on redirect
            CURLOPT_CONNECTTIMEOUT => 120,      // timeout on connect
            CURLOPT_TIMEOUT        => 120,      // timeout on response
            CURLOPT_MAXREDIRS      => 10,       // stop after 10 redirects
            CURLINFO_HEADER_OUT    => true,
            CURLOPT_SSL_VERIFYPEER => true,     // Validate SSL Cert
            CURLOPT_HTTP_VERSION   => CURL_HTTP_VERSION_1_1,
            CURLOPT_COOKIE         => $cookiesIn
        );

        $ch      = curl_init( $url );
        curl_setopt_array( $ch, $options );
        $rough_content = curl_exec( $ch );
        $err     = curl_errno( $ch );
        $errmsg  = curl_error( $ch );
        $header  = curl_getinfo( $ch );
        curl_close( $ch );

        $header_content = substr($rough_content, 0, $header['header_size']);
        $body_content = trim(str_replace($header_content, '', $rough_content));
        $pattern = "#Set-Cookie:\\s+(?<cookie>[^=]+=[^;]+)#m"; 
        preg_match_all($pattern, $header_content, $matches); 
        $cookiesOut = implode("; ", $matches['cookie']);

        $header['errno']   = $err;
        $header['errmsg']  = $errmsg;
        $header['headers']  = $header_content;
        $header['content'] = $body_content;
        $header['cookies'] = $cookiesOut;
    return $header;
}

When I try to print the whole response of the second request:
The content is clearly empty yet.
Array ( [url] => https://www.google.com/a/MY_DOMAIN_NAME/ServiceLogin?
service=wise&passive=1209600&continue=https://script.google.com/a/MY_DOMAIN_NAME/
macros/s/AKfycbwg9-Y10kLyIJxGLYhpx9BnIu5f8AHs4qKEq1rSUoD-ugjsD3c/exec&followup=
https://script.google.com/a/MY_DOMAIN_NAME/macros/s/AKfycbwg9-
Y10kLyIJxGLYhpx9BnIu5f8AHs4qKEq1rSUoD-ugjsD3c/exec [content_type] => text/html; 
charset=UTF-8 [http_code] => 200 [header_size] => 511 [request_size] => 379 [filetime] =>
 -1 [ssl_verify_result] => 0 [redirect_count] => 0 [total_time] => 0.146943 [namelookup_time
] => 0.001001 [connect_time] => 0.026204 [pretransfer_time] => 0.102405 [size_upload] => 0
 [size_download] => 1839 [speed_download] => 12515 [speed_upload] => 0 [
download_content_length] => 1839 [upload_content_length] => 0 [starttransfer_time] =>
 0.146158 [redirect_time] => 0 [redirect_url] => [primary_ip] => 172.217.16.68 [certinfo] 
=> Array ( ) [request_header] => GET /a/MY_DOMAIN_NAME/ServiceLogin?service=wise&
passive=1209600&continue=https://script.google.com/a/MY_DOMAIN_NAME/macros/s/
AKfycbwg9-Y10kLyIJxGLYhpx9BnIu5f8AHs4qKEq1rSUoD-ugjsD3c/exec&followup=
https://script.google.com/a/MY_DOMAIN_NAME/macros/s/AKfycbwg9-Y10kLyIJxGLYhpx9BnIu5f8AHs4q
KEq1rSUoD-ugjsD3c/exec HTTP/1.1 Host: www.google.com Accept: */* Accept-Encoding: deflate,
 gzip Cookie: [errno] => 0 [errmsg] => [headers] => HTTP/1.1 200 OK Content-Type: text/html;
 charset=UTF-8 X-Frame-Options: DENY Cache-control: no-cache, no-store Pragma: no-cache 
Expires: Mon, 01-Jan-1990 00:00:00 GMT Date: Sun, 08 Jul 2018 20:17:24 GMT 
X-Content-Type-Options: nosniff X-XSS-Protection: 1; mode=block Content-Length:
 1839 Server: GSE Set-Cookie: GAPS=1:qc-OvEMCdZAhgMZ6MhkFoUwlPrOIGg:_W0RGhF378FPmC1G;Path=/
a;Expires=Tue, 07-Jul-2020 20:17:24 GMT;Secure;HttpOnly;Priority=HIGH Alt-Svc: quic=":443";
 ma=2592000; v="43,42,41,39,35" [content] =>
[cookies] => GAPS=1:qc-OvEMCdZAhgMZ6MhkFoUwlPrOIGg:_W0RGhF378FPmC1G )



from Use cURL in PHP to fetch url content without redirecting (rewriting) to that url

No comments:

Post a Comment