SiteCrafting, Inc.
2 Apr
PHP cURL and SSL Connection Timeout
I've been working on a web site that uses cURL for request/response transactions and I was encountering a number of problems with empty responses from the server. This caused some frustrations, since these empty responses caused entire sections of a web page to be empty, especially on searches. Well, I couldn't have that, since people using the website would be confused on why a search for an item only worked occasionally. Trust in the web application would drop dramatically.
My first instinct was to blame the server for returning empty responses, but realized that I shouldn't jump to conclusions about other people's servers. So, I launched myself into debugging the request/response transaction.
Let's take a look at some PHP code for performing a cURL transaction. The most important thing to remember is that I can't stop the processing if an error occurs. So, if there is an error, the function returns an empty string.
My first instinct was to blame the server for returning empty responses, but realized that I shouldn't jump to conclusions about other people's servers. So, I launched myself into debugging the request/response transaction.
Let's take a look at some PHP code for performing a cURL transaction. The most important thing to remember is that I can't stop the processing if an error occurs. So, if there is an error, the function returns an empty string.
// keep looping until we get a response
$count = 0;
// try to get a good response by looping 10 times
while( $response == "" )
{
$count++;
$response = curl_request( "https://php.net", "search=curl" );
// if we looped ten times and still no response, lets quit to avoid never ending loop
if( $count == 10 )
break;
} //end while loop
function curl_request($server_url, $values)
{
// this array stores all the information for a cURL transaction
$arrCurlInfo = array();
$arrCurlOptions = array();
$arrCurlOptions[CURLOPT_HEADER] = 1;
$arrCurlOptions[] = "";
// set to 1 to include header info from response
$arrCurlOptions[CURLOPT_HEADER] = 1;
//Return the output from the cURL session rather than displaying in the browser.
//If already set, don't overwrite
$arrCurlOptions[CURLOPT_RETURNTRANSFER] = 1;
// if page redirects, curl will follow
$this->setCurlOption(CURLOPT_FOLLOWLOCATION, 1, false); //
//If already set, don't overwrite
$arrCurlOptions[ CURLOPT_USERAGENT] = $_SERVER['HTTP_USER_AGENT'];
/ attempt the request/response
try
{
$fp = curl_init();
curl_setopt_array($fp, $arrCurlOptions);
$response = curl_exec($fp);
$arrCurlInfo = curl_getinfo($fp);
} //end try
// catch any errors that occur and return an empty response
catch( Exception $e)
{
$strResponse = "";
$strErrorCode = $e->getCode();
$strErrorMessage = $e->getMessage();
log_curl_error($arrCurlInfo, $strErrorCode, $strErrorMessage);
} //end catch
curl_close($fp);
$fp = null;
return $response;
} //end function
$count = 0;
// try to get a good response by looping 10 times
while( $response == "" )
{
$count++;
$response = curl_request( "https://php.net", "search=curl" );
// if we looped ten times and still no response, lets quit to avoid never ending loop
if( $count == 10 )
break;
} //end while loop
function curl_request($server_url, $values)
{
// this array stores all the information for a cURL transaction
$arrCurlInfo = array();
$arrCurlOptions = array();
$arrCurlOptions[CURLOPT_HEADER] = 1;
$arrCurlOptions[] = "";
// set to 1 to include header info from response
$arrCurlOptions[CURLOPT_HEADER] = 1;
//Return the output from the cURL session rather than displaying in the browser.
//If already set, don't overwrite
$arrCurlOptions[CURLOPT_RETURNTRANSFER] = 1;
// if page redirects, curl will follow
$this->setCurlOption(CURLOPT_FOLLOWLOCATION, 1, false); //
//If already set, don't overwrite
$arrCurlOptions[ CURLOPT_USERAGENT] = $_SERVER['HTTP_USER_AGENT'];
/ attempt the request/response
try
{
$fp = curl_init();
curl_setopt_array($fp, $arrCurlOptions);
$response = curl_exec($fp);
$arrCurlInfo = curl_getinfo($fp);
} //end try
// catch any errors that occur and return an empty response
catch( Exception $e)
{
$strResponse = "";
$strErrorCode = $e->getCode();
$strErrorMessage = $e->getMessage();
log_curl_error($arrCurlInfo, $strErrorCode, $strErrorMessage);
} //end catch
curl_close($fp);
$fp = null;
return $response;
} //end function
The log_curl_error() is a function that simply stores the cUrl information and the error code and message to a database table. Now, let's pretend that I run this script a large number of times. Once in a while, no response is returned. I take a look at the log table and notice that I get the same error message of "SSL connection timeout" and they are always in groups of ten.
Hmm, that must mean that even though we're re-initializing the cURL each time we loop through, it must be using the same connection each time. So, once the first connection is bad, each call to curl_request() uses the same connection. After some research, I found out that the cURL connection is cached.
I'm thinking that for some reason, the web site and the server have trouble connecting using "HTTPS," but since the connection is cached, we keep getting an empty response, no matter how many times we loop. During the afore mentioned research, I also discovered a cURL option,CURLOPT_FRESH_CONNECT, for clearing the cache. Additional cURL options.
Using this new insight, I added one more cURL option to the curl_request() function.
$arrCurlOptions[CURLOPT_FRESH_CONNECT] = 1;
By adding this option, the connection is cleared each time we run the loop. For example, the first time through the loop, the SSL connection between the web site and server is bad, and we get a "SSL connection timeout" error. The second time through, the connection is refreshed, this time the SSL connection between the web site and server is good. The function is now able to return a response from the server.
From the Workbench, PHP
by Ken Foubert | 4/2/2007 2:36pm | Comments (10)
by Ken Foubert | 4/2/2007 2:36pm | Comments (10)
This article is a God send. I am going through the same problem.
Left by Daniel | Feb 4, 2008
Daniel, I'm very glad the article was of some help. Figuring out the problem was a headache and required a lot of research. Hopefully, it helped save you some time.
Left by Ken Foubert | Feb 5, 2008
Thanks!! I added
curl_setopt($ch, CURLOPT_FRESH_CONNECT, 1);
And everything is working great!
Left by PHPguy | May 19, 2008
Once a while I get an empty curl response too. After a while googleing I finally found this article and I just add the CURLOPT_FRESH_CONNECT, 1 to my code. Hopefully it will fixes the problem.
Thanks for sharing your experimental with us.
Left by Hiep | Jul 30, 2008
Thanks you for this example with CURL and SSL :)
Left by Sun Location | Oct 9, 2008
I have just added this to my code and tested with a site I knew was giving me the same SSL timeout errors and it fixed it all I can say is thank you thank you thank you.
Left by Lee | Oct 17, 2009
Hi, How are you, I need your help,
I am making a Get Request to a page by cURL. and that page return a PHP array in response, I am unable to get that response in PHP array.(By default cURL get response in String)
for example i am getting response like this Below. I want to Put it in PHP array and want to get related data from the defined indexes, Please help me in this regards. Waiting for your reply
Thanks,
Response from page is Like below.
Array
(
[status] => 1
[message] => OK
[results] => Array
(
[0] => Array
(
[country_code] => 44
[number] => 2030266266
[recommended_gold_premium] => 1075
[wholesale_gold_premium] => 537.5
[block] => e1e63af1397656ba30bb45745d6776506378ca67
[bill_class] => Carrier
)
[1] => Array
(
[country_code] => 44
[number] => 2030262262
[recommended_gold_premium] => 1050
[wholesale_gold_premium] => 525
[block] => e1e63af1397656ba30bb45745d6776506378ca67
[bill_class] => Carrier
)
}
}
Left by Shary | Nov 5, 2009
Just wanted to know if CURLOPT_FRESH_CONNECT refreshes the connection for every subsequent connection or just the next immediate connection?
Left by Rini | Jul 14, 2011
Rini,
From what I can tell, it's only for the next immediate connection.
Ken
Left by Ken Foubert | Jul 19, 2011
xOlEFN Say, you got a nice blog article.Thanks Again. Cool.
Left by Adobe OEM Software | Mar 7