DEADLINE_EXCEEDED while reading logs from Google Cloud Logging

0
votes

My code is based off the sample mentioned on this page:

use Google\Cloud\Logging\LoggingClient;

$filter = sprintf(
    'resource.type="gae_app" severity="%s" logName="%s"',
    strtoupper($level),
    sprintf('projects/%s/logs/app', 'MY_PROJECT_ID'),
);

$logOptions = [
    'pageSize' => 20,
    'resultLimit' => 20,
    'filter' => $filter,
];
$logging = new LoggingClient();

$logs = $logging->entries($logOptions);
foreach ($logs as $log) {
    /* Do something with the logs */
}

This code is (at best) slow to complete, and (at worst) times out on the foreach loop with a DEADLINE_EXCEEDED error.

How can I fix this?

1
phpgoogle-cloud-platformgoogle-cloud-logging

1 Answers

1
votes

If your query does not match the first few logs it finds, Cloud Logging will attempt to search your entire logging history for the matching logs.

If there are too many logs to filter through, the search will time out with a DEADLINE_EXCEEDED message.

You can fix this by specifying a time frame to search from in your filter clause:

// Specify a time frame to search (e.g. last 5 minutes)
$fiveMinAgo = date(\DateTime::RFC3339, strtotime('-5 minutes'));

// Add the time frame constraint to the filter clause
$filter = sprintf(
    'resource.type="gae_app" severity="%s" logName="%s" timestamp>="%s"',
    strtoupper($level),
    sprintf('projects/%s/logs/app', 'MY_PROJECT_ID'),
    $fiveMinAgo
);