firebase functions - `firebase deploy` - The service is currently unavailable

Question

I am trying to run firebase deploy but I get:

=== Deploying to ...

i  deploying functions
i  functions: ensuring necessary APIs are enabled...
✔  functions: all necessary APIs are enabled
i  functions: preparing functions directory for uploading...
i  functions: packaged functions (49.59 KB) for uploading
✔  functions: functions folder uploaded successfully
i  functions: updating Node.js 6 function sendMessageNotification(us-central1)...
⚠  functions: failed to update function sendMessageNotification
HTTP Error: 503, The service is currently unavailable.


Functions deploy had errors. To continue deploying other features (such as database), run:
    firebase deploy --except functions

Error: Functions did not deploy properly.

Having trouble? Try firebase deploy --help

FYI, this is happening after one of my service accounts was compromised because I accidentally uploaded a service account json file, with my credentials in it, to github. I went through the procedures described here after my account got suspended, and I successfully got it reinstated. This problem has only started happening after I removed and created all the necessary credentials (API key/service account/ouath client id).

I have tried firebase logout then firebase login and was unsuccessful deploying. I also have tried npm install -g firebase-tools@latest and firebase use --add <myproject> (I'm not sure what that was supposed to do). I even reran the firebase-tools install but that still didn't help.

I have also tried editing the firebase function from the google cloud console but it doesn't let me save, it gives me an undefined error, and a tracking number for the issue, I provided feedback but haven't heard anything.

Any ideas would be greatly appreciated.

UPDATE

Here is my credentials page, I'm not exactly sure if everything is configured correctly, hopefully some info can be gained from this as I think maybe the project being compromised and having to renew the keys has something to do with the firebase deploy problem:

UPDATE

I have also tried resetting gcloud credentials with no luck:

./gcloud auth application-default login

UPDATE

I ran firebase deploy --only functions --debug, here is more output:

i functions: updating Node.js 6 function sendMessageNotification(us-central1)... [2018-09-20T22:28:14.587Z] Trigger is: {"eventTrigger":{"resource":"projects/PROJECT_ID/databases/(default)/documents/messages/{messageId}","eventType":"providers/cloud.firestore/eventTypes/document.write","service":"firestore.googleapis.com"}} [2018-09-20T22:28:14.588Z] >>> HTTP REQUEST PATCH https://cloudfunctions.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/functions/sendMessageNotification query params: {"updateMask":"sourceUploadUrl,name,labels,eventTrigger.resource,eventTrigger.eventType,eventTrigger.service"} sourceUploadUrl=https://storage.googleapis.com/ name=projects/PROJECT_ID/locations/us-central1/functions/Function, deployment-tool=cli-firebase, resource=projects/PROJECT_ID/databases/(default)/documents/messages/{messageId}, eventType=providers/cloud.firestore/eventTypes/document.write, service=firestore.googleapis.com [2018-09-20T22:28:15.055Z] <<< HTTP RESPONSE 503 vary=X-Origin, Referer, Origin,Accept-Encoding, content-type=application/json; charset=UTF-8, date=Thu, 20 Sep 2018 22:28:15 GMT, server=ESF, cache-control=private, x-xss-protection=1; mode=block, x-frame-options=SAMEORIGIN, x-content-type-options=nosniff, alt-svc=quic=":443"; ma=2592000; v="44,43,39,35", accept-ranges=none, connection=close [2018-09-20T22:28:15.056Z] <<< HTTP RESPONSE BODY code=503, message=The service is currently unavailable., status=UNAVAILABLE ⚠ functions: failed to update function sendMessageNotification HTTP Error: 503, The service is currently unavailable

Interesting, if you click https://cloudfunctions.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/functions/FUNCTION (from the output), it brings you to this:

{
  "error": {
    "code": 401,
    "message": "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
    "status": "UNAUTHENTICATED"
  }
}

It says I am missing an authentication credential...

UPDATE

When I click the url in the above output, it says:

<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>
The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.
</Message>
<StringToSign>
GET 1537484293 /Function-upload-us-central1/FILE.zip
</StringToSign>
</Error>

UPDATE

From google's OAuth 2.0 Playground:

GET /FUNCTION-central1-FILE.zip?GoogleAccessId=[e-mail]&Expires= HTTP/1.1
Host: storage.googleapis.com
Content-length: 0
Authorization: Bearer 
HTTP/1.1 403 Forbidden
Content-length: 298
Expires: Thu, 20 Sep 2018 23:01:16 GMT
X-guploader-uploadid: 
Cache-control: private, max-age=0
Date: Thu, 20 Sep 2018 23:01:16 GMT
Content-type: application/xml; charset=UTF-8
<?xml version='1.0' encoding='UTF-8'?><Error><Code>AccessDenied</Code><Message>Access denied.</Message><Details>[email protected] does not have storage.objects.get access to Function-us-central1-FILE.zip.</Details></Error>

In short it says:

[email protected] does not have storage.objects.get access to gFunction-us-central1-FILE.zip

Answer 1

The problem could be that you deleted "Google App Engine's 'default service account'" as details in this S.O. post.

Unfortunately, when this happens, you may have to destroy the google cloud project and remake it. See the linked post for more info.