Google Cloud Platform Authentification: Recognized as end user anthentification despite using a service account

Question

Anyone can HELP? This one is really driving me crazy... Thank you!

I tried to use a google cloud platform API Speech-to-text.

Tools: WINDOWS 10 && GCP &&Python(Pycharm IDE)

1. I've created a service account as a owner for my speech-to-test project and generated a key from GCP console in json, then I set the environment variables. Code I ran on WIN10 Powershell && CMD:

   $env:GOOGLE_APPLICATION_CREDENTIALS="D:\GCloud speech-to-text\Speech 
   To Text Series-93e03f36bc9d.json"
   
   set GOOGLE_APPLICATION_CREDENTIALS=D:\GCloud speech-to-text\Speech To 
   Text Series-93e03f36bc9d.json
   

   PS: the added environment variables disappear in CMD and Powershell after reboot my laptop but do show in the env list if added again.

2. I've enabled the google storage api and google speech-to-text api in GCP console.

3. I've tried the explicitely showing credential method via python, same problem.

4. I've installed the google cloud SDK shell and initialized by using command to log in my account.

PYTHON SPEECH-TO-TEXT CODE(from GCP demo)

import io
import os

# Imports the Google Cloud client library
from google.cloud import speech
from google.cloud.speech import enums
from google.cloud.speech import types

# Instantiates a client
client = speech.SpeechClient()

# The name of the audio file to transcribe
file_name = os.path.join(
    os.path.dirname(__file__),
    'test_cre.m4a')

# Loads the audio into memory
with io.open(file_name, 'rb') as audio_file:
    content = audio_file.read()
    audio = types.RecognitionAudio(content=content)

config = types.RecognitionConfig(
    encoding=enums.RecognitionConfig.AudioEncoding.LINEAR16,
    sample_rate_hertz=16000,
    language_code='en-US')

# Detects speech in the audio file
response = client.recognize(config, audio)

for result in response.results:
    print('Transcript: {}'.format(result.alternatives[0].transcript))

----Expected to receive a "200OK" and the transcribed text when runing the code above (a demo of short speech to text api from GCP Document)

----But got:

1. D:\Python\main program\lib\site-packages\google\auth_default.py:66: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK. We recommend that most server applications use service accounts instead. If your application continues to use end user credentials from Cloud SDK, you might receive a "quota exceeded" or "API not enabled" error. For more information about service accounts, see https://cloud.google.com/docs/authentication/ warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)

2. google.api_core.exceptions.ResourceExhausted: 429 Quota exceeded for quota metric 'speech.googleapis.com/default_requests' and limit 'DefaultRequestsPerMinutePerProject' of service 'speech.googleapis.com' for consumer 'project_number:764086051850'.

   ANOTHER WEIRD THING: the error info shows that 'project_number:764086051850', which is different from my speech-to-text project_number on GCP (I do distinguish project number and project ID), the project_number shown in the error info also varies every time the code runs. It seems I was sending cloud requirement of the wrong project?

3. My GOOGLE_APPLICATION_CREDENTIALS system environment variables disappear after I restart my laptop next time. After adding again, it will appear in the env list but can't be stored after reboot again.

Appreciate it if someone can help, thank you!

Answer 1

try to do this:

• Run gcloud init -> authenticate with your account and choose your project
• Run gcloud auth activate-service-account <service account email> --key-file=<JSON key file>
• Run gcloud config list to validate your configuration.

Run your script and see if it's better.

Else, try to do the same thing on a micro-vm for validating your code, service account and environment (and for validating that there is a problem only with Windows)

For Windows issues, I'm on ChromeBook, I can't test and help you on this. However, I checked about EnvVar on internet, and this update the registry. Check if you don't have stuff which protect Registry update (Antivirus,....)

Answer 2

D:\Python\main program\lib\site-packages\google\auth_default.py:66: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK. We recommend that most server applications use service accounts instead. If your application continues to use end user credentials from Cloud SDK, you might receive a "quota exceeded" or "API not enabled" error. For more information about service accounts, see https://cloud.google.com/docs/authentication/ warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)

This error means that your code is not using a service account. Your code is configured to use ADC (Application Default Credentials). Most likely your code is using the Google Cloud SDK credentials configured and stored by the CLI gcloud.

To determine what credentials the Cloud SDK is using, execute this command:

gcloud auth list

The IAM Member ID, displayed as ACCOUNT, with the asterisk is the account used by the CLI and any applications that do not specify credentials.

To learn more about ADC, read this article that I wrote:

Google Cloud Application Default Credentials

google.api_core.exceptions.ResourceExhausted: 429 Quota exceeded for quota metric 'speech.googleapis.com/default_requests' and limit 'DefaultRequestsPerMinutePerProject' of service 'speech.googleapis.com' for consumer 'project_number:764086051850'.

The Cloud SDK has the concept of default values. Execute gcloud config list. This will display various items. Look for project. Most likely this project does not have the API Cloud Speech-to-Text enabled.

ANOTHER WEIRD THING: the error info shows that 'project_number:764086051850', which is different from my speech-to-text project_number on GCP (I do distinguish project number and project ID), the project_number shown in the error info also varies every time the code runs. It seems I was sending cloud requirement of the wrong project?

To see the list of projects, Project IDs and Project Numbers that your current credentials can see (access) execute:

gcloud projects list.

This command will display the Project Number given a Project ID:

gcloud projects list --filter="REPLACE_WITH_PROJECT_ID" --format="value(PROJECT_NUMBER)"

My GOOGLE_APPLICATION_CREDENTIALS system environment variables disappear after I restart my laptop next time. After adding again, it will appear in the env list but can't be stored after reboot again.

When you execute this command in a Command Prompt, it only persists for the life of the Command Prompt: set GOOGLE_APPLICATION_CREDENTIALS=D:\GCloud speech-to-text\Speech To Text Series-93e03f36bc9d.json. When you exit the Command Prompt, reboot, etc. the environment variable is destroyed.

To create persistent environment variables on Windows, edit the System Properties -> Environment Variables. You can launch this command as follows from a Command Prompt:

SystemPropertiesAdvanced.exe

Suggestions to make your life easier:

• Do NOT use long path names with spaces for your service account files. Create a directory such as C:\Config and place the file there with no spaces in the file name.
• Do NOT use ADC (Application Default Credentials) when developing on your desktop. Specify the actual credentials that you want to use.

Change this line:

client = speech.SpeechClient()

To this:

client = speech.SpeechClient().from_service_account_json('c:/config/service-account.json')

• Service Accounts have a Project ID inside them. Create the service account in the same project that you intend to use them (until you understand IAM and Service Accounts well).