Google Cloud Shell CPU throttling?

0
votes

I've been using the Cloud Shell to download some data from storage, process it, and upload it again.

My problem is that the CPU usage goes to 100% but only for about 30 seconds or so, then for some reason it limits the CPU usage at about 50%, then 25%. Then it stays at 25% but in real-time it seems to continue to decline. This happens over about 1 to 2 minutes.

If I stop the process and wait a few seconds and start it again, the CPU jumps back to 100% and then starts it decline again.

I guess this is because the Google Cloud Shell is not meant to be used for data processing?

Is there a way to pay for a more powerful Cloud Shell in Google Cloud Platform? Can you have a VM instance exactly setup like cloud shell is?

2
google-cloud-platformgoogle-cloud-shell
You might be violating the terms of use for Cloud Shell. Specifically Cloud Shell is for interactive development and not data or CPU processing. If you get flagged you won't be able to use Cloud Shell. cloud.google.com/shell/docs/limitations – John Hanley
Cloud Shell is a container that you can download and modify. You could run the container on a Compute Engine instance and not worry about usage limits. – John Hanley
Don't use cloud shell. The only thing worse than cloud shell is a Windows machine. – DUDANF
I disagree with @DuDoff Cloud Shell is a very useful tool/service when used correctly. Windows is a key software product that has had a major impact on the world. However, opinions do vary. – John Hanley
Hello SectoKia@ If you find my answer useful please approve it, otherwise please tell me what is missing from my answer or feel free to share any other doubt with us – Chris32

2 Answers

1
votes

The Google Cloud Shell is not meant to be used as a processing unit for regular workload, instead its a tool for small developer testing situation.

Actually you cannot pay for a better instance for your Google Cloud Shell but you have 2 possible workarounds for this.

  1. You can use the Boost Mode of Cloud Shell to provision a better machine. This solution is free but limited and as I just said it should be used only for small testing situations.
  2. Since Cloud Shell is indeed a Compute Engine Instance with the SDK installed you can indeed set up a Compute Engine Instance with your particular requirements and ssh into it. Here you can see all the software and tools presintalled in Google Cloud Shell by default. Keep in mind that you will need to update your libraries on your own.
0
votes

Could you share more details regarding your data processing e.g., what tool you are using, and possibly some commands to reproduce this issue?

But generally, the resources on Cloud Shell instance are limited: https://cloud.google.com/shell/docs/how-cloud-shell-works

Boost mode might be able to help in this case.