I see a scary warning everytime I try to turn off my GCE instance saying that if it doesn't stop in 2 minutes, system halt will be tried which may lead to data corruption (and asking my permission to proceed).
I wonder how do you all deal with that? Just take risks (and also have backups of the system image and attached disk)?
With GCP's per minute billing, I stop my instances often and wonder how serious is this warning.
Anupam, the warning as it describes tells you that your instance is expected to shutdown gracefully within 2 minutes. 2 minutes is sufficient time for an instance to stop/kill all user processes and system services, flush pending buffers back to disks, any cleanups and shutdown without data loss. More specifically if you have data stored in RAM but not flushed to disks, you would lose this data.
Google Compute Engine provides Snapshot functionality for persistent disks which provides taking backups at various times as you choose. This gives you freedom to go back to an earlier version if you want to.
You can also use Google Cloud Storage for files where you care about durability and availability. You can always access these files on your VM when needed.
The storage model also depends on your application. You need to plan for failures if your application needs to be resilient to them. In distributed systems, failures are unavoidable and can happen at all layers in your stack - both in hardware and software.
Here is a good guide on choosing the right Storage option on Google Cloud based on your application.
The wait time is now 90 seconds, which might not be sufficient in all cases. I am using nested virtualization with a large number of VM's. Graceful stop might easily take longer that 90 seconds. At best, the timeout should be configurable. Recommend you do a graceful stop of the services from the shell before attempting a stop from the GCP console.
