7
votes

So in the current Beta Functions of the Google Cloud CLI there is a option to import and export Firestore data. https://firebase.google.com/docs/firestore/manage-data/export-import

Now I made a Export into a Bucket, all worked fine and imported it again, worked well too. Now I have 2 different Projects and wanna import the backup into a different Project, is that possible? Would be good if I only need a billing Account for one Project too.

This line in the Docs says its possible

Once you have export files in Cloud Storage, you can import documents in those files back into your project or to another project.

But the Docs dont cover how exactly its done?

4
The documentation gives you examples of gcloud commands to run to import the backup from your storage bucket. I think it's assuming that you understand how to operate gcloud among different projects. As long as your Google account has admin access to both projects, you should be able to import from a bucket in one project to Firestore in another project. Is there something specific that you're stuck on? - Doug Stevenson
@DougStevenson Nothing specific, but what would happen if I have a Cloud Bucket with the Same Name and a Backup with the same name in both Projects, which one would it take the path of if i go for the import path syntax like in the docs - Badgy
There can't two buckets with the same name anywhere in Google Cloud. They're unique. - Doug Stevenson

4 Answers

18
votes

It is possible to import/export between projects. Here are the steps that have worked for me:

First, make sure that the gcloud command line tool is installed. Instructions on set up, and complete information on the export/import process can be read on Firebase's documentation page about Firestore Export and Import.

Before continuing, set the gcloud project to the project from which you want to source your data:

gcloud config set project [PROJECT_ID]

Then, using the Google Cloud Console web application, make sure that a Cloud Storage bucket has been created on the project that will be the source of the data.

For example, for the source bucket, you might create a bucket such as:

gs://my-source-project-export.

You can name the bucket whatever you want, as long as you choose something unique.

Exporting of the source data can then be completed using a command. For example, if you wanted to export just the cameras and radios collections to your my-source-project-export bucket, with a dated directory to identify the export, you include the optional collection-ids flag, as follows:

gcloud beta firestore export gs://my-source-project-export/export-20190113_2109 --collection-ids='cameras','radios'

Omitting the flag would copy ALL the collections.

The gcloud CLI tool should complete the export without issue.

Now, to complete the import, we first switch the gcloud project to the target for our data:

gcloud config set project [PROJECT_ID]

Then, we can attempt the import:

gcloud beta firestore import --collection-ids='cameras','radios' gs://my-source-project-export/export-20190113_2109

The operation may fail due to permission issues. If so, it will report which service account needs to have access to the bucket. To resolve the permission issues, you can simply using the Google Cloud Console Storage Browser to administer the permissions for the source bucket. The required service account must be added to the members list with the role Storage Admin.

Once the permissions are corrected, the operation can be re-attempted. For long running operations, a list of operations and their statuses can be retrieved using the following command:

gcloud beta firestore operations list

Once the import is completed, it may be wise to revoke the permissions granted to the service account, if any, to avoid any unwanted security issues.

Hope that helps.

3
votes

The accepted answer didn't work for me. No matter what permissions were granted on the source bucket the import always failed with PERMISSION DENIED: The caller does not have permission

The solution was to create another service account. I created a service account on the destination project with Cloud Datastore Import Export Admin and Storage Admin roles. I then added this service account to the source project IAM with the same roles. After doing this the following process worked for me:

gcloud auth activate-service-account --key-file=./mynewserviceaccount.json
gcloud beta firestore export gs://mysourceprojectbucket --project mysourceprojectid
gcloud beta firestore import gs://mysourceprojectbucket/WHATEVER_NAME_FROM_EXPORT --project mydestinationproject
0
votes

Main steps : Make sure billing is enabled in both the projects.

For PROJECT_1,

  1. Create Storage bucket (BUCKET_1) in Google Cloud Storage in PROJECT_1
  2. Export firestore data to BUCKET_1 using Cloud shell
  3. Get service account of PROJECT_2 and give it "Storage Admin" permission in BUCKET_1

PROJECT_2

Import data from BUCKET_1

I have put output of all the steps in this blog post

0
votes

In my case, the target database had no write rules.

Switching from:

rules_version = '2';
service cloud.firestore {
  match /databases/{database}/documents {
    match /{document=**} {
      allow read, write: if false;
    }
  }
}

To:

rules_version = '2';
service cloud.firestore {
  match /databases/{database}/documents {
    match /{document=**} {
      allow read, write: if true; /* Allow writes by setting true here */
    }
  }
}

Solved my problem