How to import all firestore documents from an export to a specific collection

Question

So I want to be able to create a snapshot of the database as it is to create a "development" collection.

I am able to export all the data currently on firestore using the tutorial here https://firebase.google.com/docs/firestore/manage-data/export-import#export_data.

Firebase only shows how to import specific collections from that stored export.

How can I import the export to a specific "development" collection within firestore?

For example:

Cloud Firestore Database Collections (day 1)

Users (as of day 1)
- Has a document for each user
Tasks (as of day 1)
- Has a document for each task
Chats (as of day 1)
- Has a document for each chat

-> export all this data to google cloud platform bucket -> import all the exported data into a “development” collection

Cloud Firestore Database Collections (day 2)

Users (as of day 2)
Tasks (as of day 2)
Chats (as of day 2)
Development
- Users (as of day 1)
- Tasks (as of day 1)
- Chats (as of day 1)

I can then use this development collection as a database snapshot for testing, then the other collections can be used in production as normal.

I'm not clear what you're asking. Could you illustrate with a specific example, and what you tried that doesn't work the way you expect? — Doug Stevenson
Please do not share pictures of text. Copy the text into the question and format it so that it's easy to read, copy, and search. — Doug Stevenson

vitooh vitooh · Accepted Answer · 2020-08-10T11:44:13

There is no such feature in Cloud SDK (using gcloud firestore) however it's quite easy to be done programmatic way. With two remarks:

Collection has to be in document, so construction collection "Users" in collection "Development" is not possible. However its possible to do "User" in document "Development" in collection "Development" (/Development/Development/Users etc.)
This will cost you reads and writes to Firestore, if you have big database it may cost both time and money. So be careful with that.

If you are OK with above it's quite easy. Here is a sample doing this thing to document "devDoc" in collection "devCol" in JS (quickstart):

const admin = require('firebase-admin');

admin.initializeApp();
const db = admin.firestore();

devRefsegments = ["devCol","devDoc"];

db.listCollections().then(colList => {
    colList.forEach(colRef => 
        colRef.get().then(colSnap => {
            colSnap.forEach(prop => {
                var path = makepath(devRefsegments,prop._ref._path.segments);
                db.doc(path).set( prop.data()).then(r => console.log(r));
            });
        })
    );
});


function makepath(devRefsegments, segments) {
    return "/"+devRefsegments.join("/")+"/"+segments.join("/");
}

I am running this in GCP Cloud Shell, however you can easily make cloud function of this, if you want. And of course you can use other languages as well. I suggest to try on small db to avoid cost at start.

How to import all firestore documents from an export to a specific collection

2 Answers