I think my question is similar to: BulkLoader -export_transform OR https://stackguides.com/questions/3220911/gae-datastore-export-transform
Basically, I'm using the bulkloader to backup and restore entities that I've created. To test I'm using the "Game" kind and outputting it to a csv file called game.csv.
This is the process I go through:
Download the game kind to game.csv using:
appcfg.py download_data --config_file=bulkloader.yaml --kind=Game --filename=game.csv --application=MyAppId --url=http://MyAppId.appspot.com/remote_api --rps_limit=500 --bandwidth_limit=2500000 --batch_size=100
Delete all game entities. I then checked the my app's admin portal datastore viewer tab and I see that there are no more entities in my data store.
Upload the game kind using the game.csv created using (command is the same as download_data but with upload_data):
appcfg.py upload_data --config_file=bulkloader.yaml --kind=Game --filename=game.csv --application=MyAppId --url=http://MyAppId.appspot.com/remote_api --rps_limit=500 --bandwidth_limit=2500000 --batch_size=100
Run a servlet that retrieves an entity by 'name' (this is the property shown in Game.java below).
The following error occurs:
Uncaught exception from servlet
java.lang.IllegalStateException: Loaded Entity has name but com.example.app.model.Game has no String @Id
at com.googlecode.objectify.impl.ConcreteEntityMetadata.setKey(ConcreteEntityMetadata.java:343)
at com.googlecode.objectify.impl.ConcreteEntityMetadata.toObject(ConcreteEntityMetadata.java:210)
at com.googlecode.objectify.impl.QueryImpl$ToObjectIterator.translate(QueryImpl.java:640)
at com.googlecode.objectify.impl.QueryImpl$ToObjectIterator.translate(QueryImpl.java:629)
at com.googlecode.objectify.util.TranslatingIterator.next(TranslatingIterator.java:35)
at com.googlecode.objectify.impl.QueryImpl.list(QueryImpl.java:470)
I suspect it has something to do with my bulkloader.yaml file not being configure correct for the __key__ property. So I've posted it below:
- import: google.appengine.ext.bulkload.transform
- import: google.appengine.ext.bulkload.bulkloader_wizard
- import: google.appengine.ext.db
- import: google.appengine.api.datastore
- import: google.appengine.api.users
transformers:
- kind: Game
connector: csv
connector_options:
# TODO: Add connector options here--these are specific to each connector.
property_map:
- property: __key__
external_name: key
export_transform: transform.key_id_or_name_as_string
- property: __scatter__
#external_name: __scatter__
# Type: ShortBlob Stats: 56 properties of this type in this kind.
- property: genre
external_name: genre
# Type: String Stats: 6639 properties of this type in this kind.
- property: name
external_name: name
# Type: String Stats: 6639 properties of this type in this kind.
- property: releasedate
external_name: releasedate
# Type: Date/Time Stats: 6548 properties of this type in this kind.
import_transform: transform.import_date_time('%Y-%m-%dT%H:%M:%S')
export_transform: transform.export_date_time('%Y-%m-%dT%H:%M:%S')
Prior to the deletion and upload of data, the "Game" kind (within the datastore viewer tab of the app's admin portal) shows the ID/NAME column with hyperlinks that look like ID=12345, ID=67890 ... for each entitiy row.
After the upload the datastire viewer shows the ID/NAME column with hyperlinks that look like NAME=12345, NAME=67890 ... for each entitiy row.
Possible cause?
I've actually posted my question on, https://groups.google.com/forum/?hl=en#!topic/objectify-appengine/FFuB2Onfnzc but we don't know the syntax to avoid transforming the __key__ property to string when uploading back the data.
Jeff (creator of Objectify3.0) says:
The error message says that the data in the datastore as a String name key, but your Game entity has a numeric Long @Id. I don't really know the syntax for the bulk loader, but the most suspect line is this one:
export_transform: transform.key_id_or_name_as_string
It looks like you are converting all numeric ids to strings here, which would be your problem. Leave them as numbers.
I've tried removing the export_transform: transform.key_id_or_name_as_string but when I do the download the csv file is empty.
Maybe a solution is to avoid using the --config_file parameter and bulkloader.yaml file and just let the bulkloader download/upload the data without any transformations?
Extra information incase it might be relevant
I'm using the objectify3.0.jar to manipulate my GAE datastore. So my Game kind looks like this:
public class Game {
@Id private Long id; //This is my key property, auto generated by objectify
private String name;
private String genre;
private Date releasedate;
//ommitting getters and setters
}
