Which credentials should I put in for Google App Engine BulkLoader at development server?

Question

I would like to ask which kind of credentials do I need to put on for importing data using the Google App Engine BulkLoader class

appcfg.py upload_data --config_file=models.py --filename=listcountries.csv --kind=CMSCountry --url=http://localhost:8178/remote_api vit/

And then it asks me for credentials:

Please enter login credentials for localhost

Here is an extraction of the content of the models.py, I use this listcountries.csv file

class CMSCountry(db.Model):
  sortorder = db.StringProperty()
  name = db.StringProperty(required=True)
  formalname = db.StringProperty()
  type = db.StringProperty()
  subtype = db.StringProperty()
  sovereignt = db.StringProperty()
  capital = db.StringProperty()
  currencycode = db.StringProperty()
  currencyname = db.StringProperty()
  telephonecode = db.StringProperty()
  lettercode = db.StringProperty()
  lettercode2 = db.StringProperty()
  number = db.StringProperty()
  countrycode = db.StringProperty()

class CMSCountryLoader(bulkloader.Loader):
  def __init__(self):
    bulkloader.Loader.__init__(self, 'CMSCountry',
                           [('sortorder', str),
                            ('name', str),
                            ('formalname', str),
                            ('type', str),
                            ('subtype', str),
                            ('sovereignt', str),
                            ('capital', str),
                            ('currencycode', str),
                            ('currencyname', str),
                            ('telephonecode', str),
                            ('lettercode', str),
                            ('lettercode2', str),
                            ('number', str),
                            ('countrycode', str)
                            ])
loaders = [CMSCountryLoader]

Every tries to enter the email and password result in "Authentication Failed", so I could not import the data to the development server.

I don't think that I have any problem with my files neither my models because I have successfully uploaded the data to the appspot.com application.
So what should I put in for localhost credentials?
I also tried to use Eclipse with Pydev but I still got the same message :(
Here is the output:

Uploading data records.
[INFO    ] Logging to bulkloader-log-20090820.121659
[INFO    ] Opening database: bulkloader-progress-20090820.121659.sql3
[INFO    ] [Thread-1] WorkerThread: started
[INFO    ] [Thread-2] WorkerThread: started
[INFO    ] [Thread-3] WorkerThread: started
[INFO    ] [Thread-4] WorkerThread: started
[INFO    ] [Thread-5] WorkerThread: started
[INFO    ] [Thread-6] WorkerThread: started
[INFO    ] [Thread-7] WorkerThread: started
[INFO    ] [Thread-8] WorkerThread: started
[INFO    ] [Thread-9] WorkerThread: started
[INFO    ] [Thread-10] WorkerThread: started
Password for [email protected]: [DEBUG   ] Configuring remote_api. url_path = /remote_api, servername = localhost:8178

[DEBUG   ] Bulkloader using app_id: abc
[INFO    ] Connecting to /remote_api
[ERROR   ] Exception during authentication
Traceback (most recent call last):
  File "D:\Projects\GoogleAppEngine\google_appengine\google\appengine\tools\bulkloader.py", line 2802, in Run
request_manager.Authenticate()
  File "D:\Projects\GoogleAppEngine\google_appengine\google\appengine\tools\bulkloader.py", line 1126, in Authenticate
remote_api_stub.MaybeInvokeAuthentication()
  File "D:\Projects\GoogleAppEngine\google_appengine\google\appengine\ext\remote_api\remote_api_stub.py", line 488, in MaybeInvokeAuthentication
datastore_stub._server.Send(datastore_stub._path, payload=None)
  File "D:\Projects\GoogleAppEngine\google_appengine\google\appengine\tools\appengine_rpc.py", line 344, in Send
f = self.opener.open(req)
  File "C:\Python25\lib\urllib2.py", line 381, in open
response = self._open(req, data)
  File "C:\Python25\lib\urllib2.py", line 399, in _open
'_open', req)
  File "C:\Python25\lib\urllib2.py", line 360, in _call_chain
result = func(*args)
  File "C:\Python25\lib\urllib2.py", line 1107, in http_open
return self.do_open(httplib.HTTPConnection, req)
  File "C:\Python25\lib\urllib2.py", line 1082, in do_open
raise URLError(err)
URLError: <urlopen error (10061, 'Connection refused')>
[INFO    ] Authentication Failed

Thank you!

Emilien Emilien · Accepted Answer · 2010-02-03T21:41:56

EUREKA: I found the way to use the bulkloader.py tool without having to manually enter login credentials.

Here are the 2 steps:

Set your app.yaml file up. Example:

- url: /remote_api
  script: $PYTHON_LIB/google/appengine/ext/remote_api/handler.py
  login: admin

You should put it BEFORE your - url: .* line in app.yaml, otherwise you will never access the /remote_api url.

Note that I've left the login: admin part, as removing it is a VERY BAD practice, since you might deploy that into production...

2 Launch this command (adapt it to your needs).

echo 'XX' | python2.5 ../google_appengine/bulkloader.py --dump --kind=NAMEOFMODEL --url=http://localhost:8080/remote_api --filename=FILENAME --app_id=APPID [email protected] --passin .

The trick is to use the combination of those 2 parameters:

--email= (you can put whichever email address you want, I use [email protected])
--passin

Specifying --email= will suppress the "enter credentials" prompt, and --passin will allow to read the password from stdin (that's where the echo 'XX' | comes into play!)

Enjoy!

P.S.: Don't forget to vote so that Google can provide an easier to use way to do that: Issue 2440.

Which credentials should I put in for Google App Engine BulkLoader at development server?

9 Answers