Accessing and Updating Models with Django on Google-App-Engine

Question

What are the different options, with pros and cons, for periodically adding records to a Django app hosted on GAE?

1. Use a custom django management command on the remote datastore
2. Write an API in Django that exposes the datastore to be updated
3. Use a cron task on GAE to update
4. (am I missing anything else?)

1: Custom Django management command on "remote"

I'm currently using #1: django-nonrel on GAE and using custom management/django-admin commands for my models. For example, this is how I call my custom management command on the remote datastore:

manage.py remote mycommand

The advantage of this command is ease of development: I can test the the management command locally and simply add "remote" to use it on GAE.

2: Write an API in Django that exposes the datastore

I would have to use an extra server with cron to update.

3: Use a cron task in Google

I don't know how GAE likes having its users run a scraper periodically. Also, GAE doesn't have a real cron -- it simply hits a URL at a set intervals.

Answer 1

Use a cron job. That's what they're designed for. Whether or not scraping is okay depends on the terms of service on the site you're scraping.