I think there's a lot of stuff that is poorly documented on the SDK site, this is one of them. The claim I'm going to make is that it seems as though it's better to default to using an application context and only use an activity context when you really need to. The only place where I've ever seen that you need an activity context is for a progress dialog. SBERG412 claims that you have to use an activity context for a toast message, yet the Android docs clearly show an application context being used. I've always used application context for toasts because of this Google example. If it's wrong to do so, then Google dropped the ball here.
Here's more to think about and review:
For a toast message, the Google Dev Guide uses the application context and explicitly say's to use it:
Toast Notifications
In the dialogs section of the Dev guide, you see that an AlertDialog.Builder uses the application context, and then the progress bar uses an activity context. This isn't explained by Google.
Dialogs
It seems like a good reason to use application context is when you want to handle configuration changes like an orientation change, and you want to retain objects which need a context like Views. If you look here: Run Time Changes
There is a caution about using an activity context, which can create a leak. This can be avoided with an application context with the views that are to be retained (at least that's my understanding). In an app I'm writing, I intend to use an application context because I'm trying to hold over some views and other things on an orientation change, and I still want the activity to be destroy and recreated on orientation changes. Thus I have to use an app context to not cause a memory leak (see Avoiding memory Leaks). To me it seems there are plenty of good reasons to use the application context instead of an activity context, and to me it almost seems like you would use it more often than an activity context. That's what many Android books I've gone through seem to do, and that's what much of the Google examples I've seen do.
The Google documentation really makes it seem like using application context is perfectly fine in most cases, and in fact appears more often than using an activity context in their examples (at least the examples I've seen). If it's really such a problem to use application context, then Google really needs to place more emphasis on this. They need to make it clear, and they need to redo some of their examples. I wouldn't blame this entirely on inexperienced developers since the authority (Google) really makes it look like it's not a problem to use application contexts.