If you have a small number of key/value pairs and the values aren't big the registry is great - and you don't care about xcopy deployment - then use the registry (I know this isn't exact, but it's usually obvious when working with the registry becomes a pain).
If you want xcopy deployment the data must be in the same folder as the program obviously - but the program can be somewhere under the AppData folder, it doesn't have to be under "program files".
Use isolated storage only when you need it or have to use it - for example ClickOnce.
Otherwise use AppData\Roaming, use Local or LocalLow only if you have a good reason.
EDIT: Here is the difference between Roaming, Local and LocalLow:
Windows has a little known feature called "roaming profiles", the general idea is that in a corporate environment with this feature enabled any user can use any computer.
When a user logs in his private settings are downloaded from the server and when he logs out his settings are uploaded back to the server (the actual process is more complicated, obviously).
Files in the User's "Roaming" folder in Vista or "Application Data" in XP move around with the user - so any settings and data should be stored there.
Files under "Local" and "LocalLow" in vista and "Local Settings" in XP do not, so it's a good place for temp files, things that are tied to the specific computer or data that can be recalculated.
In Vista, as part of the new security features we all know and love, you can have programs running in "low integrity mode" (for example IE in protected mode), those programs are running with reduced privileges and can't access files in the user's profile - except for files under the "LocalLow" folder.
So, in conclusion, files stored in "LocalLow" are inherently insecure and files in "Local"/"Local Settings" are likely to be unavailable in some large companies - so unless you have good reason and know exactly what you are doing go with "Roaming"/"Application Data".
