I just built an app with 3.1.2 that was just accepted into the app store. Building with a lower version will ensure compatibility with people that are running older versions of the OS on their iphone/ipod touch.
EDIT: you should compile with 3.2...see my comment below.