Chances are you could be using code not compatible with older versions. The only way to know is to have the app compile to that iOS version. But, in my honest opinion, you should stick with 7.0 onwards since they are the officially supported versions. You could change your deployment target to 7.0 and try to compile to see the effects.
If you set the deployment target to 7.0, Xcode will tell you which parts of your code are non compatible.
Regarding performance, that depends on what type of code you did and which device you are going to deploy. For that response you need to provide a little bit of more info about your app and code. But like I said before, change the deployment target in 7.0 to at least be sure your code will run on that version.