As chux already stated, there can be a lot of different bus width's in a computer system. That said, I assume you're talking about usual PC architectures here. Now, to your questions:
Performance difference between 32 and 64 bit systems
- The hardware usually is able to operate on bigger numbers than a 32 bit system, so it can e.g. sum two 64 bits numbers in one operation, while a 32 bit system would need at least two (plus some operations to combine the results). This means a software that does lots of operations in big numbers will probably be faster on a 64 bit system, but a software that don't need big numbers will not be faster
- A 64 bit processor usually fetch bigger blocks of data from memory than a 32 bit one. If the data bus is 64 bits instead of 32, it'll fetch double the bytes than the 32 bit system
- This is actually a negative point in 64 bit system: since you have more addressable memory, you also need more memory for pointers, so 64 bit applications will also use a little more memory than the same aplication compiled for a 32 bit system.
Memory addressing
The memory address is a number that uniquely identifies a position in memory, where data is stored. With a 32 bit number, you can adress 2^32 positions, which is roughly 4 GB. This is why 32 bit PC's cannot use more than 4 GB of memory (they actually can, with some restrictions. See PAE). Using 64 bit numbers means the computer can now address 2^64 positions, which means it could, in principle, use up to 16 exbibytes of memory. In practice, other limits prevent a PC from having all that memory.
Addressable locations vs Memory Size
Since lots of instructions should reference a memory position, this means that some of them will have to be bigger, so they have room for memory adresses.
Bigger instructions usually means bigger software code, but this is not a problem in most cases, because the difference isn't that big, and because most of software size usually is composed of data, rather than code.
Disclaimer: Not all I said is valid for every software/architecture. There are a lot of detais that may have more impact in performace and memory usage than the points I wrote here.