You have to decompose what it is that each instruction does.
Use the MIPS green card, for example.
All R-Type instructions share certain control signals, which follow from their basic instruction format — that they source/read rs & rt, and target/write to rd, for example, which means that:
RegDest is true (rd is written, not rt)RegWrite is true (some reg is written)ALUSrc is true (the 2nd alu input is from register, not from immediate)
I-Type instructions all have certain shared control signal patterns:
RegDest is false (rt is written, there is no rd for I-Types)RegWrite is true (some reg is written)ALUSrc is false (the 2nd alu input is from immediate, not from register)
No instructions touch memory except loads and stores, so MemRead and MemWrite are false for all of the other instructions. MemRead is true for loads only and MemWrite is true for stores only.
MemtoReg is true for loads as the result comes from memory, but false for most other instructions, as the result (if any) comes from the ALU instead.
For many instructions, some of these control signals don't matter, so can be classified as "don't care" rather than true or false (either of which would be fine by the hardware). For example, since a store instruction sends data to memory, it doesn't update a CPU register — thus, MemToReg for that one is don't care, since no reg is written anyway (same for branches as they also don't write/update a register).
When RegWrite is false, then MemtoReg doesn't matter (is don't care).
Unlike most of the rest, the ALUOp is one control signal that is bigger than a single boolean. So the ALU needs to know what operation to perform, and for R-Type instructions, this comes from the funct field of the instruction, whereas for I-Type instructions this comes from the opcode field itself.
For loads and stores, the ALU (in the single cycle MIPS CPU) is used to perform the addressing mode computation, so the ALU should be told to add.
For branches (in the single cycle MIPS CPU), the ALU should be told to subtract (i.e. compare).
High level: the hardware is a union of all the components necessary for all the instructions. The control signals are used to "activate" the proper hardware components for any single instruction being executed — but, the trick is that these signals don't actually activate the components, but instead choose whether to accept or ignore their output(s) after they have computed something useful or useless, respectively, given the instruction being executed.
It's kind of screwy, but that union of all the needed hardware components (for every possible instruction) are sitting there doing something all the time (unless very advanced power saving circuitry is used). So, what the control signals do — instead of turning off components that don't contribute to the current instruction — is to simply ignore their computations results, favoring something relevant instead.
For example, effectively, the immediate hardware is sign extending the I-Type 16-bit immediate on every cycle for every instruction, but that computed value is only used when the instruction actually is an I-Type, and otherwise ignored due to the control signal (ALUSrc).
While we might think of this as power inefficient, there is a parallelism performance to be appreciated, which is that things unnecessary are computed in advance, just in case they are needed and only suppressed later when more is known.
