My windows system has 8 cores.
When I use 8 CPUs with my MPI: mpiexec.exe -n 8, all of my 8 available processors are busy in task manager which makes sense.
When I use 2 cores: mpiexec.exe -n 2, I expect only 2 cores should be busy but that's not the case and I have an irregular CPU usage distributed over 8 cores.
Is this observation expected?
Yes, this behaviour is expected. A general-purpose operating system such as (most versions of) Windows moves processes around cores. One reason for this is to ensure that no process is starved of execution time. Don't forget that on most Windows computers there will be all sorts of processes running at the same time as your computational processes. Use the Task Manager to see what is going on and don't be surprised if there are dozens of processes running in addition to the 2 running your MPI program.
So, yes, with two processes running a computationally-intensive program you can expect the core usage to be irregular, but to average out at 2/8 over time.
Of course, for the special case of parallel MPI programs this behaviour may be performance damaging. Generally MPI implementations provide some way to 'pin' processes to cores. Consult the documentation for your MPI implementation for how to do this. But don't be surprised if you find that performance actually drops
