4000 colliders is excessive, specially if all start colliding at the same time. You can always optimize and "fake" things. Depending on the scope of your simulation you might not need all generated particles to collide. For example, dust and tiny particles could be voxels.
If your simulation is calculation heavy, which any Physics simulation should be, then you should delegate the number crunching to C.
In Unity you can also take advantage of the shader language and send packages to the GPU to help you render. You should also look for ways to optimize, such as caching Components. Good read on optimization in Unity.
You can also tell your simulation to ignore certain collisions between certain objects, such as, ignore collision between A and B.
Once you hit the ground or a certain condition you could also add this line of code:
rigidbody.isKinematic = true;
That will make Unity ignore Physics for that Rigibody. Info here.