Generally, it is true that thousands of mobile phones in the vicinity of one beacon are not affected by one another for the reason you say. When scanning, the phones' Bluetooth chips are in receive only mode.
In practice, however, phones are often transmitting as well for at least two reasons:
Both iOS and Android devices doing BLE scanning will transmit a scan request packet to the beacon when it is first detected, which will cause the beacon to reply with a scan response packet if it is built to do so. (Some but not all hardware beacons send a scan response. Software beacons typically do.)
See more here
Phones sometimes run other apps that themselves may be using BLE advertising in the background or even establishing BLE connections or data exchange with other phones or other BLE devices in the vicinity (e.g. a user wearing a smart watch).
The above caveats generally do not affect your use case much. Bluetooth traffic causes collisions that will affect detections, but this only reduces detection rates, it does not stop detections. In practice, detections still happen even with many hundreds of beacons transmitting in a short range to many hundreds of phones. I have seen this work for hundreds of conference attendees in the same room each wearing a personal beacon transmitter.
I also set up a beacon-based indoor back system at the Consumer Electronics Show. Thousands of people were in BLE range of transmitting beacon during keynote speeches, and many of them had the conference app that was scanning for these beacons. While I have no stats on detection rates, it generally detected the beacons just fine.