PHP has no limit, and you can do what you say and run it in a tight loop.
On linux it is sendmail, or a sendmail clone. If it starts to get overloaded it should start queueing your mails, and delivering them when it can. For Windows you have a socket connection, and I suppose the mail server might start refusing connections if it got overloaded.
Unless you are in a hurry to send them all, consider a sleep(2) between each call to mail(). If all goes smoothly then try sleep(1) the next time. Then usleep(500000) next time, etc. (UPDATE: following Matt's suggestion, put set_time_limit(10) inside the loop to avoid running out of CPU time.)
If you want them sent out even quicker you round-robin with multiple local relays. (On windows you could handle this by setting the SMTP php.ini setting; on linux you'd use on of the libraries that sends email over a socket.) Or consider a professional mail broadcasting service, where they take care of all these issues for you.
EDIT Just saw your comment about running it from your Mac. I'd assumed a server situation, with a real mail server with a global IP. If your Mac is behind a firewall you are most likely set to forward all your email to your ISPs mail server. Your ISP may well get upset about receiving 4000 emails from you in a short period of time. (E.g. they may assume your machine has been compromised by a virus and shut you down.)
Also, on shared hosting, agreeing not to send out lots of email is often part of the contract. So be careful there too.