When a change is made to a web.config, it is typically the app domain that is unloaded and not the app pool that is recycled (some info on that here). Site1, App1, and App2 should each be defined as separate applications and not just virtual directories. Each application will be running in its own app domain, so replacing or changing a web.config should not affect the others unless it is a child application that is inheriting settings from the parent web.config. Depending on the change, making a change to the Site1 web.config could end up causing the app domain for App1 and App2 to unload.
If App1 and App2 are using the same web.config (i.e. if both document roots are pointing to the same file system location) then making a change to the web.config will end up unloading the app domain for both applications.