Web Site is what you deploy to an ASP.NET web server such as IIS. Just a bunch of files and folders. There’s nothing in a Web Site that ties you to Visual Studio (there’s no project file). Code-generation and compilation of web pages (such as .aspx, .ascx, .master) is done dynamically at runtime, and changes to these files are detected by the framework and automatically re-compiled. You can put code that you want to share between pages in the special App_Code folder, or you can pre-compile it and put the assembly in the Bin folder.
Web Application is a special Visual Studio project. The main difference with Web Sites is that when you build the project all the code files are compiled into a single assembly, which is placed in the bin directory. You don’t deploy code files to the web server. Instead of having a special folder for shared code files you can put them anywhere, just like you would do in class library. Because Web Applications contains files that are not meant to be deployed, such as project and code files, there’s a Publish command in Visual Studio to output a Web Site to a specified location.
App_Code vs Bin
Deploying shared code files is generally a bad idea, but that doesn’t mean you have to choose Web Application. You can have a Web Site that references a class library project that holds all the code for the Web Site. Web Applications is just a convenient way to do it.
CodeBehind
This topic is specific to .aspx and .ascx files. This topic is decreasingly relevant in new application frameworks such as ASP.NET MVC and ASP.NET Web Pages which do not use codebehind files.
By having all code files compiled into a single assembly, including codebehind files of .aspx pages and .ascx controls, in Web Applications you have to re-build for every little change, and you cannot make live changes. This can be a real pain during development, since you have to keep re-building to see the changes, while with Web Sites changes are detected by the runtime and pages/controls are automatically recompiled.
Having the runtime manage the codebehind assemblies is less work for you, since you don't need to worry about giving pages/controls unique names, or organizing them into different namespaces.
I’m not saying deploying code files is always a good idea (specially not in the case of shared code files), but codebehind files should only contain code that perform UI specific tasks, wire-up events handlers, etc. Your application should be layered so that important code always end up in the Bin folder. If that is the case then deploying codebehind files shouldn't be considered harmful.
Another limitation of Web Applications is that you can only use the language of the project. In Web Sites you can have some pages in C#, some in VB, etc. No need for special Visual Studio support. That’s the beauty of the build provider extensibility.
Also, in Web Applications you don't get error detection in pages/controls as the compiler only compiles your codebehind classes and not the markup code (in MVC you can fix this using the MvcBuildViews option), which is compiled at runtime.
Visual Studio
Because Web Applications are Visual Studio projects you get some features not available in Web Sites. For instance, you can use build events to perform a variety of tasks, e.g. minify and/or combine Javascript files.
Another nice feature introduced in Visual Studio 2010 is Web.config transformation. This is also not available in Web Sites. Now works with Web Sites in VS 2013.
Building a Web Application is faster than building a Web Site, specially for large sites. This is mainly because Web Applications do not compile the markup code. In MVC if you set MvcBuildViews to true then it compiles the markup code and you get error detection, which is very useful. The down side is that every time you build the solution it builds the complete site, which can be slow and inefficient, specially if you are not editing the site. l find myself turning MvcBuildViews on and off (which requires a project unload). On the other hand, with Web Sites you can choose if you want to build the site as part of the solution or not. If you choose not to, then building the solution is very fast, and you can always click on the Web Site node and select Build, if you’ve made changes.
In an MVC Web Application project you have extra commands and dialogs for common tasks, like ‘Add View’, ‘Go To View’, ‘Add Controller’, etc. These are not available in an MVC Web Site.
If you use IIS Express as the development server, in Web Sites you can add virtual directories. This option is not available in Web Applications.
NuGet Package Restore does not work on Web Sites, you have to manually install packages listed on packages.config Package Restore now works with Web Sites starting NuGet 2.7