Cross-browser testing: All major browsers on ONE machine

139
votes

Aim of this guide:

  • Running multiple unmodified native versions of Internet Explorer,
    Safari, Opera, Chrome and Firefox on a single machine, side-by-side.

Covered in part 1:

  • Table of contents
  • Which browsers have to be tested?
  • How to create a Windows XP VM which boots quickly, and never expires?
  • Where can the necessary software (VM image, browsers, ... ) be downloaded?

Also covered in part 2:

  • Installation and configuration guide for IE, Firefox, Opera, Chrome and Safari.
  • Developer tools and shortcuts.
  • Time and disk space costs.

Many sections are independent. Eg, instructions on running multiple versions of a browser are generally applicable.

3
internet-explorerbrowsercross-browservirtualization
I would say, an easy alternative would be to use browserstack.com – Nakul
For anyone trying to make a Windowns 7 VM: VirtualBox by default will mount the Windows 7 vhd file as SATA, you will need to unmount this and mount it as IDE instead - otherwise Windows 7 will try to repair the installation and fail. Additionally, I was only able to get the mouse cursor working if I disabled mouse integration (from the Devices menu of the VM) – Ben
Related: stackoverflow.com/questions/574463/… – Denilson Sá Maia
@RobW as worthwhile and awesome as the content of this page is, it's not even slightly a question and liable to be something of a bad answer magnet... can you try and rearrange it into one somehow? – DaveRandom
@Dave It is not a usual question, but the concise question body shows what timeless questions are going to be answered, and in which format. That's the unambiguous objective of this page (which would suit better on a standalone blog in retrospect). There are 2 other useful answers about IE, plus a deleted answer about configuring Safari, so there are no signs of being a bad answer magnet. If you have any concrete improvements, just suggest an edit. – Rob W

3 Answers

101
votes

Contents

  1. Which browsers have to be tested?
    • Rules of thumb: Which browsers should be included?
  2. Preparation
    • Windows XP
    • Windows 7+ (for IE9+)
    • Browser downloads
      • Internet Explorer
      • Firefox
      • Opera
      • Chrome
      • Safari
      • Adobe Flash Player
    • Download summary
  3. Sandboxie

    Part 2: Installation and configuration
  4. Internet Explorer
  5. Firefox
  6. Opera
  7. Chrome
  8. Safari
  9. Developer tools (and shortcuts)
  10. Measured set-up time and disk space
    • Time needed per browser (install & config)
  11. Optimal usage
    • Homepage at http://10.0.2.2:8888/

1. Which browsers have to be tested?

Statistics are quickly outdated. For this reason, I refer to Usage share of web browsers on Wikipedia, and the following sites for the latest browser version information. Each site is suffixed by a brief usage guide.

  • Can I use - Browser usage table, based on data from StatCounter and other sources.
  • StatCounter - Statistic: Browser version | Time period: Last month screenshot.
  • W3Counter - View archived reports: January 2012 (Choose the last month). screenshot.
  • Wikimedia - Go to the bottom to open the latest report, SquidReportClients.
  • Clicky - This site offers statistics for individual versions screenshot.

Rules of thumb: Which browsers should be included?

  • Firefox: Latest ESR release + latest stable major version. Release notes | For developers | Rapid release schedule
  • Opera: 12.x + Latest stable version. Version history
    Since Opera 15, Opera uses the same engine as Chrome. Quite a lot of users are still using Opera 12 though (when Opera 15 was released, the UX significantly changed).
  • Safari: 5.1.7 (Windows) + Latest version. Release notes
    Unfortunately, Apple dropped support for Windows, so you need to get a Mac or run OS X in a VM to test pages in Safari.
  • Chrome: Latest stable version. Release notes
  • Internet Explorer: IE8+. Feature history
    This actually depends on your clients. This guide teaches how to get IE6 and 7 as well, but these browsers are already dead, or close to dead. IE8 is the last supported version of Internet Explorer on Windows XP, which is still used a lot.

Remark

These days, most web browsers use some form of rapid releases with automated updates. When this guide was written, the releases were not as frequent, and thus it was useful to set up a set of browsers without having to look back.
Unless you intentionally want to test applications in old browsers, just get the latest version of the browser and let the auto-updater take care of managing the version.

If you don't need too much flexibility, and quickly want to test a page, I recommend to take a look at BrowserStack.com. After signing up, you can get a 30-minute free trial that grants you access to a lot of desktop and mobile browsers, directly within your browser.

2. Preparation

Before setting up the machine, download all necessary files (see "Download summary" at the end of this section). All files are going to be shared with the virtual machine through a shared folderWhat? How?.

  • Virtualization software (VirtualBox is recommended, it is free even for commercial use. The instructions below are written with VirtualBox in mind.)
  • Windows XP
    • Download the IE6 XP image for VirtualBox from the modern.IE website. Extract the .ova file inside and run it to import it into VirtualBox. (Images are also available for other virtualization products such as VMware, Parallels, Virtual PC and Hyper-V)
    • Preparing the image for use:
      • VirtualBox settings: Enable the network adapter, but do not connect the VM to a real network.
      • VirtualBox settings: Create a read-only shared folder. This folder is going to be used to transfer data between the Host OS and the guest OS.
      • Run net use x: \\vboxsvr\WinShared (assuming that the shared folder is called WinShared. This will bind the network directory to the x: drive).
      • Optional: Install AntiWPA to disable the activation check if you will use the image for more than 30 days.
      • Optional: Disable the paging file (Computer > Properties > Advanced > Performance > Advanced > Virtual Memory > Change > No paging > Set [confirm]).
      • Optional: Disable unnecessary services via Start > Run > services.msc
        Order the rows by column Startup Type, and switch all "Automatic" services to "manual", according to the image. Whenever you want to install a MSI package, run net start msiServer ("Windows Installer"):
        Windows XP - services.msc
      • Optional: Disable Desktop clean-up wizard: Desktop > Properties > Desktop > Customize Desktop > Disable "Desktop Cleanup wizard every 60 days"
      • Optional: Install and configure Sandboxie (used for running IE7 and IE8 in the same IE6 virtual machine)
      • Optional: Install CCleaner, run it to configure it and clean junk.
      • Optional: Install 7-Zip (used for multiple Chrome's)
      • Shutdown the system via the guest OS (WinXP).
      • VirtualBox settings: Connect the VM to a real network.
        (Only Internet Explorer requires Internet during installation)
      • Optional: Create a snapshot of your VM, so that you can restore when you mess up in the next step.
  • Windows 7+ (for IE9+)
    • Download pre-built virtual machine images from the modern.IE website.
      These images are time-bombed, they expire 30 days after first use. When the images are expired, they shut down after an hour of use. If you do not wish to re-create the images every time, change the hardware clock of your VM before activating the image.
      In VirtualBox, for example, you can use VBoxManage modifyvm NAME_OF_VM --biossystemtimeoffset +3600000000 to set the time to 1000 hours in the future (use minus if you want to set it to some point in the past).
  • Browsers downloads

Download summary

For convenience, it's wise to have all installation files in the shared folder. Here's a brief (complete) list of all necessary files (ordered per step):

3. Sandboxie

Sandboxie is a lightweight tool which runs applications in a sandbox. It can also be used to install multiple versions of IE/Safari on a single Windows machine.

  • Download: http://sandboxie.com/index.php?DownloadSandboxie
  • Patch: The extended version is required to enable multiple sandboxes at a time. Pay them or visit YouTube.
    Running multiple IE/Safari versions simultaneously requires the extended version. If you don't mind testing one IE/Safari version at a time, the standard version is sufficient.

    Post-install:
  • Disable tips: "Configure > Tips > Hide All Tips"
  • By default, a yellow border appears around sandboxed applications. If you don't like this, go to Configure base: "Sandbox > DefaultBox > Sandbox Settings > Appearance".

    For each new IE/Safari version, you have to follow the following steps:
  • Create: "Sandbox > Create New Sandbox" - Enter a name, eg "IE8" and confirm.
  • Install: "IE8 > Run Sandboxed > Run Any Program"
    Select the desired installer, and press OK.
  • Shortcut: Upon finishing installation, use "IE8 > Explore contents", locate the binary and create a Desktop shortcut to the application.

All sandboxed file/registry changes are saved in C:\Sandbox. Applications in this directory start in a sandbox by default. Other programs can easily be started in a sandbox via the contextmenu: "Run Sandboxed" or "Send To > Sandboxie > IE8".

Cleaning up using CCleaner: Run CCleaner in the main environment, then in the individual sandboxes.

Navigate to: Part 2

60
votes

Contents of part 2 (navigate to: part 1)

 4.  Internet Explorer
 5.  Firefox
 6.  Opera
 7.  Chrome
 8.  Safari
 9.  Developer tools (and shortcuts)
 10.  Measured set-up time and disk space
    •  Time needed per browser (install & config)
 11.  Optimal usage
    •  Homepage at http://10.0.2.2:8888/

Installation & configuration

Browser configuration tips:

  • Home page
  • Disable check for: "Default browser" and "updates".
  • Activate Developer tools

4. Internet Explorer

Windows 7 does not allow running older instances of IE but see note below. That is why a Win XP VM is needed.

IE cannot be downgraded, so start with the lowest version, then upgrade IE in a separate Sandboxie. Set the preferences in the lowest browser version before upgrading, so that you only have to set the main preferences once.

  • IE6 - Installed by default in WinXP
  • IE7/IE8 - Install in Sandboxie (WinXP)
  • IE9+ - Get a pre-built Windows virtual machine images from the modern.IE website.
    These images expire 30 days after first use. When the images are expired, they shut down after an hour of use. If you do not wish to re-create the images every time, change the hardware clock of your VM before activating the image.
    E.g. If you use VirtualBox, launch the terminal and issue the following command (replace the capitalized names with something sensible):
    VBoxManage modifyvm NAME_OF_VM --biossystemtimeoffset <OFFSET IN MS, prefixed by plus or minus>

Fully offline installers for IE6 (and even IE5.5) are available at this site.

For IE6 and IE7, the Internet Explorer Developer Toolbar should be installed. IE8+ has built-in F12 Developer tools.

Configuration

  • Start IE6 settings: Home page, Security, Cookies, default browser.
  • IE7: Get rid of the First run page, via (IE7 Sandboxie) registry. See this post on Microsoft's TechNet forum.
  • IE8: Get rid of the "Set Up Windows Internet Explorer 8" dialog. This can be done by starting IE8, then clicking "Later", or modifying the registry.

It is actually possible to run IE6 - 8 in Windows 7 through application virtualization. VMWare ThinApp works great, but it takes considerably more time to set up, because you need to capture the installation of Internet Explorer. Also, the software package is very expensive (keygens are freely available, but unethical).

5. Firefox

Option 1 (lazy):
Utilu Mozilla Firefox Collection. This tool contains installers for all Firefox versions (200+ MB). Firebug, the Web Developer toolbar and Flash are also included.

Option 2:
Multiple Firefox versions can easily be installed and executed at the same time.

  1. Download Mozilla's official Profile Manager.
  2. Download the relevant Firefox versions from http://releases.mozilla.org/pub/mozilla.org/firefox/releases/. Older versions can be found at ftp.mozilla.org.
  3. Install each Firefox version. It is convenient to choose one main directory ("Firefox"), and use the version numbers as names for the subdirectories ("3.6").
  4. Extensions: Start the oldest Firefox version, and get your favourite extensions:
    • Firebug - A must-have for old Firefox versions. Not really needed for the latest Firefox versions, that have a nice set of built-in dev tools.
    • HttpFox - A simple and effective tool to measure HTTP requests.
    • Web Developer - An extremely useful toolbar for web development. Saves much time.
    • Console2 - Enhances the built-in Console (Ctrl Shift> J).
  5. Preferences: Tools > Options (Windows), Edit > Preferences (Linux)
    • General: Homepage
    • Security: Uncheck all settings, except for "Warn me when sites try to install add-ons".
      (Do not forget to view the Warning message settings, via the button at the bottom).
    • Advanced:
      • General:
        • Check for default browser: off
        • Submit crash reports: off
      • Update: Disable all updates
  6. Clean-up: Close all tabs, press Ctrl Shift Del and check everything. Then close Firefox.
  7. Firefox' Profile Manager:
    screenshot
    • Duplicate each profile via the Copy option.
    • A dialog screenshot opens. Choose an appropriate name (e.g. Firefox 3.6) and a destination.
    • Use the Firefox version option to select the default Firefox version for the profile.
    • Also check the Start new instance [-no-remote] box, to allow multiple simultaneous Firefox versions running at a time.
  8. Finishing up
    • Use the Start Firefox button to start an instance of Firefox.
    • Let the add-on compatibility check run and update where necessary.
    • Clear the cache, history, etc. using Ctrl Shift Del.
    • Repeat this step for each Firefox version.

6. Opera

Opera: A list of all installers is available on Opera.com. Multiple versions can be installed side-by-side without any problems. During the installation, choose "Custom", and a different directory.
On installation, also choose to use the same profile for all users.

Important preference: Settings > Preferences > Advanced > Security > Do not check for updates.

Note: Opera 15+ uses the same rendering and JavaScript engine as Chrome.

7. Chrome

Chrome: Stand-alone installers can be downloaded from File Hippo.
It is also possible to run multiple Chrome versions side-by-side.

Although Sandboxie can be used, it's recommended to use the next native method in order to run multiple versions side-by-side.

  1. Download the desired version(s) from File Hippo.
  2. Create a main directory, e.g. C:\Chrome\.
  3. Extract the installer (=without installing), using 7-Zip for example.
    After extracting, a chrome.7z archive is created. Also extract this file, and descend the created Chrome-bin directory.
    Now, you see chrome.exe and a dir like 18.0.1025.45.
    Move chrome.exe to 18.0.1025.45, then move this directory to C:\Chrome. The remaining files in Chrome-bin can safely be deleted.

  4. Create a shortcut for each version:

    "C:\Chrome\18.0.1024.45\chrome.exe" --user-data-dir="..\User Data\18" --chrome-version=18.0.1025.45

    Explanation of this shortcut:

    • "C:\Chrome\18.0.1024.45\chrome.exe" • This is the launcher
    • --user-data-dir="..\User Data\18" • User profile, relative to the location of chrome.exe. You could also have used --user-data-dir="C:\Chrome\User Data\18" for the same effect. Set your preferences for the lowest Chrome version, and duplicate the User profile for each Chrome version. Older Chrome versions refuse to use User profiles from new versions.
    • --chrome-version=18.0.1025.45 • Location of binaries:
      • The location (eg 18.0.1025.45) must be the name of the directory:
      • Must start and end with a number. A dot may appear in between.
      • The numbers do not necessarily have to match the real version number (though it's convenient to use real version numbers...).

Regarding configuration: All preferences can be set at chrome://settings/. I usually change the home page and "Under the hood" settings.

With the lot of Chrome releases, it is not practical to have all versions installed. I have created a VB script that automates these steps, so I can just store the installers, then run the script whenever I need to test an old version of Chrome: https://gist.github.com/Rob--W/2882558

8. Safari

Note: Safari support on Windows has been discontinued. The last supported Safari version on Windows is 5.1.7; you need a Mac or an OS X VM to test your sites in newer Safari versions.

Safari does not natively support multiple versions or user profiles side-by-side. Safari 4 is close to death, so you only have to test 5.*. All Safari versions can be downloaded from Old Apps.

  • Download and install Safari 5.0.
  • Start Safari, set your preferences.
  • Create a new Sandboxie sandbox for each additional installation.
    It's important to initially install the oldest version, so that the user profile can be adapted by newer versions.
  • For more information on installing in Sandboxie, see the Sandboxie and Internet Explorer section.

The Developer tools have to be enabled via Preferences > Advanced > Show Developer menu in menu bar. After installing Safari, uninstall Apple software Updater, via Control panel > Add/Remove software.

9. Developer tools (and shortcuts)

10. Measured set-up time and disk space

  • Setting up the base environment takes 30 minutes.
  • Installing browsers does not require much time.
  • Depending on the number of browsers, configuring them can take some time.

  • Setting up a WinXP VM with the browsers as listed in this guide + Opera 12 beta:
    • Size of downloads: 585MB
    • 12 browsers
    • Time used: 1:09 hour (32 minutes to get to IE)
    • Appliance size: 1.1G / imported size: 2.2G.
    • Detailed log: http://pastebin.com/R7WGPK99
  • Setting up a huge WinXP VM for detailed browser compatibility tests:

Time needed per browser (install & config)

  • Opera: 2 minutes
  • Internet Explorer: 3 minutes *
  • Safari: 4 minutes *
  • Firefox: 5 minutes
  • Chrome: 6 minutes

* excludes time to set up a Sandboxie Sandbox ( < 1 minute).

11. Optimal usage

When you've finished installing, use CCleaner to clean junk, then:

  • Export the appliance (this appliance can be saved elsewhere as a back-up).
  • Remove the newly created Virtual Machine
  • Import the appliance (these steps reduce the file size of the VM)
  • Create a snapshot (to be used as a checkpoint)

From now on, when you've finished testing web pages, turn off the VM, and choose "Restore snapshot". This will keep your VM tidy and fast.

Turn off machine > Restore snapshot

Homepage at http://10.0.2.2:8888/

The guest OS can access the host OS by IP address 10.0.2.2. A server running on port 8888 can be accessed at the guest through http://10.0.2.2:8888/, even when the host does not have an internet connection.

For this reason, setting http://10.0.2.2:8888/ as the home page is recommended.

Possible use cases:

  • Test a single page.
  • Automatically show a certain page, depending on the exposed User agent (e.g. by redirection).
  • Fiddling: Create a fiddle in Chrome, and test it in Firefox.
  • Share text and links.

All of the previous can easily be implemented on a simple server (I've used Node.js, for example).

11
votes

Just to pitch in, Microsoft recently started providing a free service called modern.IE:

modern.IE is an undertaking my Microsoft to make cross-browser testing for the Internet Explorer browsers easier. Microsoft has created modern.IE to provide developers and designers with a suite of tools to facilitate IE browser testing.

With modern.IE you have two methods of testing your website in IE. First, modern.IE offers you three months free usage of the web-based browser testing service BrowserStack. You just need a Facebook account to login and start testing.

The second method modern.IE offers is a virtualization image of each browser from IE 6 to IE 10, which can be run on virtualization software like VirtualBox, Virtual PC, Hyper-V or VMWare Player on Windows, Mac or Linux.

Additionally, modern.IE also provides a tool which scans your web page for common coding problems and lists them out for you to correct so that they display correctly in all IE versions.

Source: modern.IE - Cross-Browser IE Testing Tools Suite

Note: The linked article has been removed. The links now go to a Wayback Machine archive of the page.