Don't launch simulator when running unittests

52
votes

Some background:

I have iOS application with a target configured to run unitTests. And I am running build automation tool jenkins on my MacBook which automatically builds this application and run all tests (using command line xcodebuild tool).

Everything worked fine with Xcode 4. This build automation tool was running under different user and was running all these tests.

I switched to Xcode 5 recently and it started to fail, because it can't launch Simulator.

The problem

I have a scheme UnitTests which is configured to run tests (logic tests). A I run these test using one of two methods:

  • Command U in Xcode

  • Or command line "/usr/bin/xcodebuild -scheme UnitTests -sdk iphonesimulator -configuration Release clean build TEST_AFTER_BUILD=YES "

In both cases, it tries to start simulator. However, per my understand it doesn't need it. Anyway it runs on top x86 and it doesn't look like any apps are installed on Simulator.

Is there a way to get rid of this pesky simulator start (because it breaks my build automation)?

Update 1

Seems to find very similar question, but can't get it working: Run logic tests in Xcode 4 without launching the simulator

Update 2

I found VERY relevant and interesting question/answer: Apple CI / Xcode Service and Jenkins

6
iosxcodejenkins
This question might help too. The steps should be pretty much the same for an iOS app. stackoverflow.com/questions/19219706/… – Craig Siemens
@CleverError: Thanks. I will look at it. – Victor Ronin
Why don't you want to run the simulator? – quellish
@quellish: Because starting from Xcode 5, it's a problem if you are running it under different user (which doesn't UI session) – Victor Ronin
@VictorRonin , that's true on earlier versions of Xcode as well. Xcode 5, however, does support running in a headless mode for just these kinds of scenarios. Xcode Server works in this way. On earlier versions of Xcode it was possible to run the simulator as a daemon user but difficult to set up correctly. To use iPhone specific functionality like UIKit in your tests, you DO need the simulator. – quellish

6 Answers

38
votes

Using xCode 7 and xCtool.

xctool is capable of executing unit tests without the simulator.

To get this working,

1 . Update target settings to run without a host app.

Select your project --> then test target --> Set the host application to none.

enter image description here

2. Install xctool , if you don't have it.

brew install xctool

3. Run the tests using terminal with xctool. 

xctool -workspace yourWorkspace.xcworkspace -scheme yourScheme run-tests -sdk iphonesimulator
22
votes

You can create a Mac OSX Unit Test instead of an iOS unit test. This requires that you not include any iOS specific libraries in the unit tests though. You can do this via the following:

  1. Select the project -> the target drop down -> "Add Target..."
  2. Select "Mac OSX" -> "Other" -> "Cocoa Unit Testing Bundle"
  3. Create the testing bundle as you would a normal project

You can now add sources to the unit test and run it like an iOS test without launching the simulator.

14
votes

I've asked the same question to apple engineers. Unfortunately it doesn't seem you can accomplish this and stay with iOS at the same time. There are some tricks you can do to check if testing. You could put this code snippet in your AppDelegate.h or some other global class to say not load a root viewcontroller and prevent any wierdo ui stuff from corrupting your unit tests:

static BOOL isTesting() {
    BOOL isTesting = !isEmpty([[[NSProcessInfo processInfo] environment] objectForKey:@"XCInjectBundle"]);
    return isTesting;
}

I've also had an apple engineer verify this is a legitimate check. And to give credit where credit is due, this is from: Programmatically determine current target (run or test) in iOS project

EDIT: I've also had success with this and it's a little more straight forward:

static BOOL isTesting() {
    return [[[NSProcessInfo processInfo] processName] isEqualToString:@"xctest"];
}
4
votes

A osx test target can become a huge hassle because you have to manage yourself which source file to include. Putting @testable import YourAppName on top of your XCTest files is way more convenient. So just prevent your app from launching in case of a XCTest run.

In your AppDelegate put: (Swift 3 solution) 

func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplicationLaunchOptionsKey: Any]?) -> Bool
{
    if ProcessInfo.processInfo.environment["XCInjectBundleInto"] != nil {
        return false
    } 

...

This wont prevent the simulator from launching, but will save you a lot of time.

2
votes

You can run unit and ui tests in headless mode with Xcode 9 and a command from your terminal. For reference sample commands:

Building and testing a workspace

xcrun xcodebuild -workspace "YOUR_WORKSPACE_NAME.xcworkspace" -scheme "YOUR_SCHEME" -sdk "iphonesimulator12.0" -destination "OS=12.0,name=iPhone X" -configuration Debug -enableCodeCoverage YES clean build test

For project

xcrun xcodebuild -project "YOUR_PROJECT_NAME.xcodeproj" -scheme "YOUR_SCHEME" -sdk "iphonesimulator12.0" -destination "OS=12.0,name=iPhone X" -configuration Debug -enableCodeCoverage YES clean build test
0
votes

Workaround:

App will still launch but you can #if to define what you don't want to run.

Approach:

  1. Create a custom build configuration called Test by duplicating Debug (Project > Info > create new configuration)
  2. In Build Settings > Active Compilation Conditions for Test add TESTING
  3. Edit Scheme > Info > Build Configuration, set build configuration as Test
  4. Use #if !TESTING #endif around the code you don't want to execute when testing.

Frameworks:

If you have embedded frameworks, create the same build configuration in the framework, so that the framework binary is properly linked.