Performance Issues - Chutzpah Console.exe and TypeScript

Question

I am running into some performance issues trying to run the chutzpah.console.exe with my chutzpah.json file. For this project, I am using TypeScript and pre-compiling the TypeScript files into .js files before running the tests. Each test file that I have generally references one other TypeScript file that it depends on - this is so Visual Studio is happy and so that the Chutzpah extensions work. For example, my drawerBusiness.spec.ts file contains this reference:

/// <reference path="../../../../app/business/documents/drawerBusiness.ts"/>

My chutzpah.json file is as follows:

{
  "Compile": {
    "Mode": "External",
    "Extensions": [ ".ts" ],
    "ExtensionsWithNoOutput": [ ".d.ts" ]
  },
  "Framework": "jasmine",
  "TestHarnessReferenceMode": "Normal",
  "TestHarnessLocationMode": "SettingsFileAdjacent",
  "References": [
    { "Path": "./lib/jquery/jquery.js" },
    { "Path": "./lib/angular/angular.js" },
    { "Path": "./lib/angular-mocks/angular-mocks.js" },
    { "Path": "./lib/jasmine-jquery/jasmine-jquery.js" },
    { "Path": "./lib/underscore/underscore.js" },
    { "Path": "./lib/angular-bootstrap/ui-bootstrap-tpls.js" },
    { "Path": "./tests/dependencies.js" }
  ],
  "Tests": [
    { "Include": "*/tests/specs/*.ts", "ExpandReferenceComments":  "true" }
  ]
}

As you can see from above, I need to load some external libraries. I presume these get loaded once and then are used for each of the tests?

When I was using pure JavaScript, this configuration was blazing fast. It completed quite quickly in Visual Studio and through running the tests from the chutzpah.console.exe. However, since I have switched over to Chutzpah, the tests are much slower:

Tests complete: 263
=== 263 total, 0 failed, took 74.64 seconds ===

These results are even worse on our build machine, and it's timing out. It seems like there's a huge delay between each spec file that is run, and I think that is causing the overall slowness - once the spec file starts running, it completes in a few one-hundredths of a second. Does anyone have any ideas about what could possibly be configured incorrectly?

Upon further testing, I even tried removing the "Compile" section and using pure JavaScript files, and I'm still experiencing a slowdown. I am fairly certain the test harness creation for each file is causing the issues, and I believe the slowdown occurred after upgrading to a version of Chutzpah that supports non-JS files. But perhaps I am just doing something wrong with the configuration. — Brandon Bell

Matthew Manela Matthew Manela · Accepted Answer · 2016-02-27T18:01:56

Here are a few things you can try to help get to the bottom of this issue:

Can you try enableding "EnableTestFileBatching": true in your chutzpah.json file to see if it helps? There can be slowness if you have MANY test files and this setting tells chutzpah to batching them into one harness.
Can you provide the output of the /trace option so I can see all the timings?
Can you provide a repro that I can run locally?

Performance Issues - Chutzpah Console.exe and TypeScript

1 Answers