MISCELLANEOUS

This section provides various information related to Continuous Integration API.

Test Run Reports

Test run reports returned by taskComplete and taskUpdated events contain detailed information about the results of test task execution.

By default, reports are presented in the JSON format. You can change this to the JUnit or NUnit format. To specify the report format, pass an appropriate value to the runTests method in the reportFormat parameter.

JSON Reports

A JSON report consist of the following fields.

uid - the unique ID of the executed test.
name - the name of the object that owns the executed test(s) (directory name, fixture name and single test name).
status - information that indicates the state of the testing operation (pending, running, failed and succeeded).
startedAt - the date and time when testing started.
completedAt - the date and time when testing completed.
time - specifies the time, in seconds, of tests execution.
workerNames - identifies the browsers for test execution.
browserVersions - specifies the versions of the browsers used for test execution.
testCount - the total number of executed tests.
failed - the number of failed tests.
passed - the number of passed tests.
testErrReports - information about the failed tests.
passedTests - information about the passed tests.
unstableTests - information about unstable tests.

The code below illustrates the result of the test execution that the report object can contain.

{
    uid: '842a36e2-a12c-48d8-a946-f878896ad53c',
    name: '"Root" directory tests',
    status: 'succeeded',
    startedAt: ['07/29/2013', '12:36:12', 'PM'],
    completedAt: ['07/29/2013', '12:37:15', 'PM'],
    workerNames: ['Google Chrome', 'Internet Explorer', 'Mozilla Firefox'],
    browserVersions: { "Google Chrome": "41.0.2272", "Internet Explorer": "11.0.9600.17690", "Mozilla Firefox": "36.0.4"},
    testCount: 10,
    failed: 0,
    passed: 10,
    testErrReports: {},
    passedTests: {},
    unstableTests: {}
}

As you can see, the example above suggests that there were no test execution errors. In this instance, the testErrReports field is empty. The code below lists the information you can obtain from the testErrReports field when tests fail.

"testErrReports": {
    'f60f7573-fa42-4f2e-b760-33883eff9a05': {
        name: 'Click a label that does not exist',
        fixtureName: 'My Page',
        fixturePath: '',
        time: 10,
        errs: [{
            msg: 'Error on step "Click label": "click" action target does not contain DOM-elements.',
            workerName: 'Internet Explorer',
            browserVersion: 'Internet Explorer': '11.0.9600.17690'
        }]
    }
}

passedTests field provides the details about tests that passed successfully.

"passedTests": {
    'd72022c7-a694-4db5-8fef-96a2cc9be988': {
        fixtureName: 'Data Binding',
        fixturePath: '',
        name: 'Filter',
        time: 7
    }
}

The following example illustrates the information contained in the unstableTests field if there are tests that have been marked as unstable in the quarantine.

"unstableTests": {
    'fd93159b-db41-48e3-9df3-11504fa765de': {
        name: 'Click a label',
        fixtureName: 'My Fixture',
        fixturePath: '',
        workerNames: [
            'Mozilla Firefox'
        ]
        browserVersions:{
            "Mozilla Firefox": "36.0.4"
        }
    }
}

JUnit Reports

When the report format is set to JUnit, the report is returned as a string value containing data in the standard JUnit format.

<?xml version="1.0" encoding="UTF-8" ?>
<testsuites>
    <testsuite name="&quot;My Fixture&quot; fixture tests" errors="1" failures="1" tests="3" time="23">
            <testcase name="'/My Fixture' - Should show header" time="2">
                <error>
                        Worker: Google Chrome (version 41.0.2272)
                        Message: Assertion failed at step &quot;1.Assert&quot;: eq($(&quot;.article-header&quot;).is(&quot;:visible&quot;), false);
                                     Expected: false
                                     Actual:   true


                </error>
            </testcase>
            <testcase name="'/My Fixture' - Should prevent unauthorized access"  time="15" />
            <testcase name="'/My Fixture' - Should validate entered data"  time="6" />
    </testsuite>
</testsuites>

NUnit Reports

When report format is set to NUnit, the report is returned as a string value containing data in the standard NUnit format.

<?xml version="1.0" encoding="UTF-8" ?>
<test-results name="&quot;My Fixture&quot; fixture tests" total="3" error="1" failures="1" not-run="0" time="23">
    <test-suite name="&quot;My Fixture&quot; fixture tests" success="False" executed="True" time="23" >
        <results>
            <test-case name="'/My Fixture' - Should show header" success="False" executed="True" time="2">
                <failure>
                    <message>
                            Worker: Google Chrome (version 41.0.2272)
                            Message: Assertion failed at step &quot;1.Assert&quot;: eq($(&quot;.article-header&quot;).is(&quot;:visible&quot;), false);
                                     Expected: false
                                     Actual:   true


                    </message>
                </failure>
            </test-case>
            <test-case name="'/My Fixture' - Should prevent unauthorized access" success="True" executed="True" time="15" />
            <test-case name="'/My Fixture' - Should validate entered data" success="True" executed="True" time="6" />
        </results>
    </test-suite>
</test-results>