CodedUi Testing

0
votes

Sorry for such a loooong post!!

I am brand new to coding/automated testing. I have VS 2013 Ultimate and created a CodedUI test with an assertion and it runs fine. However, when the assertion fails, the test stops. I know this question has been asked many times and I have tried the solutions shown but am having no luck. I want the test to run even after the failure occurs.

I am testing a .net application, in the app we have several search fields, I want to verify that if I search for the word 'test' that the search result will have the word 'test' in it. I did an assertion on a cell that did not have the word 'test' in it and as expected, the assert failed.

I want the test to continue running so that I can search all the other fields but it stops as soon as the assert fails.

When I did the assert on the cell I used 'Contains' and not 'AreEqual' seemed to make sense since I am looking for 'test' in any part of the address (even if it said something like '1 wetester dr')

here is the part of the code from the codedUITest1.cs script:

//From the CodedUITest1.cs file 

[TestMethod]
        public void TestAddressSearchFields()
        {            
            this.UIMap.SearchAddressFor_test();
            this.UIMap.Assert_on_FirstAddressCellForTheWord_test();
            this.UIMap.SearchNextThing();
            this.UIMap.SearchAnotherThing();                      
        }
---------------------------------------------------------------------------------------

// From the UIMap.Designer.cs file (this is the definition for the assert)

 public void Assert_on_FirstAddressCellForTheWord_test()
        {
            #region Variable Declarations
            WinCell uIItem800SROLLINGRDCell = this.UIDPSClaimantSearchWindow.UIGrdClaimantSearchWindow.UIDataGridViewTable.UIRow6Row.UIItem800SROLLINGRDCell;
            #endregion

            // Verify that the 'Value' property of '800 S ROLLING RD' cell contains 'test'
            StringAssert.Contains(uIItem800SROLLINGRDCell.Value, this.Assert_on_FirstAddressCellForTheWord_testExpectedValues.UIItem800SROLLINGRDCellValue, "Address does not contain the word \"test\"");


        }


//And then the definition for the cell value

[GeneratedCode("Coded UITest Builder", "12.0.30501.0")]
    public class Assert_on_FirstAddressCellForTheWord_testExpectedValues
    {

        #region Fields
        /// <summary>
        /// Verify that the 'Value' property of '800 S ROLLING RD' cell contains 'test'
        /// </summary>
        public string UIItem800SROLLINGRDCellValue = "test";
        #endregion
    }

I tried the following but it didn't work, once the assertion failed, the test stopped (is it because the failure is not an exception?)

     this.UIMap.SearchAddressFor_test();
                try
                {
                    this.UIMap.Assert_on_FirstAddressCellForTheWord_test();
                }

                catch (Exception theword_test_not_found)
                {
                    throw (theword_test_not_found);
                }
                {
                    Playback.PlaybackSettings.ContinueOnError = true;
                }

Sorry for such a long post, remember I am brand new to this, please be gentle!

3
c#coded-ui-tests
The Assert class throws an AssertFailedException if the assertion fails. This exception is handled by the unit test engine to indicate an assert failure. What you should be doing is creating stand alone tests for each condition you wish to test. - Kevin
Thanks so much Kevin, again, new guy here, so do you mean add a new codedui test for each thing I want to do? For example, I have a search screen where there are 5 or 6 search fields (search name, search address, search some ID etc.) so I create a new project and record 'search name', then stop recording then add a new test (same proj) and record 'search ID' stop and then repeat that for all search fields? That way if I have asserts on each search result, if one fails, then the test will continue because it goes to the next test I created? - Rob W
Yes that is exactly it. Your tests should be individual per thing to be tested and independent of each other (so you can run one test at a time if needed). - Kevin

3 Answers

0
votes

If you really want to test multiple conditions in one test you could add this line to the top of the test.

Playback.PlaybackSettings.ContinueOnError = True;

0
votes

It can be quite tedious to break apart the tests like that and potentially quite time consuming to run.

I would recommend a few things. Firstly, consider writing your CodedUI tests by hand and not using the record and play. That will let you have more configurable, reusable blocks.

Code First is a great start.

I have some basic examples as well that show a few techniques for writing reusable tests.

Now, regarding writing a test for each of those things you describe. We find that it is quite alright to test several conditions in a single test if they are recoverable tests. I would still recommend breaking the tests up as much as possible.

In your case, I would say it is fine to aggregate the exceptions if the time to get to the page you are testing is high, or the time to satisfy the preconditions is high, or if the likely hood of coupled errors is high. It would be annoying to trackdown and fix an error with the name field just to find that the state field also has an error that can be caught by the same test.

[TestInitialize]
public void GivenTimeConsumingPreConditions()
{
   // do time consuming steps
}

[TestMethod]
public void WhenManipulatingTabs_ThenSearchFiltersResults()
{
    // depending on how you want to iterate, this is a super simple example
    string[] tabs = new ["Products", "Customers"];

    List<Exception> exs = new List<Exception>(tabs.Length);
    foreach(string tabName in tabs){
        OpenTab(tabName);
        EnterTextIntoSearchBox("test");
        ClickSearch();

        try {
            Assert.IsTrue(AllResultsContain("test"));
        } catch(Exception e) {
            exs.Add(e);
        }
    }

    if(exs.Any()){
        throw new AggregateException(exs);
    }
}

Doing it this way ensures that if the test fails for some other reason (eg, tab can't open), the test still fails immediately so you are only aggregating exceptions which are expected to potentially group together. (eg, search doesn't work propertly on all tabs vs single tab).

0
votes

This might help. Mark your test method to expect an exception (when you did not find that "test" word) and catch it, then you can do with it whatever you want. Here is how I handle expected exceptions in unit test -- 

    [TestMethod]
    [ExpectedException(typeof(ArgumentException))]
    public void HoofNotesUtilitiesCreateDirectoryTest()
    {

        try
        {
            // Test whether it throws an exception
            // It'd better, * is not a valid directory name
            // when you are creating one
            HoofNotesUtilities.CreateDirectory("*");
        }
        catch (System.ArgumentException e)
        {
            if (e.Message == "Illegal characters in path.")
            {
                // You got the exception that you expected - GREAT
                // Do whatever you need to do with it here. 
                // THEN rethrow it, because the test expects it.
                // If you don't, the test will fail and stop. 
                throw;
            }
            else
            {
                // Wrong exception, fail the test
                Debug.Assert(false);
            }
        }
    }