Image Recognition

Introduction

BELLATRIX Image Recognition library can be used to verify hard-to-assert functionalities such as PDFs, charts, and similar. But our computer vision library is much more than that. You can even interact with everything on your screen. Perform clicks, set text, drag and drop, double click, and more. Our library uses a machine learning engine for comparing images of your screen. Basically, you add screenshots to your project and embed them to it. Afterward, you need to mention the name of the picture.

To use the library you need to install the Bellatrix.ImageRecognition.SikuliX NuGet package.

Example

[TestFixture]
public class ImageRecognitionTests : WebTest
{
    private static GridTestPage _gridTestPage;
    private static Grid TestGrid => _gridTestPage.SampleGrid;

    public override void TestInit()
    {
        _gridTestPage = App.Create<GridTestPage>();
        App.Navigation.NavigateToLocalPage("TestPages\\Grid\\Grid.html");
    }

    [Test]
    public void AssertPrintPreview()
    {
        var firstOrderCell = TestGrid.GetRow(0).GetCell(0).As<TextField>();
        firstOrderCell.SetText("BELLATRIX IS beautiful");

        Screen.Click("chrome-dots-button");
        Screen.Click("chrome-print-button");

        Screen.ValidateIsVisible("chrome-print-preview-grid", similarity: 0.7, timeoutInSeconds: 30);

        Screen.Click("print-preview-cancel-button");

        Screen.ValidateIsNotVisible("chrome-print-preview-grid");
    }
}

Explanations

var firstOrderCell = TestGrid.GetRow(0).GetCell(0).As<TextField>();
firstOrderCell.SetText("BELLATRIX IS beautiful");

Here we combine the BELLATRIX web library with the image recognition library. First, we navigate and perform actions on our web page. After that, we open the print-preview window of the Chrome browser by clicking the “dots” button and afterward the “Print…” menu item. Since Chrome is a native app, you cannot use WebDriver to interact with its menus. This is where we use our Image Recognition library.

Screen.Click("chrome-dots-button");
Screen.Click("chrome-print-button");

To perform visual actions, use the class Screen and its methods.

Screen.ValidateIsVisible("chrome-print-preview-grid", similarity: 0.7, timeoutInSeconds: 30);

Check whether the provided image is part of the current screen with similarity, and we set the timeout.

Screen.ValidateIsNotVisible("chrome-print-preview-grid");

We can check that a particular image is not visible anymore.

Configuration

"imageRecognitionSettings": {
    "timeoutInSeconds": "5",
    "defaultSimilarity": "0.7"
},

You can find a dedicated section about Image Recognition in testFrameworkSettings.json file under the imageRecognitionSettings section. There you can set the default timeout and similarity.