Testing can be performed in a manual as well as automated manner. Irrespective of the type of testing approach being used, it is necessary to know the point at which the tests have to be halted (or stopped).
When doing Selenium automation testing, we might come across a number of scenarios where a decision needs to be made on the subsequent execution of the tests. This is especially important in cases where the result of the previous test execution has resulted in a failure. If the issue encountered is a minor one, we can still proceed with the execution; otherwise, it is recommended to halt the test execution.
This is where Asserts in Selenium WebDriver comes to the rescue. The test execution halts when the condition (part of the assert) is not met. Thus, when performing Selenium automation testing with TestNG, assertions can be used for taking relevant steps when an issue is encountered.
In this Selenium TestNG tutorial, we look at what are assertions in TestNG, different types of TestNG assertions, and how they can be used in scenarios pertaining to Selenium automation testing.
What are Assertions in TestNG?
Every test automation framework (e.g., TestNG, JUnit, PyTest, etc.) provides a mechanism to raise asserts. Hence, the generic concept of Asserts and Verify in Selenium is synonymous across different automation frameworks.
As far as the TestNG framework is concerned, asserts in TestNG are used for validating the scenario under test. The outcome of the test can be decided based on the assertion status. TestNG assertion is raised if the ‘achieved test result’ is not the same as the ‘expected test result.’
Assert Class in TestNG provides different methods that can be used to raise asserts. For using TestNG assertions, all you need to do is import the org.testng.Assert package.
Let’s take a simple test scenario where the page title of the LambdaTest homepage has to be validated before further tests can be executed.
Here expected result would be ‘Most Powerful Cross Browser Testing Tool Online | LambdaTest.’ The following cases can arise during the process of test execution:
Case 1
- Fetched Title: Most Powerful Cross Browser Testing Tool Online | LambdaTest
- Expected Title: Most Powerful Cross Browser Testing Tool Online | LambdaTest
- Test case result: Passed, as fetched title matches the expected title
Case 2
- Fetched Title: Most Powerful Cross Browser Testing Tool Online | LambdaTest
- Expected Title: Most Powerful Cross Browser Testing Tool Online
- Test case result: Failed, as fetched title does not match with the expected title
The above example that demonstrated TestNG assertions gives an overall idea about the what & how of assertions. Let’s look at the syntax and other internals of assertions in TestNG.
Syntax of TestNG assertions
The Assert package in TestNG provides methods(or options) to raise assertions. Shown below is the generic syntax of TestNG assertions:
Assert.methodName(actual, expected);
- Assert is the Class provided by the TestNG framework
- methodName is the name of the method that can be used for implementing TestNG assertions
- actual is the first parameter that describes the value that the user gets during the execution of the test script
- expected is the second parameter that describes what the user should get to validate the functionality of the test case
What are Assertion errors?
An Assertion Error, which is a subclass of the Error class, is thrown whenever an issue is encountered during the process of Selenium automation testing.
Let’s take a look at a sample implementation on TestNG assertions where the Selenium test raises an Assertion Error due to an error in the code.
The example shown below is run on the LambdaTest cloud Selenium Grid. A cloud-based Selenium Grid helps improve browser and test coverage by helping run tests on a range of browser and OS combinations. In addition, the overall process of Selenium Java automation testing using TestNG can be expedited by leveraging parallel testing in TestNG with Selenium.
package com.lambdatest;
import java.net.URL;
import java.util.ArrayList;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.AfterTest;
import org.testng.annotations.Test;
public class AssertionTestOnLambdatest
{
private static RemoteWebDriver driver;
private static String status;
@Test
public void hardAssertion() throws Exception
{
String username = "Enter your user name";
String accesskey = "Enter your access key";
String gridURL = "@hub.lambdatest.com/wd/hub";
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("browserName", "chrome");
capabilities.setCapability("version", "latest");
capabilities.setCapability("platform", "win10");
capabilities.setCapability("build", "AssertionError Test");
capabilities.setCapability("name", "AssertionError Test");
capabilities.setCapability("visual", true);
driver=new RemoteWebDriver(new URL("https://"+username+":"+accesskey+gridURL), capabilities);
driver.get("https://www.lambdatest.com/");
String expectedtitle="Most Powerful Cross Browser Testing Tool Online | LambdaTe";
String actualtitle = driver.getTitle();
ArrayList<String> exceptionCapture = new ArrayList<>();
try
{
Assert.assertEquals(actualtitle, expectedtitle);
status="passed";
}
catch(AssertionError e)
{
status = "failed";
exceptionCapture.add(e.getMessage());
((JavascriptExecutor) driver).executeScript("lambda-exceptions", exceptionCapture);
}
}
@AfterTest
public void tearDown() throws Exception
{
if (driver != null) {
((JavascriptExecutor) driver).executeScript("lambda-status=" + status);
driver.quit();
}
}
}
Since the tests are running on a cloud Selenium Grid, an instance of Remote WebDriver in Selenium is created before implementing the actual test logic. Then, relevant Selenium methods are used to access and perform operations on the WebElements in the DOM.
The respective methods are placed under relevant TestNG annotations, thereby improving the readability of the tests. However, as seen in the implementation, the fetched title does not match with the expected title, which eventually results in an assertion error.
The rest of the implementation is pretty straightforward, hence we are deep diving into those aspects of the code 🙂
Invoke the following command on the terminal to kick-start the test execution:
mvn test –P single
The test resulted in a failure as a TestNG assertion was raised since the read title did not match with the expected title.
Navigate to the Exception tab in LambdaTest Automation Dashboard to check the details of the TestNG Assertion error.
In the above images, we can clearly see that the expected title and actual title did not match; hence AssertionError is displayed. The noticeable point here is we do not see any clear failure message in all the above outputs. Let’s resolve this problem in the next section.
Here’s a short glimpse of the TestNG certification from LambdaTest:
Note:- TimeBombSkipException - This Class is an extension of SkipException which change the SkipException to Failed status based on time when test trigger timing.
Different Methods in TestNG assertions
TestNG provides different methods to validate the test result during script execution. In this section, we will see the most commonly used assertion methods available for TestNG assertions
| Methods | Description |
|---|---|
| assertEquals(String actual, String expected, String message); | Asserts that two strings are equal. If not, then AssertionError is thrown. Parameters- 'actual: actual value' 'expected: expected value' 'message: message that we want to display in case of failure' |
| assertEquals(boolean actual, boolean expected, String message); | Asserts that two boolean values are equal. If not, then AssertionError is thrown. Parameters- 'actual: actual value' 'expected: expected value' ' message: message that we want to display in case of failure' |
| assertEquals(Collection <?> actual, Collection <?> expected, String message); | Asserts that two Collection type objects hold the same elements and in the same order. If not, an AssertionError is thrown. Parameters- 'actual: actual value' 'expected: expected value' 'message: message that we want to display in case of a failure' |
| assertEqualsNoOrder(Object[] actual, Object[] expected, String message); | Asserts that two arrays contain the same elements in no particular order. If not, an AssertionError is thrown. Parameters- 'actual: actual value' 'expected: expected value' 'message: message that we want to display in case there is a failure' |
| assertTrue(boolean condition, String message); | Asserts that condition is true. If not, then AssertionError is thrown. Parameters: 'condition: evaluate condition' 'message: Assertion error message in case of a failure.' |
| assertFalse(boolean condition, String message); | Asserts that condition is false. If not, then AssertionError is thrown. Parameters- 'condition: evaluate the condition' 'message: Assertion error message in case of a failure' |
| assertNotNull(Object object, String message); | Asserts that an object is not null. If not, then AssertionError is thrown. Parameters- 'object: assertion object' 'message: message that we want to display in case of a failure' |
| assertNull(Object object, String message); | Asserts that an object is null. If not, then AssertionError is thrown. Parameters- 'object: assertion object' 'message: message that we want to display in case of a failure' |
| assertNotSame(Object actual, Object expected, String message); | Asserts that two objects do not refer to the same objects. If not, an AssertionError is thrown. Parameters- 'actual: actual value' 'expected: expected value' 'message: message that we want to display in case of a failure' |
| assertSame(Object actual, Object expected, String message); | Asserts that two objects refer to the same objects. If not, an AssertionError is thrown. Parameters- 'actual: actual value' 'expected: expected value' 'message: message that we want to display in case of a failure' |
The “message” parameter is optional in all the above methods.
Assertion Messages in TestNG Tests
TestNG provides the flexibility to add a custom message as a parameter inside the test method. The message helps in better understanding and tracking the reason behind the test failure.
Assert.methodName(actual, expected, message);
We have already discussed the major parameters in the earlier section. The newest parameter, ‘message’ is used for defining the custom message that is displayed in case the test results in a failure.
We now modify the already demonstrated example where a custom ‘message’ is displayed when the TestNG assertion is raised.
package com.lambdatest;
import java.net.URL;
import java.util.ArrayList;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.AfterTest;
import org.testng.annotations.Test;
public class AssertionTestOnLambdatest
{
private static RemoteWebDriver driver;
private static String status;
@Test
public void hardAssertion() throws Exception
{
String username = "Enter your user name";
String accesskey = "Enter your access key";
String gridURL = "@hub.lambdatest.com/wd/hub";
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("browserName", "chrome");
capabilities.setCapability("version", "latest");
capabilities.setCapability("platform", "win10");
capabilities.setCapability("build", "AssertionError Test");
capabilities.setCapability("name", "AssertionError Test");
capabilities.setCapability("visual", true);
driver=new RemoteWebDriver(new URL("https://"+username+":"+accesskey+gridURL), capabilities);
driver.get("https://www.lambdatest.com/");
String expectedtitle="Most Powerful Cross Browser Testing Tool Online | LambdaTe;
String actualtitle = driver.getTitle();
ArrayList<String> exceptionCapture = new ArrayList<>();
try
{
Assert.assertEquals(actualtitle, expectedtitle);
status="passed";
}
catch(AssertionError e)
{
status = "failed";
exceptionCapture.add("Title not matched"+" "+e.getMessage());
((JavascriptExecutor) driver).executeScript("lambda-exceptions", exceptionCapture);
}
}
@AfterTest
public void tearDown() throws Exception
{
if (driver != null) {
((JavascriptExecutor) driver).executeScript("lambda-status=" + status);
driver.quit();
}
}
}
As seen in the execution snapshot, the custom message is printed when the corresponding TestNG assert is raised.
As seen in the ‘Exception’ Tab of the Automation dashboard, the custom message ‘Title not matched’ is printed instead of the rudimentary message that we witnessed in the earlier screenshot.
Different types of TestNG Assertions
Soft Assertions and Hard Assertions are the two main types of Assertions in TestNG. The different types of asserts are defined based on the necessity that the user wants to halt the execution or continue the execution in case there is a failure.
Hard Assertions in TestNG
Hard TestNG Assertions should be used when failure in test execution should halt the overall execution of the test suite. By default, TestNG assertions are ‘hard’ in nature. Hence, you need to import the org.testng.asserts.SoftAssert package if you want to raise Soft TestNG Assertions in your test.
Let’s understand the above concept with the help of an example where the login has to be done before performing any actions on the site. Therefore, there is no point in validating the Home page until a successful login is done. In this case, we can use hard assertions on the Login page so that we would be able to stop our test execution immediately if the login is unsuccessful.
package com.lambdatest;
import java.net.URL;
import java.util.ArrayList;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.AfterTest;
import org.testng.annotations.Test;
public class HardAssertTest
{
private static RemoteWebDriver driver;
private static String status;
@Test
public void hardAssertion() throws Exception
{
String username = "user_name";
//Enter your access key
String accesskey = "access_key";
//Enter Grid URL
String gridURL = "@hub.lambdatest.com/wd/hub";
//Set Desired capabilities to run on Lambdatest platform
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("browserName", "chrome"); //Set Browser Name
capabilities.setCapability("version", "latest"); //Set browser version
capabilities.setCapability("platform", "win10"); // If this cap isn't specified, it will just get the any available one
capabilities.setCapability("build", "Hard Assert Test");
capabilities.setCapability("name", "Hard Assert Test");
capabilities.setCapability("visual", true);
driver = new RemoteWebDriver(new URL("https://"+username+":"+accesskey+gridURL), capabilities);
driver.get("https://www.lambdatest.com/");
// Define expected title
String expectedtitle = "Most Powerful Cross Browser Testing Tool Online | LambdaTe";
// Extract Actual title
String actualtitle = driver.getTitle();
ArrayList<String> exceptionCapture = new ArrayList<>();
try
{
//Assertion applied to validate the result
Assert.assertEquals(actualtitle, expectedtitle);
System.out.println("This statement will not be executed because the previous statement is failed");
status="passed";
}
catch(AssertionError e)
{
//Logic to capture Assertion error message with user customized message in exception tab on lambdatest
status = "failed";
exceptionCapture.add("Title not matched"+" "+e.getMessage());
((JavascriptExecutor) driver).executeScript("lambda-exceptions", exceptionCapture);
}
}
@AfterTest
public void tearDown() throws Exception
{
if (driver != null)
{
((JavascriptExecutor) driver).executeScript("lambda-status=" + status);
driver.quit();
}
}
}
As seen in the screenshot from the automation dashboard, it is evident that the test has failed, and the custom message is printed as a part of the exception message.
Soft Assertions in TestNG
Soft TestNG Assertions are preferred in scenarios where a failure in the test should not result in halting the execution of the other tests in the test suite. Use soft assertions when the failure is not a major one, thereby having minimal (to no) impact on the subsequent tests.
Soft Assert not only asserts a given assertion statement but also provides the flexibility to the user for continuing the execution of the test script even after assertion failure. For implementation, we use the SoftAssert class available in TestNG, and this SoftAssert class extends the Assertion class, which has almost all the assertion methods.
In the case of Soft Assert, TestNG always executes only those test steps that are in between assert statement and assertAll statement. The steps present after the assertAll method will not be executed.
package com.lambdatest;
import java.net.URL;
import java.util.ArrayList;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.AfterTest;
import org.testng.annotations.Test;
import org.testng.asserts.SoftAssert;
public class SoftAssertTest
{
private static RemoteWebDriver driver;
private static String status;
@Test
public void softAssertion() throws Exception
{
String username = "user_name";
//Enter your access key
String accesskey = "access_key";
//Enter Grid URL
String gridURL = "@hub.lambdatest.com/wd/hub";
//Set Desired capabilities to run on Lambdatest platform
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("browserName", "chrome"); //Set Browser Name
capabilities.setCapability("version", "latest"); //Set browser version
capabilities.setCapability("platform", "win10"); // If this cap isn't specified, it will just get the any available one
capabilities.setCapability("build", "Hard Assert Test");
capabilities.setCapability("name", "Hard Assert Test");
capabilities.setCapability("visual", true);
driver = new RemoteWebDriver(new URL("https://"+username+":"+accesskey+gridURL), capabilities);
driver.get("https://www.lambdatest.com/");
// Creating the object of SoftAssert Class
SoftAssert softassert = new SoftAssert();
// Define expected title
String expectedtitle = "Most Powerful Cross Browser Testing Tool Online | LambdaTe";
// Extract Actual title
String actualtitle = driver.getTitle();
// Soft Assertion applied to validate the result
softassert.assertEquals(actualtitle, expectedtitle,"Title is not Matched");
System.out.println("This statement will be executed even the previous statement is failed");
}
@AfterTest
public void tearDown() throws Exception
{
if (driver != null)
{
driver.quit();
}
}
}
The execution output shows that a soft assert was raised, and the statement following the assert was also printed on the output console.
Note- CouldNotReadCoreException -It throws when Max value can’t be read by the MaxCore Serialization
Custom Soft Assertions in TestNG
In the previous sections, we demonstrated the implementation of different TestNG asserts, but there might be a need to write our assertions (called Custom Assertions).
We will first create a CustomAssertion class, which extends the SoftAssert class so that we can use all assertion methods in TestNG. For demonstration purposes, we will take the example of the LambdaTest website in which we will validate whether the “Free start testing” button is a button web element or not.
package com.lt.assertions;
import org.openqa.selenium.WebElement;
import org.testng.asserts.SoftAssert;
public class CustomAssertion extends SoftAssert
{
// We created assertButton in which we will validate whether element is button or not
public void assertButton(WebElement element)
{
// Extract the tagname as well as type of element
String tagname = element.getTagName();
String type = element.getAttribute("type");
// Verify whether tag is button and type is submit
if(tagname.equals("button") && type.equals("submit"))
{
System.out.println("Element is Button, So Assertion Passed");
}
else
{
// If assertion is failed then we will throw AssertionError
throw new AssertionError("Element is not Button, So Assertion Failed");
}
}
}
Now, we will create the main class where we test the recently created custom assertion. Here, we will inspect elements that we want to test and create the object of the above-created custom assertion class. Finally, with the help of the created object, we will call the method that we created in the custom assertion.
package com.lt.assertions;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
public class TestAssertion
{
public static void main(String[] args)
{
RemoteWebDriver driver;
String username = "user_name";
//Enter your access key
String accesskey = "access_key";
//Enter Grid URL
String gridURL = "@hub.lambdatest.com/wd/hub";
//Set Desired capabilities to run on Lambdatest platform
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("browserName", "chrome"); //Set Browser Name
capabilities.setCapability("version", "latest"); //Set browser version
capabilities.setCapability("platform", "win10"); // If this cap isn't specified, it will just get the any available one
capabilities.setCapability("build", "Hard Assert Test");
capabilities.setCapability("name", "Hard Assert Test");
capabilities.setCapability("visual", true);
driver = new RemoteWebDriver(new URL("https://"+username+":"+accesskey+gridURL), capabilities);
driver.get("https://www.lambdatest.com/");
// Inspect the “Free Testing Button” element
WebElement freetestingbutton = driver.findElement(By.xpath("(//button[contains(text(),\"Start Free Testing\")])[1]"));
// Creating the object of CustomAssertion class
CustomAssertion customassert = new CustomAssertion();
// Calling the custom created method with the help of object
customassert.assertButton(freetestingbutton);
}
}
If we run the main class, then we can see our assertion is passed, and the success message is displayed on the console.
If we want to fail the above TestNG assertion, add the following change in the existing implementation:
// Inspect the “Free Testing Button” element
WebElement freetestingbutton = driver.findElement(By.tagName("p"));
Conclusion
In this TestNG tutorial with a focus on TestNG assert, we looked at different asserts and constraints helpful in Selenium automation testing. The reason for using assert in test code is to pause the execution as soon as assert is encountered.
Selenium testing is an essential element in ensuring the effectiveness of test automation. With the combination of cloud-based Selenium testing tools like LambdaTest, development teams can verify product functionalities across several browsers & operating system combinations. However, the core business logic remains intact, with minimal changes required to port the TestNG test code to the remote Selenium Grid.
Let us know the TestNG asserts you have been using in the comments section to learn from each other’s experience.
Happy Testing
Top comments (0)