Prospect or Usage of ChatGPT in Test Automation

As ChatGPT shattered Internet with its launch, So I tried to explore it in terms of Test Automation

What is ChatGPT?

ChatGPT is a natural language processing (NLP) model developed by OpenAI that can generate human-like responses to a given prompt. In recent years, the use of ChatGPT has gained popularity in various industries, including test automation. In this post, we'll explore the prospect of ChatGPT in test automation and how it can be used to improve the testing process.

How to use ChatGPT in Test Automation?

Let's start with first step of software testing i.e Requirement Understanding and Test Case Creation. So I have send basic requirement of ecommerce search and asked it to write test cases for it.
Query to ChatGpt:

Feature: User is able to search on different products in ecommerce site
Inputs: Price Range, Item Category, Size, Color, Brand

Can you write all test cases for it?

And response by ChatGPT is truly amazing.. You wanna know?

It didn't stop here, you can also get Cucumber Feature file.

Feature: Search for products on ecommerce site
  As a user, I want to be able to search for products on the ecommerce site by specifying different search criteria such as price range, item category, size, color, and brand.

Scenario: Search for products by price range
  Given I am on the ecommerce site's home page
  When I enter a price range in the search bar and press enter
  Then I should see a list of products within the specified price range

Scenario: Search for products by item category
  Given I am on the ecommerce site's home page
  When I select an item category from the dropdown menu and press search
  Then I should see a list of products in the specified category

Scenario: Search for products by size
  Given I am on the ecommerce site's home page
  When I select a size from the size dropdown menu and press search
  Then I should see a list of products in the specified size

Scenario: Search for products by color
  Given I am on the ecommerce site's home page
  When I select a color from the color dropdown menu and press search
  Then I should see a list of products in the specified color

Scenario: Search for products by brand
  Given I am on the ecommerce site's home page
  When I enter a brand name in the search bar and press enter
  Then I should see a list of products from the specified brand

Reaction:

Then my expectation increase more, and I asked it to automate APIs. So below is my query to it.

Can you automate below curl request using rest assured and java?

curl --location --request POST 'https://localhost:8080/login' \
--header 'Content-Type: application/json' \
--data-raw '{
    "username":"akansh@yopmail.com",
    "password":"12222"
}'

And response can shock you:

Till now it is meeting my expectations. So as per human basic nature, you can't be satisfied with what you have, you always need more. So I pass query to find xpath:

Can you write xpath of 2nd <li> tag in below html?

"<div>
<li id='akansh'>Abc</li>
<div>
<li>Abc</li>
</div>
</div>"

What do you think it meets my expectation?

No, this tool doesn't meet my expectation here, since it doesn't provide me relative xpath. Moreover, here I provide a dummy HTML block, in real life situation we have 1000s Line of code and it has word limit.

It is definitely reduce our effort but it can not completely take over us right now. Following are reason why I think so:

1. Maintainability & Reusability of code:
   As we all work on some sort of framework (TDD, BDD etc.) but it provide us solution that uses methods which make us framework useless. Moreover there will be duplicity of code which in return increase maintenance cost.

2. Exploration of locator:
   As we seen above, it is unable to find relative xpaths/locators, which are basic building block of UI Automation.

3. Database Validation:
   As each organization wants to keep their Database schema private, so it will be conflict of interest if someone tries to write DB query from it or want to validate data from it.

4. Integration Test Cases:
   You can't rely on ChatGPT to write integration test cases for you. It can't map the impact area between modules and features.

5. Can assume things/Biased:
   As tester can never assume requirement or flows, but ChatGPT can assume things since it uses AI and which can lead to miss of potential bug. So we can't totally rely on it. It can add biases in test case formation or data generation on the basis of its AI model.

Conclusion:

ChatGPT is optimzed version of Google Search or Stack Overflow where you can put your query and you will get some instant solution instead of moving to different web links. And it also uses some AI to predict particular solution as per your requirement while current solutions are based on static webpages. ChatGPT can be used in test automation to generate test cases, test data, and test scripts. This can save time and effort in writing and maintaining tests.