IAENG International Journal of Computer Science, 47:3, IJCS_47_3_24
______________________________________________________________________________________
A GA-Based Approach to Automatic Test Data
Generation for ASP.NET Web Applications
Islam T. Elgendy, Moheb R. Girgis, and Adel A. Sewisy
Abstract—One of the major challenges in software testing is to cover a certain testing criterion requires experience and
the generation of test data automatically that satisfy a specified time. Automating the process of test data generation can
adequacy criterion. This paper presents a GA-based approach solve these issues. One possible solution is generating test
and a supporting tool for data-flow test data generation for
ASP.NET web applications. The proposed tool accepts as input data at random, which can speed up the process significantly.
the web application under test, instruments it, and performs However, there is no guarantee that the generated data will
static analysis to compute the definition-use pairs. The proposed cover the testing criterion, or the data will be useful in
GA conducts its search by constructing new test data from error detection. Therefore, a more sophisticated and intel-
previously generated test data that are evaluated as effective ligent technique is required. Search-based software testing
test data. In this GA, the chromosome is a collection of user
interface control objects, where each control is considered as a formulates testing as an optimization problem, which can be
gene. Therefore, novel crossover and mutation operators are solved using computational search techniques from the field
developed to manipulate the chromosome, which are called of Search Based Software Engineering.
block crossover and control-based mutation operators. The This paper presents an automated test data generation ap-
proposed GA accepts as input the instrumented version, the proach based on genetic algorithm (GA) for data flow testing
list of definition-use pairs to be covered, and input controls
related information. The tool produces a set of test cases, the of ASP.NET webApps. The paper is organized as follows:
set of definition-use pairs covered by each test case, and a list of Section 2 presents the related work. Section 3 briefly covers
uncovered definition-use pairs, if any. Also the paper presents a the concepts of GAs and some of the used utilities. Section 4
case study to illustrate how the tool works. Finally, it presents presents the proposed test data generation approach. Section
the results of the empirical evaluation that is performed to 5 presents a case study. Section 6 presents the experimental
evaluate the effectiveness of the generated test data in exposing
web application errors. results. Section 7 presents the conclusion of this work.
Index Terms—Software testing, Data Flow Testing, Automatic II. R ELATED W ORK
test data generation, Automated Testing Tool, Web Applications
Testing. As this paper focuses on automatic test data generation
for webApps using GA, this section reviews some of the
research work in the field of search-based test data generation
I. I NTRODUCTION
and webApp testing. Sharma et al. [6] presented a survey of
S OFTWARE testing has been widely used in the industry
as a quality assurance technique for the various artifacts
in a software project. However, software testing is very labor-
GA approach for addressing the various issues encountered
during software testing, and discussed the applications of
GA in different types of software testing. They concluded
intensive, time-consuming and expensive; almost 50% of the that using GA, the results and the performance of testing
cost [1], [2] and 40% of the time of a software system can be improved. Li et al. [3] presented a broad survey of
development is spent on software testing [3]. Software testing testing advances in webApps and discussed the techniques
has two main aspects: test data generation and application employed, targets, goals, inputs/outputs and stopping criteria.
of a test data adequacy criterion. A test data generation They stated that there are various testing goals, such as
technique is an algorithm that generates test cases, whereas ensuring testing adequacy, finding faults, etc. and the choice
an adequacy criterion is a predicate that determines whether of a testing technique depends on the testing goal. Lakshmi
the testing process is finished [4]. and Mallika [7] presented a comparative study of some of
Web applications (WebApps) are being used extensively in the techniques, prominent tools, and models for WebApp
business, social, organizational, and governmental functions. testing, and highlighted the research directions for some of
The continual availability of WebApps is one of the advan- the WebApp testing techniques.
tages to use them by many users without regard to their Girgis [8] presented an automatic test data generation tech-
location or time limitations [5]. However, this demands high nique based on GA, which is guided by the data flow
reliability of WebApps. Inadequate testing poses huge risks dependencies in the program, to search for test data to fulfil
including downtime and loss of users’ trust and convenience. the all-uses criterion. The input to the GA is an instrumented
The Web’s ubiquity and users’ reliance on it have made it version of the program under test, number of input variables,
crucially important to ensure the quality, correctness, and the domain and precision of each input variable, and list of
security of WebApps. definition-use (def-use) associations. The output of the GA is
The process of constructing or choosing test data manually a set of test cases and the list of def-use associations covered
Manuscript received August 29, 2019; revised February 3, 2020 by each test case, and a list of uncovered def-use associations,
Islam T. Elgendy and Adel A. Sewisy are with the Department of Com- if any.
puter Science, Faculty of Computers and Information, Assiut University, Alshahwan and Harman [5] adapted a set of related search
Egypt. e-mail:
Moheb R. Girgis is with the Department of Computer Science, Faculty based testing algorithms for webApp testing, and imple-
of Science, Minia University, Egypt. mented an automated test data generation approach for PHP
Volume 47, Issue 3: September 2020
, IAENG International Journal of Computer Science, 47:3, IJCS_47_3_24
______________________________________________________________________________________
webApps. Setiadi et al. [9] proposed a method to reduce III. G ENETIC A LGORITHM C ONCEPTS AND U TILITIES
the number of test cases for detecting errors in a concurrent This section presents the basic concepts of genetic al-
program. They used data flows and branch structure to gorithms (GAs), and some utilities for automated testing,
reduce redundant test cases, identifying only the interleavings namely Microsoft Coded User Interface Testing (CUIT) and
that affect the branch outcomes rather than identifying all Genetic Algorithm Framework (GAF).
interleavings affecting shared variables. Setiadi et al. [10]
further improved their work by analyzing data dependency to
A. Genetic algorithms
generate test cases affecting sequences of locks and shared
variables, reducing the required memory space. Also, they The basic concepts of GAs were developed by Holland
generated test cases for detecting race conditions caused [18]. GA is a search-based optimization technique inspired
by accesses through reference variables. Furthermore, by from the natural search and selection processes that leads
exploiting previous test results, they reduced the effort for to the survival of the fittest individuals. GAs generate a
checking race conditions. sequence of populations by using a selection mechanism,
Takamatsu et al. [11] extended the Seeker tool developed and use crossover and mutation as search mechanisms [19].
by Thummalapenta et al. [12], which is based on branch The idea behind GAs is that they construct a population of
coverage, focusing on multiple targets, identifying all the in- individuals represented by chromosomes, which is normally
volved targets in uncovered branches and evaluating method a character string similar to the DNA chromosomes. These
sequences according to a fitness function, while applying a chromosomes represent solutions to a problem. The chro-
search strategy to suppress combinatorial explosion. Bous- mosomes then evolve according to the selection, mutation
sasa [13] introduced the use of Novelty Search (NS) algo- and reproduction rules. The fitness of each individual (chro-
rithm to the test data generation problem based on statement- mosome) in the environment is measured. Individuals with
coverage criteria. The NS adaptation attempted to exploit high fitness values in the population are selected by repro-
the large search space of input values and catch relevant duction, and a new population is derived through crossover
test cases that may cover as much as possible the executed and mutation, in which individuals may have better fitness
statements. in their environment. In crossover two chromosomes swap
Girgis et al. [14] have presented an approach to data flow genetic information as in the process of sexual reproduction.
testing of WebApps. They presented an approach that in- Mutation makes slight modifications in a small part of the
cludes the construction of a WebApp data flow model to population that represents an evolutionary step. The structure
aid WebApps data flow analysis. In this approach, testing is of a simple GA is given in algorithm 1.
conducted in four different levels, Function, Inter-procedural,
Page, and Inter-Page levels. In each level, the def-use pairs Algorithm 1: Basic GA
of the variables are obtained. Then, selecting test data that 1 begin
cover these def-use pairs ensures the fulfilment of the all- 2 initialize population;
uses criterion. 3 evaluate population;
Akhter et al. [15] proposed a GA based-test data generation 4 while termination criterion not reached do
technique to achieve path coverage of the program under test. 5 select solutions for next population;
The proposed technique is a multiple population algorithm, 6 perform crossover and mutation;
in which small changes are performed and the populations 7 evaluate population;
are combined to find the fittest test data. The fittest solutions 8 end
from all the populations form a new population. Crossover is 9 end
performed to have a new population which is also combined
with the fittest population and the process continues. After all
iterations, the set of fittest test data is obtained for maximum The termination criterion of the algorithm can be either
path coverage. Azam et al. [16] presented a toolkit for reaching a solution to the problem, or reaching the maximum
automated test data generation, and test case prioritization. number of iterations, meaning that a solution cannot be found
They used GA and fuzzy rule based system to generate given the available resources.
automated test data with focus on boundary values testing.
Scalabrino et al. [17] presented a search-based tool, called B. CUIT and GAF
Ocelot, for the automatic generation of test cases in C. Run- In Visual Studio, automated tests that drive an application
ning Ocelot consists of two distinct macro-phases: build and through its user interface (UI) are known as coded UI tests
run. In the build phase, the target program is instrumented, (CUITs) [20]. They are frequently used to verify that the
and then compiled. The output of such a phase is a static application is working correctly including its user interface,
library, which is linked by the tool and used in the run phase, and to automate an existing manual test. CUIT is used to run
where a search-based algorithm is exploited to identify a set the webApp with certain data generated from the proposed
of inputs that maximize the code coverage. The output of tool to test the viability of the test data and the achieved
this phase is a set of test data that can be used to test the coverage.
target program. The Genetic Algorithm Framework (GAF) is a .Net/Mono
As far as the authors are aware, none of these researches assembly, freely available via NuGet, that allows a GA based
have used GAs to generate test data for data flow testing of solution to be implemented in C# [21]. The population,
webApps. which is the collection of possible solutions, can be initial-
ized randomly, or a set of starting possible solutions can be
Volume 47, Issue 3: September 2020
______________________________________________________________________________________
A GA-Based Approach to Automatic Test Data
Generation for ASP.NET Web Applications
Islam T. Elgendy, Moheb R. Girgis, and Adel A. Sewisy
Abstract—One of the major challenges in software testing is to cover a certain testing criterion requires experience and
the generation of test data automatically that satisfy a specified time. Automating the process of test data generation can
adequacy criterion. This paper presents a GA-based approach solve these issues. One possible solution is generating test
and a supporting tool for data-flow test data generation for
ASP.NET web applications. The proposed tool accepts as input data at random, which can speed up the process significantly.
the web application under test, instruments it, and performs However, there is no guarantee that the generated data will
static analysis to compute the definition-use pairs. The proposed cover the testing criterion, or the data will be useful in
GA conducts its search by constructing new test data from error detection. Therefore, a more sophisticated and intel-
previously generated test data that are evaluated as effective ligent technique is required. Search-based software testing
test data. In this GA, the chromosome is a collection of user
interface control objects, where each control is considered as a formulates testing as an optimization problem, which can be
gene. Therefore, novel crossover and mutation operators are solved using computational search techniques from the field
developed to manipulate the chromosome, which are called of Search Based Software Engineering.
block crossover and control-based mutation operators. The This paper presents an automated test data generation ap-
proposed GA accepts as input the instrumented version, the proach based on genetic algorithm (GA) for data flow testing
list of definition-use pairs to be covered, and input controls
related information. The tool produces a set of test cases, the of ASP.NET webApps. The paper is organized as follows:
set of definition-use pairs covered by each test case, and a list of Section 2 presents the related work. Section 3 briefly covers
uncovered definition-use pairs, if any. Also the paper presents a the concepts of GAs and some of the used utilities. Section 4
case study to illustrate how the tool works. Finally, it presents presents the proposed test data generation approach. Section
the results of the empirical evaluation that is performed to 5 presents a case study. Section 6 presents the experimental
evaluate the effectiveness of the generated test data in exposing
web application errors. results. Section 7 presents the conclusion of this work.
Index Terms—Software testing, Data Flow Testing, Automatic II. R ELATED W ORK
test data generation, Automated Testing Tool, Web Applications
Testing. As this paper focuses on automatic test data generation
for webApps using GA, this section reviews some of the
research work in the field of search-based test data generation
I. I NTRODUCTION
and webApp testing. Sharma et al. [6] presented a survey of
S OFTWARE testing has been widely used in the industry
as a quality assurance technique for the various artifacts
in a software project. However, software testing is very labor-
GA approach for addressing the various issues encountered
during software testing, and discussed the applications of
GA in different types of software testing. They concluded
intensive, time-consuming and expensive; almost 50% of the that using GA, the results and the performance of testing
cost [1], [2] and 40% of the time of a software system can be improved. Li et al. [3] presented a broad survey of
development is spent on software testing [3]. Software testing testing advances in webApps and discussed the techniques
has two main aspects: test data generation and application employed, targets, goals, inputs/outputs and stopping criteria.
of a test data adequacy criterion. A test data generation They stated that there are various testing goals, such as
technique is an algorithm that generates test cases, whereas ensuring testing adequacy, finding faults, etc. and the choice
an adequacy criterion is a predicate that determines whether of a testing technique depends on the testing goal. Lakshmi
the testing process is finished [4]. and Mallika [7] presented a comparative study of some of
Web applications (WebApps) are being used extensively in the techniques, prominent tools, and models for WebApp
business, social, organizational, and governmental functions. testing, and highlighted the research directions for some of
The continual availability of WebApps is one of the advan- the WebApp testing techniques.
tages to use them by many users without regard to their Girgis [8] presented an automatic test data generation tech-
location or time limitations [5]. However, this demands high nique based on GA, which is guided by the data flow
reliability of WebApps. Inadequate testing poses huge risks dependencies in the program, to search for test data to fulfil
including downtime and loss of users’ trust and convenience. the all-uses criterion. The input to the GA is an instrumented
The Web’s ubiquity and users’ reliance on it have made it version of the program under test, number of input variables,
crucially important to ensure the quality, correctness, and the domain and precision of each input variable, and list of
security of WebApps. definition-use (def-use) associations. The output of the GA is
The process of constructing or choosing test data manually a set of test cases and the list of def-use associations covered
Manuscript received August 29, 2019; revised February 3, 2020 by each test case, and a list of uncovered def-use associations,
Islam T. Elgendy and Adel A. Sewisy are with the Department of Com- if any.
puter Science, Faculty of Computers and Information, Assiut University, Alshahwan and Harman [5] adapted a set of related search
Egypt. e-mail:
Moheb R. Girgis is with the Department of Computer Science, Faculty based testing algorithms for webApp testing, and imple-
of Science, Minia University, Egypt. mented an automated test data generation approach for PHP
Volume 47, Issue 3: September 2020
, IAENG International Journal of Computer Science, 47:3, IJCS_47_3_24
______________________________________________________________________________________
webApps. Setiadi et al. [9] proposed a method to reduce III. G ENETIC A LGORITHM C ONCEPTS AND U TILITIES
the number of test cases for detecting errors in a concurrent This section presents the basic concepts of genetic al-
program. They used data flows and branch structure to gorithms (GAs), and some utilities for automated testing,
reduce redundant test cases, identifying only the interleavings namely Microsoft Coded User Interface Testing (CUIT) and
that affect the branch outcomes rather than identifying all Genetic Algorithm Framework (GAF).
interleavings affecting shared variables. Setiadi et al. [10]
further improved their work by analyzing data dependency to
A. Genetic algorithms
generate test cases affecting sequences of locks and shared
variables, reducing the required memory space. Also, they The basic concepts of GAs were developed by Holland
generated test cases for detecting race conditions caused [18]. GA is a search-based optimization technique inspired
by accesses through reference variables. Furthermore, by from the natural search and selection processes that leads
exploiting previous test results, they reduced the effort for to the survival of the fittest individuals. GAs generate a
checking race conditions. sequence of populations by using a selection mechanism,
Takamatsu et al. [11] extended the Seeker tool developed and use crossover and mutation as search mechanisms [19].
by Thummalapenta et al. [12], which is based on branch The idea behind GAs is that they construct a population of
coverage, focusing on multiple targets, identifying all the in- individuals represented by chromosomes, which is normally
volved targets in uncovered branches and evaluating method a character string similar to the DNA chromosomes. These
sequences according to a fitness function, while applying a chromosomes represent solutions to a problem. The chro-
search strategy to suppress combinatorial explosion. Bous- mosomes then evolve according to the selection, mutation
sasa [13] introduced the use of Novelty Search (NS) algo- and reproduction rules. The fitness of each individual (chro-
rithm to the test data generation problem based on statement- mosome) in the environment is measured. Individuals with
coverage criteria. The NS adaptation attempted to exploit high fitness values in the population are selected by repro-
the large search space of input values and catch relevant duction, and a new population is derived through crossover
test cases that may cover as much as possible the executed and mutation, in which individuals may have better fitness
statements. in their environment. In crossover two chromosomes swap
Girgis et al. [14] have presented an approach to data flow genetic information as in the process of sexual reproduction.
testing of WebApps. They presented an approach that in- Mutation makes slight modifications in a small part of the
cludes the construction of a WebApp data flow model to population that represents an evolutionary step. The structure
aid WebApps data flow analysis. In this approach, testing is of a simple GA is given in algorithm 1.
conducted in four different levels, Function, Inter-procedural,
Page, and Inter-Page levels. In each level, the def-use pairs Algorithm 1: Basic GA
of the variables are obtained. Then, selecting test data that 1 begin
cover these def-use pairs ensures the fulfilment of the all- 2 initialize population;
uses criterion. 3 evaluate population;
Akhter et al. [15] proposed a GA based-test data generation 4 while termination criterion not reached do
technique to achieve path coverage of the program under test. 5 select solutions for next population;
The proposed technique is a multiple population algorithm, 6 perform crossover and mutation;
in which small changes are performed and the populations 7 evaluate population;
are combined to find the fittest test data. The fittest solutions 8 end
from all the populations form a new population. Crossover is 9 end
performed to have a new population which is also combined
with the fittest population and the process continues. After all
iterations, the set of fittest test data is obtained for maximum The termination criterion of the algorithm can be either
path coverage. Azam et al. [16] presented a toolkit for reaching a solution to the problem, or reaching the maximum
automated test data generation, and test case prioritization. number of iterations, meaning that a solution cannot be found
They used GA and fuzzy rule based system to generate given the available resources.
automated test data with focus on boundary values testing.
Scalabrino et al. [17] presented a search-based tool, called B. CUIT and GAF
Ocelot, for the automatic generation of test cases in C. Run- In Visual Studio, automated tests that drive an application
ning Ocelot consists of two distinct macro-phases: build and through its user interface (UI) are known as coded UI tests
run. In the build phase, the target program is instrumented, (CUITs) [20]. They are frequently used to verify that the
and then compiled. The output of such a phase is a static application is working correctly including its user interface,
library, which is linked by the tool and used in the run phase, and to automate an existing manual test. CUIT is used to run
where a search-based algorithm is exploited to identify a set the webApp with certain data generated from the proposed
of inputs that maximize the code coverage. The output of tool to test the viability of the test data and the achieved
this phase is a set of test data that can be used to test the coverage.
target program. The Genetic Algorithm Framework (GAF) is a .Net/Mono
As far as the authors are aware, none of these researches assembly, freely available via NuGet, that allows a GA based
have used GAs to generate test data for data flow testing of solution to be implemented in C# [21]. The population,
webApps. which is the collection of possible solutions, can be initial-
ized randomly, or a set of starting possible solutions can be
Volume 47, Issue 3: September 2020
