Test And Automated Build (TAB) Framework
1. Why TAB Framework?
In software development life cycle we basically divide our time in different slots, so that software get developed and thoroughly tested in scheduled timelines. Various tasks involved in SDLC include: requirement gathering & analysis, designing & prototyping, development, Unit testing, UAT & Automated testing for interfaces and other functionality and finally the deployment. All these tasks can be completed quickly by following firm process and using some automated tools. Tools reduces the human intervention for iterative tasks, reduces the time required for completing the task and up to a certain extend we can say that it maintains consistency of task as well.
TAB frame is focusing on one of the task mentioned above, and that is Unit Testing. TAB framework is focused on automating the chain of iterative tasks and that includes:
(a) Extracting of latest copy of code from versioning server (SVN/Development environment)
(b) Compilation & Creating of deployment build.
(c) Execution of Unit Test Cases.
(d) Management and e-mailing of test result.
2. Concepts & Terminologies
2.1 What is Unit Testing?
Unit testing deals with testing a unit as a whole. This would test the interaction of many functions but confine the test within one unit. The exact scope of a unit is left to interpretation. Supporting test code, sometimes called scaffolding, may be necessary to support an individual test. The architecture and implementation teams drive this type of testing. This focus is also called black box testing because only the details of the interface are visible to the test. Limits that are global to a unit are tested here.
Internal and unit testing can be automated with the help of custom tools. A custom tool analyzes the source code and tests it. It is still up to the programmer to combine these tests into meaningful cases to validate the result of each thread of execution. Typically, the custom tool is used in a slightly different way. First the coverage tool is used to augment the source by placing informational prints after each line of code. Then the testing suite is executed generating an audit trail. This audit trail is analyzed and reports the percent of the total system code executed during the test suite. If the coverage is high and the untested source lines are of low impact to the system's overall quality, then no more additional tests are required.
2.2 What is Junit?
JUnit is a regression-testing framework written by Erich Gamma and Kent Beck. The developer who implements unit tests in Java uses it. JUnit is Open Source Software, released under the Common Public License Version 1.0 and hosted on SourceForge.
3. The TAB Framework
TAB Framework can be used to automate the process of Building deployment package, Unit Testing using Junit and Deployment. The figure below shows the high level overview of TAB Framework. TAB Framework basically interact with SVN or Workspace to get the latest copy of code, compiles it and executes Junit Test cases and generate a test report with testing results.
Feature of TAB Framework
(a) Automated SVN/Workspace interaction for latest build.
(b) Scheduled Execution of Unit Test cases.
(c) TAB Framework maintains an Excel report for executed test cases.
(d) E-mailing of Reports.
For information on each of these functionalities please see the TAB Framework design section. This is very first cut of TAB Framework and missing few functionalities and features, those will be included once the design is completed. (Please see the section: Improvements & Yet to come.)
TAB Frame work can be divided in three different sections and those are a) TAB Tasks b) TAB Configuration & Data c) TAB Test Suit
4. Components of TAB Framework
TAB Frame work can be divided in three different sections and those are
a) TAB Tasks
b) TAB Configuration & Data
c) TAB Test Suit
Figure below shows the three different sections with individual components, this is very first design, so might be it needs some changes as and when required, so please don't consider this as final design. If things can be added please share you precious suggestions.
Lets discuss each component in detail:
4.1 TAB Tasks
These are the various tasks/operations performed by the TAB framework, which includes:
(a) SVN/Source code client: This is a configurable source code client; we can configure this client to get the latest copy of code from either SVN repository or from development/workspace environment. This client is responsible for reading the configuration, getting the latest copy of code and copying it to test environment.
Once the latest code is available in the test environment build script (ant script) will call the compilation task to compile the latest code and subsequently a new JAR file (application deployment package) will be created. So now the test environment contains the latest build and now we will be firing all Junit test cases on this latest build to make sure that it (build) is not failing.
(b) Test Environment creator - Before actually firing unit test cases on latest build we should clean/reset the folders and database tables to make sure that we don't have any garbage files/data. If we have any garbage files/data we might not get correct unit test results. Test Environment creator will perform these tasks and it will make sure that folders/tables used during Unit Testing is clean and don't contain any garbage file/data.
(c) Test Report Mailer - This is simple mail handler, which will read e-mail configurations (server address, e-mail address etc) and will send out e-mail alert with an attached copy of test results.
(d) Test Case Report Manager - report manager will collect results of all test cases executed, update the test case report template with these results.
(e) TAB Scheduler (TODO Task) - If we want to have a nightly build functionality then we need to configure the TAB Scheduler, so that complete cycle of build and unit testing will be performed automatically at scheduled time.
4.2 TAB Configurations & Data
Here we have few configuration files, which will be used by the framework. These files will basically guide the framework about from where to read data files, where is the SVN/workspace located, where is the report test case template etc.
4.3 TAB Test Suit
Test suit basically contains two entities one is the Test Suit itself and another is set of test cases. Test cases are basically java files and it contains logic to test business classes. Test suit holds all test cases together and run them in threaded environment and finally updates the results in report template.
5. TAB Control flow
The figure below shows control flow of TAB framework. Main heart of this framework is a build script which controls/calls different tasks one after another. Build script is responsible for following:
(A) Class path settings.
(B) Locating application dependent libraries.
(C) Executing and controlling different tasks.
(D) Build deployment package (JAR file).
Flow chart shows how different tasks/operations are called one after another.
Lets explore this control flow in detail:
(a) Before everything the scheduler will invoke and it will check the configurations. It will run the tasks of TAB framework if current time is matching with configurations, else it will go into sleep mode.
(b) Once the scheduler has started executing task, the very first thing it will do is: it will grab the latest code from SVN/Workspace source location (based on configurations). Once the source code is copied to test environment, build script will compile it and finally a deployment package (JAR) will be created.
(c) Once we have the deployment package (JAR) available with us then we need to reset the test environment, so this will be done by another TAB task i.e. "Test Environment creator". Environment creator will clean all the test data folders and will remove the garbage file and at the same time it will flush the garbage records form the database.
(d) Now, we have deployment package and test environment ready, and its time to execute test cases. We have a test suit, which holds all test cases together and will run them. Test Suit will run each test case in threaded environment and at the same time it will collect results of each test case executed, and these results will be updated in test report template.
(e) Once Test Suit execution is finished, test case results will be updated in the test case report template and the report will be mailed to configured e-mail addresses.
(f) If all test cases have passed status then only the build will be deployed on server, else no
In software development life cycle we basically divide our time in different slots, so that software get developed and thoroughly tested in scheduled timelines. Various tasks involved in SDLC include: requirement gathering & analysis, designing & prototyping, development, Unit testing, UAT & Automated testing for interfaces and other functionality and finally the deployment. All these tasks can be completed quickly by following firm process and using some automated tools. Tools reduces the human intervention for iterative tasks, reduces the time required for completing the task and up to a certain extend we can say that it maintains consistency of task as well.
TAB frame is focusing on one of the task mentioned above, and that is Unit Testing. TAB framework is focused on automating the chain of iterative tasks and that includes:
(a) Extracting of latest copy of code from versioning server (SVN/Development environment)
(b) Compilation & Creating of deployment build.
(c) Execution of Unit Test Cases.
(d) Management and e-mailing of test result.
2. Concepts & Terminologies
2.1 What is Unit Testing?
Unit testing deals with testing a unit as a whole. This would test the interaction of many functions but confine the test within one unit. The exact scope of a unit is left to interpretation. Supporting test code, sometimes called scaffolding, may be necessary to support an individual test. The architecture and implementation teams drive this type of testing. This focus is also called black box testing because only the details of the interface are visible to the test. Limits that are global to a unit are tested here.
Internal and unit testing can be automated with the help of custom tools. A custom tool analyzes the source code and tests it. It is still up to the programmer to combine these tests into meaningful cases to validate the result of each thread of execution. Typically, the custom tool is used in a slightly different way. First the coverage tool is used to augment the source by placing informational prints after each line of code. Then the testing suite is executed generating an audit trail. This audit trail is analyzed and reports the percent of the total system code executed during the test suite. If the coverage is high and the untested source lines are of low impact to the system's overall quality, then no more additional tests are required.
2.2 What is Junit?
JUnit is a regression-testing framework written by Erich Gamma and Kent Beck. The developer who implements unit tests in Java uses it. JUnit is Open Source Software, released under the Common Public License Version 1.0 and hosted on SourceForge.
3. The TAB Framework
TAB Framework can be used to automate the process of Building deployment package, Unit Testing using Junit and Deployment. The figure below shows the high level overview of TAB Framework. TAB Framework basically interact with SVN or Workspace to get the latest copy of code, compiles it and executes Junit Test cases and generate a test report with testing results.
Feature of TAB Framework
(a) Automated SVN/Workspace interaction for latest build.
(b) Scheduled Execution of Unit Test cases.
(c) TAB Framework maintains an Excel report for executed test cases.
(d) E-mailing of Reports.
For information on each of these functionalities please see the TAB Framework design section. This is very first cut of TAB Framework and missing few functionalities and features, those will be included once the design is completed. (Please see the section: Improvements & Yet to come.)
TAB Frame work can be divided in three different sections and those are a) TAB Tasks b) TAB Configuration & Data c) TAB Test Suit
4. Components of TAB Framework
TAB Frame work can be divided in three different sections and those are
a) TAB Tasks
b) TAB Configuration & Data
c) TAB Test Suit
Figure below shows the three different sections with individual components, this is very first design, so might be it needs some changes as and when required, so please don't consider this as final design. If things can be added please share you precious suggestions.
Lets discuss each component in detail:
4.1 TAB Tasks
These are the various tasks/operations performed by the TAB framework, which includes:
(a) SVN/Source code client: This is a configurable source code client; we can configure this client to get the latest copy of code from either SVN repository or from development/workspace environment. This client is responsible for reading the configuration, getting the latest copy of code and copying it to test environment.
Once the latest code is available in the test environment build script (ant script) will call the compilation task to compile the latest code and subsequently a new JAR file (application deployment package) will be created. So now the test environment contains the latest build and now we will be firing all Junit test cases on this latest build to make sure that it (build) is not failing.
(b) Test Environment creator - Before actually firing unit test cases on latest build we should clean/reset the folders and database tables to make sure that we don't have any garbage files/data. If we have any garbage files/data we might not get correct unit test results. Test Environment creator will perform these tasks and it will make sure that folders/tables used during Unit Testing is clean and don't contain any garbage file/data.
(c) Test Report Mailer - This is simple mail handler, which will read e-mail configurations (server address, e-mail address etc) and will send out e-mail alert with an attached copy of test results.
(d) Test Case Report Manager - report manager will collect results of all test cases executed, update the test case report template with these results.
(e) TAB Scheduler (TODO Task) - If we want to have a nightly build functionality then we need to configure the TAB Scheduler, so that complete cycle of build and unit testing will be performed automatically at scheduled time.
4.2 TAB Configurations & Data
Here we have few configuration files, which will be used by the framework. These files will basically guide the framework about from where to read data files, where is the SVN/workspace located, where is the report test case template etc.
4.3 TAB Test Suit
Test suit basically contains two entities one is the Test Suit itself and another is set of test cases. Test cases are basically java files and it contains logic to test business classes. Test suit holds all test cases together and run them in threaded environment and finally updates the results in report template.
5. TAB Control flow
The figure below shows control flow of TAB framework. Main heart of this framework is a build script which controls/calls different tasks one after another. Build script is responsible for following:
(A) Class path settings.
(B) Locating application dependent libraries.
(C) Executing and controlling different tasks.
(D) Build deployment package (JAR file).
Flow chart shows how different tasks/operations are called one after another.
Lets explore this control flow in detail:
(a) Before everything the scheduler will invoke and it will check the configurations. It will run the tasks of TAB framework if current time is matching with configurations, else it will go into sleep mode.
(b) Once the scheduler has started executing task, the very first thing it will do is: it will grab the latest code from SVN/Workspace source location (based on configurations). Once the source code is copied to test environment, build script will compile it and finally a deployment package (JAR) will be created.
(c) Once we have the deployment package (JAR) available with us then we need to reset the test environment, so this will be done by another TAB task i.e. "Test Environment creator". Environment creator will clean all the test data folders and will remove the garbage file and at the same time it will flush the garbage records form the database.
(d) Now, we have deployment package and test environment ready, and its time to execute test cases. We have a test suit, which holds all test cases together and will run them. Test Suit will run each test case in threaded environment and at the same time it will collect results of each test case executed, and these results will be updated in test report template.
(e) Once Test Suit execution is finished, test case results will be updated in the test case report template and the report will be mailed to configured e-mail addresses.
(f) If all test cases have passed status then only the build will be deployed on server, else no
Comments