Agile Testing

How does testing happen in Agile environment?

James Lyndsay in his white paper in workroom-productions.com explained the process of testing on Agile Projects.

There are many agile methodologies used in testing, however, they are not unique and original. The effectiveness of a methodology does not depend on individual elements but depends on a combination of elements that are combined into a framework.

Automated testing is at the heart of agility

Automated testing is fundamental in agile testing and without them; the project cannot be called agile. Thus, the popular tests on an agile platform are automated. This type of testing focuses on automated verification to enable agile software development.

Comprehensive testing, but not by testers: Coders and not testers write for Automated Unit Tests. In Test Driven Development, once the test is written, they are executed so as they fail as a control. Then they are executed once again after coding to demonstrate that the previously failing tests now passes. Though slow, it generates accurate results.

Refactoring: With an aim to improve system flexibility, the system is re-structured without any changes to its functions. These changes should not be visible to testers, just to confirm that the system works as before.

Continuous Integration: Codes are being built regularly either hourly or daily. It continuously implements quality control processes to ensure quality software. This in turn saves the delivery time of software as opposed to traditional testing where testing is done when the product is developed.

Why do organisations prefer Agile methodology over Waterfall or any other methodologies? 

Transparency Get an opportunity for client throughout the project and see work in progress by adding new changes.

Early and predictable delivery

Predictable cost and schedule

Allow changes for in an each iteration

Focusing on client business value and deliver product with an importance to business

What are the key benefits for a team that follows Agile methodology

Stakeholder Engagement

Agile provides multiple opportunities for stakeholder and team engagement – before, during, and after each Sprint. By involving the client in every step of the project, there is a high degree of collaboration between the client and project team, providing more opportunities for the team to truly understand the client’s vision. Delivering working software early and frequently increases stakeholders’ trust in the team’s ability to deliver high-quality working software and encourages them to be more deeply engaged in the project.

Transparency

An Agile approach provides a unique opportunity for clients to be involved throughout the project, from prioritizing features to iteration planning and review sessions to frequent software builds containing new features. However, this also requires clients to understand that they are seeing a work in progress in exchange for this added benefit of transparency.

Early and Predictable Delivery

By using time-boxed, fixed schedule Sprints of 1-4 weeks, new features are delivered quickly and frequently, with a high level of predictability. This also provides the opportunity to release or beta test the software earlier than planned if there is sufficient business value.

Predictable Costs and Schedule

Because each Sprint is a fixed duration, the cost is predictable and limited to the amount of work that can be performed by the team in the fixed-schedule time box. Combined with the estimates provided to the client prior to each Sprint, the client can more readily understand the approximate cost of each feature, which improves decision making about the priority of features and the need for additional iterations.

Allows for Change

While the team needs to stay focused on delivering an agreed-to subset of the product’s features during each iteration, there is an opportunity to constantly refine and reprioritize the overall product backlog. New or changed backlog items can be planned for the next iteration, providing the opportunity to introduce changes within a few weeks.

Focuses on Business Value

By allowing the client to determine the priority of features, the team understands what’s most important to the client’s business, and can deliver the features that provide the most business value.

Focuses on Users

Agile commonly uses user stories with business-focused acceptance criteria to define product features. By focusing features on the needs of real users, each feature incrementally delivers value, not just an IT component. This also provides the opportunity to beta test software after each Sprint, gaining valuable feedback early in the project and providing the ability to make changes as needed.

Improves Quality

By breaking down the project into manageable units, the project team can focus on high-quality development, testing, and collaboration. Also, by producing frequent builds and conducting testing and reviews during each iteration, quality is improved by finding and fixing defects quickly and identifying expectation mismatches early.

Why do organisations need a separate testing team and what are the key roles for a tester?

It is very difficult for a developer to find out the defects in their own product so crack the errors or loop holes in the software, It is crucial to have independent  testing team . Imagine, a developer never want to break is own system. He will test his system in safe way. He will not  test application from end user perspective. So its quite possible there may be some severe defects available in software. Dedicated testing team takes the role of real user and try to break the system from different viewpoints. This is the main reason organizations need dedicated software testing team.

Testers plays vital role in software development process. These are the roles and responsibilities  testers can perform :-

Testers can contribute to test planning, test design , requirement  Analysis and design specifications.

Identify and prepare test conditions , test cases and test data.

Review test cases and test data.

Test execution  and defect logs.

Creating reports and defect management.

What is release note?

Release notes are communication documents shared with customers and clients of an organization detailing the changes or enhancement made to the features of service or product the company provides. Thus this communication document is usually circulated only after the product or service is thoroughly tested and approved against the specification provided by the development team.

When do you do impact analysis and how does it help improve the quality of software developed?

According to ISTQB, impact analysis is an estimation of consequences a change has on all levels of testing and development documentation, including registering them in corresponding requests. This definition draws our attention to a more practical side of impact analysis - changes should always be documented.

By understanding what impact analysis really is, we can now understand when it should be used. Impact analysis should be employed in the following situations:

Changes are introduced to requirements

Request for product changes is submitted

New, planned functionality introduced

Existing modules and product features are changed

 Impact analysis helps us in the following way:

How the changes will affect existing modules and features, and what those affected modules are.

Determine new test cases required for testing new modules or features.

Examine how the testing process will be affected by the changes and whether existing process should be corrected.

Determine, what effect these changes will have on the budget

Can you list few common challenges faced by software testers in general?

There are many issues that testers have to deal with. Listed a few here:

1. Subject awareness: A major challenge in the domain of testing in general is the lack of skilled testers. Software testing can only achieve its intended purpose when testers are aware of their customer’s business. In order to understand this, one must have good knowledge of Customer’s business domain.  Poor domain knowledge of testers can end up in ineffective Test Scenarios, test scripts and post implementation defects.

2. Time:  Time has always been a limiting factor for Software testers and is challenging especially when it comes to execution of Regression tests.  During testing phase, testers are asked to perform multiple rounds of Regression tests and the time taken for these regression test cases could range from 100s to 1000s depending on the size of the application.  The best way to finish within the deadline and manage several rounds of regression testing is by automating regression tests that are not subject to direct change as part of the current release.

3. Test Estimation: There are many established Test Estimation methods.  Deriving accurate test estimates is a vital factor for successful Software Testing Phase.  Good Test Estimates will include an estimate time for all the activities performed during Software Test Life Cycle irrespective of the size of the task.

4. Test Data setup: Test Data setup will be more difficult when data setup deals with mainframe or multiple applications or databases. Test Data creation for Load or performance testing would involve writing scripts or SQL Procedures.

5. Setting up Test environment: In most of the bigger organizations, Test environment setup is done by Operations / Technical support team.  However in smaller or midsized companies, Test environment setup could seem extremely difficult.  Environment setup is one the tasks that necessitates knowledge of various software and testers find it most challenging.

6. Unavailability of the best Tools:  Project budget availability and funding affects availability of right tools.  Usage of right tools can aid testers in cutting short the time required to perform their tasks.

7. Team at different locations:  If Business Analysts and development teams are located in locations other than the time zone of testing team, the testers might have to extend their working hours to get clarifications or attend meetings.  Difference in time zone could result in longer wait time and general delays.

What is Risk Based Testing?

Risk based testing is prioritizing the feature's, modules and functions of the Application under Test based on impact and likelihood of failures. It involves assessing the risk based on the complexity, business criticality, usage frequency, visible areas, Defect prone areas, etc.

How to overcome the challenge of not having input documentation for testing?

Without having documented requirements will be a challenge for the testing team to test the software. But there are cases were the documented software requirement is too little or not clear. In those cases, we can test the system with our knowledge on the domain. Also we can approach the Customer or Business analyst to know the functionality and later we can document our understanding get it verified by the BA/customer and continue with the testing and these are few ways we can overcome the challenge of not having documented input requirements.

How do you plan your test case execution?

First and foremost make the understanding clear of what to be tested and document your understanding. Testing team should give their input to the test manager in the effort required for testing the software. Once the test plan prepared, reviewed and approved the testing team should come out with the detailed test condition and test cases for the software to be tested. Then start executing the test cases by giving high priority to defect prone or critical functionality.

Test Planning

Ideally, who prepares the Test Plan document?

Test Manager is the one who prepares the Test Plan document.

What is the process to create a Test Plan document?

Analyze the product

Design the Test Strategy

Define the Test Objectives

Define Test Criteria

Resource PlanningPlan

Test Environment

Schedule & Estimation

Determine Test Deliverables

What is the difference between a Test Plan and Test Strategy?

Test Plan	Test Strategy
A test plan for software project can be defined as a document that defines the scope, objective, approach and emphasis on a software testing effort.	Test Strategy is a set of guidelines that explains test design and determine how testing needs to be done
It is defined at project level	It is set at organization level and can be used by multiple projects
Test planning is done to determine possible issues and dependencies in order to identify the risks.	It is a long-term plan of action. You can abstract information that is not project specific and put it into test approach
Test plan narrates about the specification	Test strategy narrates about the general approaches

What is Defect rejection ratio and Defect leakage ratio?

Defect rejection ratio is the ratio of total defects rejected to the total defects raised by the testing team.

Defect Leakage ratio is the total defects raised by the customer during user acceptance and beta testing to the total defects found combined by the testing team and the customer.

What is Entry Criteria, Exit Criteria and Suspension Criteria?

Entry Criteria is a Criteria defined in the test plan for starting the testing and the criteria can include:

Test Plan should be reviewed and approved.

Test cased and Test data should be available.

Test Environment should be ready

Core functionality should be completed.

Exit Criteria is a Criteria defined for exiting the testing phase. This depends on

Ensure all the critical test cases are passed

There are no Critical and high severity defects opened

Ensure 100% functional coverage is done during testing.

Suspension Criteria : When there are too many test cases fail or when there is too many  and when there are too many defects blocking the testing could be another criteria for suspending testing.

When do you start and stop testing?

An early start to testing reduces the cost, time to rework and error free software that is delivered to the client. However in Software Development Life Cycle (SDLC) testing can be started from the requirements Gathering phase and lasts till the deployment of the software. However it also depends on the development model that is being used. For example in Water fall model formal testing is conducted in the Testing phase, but in incremental model, testing is performed at the end of every increment/iteration and at the end the whole application is tested.

Testing is done in different forms at every phase of SDLC like during Requirement gathering phase, the analysis and verifications of requirements are also considered testing. Reviewing the design in the design phase with intent to improve the design is also considered as testing. Testing performed by a developer on completion of the code is also categorized as Unit type of testing.

Unlike when to start testing it is difficult to determine when to stop testing, as testing is a never ending process and no one can say that any software is 100% tested. Following are the aspects which should be considered to stop the testing:

Testing Deadlines.

Completion of test case execution.

Completion of Functional and code coverage to a certain point.

Bug rate falls below a certain level and no high priority bugs are identified.

Management decision.

What is the purpose of having a test plan document?

It gives the overall direction for testing on what is the schedule, budget, resource planning, risks and issues, In Scope and Out of scope of requirements, test environment requirement, entry and exit criteria and much more.

How do you perform test estimation?

Mention what is the difference between Beta and Pilot testing?

Pilot testing : This is a real world testing executed by group of end users, before its full deployment in order to find defects so as to improve the quality of the application. The purpose of pilot testing is to find potential problems as early as possible before they become costly mistakes. The level of pilot testing depends on the size of the project. For larger projects, a planned pilot is necessary.

Beta Testing: This is a testing stage followed by the internal full alpha test cycle. This is the final testing phase where the companies release the software to few external user groups outside the company test teams or employees. This initial software version is known as the beta version. Most companies gather user feedback in this release.

Mention what are the risk to be avoided while testing?

Never test in the development environment

Try to have proper estimation to ensure not to miss the project deadline.

Ensure not overshoot on agreed budget.

Not to leave Critical functionality for test till the last phase of testing.

Never miss out on regular status reporting to the management.

Mention what are the counter measures that test manager should take against risks?

Avoidance: Eliminate the risk factor that is involved

Reduction: Mitigate plan to decrease the impact of risks and to take corrective measures

Sharing: Transfer the risk to another resource such as insource or insure

Accept: Accept the risk and prepare a plan budget for this risks

SDLC common interview questions

What is the difference between a use case and test case?

A use case captures business and user requirements related to system functions—that is, how the users interact with the system. The goal of a use case is to help the development team understand precisely what the users will expect the system to do.

A use case describes all the possible paths through a given user/system interaction, including the basic path and any alternative or exception paths. The basic (or "happy") path is the one that meets the user's needs. "Alternative paths" are additional paths that are acceptable but aren't the most common, frequent, or desirable. "Exception paths" are those that fail to meet the user's needs because of errors like missing information or invalid data. A single use case may describe many different paths.

Test cases are used to validate that the requirements have been met. The quality assurance analyst will probably want to test the system thoroughly by setting up an individual test case for each path described in a use case. At a minimum, they would set up separate test cases for the "happy path," each alternative path, and each exception path. There would probably also be multiple test cases for the happy path—one for each situation that would cause different business rules to be invoked.

Use cases are provided to developers so that they can develop the solution, and test cases are provided to testers so that they can validate that the solution matches the requirements. Thus, use cases often supply input for developing test cases. But while the two documents may overlap quite a bit, they aren't exactly the same thing.

Which phase of SDLC does the testers begin to write test case?

Testers start writing test case when the requirement are base lined and the functional requirement / use case are available and generally this is ready by Design phase or beginning of coding and unit testing phase. Testers start writing test case during Design or coding and unit testing phase.

What is the difference between functional document and business document?

Business document is the requirement gathered from the customer about their business requirements. This can also come from the competitor products from the marked and also through market research. Whereas the functional requirement is exact workflow of the agreed business requirements which the end user will be using when the project/product is developed.

Who are the different stakeholders involved in different phases of SDLC?

Requirements Gathering : Business Analyst / Product owner

Planning : Senior Management, Product owner, Business Analyst and project manager

Design : System Architect, Senior Developers and Project Managers

Coding and unit testing : Software Developer / Development Team

Testing : Tester / Testing Team

Implementation :  Operational Team

Maintenance:  Production Support Team

Do we need a separate environment for the developers and testers?

Developers when developing the system, there will be many changes happening to the system from the development team and it will be difficult for the testing team to test the system here as it is not stable. Hence testing team need a stable environment and it should be close to production environment if possible.

What is the role of a Business Analyst in different phases of SDLC?

Business Analyst has the key role in the requirement elicitation phase where he is directly responsible to get the requirements from customer. He also has the responsibility of sharing the knowledge with the rest of the team. During other phases of the SDLC, whenever the team has any doubts on the requirement, he has the responsibility of clarifying their doubts. During development, if any new requirement is pushed or any change to the existing requirement is identified he has to discuss with the team and see the impact of the change and feasibility of implementation of the change after discussing with the architects, project managers, test managers and senior developers. When the project / product is ready for release he can always get involved in system testing to ensure the system is working as per the customer requirements. When the defects are raised by the testing team during testing and if there are any conflicts between development and testing team on the relevance of the defect, he can always give his input on the defect under discussion.

What is the role of a Developer in different phases of SDLC?

Requirement Elicitation: Ensure the requirements are understood clearly and demonstrate his understanding to the team (Medium: Presentation, discussion, understanding documents..etc.)

Planning : Provide the input to the project manager on the initial estimation of the requirements.

Design Phase: Come out with the low level design from the High level design documents.

Coding and unit testing: Implement the system and do the self-review, code review and unit testing.

Formal Testing: Fix the defect raised by the testing team.

Maintenance: Ensure the issues raised in production are verified, do the impact analysis, estimate the time for fixing and do the production fix.

What is the role of a Test Analyst in different phases of SDLC?

Requirement Elicitation: Ensure the requirements are understood clearly and demonstrate his understanding to the team (Medium: Presentation, discussion, understanding documents..etc.)

Planning Phase:  Help the test lead in providing the estimation of each testing task. 

Design Phase:  Start writing the test cases for the finalized requirements.

Coding and unit testing: Continue on test case development and also involve in test case review of peers.

Formal Testing: Execute the test cases, analyze any defects found, report the defect and get the acknowledgement of the defect from the development team. When the defects are fixed by the dev team, ensure they are re-tested and perform the regression testing to ensure no other functionality is broken. Also test the non-functional requirements identified by the customer and the team.

Maintenance: Regression testing, non-functional testing and the production fix test.

What is the role of a Customer/ Client in different phases of SDLC?

Requirement Elicitation: Provide the necessary business requirements to the development team. Ensure all the clarifications from the team (business analyst and rest of the team members) are addressed in timely manner.

During other phase of SDLC, the customer should always be available for clarification and also participate during the sprint or iteration demo. During UAT, participate in the UAT testing and give their feedback on the system.

Defect Management

Why is defect management process important in software development teams?

This is a formal way of identifying, fixing and tracking the defects in the system. If defect management process is not followed it will be very hard to know the quality of the system and also will be very difficult to know the status of the defect like how many defects raised, whether any developer working on these defects and also will be difficult to know if anyone re-tested the defect.

What are the fields in a bug report? • Why do defects have priority and severity?

Different field in bug report are Defect ID, Defect Description, Version, Steps to reproduce the defects, Date Raised, Reference (Like requirement doc, design doc or screenshots), Detected by, Status (New, Assigned, Deferred, Fixed, re-test, re-open, close), Fixed By, Date Closed, Severity and Priority.

Severity and Priority helps to understand the criticality of the defect and the impact on the system.

When does a defect gets deferred? • How do you deal with an inconsistent defect?

Defect gets deferred when there are other priority defects to be fixed and also when the impact of the deferred defect on the system is low. There could be also management/customer decision to defer the defect to the next version.

Dealing with Inconsistent defect: Always have detailed steps in your test case, if possible record the flow or take screenshots as for as possible. Do not sit on inconsistent defect, discuss with senior testers and test managers / lead and their experience will help you to deal with these kind of defects.

How do you log a defect? • Can you name a defect that has high priority and low severity?

Defect logging is generally done through a tool like Jira, HP QC, ALM , Microsoft TFS etc.. or if the organization is very small you can use excel spreadsheet as well.

The defect logged should have at least Defect ID, Defect Description, Version, Steps to reproduce the defects, Date Raised, Reference (Like requirement doc, design doc or screenshots), Detected by, Status (New, Assigned, Deferred, Fixed, re-test, re-open, and close), Fixed By, Date Closed, Severity and Priority.

The logo of the company is set wrong can be a good example of High priority and low severity.

When does a defect gets rejected by the developer?

Bug gets rejected due to one or combination of the below reasons

1. Defects not reproducible
2. Requirement not clear (everyone have their own understanding)
3. Duplicate Defect
4. Test Description and test steps are not clear
5. Test data used was not correct
6. Test Environment issue (Hardware, software, wrong version/build  ...etc)
7. Wrong understanding of the requirement by the tester
8. Requirement got changed but not informed to the test team (Communication issue)

 

What is a change request and how does a change request gets handled?

If any changes comes after the requirement is baselined will be called as change request.

Change request will go to the change control board (CCB) which will have the stakeholders from the customer and the software development team. First the change will be discussed on the business impact, time line and changes required to the system. Once the development team does the impact analysis of the change and give their feasibility report, the CCB will approve or reject the defect.

How do you analyze the root cause of a defect? • What is defect triaging?

Either by 5 why technique, Brain storming or using fish bone diagram we can do a root cause analysis of software defects. First step is to categorize the defects and group the similar defects and use pareto analysis and find those 20% defect category causing the 80% of the defects. Use the technique like 5 why, brain storming or fish bone to arrive at the root causes for these 20% of the defects. Have an action item for these defects and do the systemic changes so the defects will not reoccur in the future releases.

Defect Triaging : Defect triaging is a meeting initiated by the test lead and attended by Project managers, developers , testers , BA and product owner. Here each defect will be discussed, and the priority and severity will be set and also the testing team will give their view of any defects blocking their testing process. Even the defects rejected reason will be discussed here. Also the developers will give their timeline in fixing the agreed defects in this meeting. Any defects to be deferred to the next release will also be discussed here.

Testing Principles and Testing Types

How do you identify test cases for regression testing?

Following may be few reasons how we can choose regression test cases

Test cases which verify the core functionality of the project

Test cases related to the functionality changes undergone recently

Commonly used production scenarios to be included as part of regression test cases

If there are automated test cases for IT and ST included all the test cases as part of regression.

How to avoid re-occurring defects?

Every defects need to be discussed and identify the root cause of the defects. Then come out with the action plan which is systemic and the changes have to be implemented so that the defects will not re-occur.

How to optimize test cases?

How to write re-usable test cases?

What is the difference between alpha and beta testing?

Alpha Testing: This is a form of internal acceptance testing performed mainly by the in-house software QA and testing teams. Alpha testing is the last testing done by the test teams at the development site after the acceptance testing and before releasing the software for beta test. Alpha testing can also be done by the potential users or customers of the application. But still, this is a form of in-house acceptance testing.

Beta Testing: This is a testing stage followed by the internal full alpha test cycle. This is the final testing phase where the companies release the software to few external user groups outside the company test teams or employees. This initial software version is known as the beta version. Most companies gather user feedback in this release.

What is the difference between white box and black box testing?

White box testing is generally done by the developers who know the internal algorithm or know how of the code and they test each path, branch and statements to ensure the code works in different test condition. Their main aim is to cover as much code as possible during their testing and ensure the functionality works as per the given requirements.

Whereas black box testing is done by the testing team after the code freeze and here the team may not look into how the code is written and their main focus is to ensure all the requirements / scenarios works as per the expected results. Testing done post coding and unit testing like Integration testing, System testing, Performance testing, User acceptance testing falls under black box testing.

Difference between functional and non-functional testing

Functional testing is done to ensure the system works as per the agreed requirements and this is done by the testing team post coding phase. There are 2 types of functional testing one is positive functional testing as explained earlier which ensures when valid inputs are provided to the system, the system works as per the requirements. Second type of the functional testing is called as negative functional testing where the testers gives invalid inputs, unanticipated operating condition and other invalid operations.

Non-functional testing is designed to figure out if your product will provide a good user experience. For example, non-functional tests are used to determine how fast the product responds to a request or how long it takes to do an action. Examples of non-functional tests include:

Load/Performance Testing

Compatibility Testing

Localization Testing

Security Testing

Reliability Testing

Stress Testing

Usability Testing

Compliance Testing 

Difference between system testing and system integration testing?

Integration testing is done right after the coding and here all the modules are integrated and tested to ensure the system works as per the requirements and here stubs and drivers are used to replace the requirements of external feeds or systems.

System testing is done after integration testing where the scope of the testing is to test the system end to end with all the modules and external systems in place. This test ensures all the functional and non-functional requirement are tested and system meets all these requirements.

Difference between smoke testing and sanity testing?

Smoke Testing is a kind of Software Testing performed after software build to ascertain that the critical functionalities of the program is working fine. It is executed "before" any detailed functional or regression tests are executed on the software build. The purpose is to reject a badly broken application, so that the QA team does not waste time installing and testing the software application.

In Smoke Testing, the test cases chosen cover the most important functionality or component of the system. The objective is not to perform exhaustive testing, but to verify that the critical functionalities of the system is working fine.

For Example a typical smoke test would be - Verify that the application launches successfully, Check that the GUI is responsive ... etc.

Sanity testing is a kind of Software Testing performed after receiving a software build, with minor changes in code, or functionality, to ascertain that the bugs have been fixed and no further issues are introduced due to these changes. The goal is to determine that the proposed functionality works roughly as expected. If sanity test fails, the build is rejected to save the time and costs involved in a more rigorous testing.

The objective is "not" to verify thoroughly the new functionality, but to determine that the developer has applied some rationality (sanity) while producing the software. For instance, if your scientific calculator gives the result of 2 + 2 =5! Then, there is no point testing the advanced functionalities like sin 30 + cos 50

STLC Common Interview Questions

What is the testing process followed in a company ideally?

The ideal testing process followed in the company are:

Test Planning: Here in this phase the test manager or test lead will come out with the overall test strategy. He will be also involved in estimation of testing effort and cost. Other important decision likes what will be in scope and out of scope for the testing requirements. What percentage of automation will be involved in the project will be also decided here.

Test Design (Test Condition and Test Case Development):

Once the test plan is approved, based on the requirement analysis, test scenarios and detailed test cases will be developed for all the identified components. The test cases are reviewed by the testing team and will be approved by the lead to start execution of these test cases.

Test Environment Setup: This is an independent activity which can be done by the development team or the customer. Here the environment is build and the required test data will also be kept ready. Testing team should meanwhile prepare the smoke test case to test the environment to start the formal test execution.

Test Execution: Here the team execute the test case as per the test plan and the test case developed earlier and any defect found will be logged in the defect tracking system. Each failed test cases will be tagged with one or more defects and these defects will be analyzed and assigned to the development team. Once the defects are fixed, the team will re-execute the failed test cases. Team also prepare the report on number of test cases executed, how many passed, failed, blocked, not run and pass on this information to the test lead.

Test Cycle Closure: Team will conduct a meeting to evaluate the cycle completion criteria based on test coverage, quality, cost, effort and critical business objective and software.

Team will discuss what went well, what went wrong so that can improve the process in the next testing cycle. Test case and the defect report will be analyzed to find the defect distribution by type and severity. Once the test cycles are complete the closure report along with the test metrics will be prepared.

What are the activities performed in test design and test execution?

Test Design (Test Condition and Test Case Development):

Once the test plan is approved, based on the requirement analysis, test condition and detailed test cases will be developed for all the identified components. The test cases are reviewed by the testing team and will be approved by the lead to start execution of these test cases.

Test Execution: Here the team execute the test case as per the test plan and the test case developed earlier and any defect found will be logged in the defect tracking system. Each failed test cases will be tagged with one or more defects and these defects will be analyzed and assigned to the development team. Once the defects are fixed, the team will re-execute the failed test cases. Team also prepare the report on number of test cases executed, how many passed, failed, blocked, not run and pass on this information to the test lead.

Explain the difference between an SDLC and STLC?

SDLC is a verification process and whereas STLC is a validation process.

What is test scenario, test condition and test case?

Test Scenario: Gives the idea of what we have to test. It’s like a high level test case. It’s a possible ways to test the system. A scenario can be tested by single or group of test cases.

Test Condition: Test condition is a constraint that you should follow to test the application.

Test condition can be a piece of functionality or in simple term goal of the test case.

Test Cases: Test cases helps how to test. Test cases are the set of positive and negative executable steps of a test scenario which has a set of pre-conditions, test data, expected result, post conditions and actual result.

What is the process to write a manual test case?

A test case is a document that describes step by step process on how to test the application.

They should be independent with each other and pass/fail independently from one another.

A test case should include Test Case ID, Test Description, Test Steps, Expected result, Actual result, Pass/Fail status and remarks.

What is a test cycle and why it is important to carry out testing in cycles?

Product / Project cannot be stable if you do only one round of testing as it is hard to get the test case pass percentage to the agreed value. The main reason for this is, number of defects found in the initial round of testing will be generally more and as and when the developers fix the defects and re-tested by the testing team, the number of defect raised starts reducing in the next rounds of testing cycles. Number of rounds of testing cannot be fixed and this vary company to company and it depends on what exit criteria you have defined in your test plan and the past performance of the company in handling similar kind of project. In a nut shell, testing done in multiple rounds ensure better quality and stable product /project.

Explain Requirement Traceability Matrix (RTM) in simple terms.

Every requirement needs to be mapped to the high level design, low level design, the code which is implemented and finally to the test cases which are validating this requirement. By doing this, it is easy to handle changes and also helps during audit to show case the particular scenario and show the evidence related to the scenario. This kind of mapping of requirement down to the test cases in known as traceability matrix.

Software Testing Life Cycle (STLC)

Software testing life cycle identifies what testing activities to carry out and when to accomplish those test activities. Even though testing differs organization to organization, there is a testing life cycle.

Requirement Study : 

Testing Life cycle starts with the study of customer requirements.

Understanding of the customer requirements is very important for testing the Product.

Analysis and Planning :

Test Objective and Coverage

Overall Schedule

Resource required, including necessary training

Roles and responsibilities of the team members

Test Case Design and Development :

Component Identification 

Test Specification Design

Test Specification review

Test Case Execution :

Code review (few companies expect this to be done by the tester)

Test Case Execution and evaluation

Performance and Simulation

Test Closure :

Test Closure report

Project Documentation (User Manual)

Test Process Analysis :

Analysis done on the reports and improving the applications performance by implementing new technology and additional features

Success of testing team in agile world

Gone are those days where the team use to work on a project for a long time and the customer would get their required project delivered after a very long time in one go. These were the days where water fall model was predominantly used where fixed phases of software development like Planning, Requirement elicitation, Design, coding and unit testing, Integration testing, System testing and Customer acceptance testing were followed very rigidly. In these kind of software development model, testing team used to have dedicate phase to test the software and any major issues found in the system under test would have huge impact on the project or product quality and on the timeline. Pressure on the entire team would be high to make change and retest the system and deliver on time with good quality. Even handling changes in requirements used to have huge impact on the system to be developed. These are the few reasons which lead to the evolution of agile development model. 

Agile model encourages continuous integration and continuous deployment of the software and the team develops the system in multiple sprints and generally there is no dedicated testing phase at the end of the project. Now the question arises how will the testers contributes in this kind of software development model and ensure they are adding value to the team.

Unlike in water fall model, here the testers have to be involved or engaged throughout the project development from the inception till the delivery of the project. Let’s see how the testers can contribute in agile world.

Sprint planning Session

A member of testing team should always attend planning sessions. This ensures QA is in sync with the development team from the start, and allows QA to identify possible problem areas and risks early on. Just like developers estimate the effort it will take for them to write code, QA should estimate their effort required for testing the code during the planning session. When QA is left out of the planning session, testing effort / time is often overlooked and not included in the sprint’s overall estimates.

Daily stand ups.

A member of testing team should attend daily stand ups. This promotes a collaborative team environment, making QA feel involved and a part of the team. Additionally, by QA being involved in stand ups, they’re able to stay up to date with how the sprint is going and allows them to plan their workload. If a tester has a blocker, they can bring this up during the stand up. QA’s presence in stand ups also gives them a chance to give an update on known issues, which in turn allows developers to keep up to speed on testing progress and plan their own workload

Testing throughout the sprints

In order to deliver high quality software in a short amount of time, testing team need to work efficiently. With testing happening throughout the entire duration of the sprint, the workload for QA is spread out and allows for issues to be found earlier instead of only at the end of the sprint. If you find all the bugs at the end of the sprint, it’s too late. By integrating testing and development, it allows the two teams to work together and resolve issues faster, leading to higher quality results.

Short demo of the system developed so far

It’s hard to argue the value of in-person communication. Assuming QA and development work in the same location, schedule a quick face-to-face meeting to get the demo of each feature developed so far. This allows QA to see exactly how the new feature works and is a good time for them to ask the developer any questions. These hand-offs can also bring to light issues the developer may not have thought of yet. These interactions also help shorten the feedback loop between development and QA.

Sprint Retrospective

Ensure testing team members are part of Sprint retrospectives and help in identifying weaknesses in the system and contribute in determining solutions. QA needs to be involved in these discussions to ensure any concerns they have are addressed before the start of the next sprint. For example, maybe a lot of the work was delivered to QA late in the sprint, leading to a rushed testing effort. QA might raise this concern to avoid it happening again in the next sprint.

Document test cases.

Never skip documentation since the agile manifesto / principles says so. Always have the minimum required testing documentation in place.

Team have to embrace the above discussed points to be successful in agile development world.