CWDS

Test Management Approach

Executive Summary

The purpose of the CWDS Test Management Approach is to provide an overall framework and a set of principles, best practices, and guidelines for how testing will occur across all CWDS digital services in delivering code to production frequently as part of the continuous delivery pipeline. This document defines the overall testing strategy, but does not contain a complete set of detailed processes and procedures for each type of test. The primary objective of testing is to validate that the CWS-NS meets the approved user stories and requirements and that it delivers a quality application with minimal defects into production.

The Test Management Approach achieves the following objectives:

• Defines the goals and approach of the CWS-NS Testing processes.
• Defines the test management methodologies, best practices, roles and responsibilities, training and communication required throughout the continuous delivery pipeline.
• Identifies the process for defect management.

Testing is iterative, incremental, and continuous. This rapid turnaround of test results insures the product meets quality standards and facilitates moving the project forward. By providing feedback of test results on an on-going basis, developers are insured the product quality meets the business requirements.

• Continuous Delivery Pipeline – The pipeline represents the project’s software delivery process for delivering useful, high-quality, working software to users in an efficient, fast, reliable, and repeatable manner. The pipeline breaks down the software delivery process into stages. Each stage is aimed at verifying the quality of the product from a different perspective and prevent identified bugs from affecting users.

• Test Driven Development - TDD is a software development process that relies on the repetition of a very short development cycle: requirements are turned into very specific test cases, then the software is improved to pass the new tests, only.

• Continuous Integration - CI is a software development process of integrating developers’ work frequently (at least daily) using an automated suite of tools. There are three important and foundational elements that underpin any Integration system: (1) Automated Tests; (2) Never break the build; and (3) Version Control.

Various Testing Types include:

1. Code Quality (development) – Development teams may use an automated tool to evaluate the code base and ensure compliance with the code-style guideline.

2. Unit Tests (development) - tests verify functionality of a small subset of the system, such as an object or method. Automated unit tests are created by the developers to validate the internal function of the software at the service or component level. They are added to the regression test suite within the CI process as they are developed.

3. System Tests (development) - Automated system tests validate an entire module and will be executed automatically as part of the CI process after all unit tests are passed.

4. Code Coverage (development) - Code coverage measures how much application code is executed by test scripts. Each digital service team uses automated code coverage tools as part of their CI build process to generate reports that indicated the amount of coverage.

5. Browser Tests (development) - Each of the front-end development teams are expected to run automated tests to verify that their components are compatible with various browsers.

6. Accessibility Tests (development) - Accessibility testing is a subset of usability testing where the users have disabilities that affect how they use the application. This includes (but is not limited to) ensuring visibility, keyboard navigability, language clarity, and screen reader compatibility.

7. Smoke Tests (all) - Smoke tests, also known as “Build Verification Tests”, are comprised of a non-exhaustive set of tests that aim at ensuring that the most important functions work. The results of this type of testing is used to decide if a build is stable enough to proceed with further testing.

8. Integration tests (Pre-Int Integration) - Integration tests verify the interaction between code components. They run without the intervention of the GUI web interface.

9. Acceptance tests (Pre-Int Integration) - Acceptance tests define the expected behavior for the code to be delivered by the story during the sprint. They are scenarios which are derived from acceptance criteria outlined in the user stories.

10. Legacy Tests (Integration, Performance, PreProd) - The legacy systems, specifically CWS/CMS, have many difficult to see “hidden” rules, which requires legacy experts to help identify. Initially, legacy verification will require a lot of people to develop test scenarios and manually execute tests and participate in the acceptance process.

11. Exploratory Tests (Pre-Int, Preview, Integration, PreProd) - Exploratory tests are a complimentary activity to automated testing and used verify the quality of the software using a guided methodology that focuses on scenarios that emulates the way a real-world user would use the application.

12. Usability Testing (Development, Preview) - Usability tests help evaluate how real users experience the application under realistic conditions. It provides an opportunity to watch real people using the product.

13. Load and Performance Tests (Performance) - Load testing evaluates the robustness of the application under higher-than-expected loads. Performance testing identifies bottlenecks in a system.  

Introduction

In the world of software development, the term agile typically refers to any approach that strives to unite teams around the principles of collaboration, flexibility, simplicity, transparency, and responsiveness to feedback throughout the entire process of developing a new program or product. Agile Testing generally means the practice of testing software for bugs or performance issues within the context of an agile workflow.

Agile suggests that development and testing functions be merged—not necessarily in terms of people, but in terms of time and process—thus bridging the divide between code creators and code breakers, and even reducing the need for robust testing teams, while still respecting the necessity of both roles. In Agile, developers are encouraged to think more like testers, continually checking their own code for potential errors, and testers are encouraged to think more like developers, tempering their natural destructive tendencies to engage more fully in the creative process itself.

Testing becomes an essential component of each and every phase of the developmental process, with quality being “baked in” to a product at every stage of the continuous delivery pipeline. Software developers, testers, and quality-assurance personnel wear each other’s hats from time to time, and while there may be a select group of people running most of the tests, the notion of a separate testing team disappears entirely for many product teams.

Purpose

The purpose of the CWDS Test Management Approach is to provide an overall framework and a set of principles, best practices, and guidelines for how testing will occur across all CWDS digital services in delivering code to production frequently as part of the continuous delivery pipeline. This document defines the overall testing strategy, but does not contain a complete set of detailed processes and procedures for each type of test.

The primary objective of testing is to validate that the CWS-NS meets the approved user stories and requirements and that it delivers a quality application with minimal defects into production.

The Test Management Approach describes the methods that are used to test and evaluate the CWDS system. The approach provides guidance for the management of test activities, including organization, relationships, and responsibilities. The methodology provides a basis for verification of test results of the system. The validation process ensures that the system conforms to the functional requirements and ensures other applications or subsystems are not adversely affected.

Scope

The Test Management Approach achieves the following objectives:

• Defines the goals and approach of the CWS-NS Testing processes.
• Defines the test management methodologies, best practices, roles and responsibilities, training and communication required throughout the continuous delivery pipeline.
• Identifies the process for defect management.

Testing Vision

Establish a flexible, scalable, comprehensive, and integrated testing process with the goal of “exactly good enough” using lightweight processes, documentation, tools, and techniques.

Testing Principles

• All Digital Service team members are responsible for testing and delivering high- quality software

• Quality is built in at the start and tested throughout the life of each Digital Service

• Developers will use the Test-Driven Development (TDD) approach in developing code

• A feature story may not be accepted or meet the Definition of Done until it passes all its acceptance tests

• Bugs identified during testing will be made available to view real-time and on a historical basis

• Provide adequate test coverage to test the software application completely, and make sure it is performing as per the requirements specifications

Testing Best Practices

• Execute the simplest tests using the simplest tools possible

• Focus on testing the core functionality and “happy path” over edge cases and boundary tests

• Manage test artifacts in source code control repository

  • Map versions of test scripts to the versions of code they test

  • Organize the repository for application code, unit test scripts, and higher-level test scripts

• Set team rules, like the following:

  • Focus on finishing one before starting another

  • The number of open bugs in a sprint should never get higher than 10 at any one time

• Emphasize conversation and collaboration to clarify requirements over specifications and documentation

• Try to have automated tests that can run completely in memory and minimize tests that require database access or have to go through a presentation layer

• Ensure that test coverage remains above the agreed-upon baseline level

• Ensure running times of the automated test suites don't exceed agreed-upon upper bounds

• Conduct an “engineering sprint” or “refactoring iteration” every few months to reduce technical debt, upgrade tool versions, and try out new tools, rather than delivering new functionality/features

• Include time for writing tests, fixing bugs, and preparing test data when planning a sprint

• Whenever a test fails during the continuous integration (CI) build process, the team’s highest priority should be to get the build passing again

Integration with other CWDS Plans

This plan integrates with the Release Management processes.

Release Management Plan

The Release Management Plan (Note: This link is temporarily for OSI internal use only) describes the process how the Project will plan, build, monitor and report out progress of CWS-NS releases. The purpose of this plan is to document how the Project will plan, build, test, manage, and govern releases as part of the continuous delivery pipeline.

Assumptions

Document Maintenance

The CWDS Test Management Approach will be updated as processes and procedures change. A minor version change does not change the intent of the document and consists of spelling, grammatical and minor corrections. A major version is when a document’s content is changed and represents a change in intent, change in process, or procedures. Please refer to the CWDS Configuration Management for further detail on version control.

Roles and Responsibilities

Testing is primarily carried out by the digital service development teams, but is also supported by other project roles on the project using a “whole team” approach to testing. Each digital service team’s Product Owner is ultimately responsible for ensuring that the appropriate level of quality is being applied and validating that all stories have met the defined story acceptance criteria. The following table describes the roles and responsibilities of the CWS-NS Project stakeholders in the test management arena. 

 Table 3 - Testing Roles and Responsibilities

 

---

Test Management Approach

This section describes the approach for supporting testing and addresses the various types of testing and quality assurance activities, that occurs throughout the continuous delivery pipeline in getting releases out the door and into user’s hands. Testing is iterative, incremental, and continuous. This means that each increment of coding is tested as soon as it is finished. The team builds and tests a little bit of code, making sure it works correctly, and then moves on to next piece that needs to be built. This rapid turnaround of test results insures the product meets quality standards and facilitates moving the project forward. By providing feedback of test results on an on-going basis, developers are insured the product quality meets the business requirements.

Continuous Delivery Pipeline

As described in the CWDS Release Management Plan, the Continuous Delivery (CD) Pipeline (hereinafter referred to as the “pipeline”) represents the project’s software delivery process for delivering useful, high-quality, working software to users in an efficient, fast, reliable, and repeatable manner. The pipeline breaks down the software delivery process into stages. Each stage is aimed at verifying the quality of the product from a different perspective and prevent identified bugs from affecting users. Releases are not allowed to move forward to the next stage unless they pass all Stage Acceptance Criteria or have an approved exception.

Figure 1 – Logical Continuous Delivery Pipeline describes the logical migration path that a release candidate will take from one stage to the next on its journey to production (aka Live).

Figure 1 – Logical Continuous Delivery Pipeline

Figure 2 – Physical Continuous Delivery Pipeline describes the actual migration path that code will take from one stage (environment) to the next on its journey to production. The pipeline starts in the Development stage/environment with each development team building, testing and deploying code following a defined continuous integration (CI) process. At each subsequent stage in the CD pipeline, the application is tested to ensure that it meets all desired system qualities appropriate for the environment/stage that it is in. Only once the application passes each stage can the feature be released to production.

    Figure 2 – Physical Continuous Delivery Pipeline

Test Driven Development

During the Development/Build stage, each of the development teams are utilizing a software development process called Test Driven Development (TDD). TDD is a software development process that relies on the repetition of a very short development cycle: requirements are turned into very specific test cases, then the software is improved to pass the new tests, only. TDD uses the following steps:

• Write the test

• Run the test (there is no application code, test does not pass)

• Write just enough application code to make the test pass

• Run all tests (tests pass)

• Refactor

• Repeat

Step 1: Write the Test. In test-driven development, each new feature begins with writing a test to define a function or improvements of a function. To write a test, the developer must clearly understand the feature's specification and requirements. The developer can accomplish this through user stories to cover the requirements and exception conditions, and can write the test in whatever testing framework is appropriate to the software environment. It could be a modified version of an existing test. This is a differentiating feature of test-driven development versus writing unit tests after the code is written: it makes the developer focus on the requirements before writing the code, a subtle but important difference

Step 2: Run the test and see if it fails. This validates that the test harness is working correctly. A test harness is a collection of software and test data configured to test a program unit by running it under varying conditions and monitoring its behavior and outputs. Step 2 also shows that the new test does not pass without requiring new code because the required behavior already exists, and it rules out the possibility that the new test is flawed and will always pass. The new test should fail for the expected reason. This step increases the developer's confidence in the new test.

Step 3: Write the code. The next step is to write some code that causes the test to pass. The new code written at this stage may not be efficient or elegant, but the test is passed.” That is acceptable because it will be improved and honed in Step 5.

Step 4: Run tests. If all test cases now pass, the developer can be confident that the new code meets the test requirements, and does not break or degrade any existing features. If they do not, the new code must be adjusted until they do.

Step 5: Refactor Code. The growing code base must be cleaned up regularly during test-driven development. New code can be moved from where it was convenient for passing a test to where it more logically belongs. However, duplication must be removed. Object, class, module, variable and method names should clearly represent their current purpose and use, as extra functionality is added. As features are added, method bodies can get longer and other objects larger. They benefit from being split and their parts carefully named to improve readability and maintainability, which will be increasingly valuable later in the software lifecycle. Inheritance hierarchies may be rearranged to be more logical and helpful, and perhaps to benefit from recognized design patterns. There are specific and general guidelines for refactoring and for creating clean code. By continually re-running the test cases throughout each refactoring phase, the developer can be confident that process is not altering any existing functionality.

The concept of removing duplication is an important aspect of any software design. In this case, however, it also applies to the removal of any duplication between the test code and the production code—for example magic numbers or strings repeated in both to make the test pass in Step 3.

Step 6: Repeat. Starting with another new test, the cycle is then repeated to push forward the functionality. The size of the steps should always be small, with as few as 1 to 10 edits between each test run. If new code does not rapidly satisfy a new test, or other tests fail unexpectedly, the programmer should undo or revert in preference to excessive debugging. Continuous integration helps by providing revertible checkpoints. When using external libraries it is important not to make increments that are so small as to be effectively merely testing the library itself, unless there is some reason to believe that the library is buggy or is not sufficiently feature-complete to serve all the needs of the software under development.

Continuous Integration (CI)

A key component of the pipeline is Continuous Integration (CI), which is a software development process of integrating developers’ work frequently (at least daily) using an automated suite of tools.

There are three important and foundational elements that underpin any Continuous Integration system.

1. Automated Tests. The commitment of the development team to produce a comprehensive automated test suite at the unit level and functional level together with their code. It is essential that the automated tests are run with each build to identify potential bugs in a timely manner.

2. Never break the build. The goal is that the application must be ready to be built, packaged and deployed on every committed change. This means that broken or untested code should never be committed.

3. Version Control. The code has to be managed by strict version control policies which includes the practices of frequent commits to a version control repository, fixing broken builds immediately, and using a separate integration build machine.

Pair Programming

Another strategy for ensuring high-quality code is the approach of pair programming, which entails two programmers working together to complete a set of code. One, the driver, writes code while the other, the observer or navigator, reviews each line of code as it is typed in. The two programmers switch roles frequently. Each of the development teams are expected to implement some form of this in meeting their team’s sprint definition of done. The following is an example of the pair programming process that the Intake team is following

• A pair of Developers usually pick up story from top of Pivotal Tracker backlog.

• Assume all stories are prioritized and estimated in Pivotal tracker backlog

• Pair starts the story in the Pivotal Tracker

• Pair creates feature branch off latest master branch

• Completes story using Test Driven Development

• Submits pull request to Tech lead and peer developers

• Pull request is approved after code review by lead or peer developers

• Merge changes to master branch

• Pair finishes story in Pivotal Tracker

• Continuous Integration runs automated tests and deploys changes to acceptance environment

• If Continuous Integration is green, then pair delivers story in Pivotal Tracker

Quality Acceptance

Story Acceptance Criteria

Acceptance Criteria is the minimal amount of documentation needed to specify the expected behavior of each story/feature.

Continuous Delivery (CD) Stage Acceptance Criteria

CD Acceptance Criteria specifies the criteria that each software candidate release must meet in order to exit one of the CD stages and be promoted to the next stage. The criteria is different for each stage based on the types of testing that occurs in the stage. Details of each stage’s acceptance criteria can be found in the CWDS Release Management Plan.

Definition of Done

Each Scrum Team, along with the Product Owner and Scrum Master, commonly defines the “Definition of Done” to claim that something is completed (story, sprint, release). Usually it includes incremental product architectural design, feature design, development, unit testing, integration testing, functional testing, and documentation.

The Statement of Work (SOW) for each development team includes a requirement to define the sprint-level Definition of Done that includes the following concepts:

• Code produced (all ‘to do’ items in code completed)

• Code commented, checked in and run against current version in source control

• Peer reviewed (or produced with pair programming) and meeting development standards

• Builds without errors

• Unit tests written and passing

• Deployed to system test environment and passed system tests

• Passed Service Manager Acceptance Testing

• Any build/deployment/configuration changes Implemented/documented/ communicated

• Relevant documentation produced/updated (e.g., how the API services supports service module needs and user stories, user stories, sketches, wireframes, clickable prototypes)

• Remaining hours for task set to zero and task closed

Automated Testing 

All team members, in particular, developers should be involved in creating and maintaining automated tests, and should stop what they are doing and fix them whenever there is a failure. Furthermore, each of the digital service vendors are expected to implement as much automated testing as possible. The Statement of Work (SOW) for each development team includes the following requirements for automated testing:

• The Contractor shall create and execute automated unit testing

• The Contractor shall create and execute automated system tests to verify all Features of the software module

• The Contractor shall create and execute automated Service Manager Acceptance testing to verify all user facing functionality

• The Contractor shall run tests automatically on code merged into version control

• The Contractor shall create and execute automated integration testing with other contractor developed modules

• The Contractor shall ensure that all code will be written to a language specific code-style guideline (e.g., PEP8 for Python)
• The Contractor shall use an automated tool to evaluate the codebase and ensure compliance with the code-style guideline (e.g., if the Contractor uses Python, PyLint may be used)
• The Contractor shall use an automated tool that measures the amount of the codebase that is covered by tests (e.g., RCov may be used to measure test coverage of Ruby code)

Bug Tracking 

Currently, all bugs are entered into each team’s Pivotal Tracker icebox as a Bug story. The bug story should have the following information:

• Date Detected: MM/DD/YYYY

• Description: Explain the defect, what screen(s) the defect is found on.

• Steps: Describe steps taken to recreate the defect.

• Environment: e.g., Pre-Int, Integration, Performance

• Screenshots: If appropriate, attach screenshots

• Expectations: If known, enter the expected behavior

While it is up to each team to determine the triage process that works best for their team, every bug story must be assigned a one of the following PivotalTracker labels that classifies its severity:

• Bug - Critical: Severely limits the users' ability to perform their job. Must be fixed before releasing into Production.

• Bug - Moderate: limits the users' ability to perform their job. Possible workaround available. May need escalation to determine whether acceptable to release system into production without this bug being fixed. Timeline for fix identified.

• Bug - Minor: does not impact user's ability to do their job. Examples: typos, cosmetic errors, text misalignment, etc.

The Product Owner treats each bug like any other story in their backlog and determines its priority.

Test Reporting

Test reporting supports decision making and provides visibility to the project. The following are examples of the types of metrics that are reported in the Sprint Review Demonstrations:

• # of Bugs incurred in the sprint

• # of open bugs at the end of the sprint

• # of Bugs resolved in the sprint  

• # of unit tests run each day

• # of test written, run, and passed at each level (e.g., unit, functional, story, load)

• Code coverage

• Build duration

Each team maintain a dashboard for reporting all major test components and defining a threshold for each component within each digital service team.

Testing Tools

The CWDS front-end digital service teams (e.g., Intake) use the following tools to assist in testing and code analysis:

• RSpec for Behavior Driven Development improving on Test Driven Development and Unit Testing.
• RSpec with Selenium and Capybara for Integration testing (Behavior Driven Development) and Acceptance Testing
• Selenium with Capybara using RSpec for performance testing
• CodeClimate along with Code Review to Check Code Quality
• Jasmine for Test Driven Development and Unit testing JavaScript Code
• Karma for executing JavaScript Code in multiple real browsers and Test Drive JavaScript Code.
• RuboCop as static code analyzer that enforce guidelines outlined in community Ruby Style Guide
• ESLint for static analyzing JavaScript Code to find problematic patterns or code that doesn't adhere to certain style guidelines
• Capybara-accessible is being used for accessibility tests along with our Rspec integration tests which use Capybara, helping you to capture existing failures and prevent future regressions.
• simplecov is a code coverage tool used to analyze Ruby code.
• istanbul is a tool used to report on the level of javascript code coverage.

The CWDS back-end digital service teams (TPT.1) use the following tools to assist in testing and code analysis:

• SonarQube - Quality Check

• JMeter – Unit, Functional and Performance Testing

• JUnit – Unit Testing

• Cobertura – Code Test Coverage

• Unified Functional Testing (UFT) – Automated Legacy (CWS/CMS) Testing

Test Data

In the early sprint cycles, dependencies between components that are still under development may require mocking up of test data. Later on, in order to perform certain tests, such as performance testing, production user IDs or data may be needed. This section will identify when test data may need to be created and who is responsible for creating it.

The legacy CWS/CMS system maintains a predefined set of test data that will be leveraged. In addition, each digital service team may come up with their own set of test data for specific test scripts that they develop.

Testing Types

The types of testing that will support the CWDS digital service development can be broken out into different types of testing that accomplish different purposes. Next to each type of test listed below is the environment (e.g., Development, Pre-Int) where this testing is typically conducted.

Code Quality (Development)

In addition to following The Contractor shall use an automated tool to evaluate the codebase and ensure compliance with the code-style guideline (e.g., if the Contractor uses Python, PyLint may be used).

Unit Tests (Development)

Unit tests verify functionality of a small subset of the system, such as an object or method. Automated unit tests are created by the developers to validate the internal function of the software at the service or component level. They are added to the regression test suite within the CI process as they are developed. Each time new code is checked in, the CI executes the total suite of automated unit tests (regression tests), ensuring that any code that breaks the integration will be caught early in the process. The CI process is usually configured to send automated notifications as build processes are executed – especially failures, but any outcome and/or previous state can trigger a notification (Pass, Fail, Breaks, All, etc.).

System Tests (Development)

Automated system tests validate an entire module and will be executed automatically as part of the CI process after all unit tests are passed.

Code Coverage (Development)

Code coverage measures how much application code is executed by test scripts. Each digital service team uses automated code coverage tools as part of their CI build process to generate reports that indicated the amount of coverage. While the overall goal is to strive for high unit test coverage, 100% is not an expected outcome. The teams are expected to focus on tests in critical areas such as business logic and worry less about tests around supporting code such as Plain Old Java Objects (POJOs) for java-based code.

As an example, Intake’s Ruby code coverage can be found by navigating to the Intake CI and clicking on 'Ruby Code Coverage'. Their javascript code coverage can be found by selecting the 'JS Code Coverage'.

Accessibility Tests (Development)

Accessibility testing is a subset of usability testing where the users have disabilities that affect how they use the application. Accessibility testing will be done throughout the design and development process ensure the system meets Section 508 of the Rehabilitation Act’s most recent guidelines (which as of Jan 18, 2017 incorporates WCAG 2.0 guidelines). This includes (but is not limited to) ensuring visibility, keyboard navigability, language clarity, and screen reader compatibility.

As stated in the project’s Research-Design wiki page, the project will apply the following design strategy:

• All components of the CWDS Pattern Library meet accessibility requirements. Refer to the 18F Accessibility Guide Checklist.

• Components combined together into page designs are tested holistically to ensure accessibility is maintained.

• Regular usability testing is conducted with disabled users.

• Any gaps in usability will be identified and corrected, or tagged with “accessibility-needs” in our product backlogs and prioritized.

• Periodic review and recommendations by CA Dept of Rehabilitation (DOR).

In addition, the front-end development teams will use an automated accessibility testing tool (Pa11y) to validate compliance as part of their CI build process.

Smoke Tests (All)

Smoke tests, also known as “Build Verification Tests”, are comprised of a non-exhaustive set of tests that aim at ensuring that the most important functions work. The results of this type of testing is used to decide if a build is stable enough to proceed with further testing. Automated smoke tests will be included with the deployment of new builds to each of the stages/environments.

Integration Tests (Pre-Int and Integration)

Integration tests verify the interaction between code components. They run without the intervention of the GUI web interface.

Acceptance Tests (Pre-Int and Integration)

Acceptance tests define the expected behavior for the code to be delivered by the story during the sprint. They are scenarios which are derived from acceptance criteria outlined in the user stories. Each acceptance criteria can have one or more acceptance tests. There must be at least one test case created for 100% of the approved acceptance criteria. Testers will also test for a negative and positive outcome for each acceptance criteria. All test cases and subsequent test scripts and expected results are directly related to at least one acceptance criteria for each user story. Acceptance testing best practices include the following:

• Becomes “living” documentation about how a piece of functionality works

• Steps for developing acceptance tests include the following:

  • Write high-level test cases with examples of desired and undesired behavior before coding

    • Start discussions about features and stories with a realistic example

    • Identify which features are critical and which ones can wait

    • Start with happy path

    • Identify all of the parts of the system that might be affected by a story

    • Review test cases with customers

  • Sit down with developers to review the high priority stories and their associated test cases

  • As soon as testable chunks of code are available and the automated tests pass, explore the functionality more deeply by trying different scenarios and learning more about the code’s behavior. The story is not “done until all these types of tests are complete

  • Write detailed test cases once coding starts

• Additional acceptance testing best practices include the following:

  • Identify high-risk areas and make sure that code is written to solidify those risk areas

  • Ensure that the most obvious use-case is working first before tackling challenging areas

  • Understand the domain and the impact of each story. Don’t forget the big picture.

  • Confine each test to one business rule or condition

  • If a developer has started working on a story, there should be a tester working on the testing tasks for that story

  • Time-box the time spent writing tests