Process · Test Execution

Run the tests, capture the data, adjust daily.Eight steps for turning a test release into a verdict on quality.

Test execution is where the risk register becomes results. These eight steps cover selection, assignment, the per-test inner loop, blocking-issue resolution, and the daily replanning that keeps a cycle on the rails.

Sub-steps
13
Inner loop stages
6
Replanning cadence
Daily
A test cycle that is not being re-planned daily is a test cycle that is already out of date.

Key takeaways

Four things to remember.

  1. 01

    Select by risk, not by habit

    Every cycle starts with a fresh selection decision. The suites that were right last cycle may not be right this one.

  2. 02

    The per-test inner loop is six steps

    Set up initial state, provoke a condition, observe, report product bugs, resolve test system bugs, capture the record. Run it with discipline on every test.

  3. 03

    Status daily; replan daily

    Report, adjust assignments, reconsider plans and priorities every day. Anything less frequent and the cycle drifts.

  4. 04

    Drop in reverse-priority order

    When the schedule compresses, cut from the bottom of the risk register, not from whatever tests are in front of you. Make the compression visible.

Why this exists

The problem this process fixes.

Test execution is the most visible phase of the testing process, and also the most mismanaged. Teams execute without a selection strategy, skip steps in the per-test loop, fail to surface blockers, and present status once a week instead of once a day.

These eight steps make that failure mode unavailable. Selection, assignment, the six-step per-test inner loop, and daily replanning are all made explicit, so that "we are in execution" means a specific, observable set of activities rather than a generic label.

The checklist

14 steps, in order.

  1. 1

    Based on risk prioritization, project constraints, and any other pertinent considerations, select the test suites (from the test set) that should be run in this test cycle.

  2. 2

    Assign the test cases in each test suite to testers for execution.

  3. Phase 3

    Execute the test cases, report bugs, and capture information about the tests continuously, taking into account previous test results for each subsequent test.

  4. 3.A

    Put the system under test and the test system into appropriate initial states. If this initial state is useful across multiple tests or multiple iterations of this test, save the initial states for subsequent re-use.

  5. 3.B

    Through data inputs and other stimulus, provoke the system under test into a desired test condition.

  6. 3.C

    Observe and evaluate the resulting outputs, behaviors, and states. Research any deviations from expected results.

  7. 3.D

    If appropriate, report problems in the system under test.

  8. 3.E

    If appropriate, report and/or resolve problems in the test system.

  9. 3.F

    Capture and report information about the test just executed.

  10. 4

    Resolve blocking issues as they arise.

  11. 5

    Report status, adjust assignments, and reconsider plans and priorities daily.

  12. 6

    If appropriate, eliminate unrealizable or redundant tests in reverse-priority order (drop lowest priority tests first, highest priority tests last).

  13. 7

    Periodically report test cycle findings and status.

  14. 8

    Check any status documents, initial states, updated testware or other test system elements, or other useful permanent records produced into the project library or configuration management system. Place the item(s) under change control.

One more thing

Most execution problems are actually discipline problems. The selection, the inner loop, the daily replanning, and the drop-in-reverse-priority rule are all cheap to do once and expensive to forget. Make them habits and the cycle pays for itself in the time you stop losing to churn.

Take it with you

Download the piece you just read.

The library is free. Tell us who you are so we can send you updated versions. One form; this browser remembers you after that.

In the library

Pair this with.

Keep reading

Related reading

Practices

Where this leads

Working on something like this?Talk to the people who wrote it.

Book a call