Since 1994

Create your own luck.

The four organizational factors that decide whether a test team succeeds · Rex Black, Inc.

There is no luck

in testing.


Every successful test team has four things in common.
Every failed one is missing at least two.


None of them are about your stack, your domain, or your team size.
All four are under the test lead's influence.

REX BLACK, INC. · CREATE YOUR OWN LUCK
What this is about

In plain English.


Why do some test teams look lucky — cooperative developers, early involvement, tools that just work? And others look cursed no matter how hard they try?


This talk identifies the four organizational factors that separate the two:

  • Who owns what. Clear roles and hand-offs.
  • When testing gets started. Day one or too late.
  • Whether tools and artifacts are shared between dev and test.
  • Whether leadership actually values what testing delivers.

Written for test managers, test leads, and QA directors. Useful for any leader deciding how to staff or scope a test effort.

REX BLACK, INC. · CREATE YOUR OWN LUCK
Factor 01 of 04

Clearly defined

roles and interfaces.

REX BLACK, INC. · CREATE YOUR OWN LUCK
Factor 01 · Roles

Test teams depend on everyone.


SUT from developers · environment from sysadmin · bug hand-offs to developers · status reports to PMs.


When interfaces are undefined, chaos follows. Missed hand-offs. Blame games. A test team that quietly absorbs everyone else's failures.

  • Define roles and interfaces in the test plan.
  • Build support for those roles before they're needed.
  • Reinforce the boundaries tactfully. Do not absorb responsibility that belongs somewhere else.
REX BLACK, INC. · CREATE YOUR OWN LUCK
Factor 01 · Two engagements

Same factor. Opposite outcomes.

Undefined (bad)

No one owned installable releases from source. Test manager took it by default because she needed builds. Added workload made planned testing impossible. Test effort blamed for upstream failure.

Defined (good)

Different test manager defined release + environment processes up front. Builds arrived on time. Install issues were mutual problems, not finger-pointed. No unanticipated lab downtime.

REX BLACK, INC. · CREATE YOUR OWN LUCK
Factor 02 of 04

Early test-team

involvement.

REX BLACK, INC. · CREATE YOUR OWN LUCK
Factor 02 · Early involvement

The cost of fixing rises

throughout the project.


Some test work takes a long time. Quality risk analysis. Test tools. Test harnesses. Environment build-out.


Relationships are easier to build under low stress than under deadline pressure. All of it argues for day-one engagement.

  • Brief the project on advantages of early involvement.
  • Start test-team work on day one — not integration, not system test.
  • Have the test team review requirements and design. Finding errors there is the cheapest finding you will ever do.
REX BLACK, INC. · CREATE YOUR OWN LUCK
Factor 02 · Two engagements

Timing decides contribution.

Late (bad)

Test manager occupied with prior maintenance releases. Free time spent haranguing process. Testers didn't engage with the new release until integration was in progress. Seen as an extraneous distraction.

Up-front (good)

Two test engineers allocated at start of development. Reviewed early specs, found ~100 errors before a single module shipped. Full test context ready on day one of integration test. Seen as a major player.

REX BLACK, INC. · CREATE YOUR OWN LUCK
Factor 03 of 04

Shared tools,

cases, and data.

REX BLACK, INC. · CREATE YOUR OWN LUCK
Factor 03 · Shared artifacts

Shared artifacts compound.

Unshared ones get rebuilt.


Test tools, cases, and data take real effort. When artifacts built for unit or component testing can be leveraged at integration and system test, that effort compounds.


When they can't — because each team built their own harness in isolation — you will rebuild everything. The rebuild never quite catches up.

  • Test engineers work alongside developers on unit and component tests.
  • Design tools, cases, and data for re-use from the start.
  • Build harnesses out of whatever your team can actually maintain.
REX BLACK, INC. · CREATE YOUR OWN LUCK
Factor 03 · Two engagements

Duplication vs. collaboration.

Unshared (bad)

One dev team built a unit-test harness with no test participation — special-skill tools, not reusable. Another built a load generator over test objections — too intrusive for performance testing. Over a person-year of duplicated effort.

Shared (good)

Dev + test engineer adapted a harness together for nightly smoke. 200+ queries, baseline comparison, emailed report to both teams. Regressions greatly reduced. Test cycles shortened. Monthly maintenance releases possible.

REX BLACK, INC. · CREATE YOUR OWN LUCK
Factor 04 of 04

A culture

that values testing.

REX BLACK, INC. · CREATE YOUR OWN LUCK
Factor 04 · Culture

Culture is sold, not given.


Test resources are almost always insufficient. Expectations about what testing actually delivers are almost always unclear.


A test-friendly culture promotes important testing, adequate resources, and use of test results for project tracking. It is not optional.

  • Pitch in their language: risk, cost, schedule, quality — not test-case counts.
  • Distinguish assessing quality from assuring it. You measure the product, you don't insure it.
  • Involve the right stakeholders in test design and planning.
  • Use FMEA / ISO 9126 so priority is data-driven, not anecdotal.
REX BLACK, INC. · CREATE YOUR OWN LUCK
Factor 04 · Two engagements

The expectations mismatch.

Unrealistic (bad)

Client wanted a test engineer to plan test process, create test context, build automated harnesses for web and legacy, train developers — at half her usual rate, in six weeks. Executive called his test manager "the QA manager" and expected testing to make quality problems go away.

Pervasive (good)

Different test manager clarified expectations first. Used QRA with the project team to define scope. Dev + test built test tools, data, and cases together. Marketing, support, and dev helped define "correct." Test dashboard became the project's quality indicator. Test exit criteria became the ship criteria.

REX BLACK, INC. · CREATE YOUR OWN LUCK
Takeaways

Four factors.

No luck required.

REX BLACK, INC. · CREATE YOUR OWN LUCK
Takeaways · 1 of 2

Clarity and timing.


  • Context-independent. Stack, domain, team size don't change the four factors.
  • Clarity is the test lead's job. No one writes your role definitions for you.
  • Get in on day one. Every week late costs a week of leverage.
REX BLACK, INC. · CREATE YOUR OWN LUCK
Takeaways · 2 of 2

Shared work, sold mission.


  • Shared artifacts compound. Rebuilt ones never catch up.
  • Sell the testing mission. In their language, every quarter, forever.
  • Plan, persuade, repeat.

There's no luck involved at all — just careful planning, calm and reasoned persuasion, and lots of attention to organizational details.

REX BLACK, INC. · CREATE YOUR OWN LUCK
Since 1994

Thank you.

Rex Black, Inc. · rexblack.com/resources/talks/create-your-own-luck