Since 1994

Shoestring

manual testing.

When you can't afford automation, yet · Rex Black, Inc.

Total automation

is a mirage.


For most real projects, at least one obstacle kills the dream of "start the test, leave it running, come back for results."


The answer is not to give up on testing.
The answer is a well-run manual program.

REX BLACK, INC. · SHOESTRING MANUAL TESTING
What this is about

In plain English.


You don't have the budget, tools, headcount, or stable system under test that automation needs. The product ships anyway. How do you run real testing on a shoestring?


This talk answers:

  • How to size the manual effort honestly — with two rules of thumb your CFO will believe.
  • Where to find good testers cheaply — and what each hiring pool's weakness is.
  • How to train, shift, and supervise a lean manual team.
  • Which tests you cannot run manually, no matter how tight the budget.

Written for founders, engineering leads, and QA managers at small or early-stage companies.

REX BLACK, INC. · SHOESTRING MANUAL TESTING
Manual vs. automated

The field test.

Automated

Tester starts the run, walks away, comes back for results.
Unattended execution. Machine compares actual to expected.

Manual

Tester enters data, observes behaviors, actively runs the tests.
Tools are fine. Tools ≠ automation. Unattended is automation.

REX BLACK, INC. · SHOESTRING MANUAL TESTING
Why automation fails teams

Five obstacles.


  • SUT changes frequently — scripts break faster than you write them.
  • No budget or calendar for script development.
  • No suitable tool at a justifiable price.
  • Team lacks automation skill or experience.
  • Critical risks aren't amenable. Usability, localization, error handling, installation, docs — cheaper and more honest manually.
REX BLACK, INC. · SHOESTRING MANUAL TESTING
Sizing

Two rules of thumb.

Nobody wants to hear them.

REX BLACK, INC. · SHOESTRING MANUAL TESTING
Rule 1

Six productive test hours

per 8–10 hour day.


Testers do not test for eight hours.
Testers test for six, and spend the other two fighting their environment.


Bug reporting, status documentation, communication, breaks, blocking issues, debugging — it's part of the job, not an excuse.

REX BLACK, INC. · SHOESTRING MANUAL TESTING
Rule 2

25% downtime on good builds.

75% downtime on bad ones.


Four days of calendar to get one day of test coverage, on a bad build.


Budget the worst case up front. Do not promise a schedule without it.

REX BLACK, INC. · SHOESTRING MANUAL TESTING
Team shape

One manager.

A few engineers.

Many technicians.

REX BLACK, INC. · SHOESTRING MANUAL TESTING
Team structure

Three roles. Leveraged.

Test manager

Overall responsibility for the team, the schedule, and the release recommendation. One per program.

Test engineers

Technical leadership. Design tests. Troubleshoot hardest failures. Supervise several technicians each.

Test technicians

Most of the actual execution. Where the coverage comes from.

The leverage

One or two engineers ramp and supervise several technicians — coverage expands without tripling salary cost.

Career ladder: technician → engineer.

REX BLACK, INC. · SHOESTRING MANUAL TESTING
Hiring

Four cheap sources.

Students (CS / engineering)

Eager. Cheap.
Sometimes unavailable.

Support crossovers

Understand customer problems.
Sometimes too solution-focused.

Moonlighters

Real experience from other jobs.
Sometimes distracted.

Data-entry staff

Fluent computer users.
Sometimes low-curiosity.


Pick the mix that fits your product and your office culture.

REX BLACK, INC. · SHOESTRING MANUAL TESTING
Training

Universal skills vs. company rituals.

Universal — train hard

Manual test-case execution.
Bug reporting.

Written docs, training, coaching.
Every technician. Every source.

Company-specific — document

Network login, badge, email, phone system, test repo.

Documentation or a mentor.
Not a skill. A ritual.


The technician who cannot reproduce and document a bug is broken in a way a second badge tour does not fix.

REX BLACK, INC. · SHOESTRING MANUAL TESTING
The single most important skill

Ten-step bug report.


Every technician has to be fluent in one thing above all others.


Investigation half: structure · reproduce · isolate · generalize · compare.
Writing half: summarize · condense · disambiguate · neutralize · review.


Grammar and spelling count. A poorly written report can render a technically correct one ridiculous.

See /resources/qa-library/bug-reporting-process for the full checklist.

REX BLACK, INC. · SHOESTRING MANUAL TESTING
Multiple shifts

Same salary envelope.

More coverage.


Night (~4 PM–1 AM) and graveyard (~midnight–9 AM) shifts stretch a shoestring team.


Why

  • Schedule — more tests per day.
  • Resources — one hardware platform covers two or three eight-hour blocks.
  • Preference — some technicians genuinely want off-hours work.
  • Politics — coverage without burning out day shift.

Get it right

  • Explicit at interview.
  • Undesirable-hours bonus.
  • Overlap shifts for hand-offs.
  • Invest in team atmosphere — out of sight is out of mind until the first conflict.
REX BLACK, INC. · SHOESTRING MANUAL TESTING
What NOT to run manually

The line you do not cross.

Manual works

Functional · use cases · error handling · localization · UI · configs · operations · date/time · installation · documentation.

Humans judge better than machines here.

Automation-only

Monkey/random · load/volume/capacity · reliability/MTBF · code coverage · performance · standards compliance.

Running these manually either doesn't work or misleads people about coverage.


Even a shoestring program buys or builds some automation for the right column.

REX BLACK, INC. · SHOESTRING MANUAL TESTING
Sell it to management

It's a solution, not a concession.


Bring a plan, not a request for more money.


  • Flexibility and responsiveness — technicians ramp fast.
  • Cost and coverage — cheaper bodies = more coverage on the same budget.
  • Schedule — test cycle time goes down.
  • Staff retention — test engineers burn out less when they have technicians to delegate to.
  • Career development — helps inexperienced people jump-start technical careers.
REX BLACK, INC. · SHOESTRING MANUAL TESTING
Takeaways

Enough signal to ship safely.

That is the goal.

REX BLACK, INC. · SHOESTRING MANUAL TESTING
Takeaways

Four to hold against.


  • Total automation is a goal, not a precondition. Manual program bridges "nothing" and "right automation, right time."
  • Budget the overhead before the test count. 6 hours, 75% downtime. Plan them in.
  • Leverage from the org shape. Manager + engineers + many technicians. Flat senior teams run out of money before tests.
  • Pick manual for what humans judge best. Usability stays manual. Load stays automated. Even on a shoestring.

Using manual testing inappropriately can mislead people about the extent of test coverage. Using automation inappropriately usually just fails.

REX BLACK, INC. · SHOESTRING MANUAL TESTING
Since 1994

Thank you.

Rex Black, Inc. · rexblack.com/resources/talks/shoestring-manual-testing