Since 1994

A test team's capability

is not a single number.

Four roles, three domains, 137 rows, the matrix that makes gaps and redundancy visible. · Rex Black, Inc.

"How strong is the team?"

is the wrong question.


The right one: where is nobody capable?
And: where is the team so thin one resignation breaks coverage?

The skills matrix answers both, row by row.

REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
What this is about

In plain English.


A skills matrix is one row per skill, one column per team member, and a 0–3 rating in every cell. Two roll-up columns matter more than anything else: team minimum (weak-link view) and team average (depth view).

Use the two roll-ups to:

  • Write hiring specs that mean something, required skills are the rows where team minimum is 0.
  • Spot emergencies before they happen, thin-depth skills are one resignation from a crisis.
  • Steer development, quarterly delta shows the team's capability trajectory.

This template has ~137 rows across three domains and four roles. Adapt every row; keep the structure.

REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
The structure

Three domains.

Testing Skills

The role-specific craft. Test design, estimation, execution discipline, bug reporting, metrics.

~31 rows.

Domain Knowledge

Customer / product empathy. Product area coverage, workflows, regulatory understanding.

15–25 rows, fully customizable.

Technical Expertise. Programming, OS, networking, cloud, security, observability, AI/ML literacy. The 2026 domain most commonly underinvested.

REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
The structure

Four roles.


  • TT, Test Technician. Execution-heavy, low autonomy, growing into MTE or ATE.
  • MTE, Manual Test Engineer. Design + exploratory + results-reporting autonomy.
  • ATE, Automated Test Engineer. Automation craft, CI/CD integration, tool-smithing.
  • TM, Test Manager. Planning, estimation, risk analysis, people, policy.

Role minimums (R / D) per row tell the recruiter what "required" means. Desirables are negotiable.

REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
The scale

Zero. One. Two.

Three.

REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
The scale · 0 through 3

Deliberately coarse.


  • 0, I cannot do this.
  • 1, I have done this with support.
  • 2, I can do this unsupervised.
  • 3, I can teach this to others.

Finer scales (5-point, 10-point) invite calibration disputes without adding signal. 0–3 is load-bearing. Don't "improve" it.

REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
The scale · R / D

Required. Desirable.

Required, R

A role-player below R on a required skill needs training or re-assignment. Non-negotiable for autonomy.

Desirable, D

Expands the role. A role-player who clears many desirables is promotion-track.

The R / D distinction is what keeps the matrix from collapsing into a single averaged number.

REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
Two roll-ups

Team minimum.

Team average.

REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
Roll-up · team minimum

The weak-link view.


  • Team minimum = 0 on an R skill for that role: nobody can cover this skill.
  • Surfaces as emergencies. "Nobody knows our performance-testing stack. Alice is on leave."
  • Fix by cross-training or hiring for it explicitly.

Red squares on the matrix. Triage first, every quarter.

REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
Roll-up · team average

The depth view.


  • Team average ≤ 1.5 on a broadly-required skill: skill is technically present but thin.
  • One resignation away from becoming a team minimum = 0.
  • Fix before it becomes a crisis: assignments, paired work, training, stretch projects.

Yellow squares on the matrix. Invest before they turn red.

REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
2026 rewrites

The Technical Expertise domain

has to catch up.

REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
Technical · 2026 substitutions

Five new row categories.


  • Cloud platform literacy, AWS / GCP / Azure basics for the platform you actually ship on.
  • Observability stack, OTel, Datadog, Honeycomb, span correctness, dashboard accuracy.
  • Security fundamentals, OWASP Top 10, SAST/DAST tooling, IAM basics, SBOM literacy.
  • AI/ML evaluation, held-out eval sets, golden sets, calibration, slice-based metrics, prompt-injection testing.
  • Contract & property-based testing, Pact / Spring Cloud Contract, property-based for pure functions, consumer-driven contracts.

The 1998–2010 test profession grew a rich Testing-axis skill base. Most teams under-invested in the Technical axis. Funding this axis outperforms funding more of the same.

REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
Testing · 2026 additions

Two new testing-craft rows.

AI-system evaluation

Probabilistic-output testing. Eval-harness design. Held-out set construction. Slice-based metrics. Adversarial input coverage.

Property-based / fuzz testing

Generator-driven test authoring. Shrinking. Invariant-based testing. Different skill from example-based assertion testing.

Both rows: Desirable for MTE, Required at 2+ for ATE on AI-backed products.

REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
Cadence

Quarterly. Not annual.

REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
Cadence · why quarterly

The matrix is maintained, not filed.


  • Annual ritual = filed artifact. Loses signal. Becomes an HR exercise.
  • Quarterly = planning artifact. Captures growth (and sometimes decline when a skill is no longer exercised).
  • Four-quarter trend = team-health telemetry. Rising team average on Technical = your development plan is working.

30 minutes per person per quarter after the first 90-minute baseline. Cheapest capability telemetry in the organization.

REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
How to use it

Five steps.

REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
How to use it · 1–3

Customize, minimums, ratings.


  1. Customize the row set. Drop rows that don't apply (no mainframe? drop the mainframe row). Add rows for your 2026 stack (Testcontainers, Playwright, Pact, OTel, LLM eval).
  2. Set per-role minimums (R / D). If your role labels differ (SDET / QE / Principal QE), rename the columns, keep the minimum-setting discipline.
  3. Rate every team member on every row. Blank = "not yet rated." 0 = "no knowledge." First time: 90 minutes per person. Quarterly updates: 30.
REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
How to use it · 4–5

Triage and act.


  1. Roll up team minimum and team average for every row. Highlight reds (min = 0 on R) and yellows (min = 1 on R, or average ≤ 1.5 on broadly-required skills).
  2. Act on the reds first. Cross-train. Hire. Pair. Yellows become the next quarter's learning goals and stretch assignments.
REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
Failure mode

"We did it last year."


An annual snapshot is not a skills matrix.
It is a PDF in a drawer.

REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
Takeaways

Four. That's the discipline.

REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
Takeaways · 1 of 2

Role minimums, not averages.


  • Hiring specs come from role minimums. "Required" = rows marked R at 2+ for the role.
  • Team minimum is the weak-link view. A zero here = next emergency.
  • Team average is the depth view. A 1.0 here = one resignation from a zero.
REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
Takeaways · 2 of 2

Quarterly. Or it dies.


  • Annual ritual collapses the signal. Quarterly captures growth and turns the matrix into capability telemetry.
  • Technical axis is the 2026 gap. Cloud / observability / security / AI eval, the teams that fund this outperform.
  • Two actions per quarter. Reds first (triage). Yellows second (invest).
REX BLACK, INC. · TEST TEAM SKILLS ASSESSMENT
Since 1994

Thank you.

Rex Black, Inc. · rexblack.com/resources/qa-library/test-team-skills-assessment