Renato Garay
EN / ES

Labs / Lab 003

Does a design skill beat using no skill at all?

2 min read Claude Code workflows · Evals Leer en español

Example lab with invented data. It exists only to preview the design and must not be published.

Question
Does a design skill beat using no skill at all?
Verdict
Inconclusive. The gap between skills was smaller than the gap between two runs of the same skill.
How it was measured
6 skills and a no-skill control, 3 runs each, blind vote by 40 people.
Repository
github.com/renatogm24/lab-003-design-skills

Hypothesis

Design skills promise better-looking pages from the same prompt. If that holds, a page built with a skill should win a blind vote against a page built with none, and the ranking of skills should stay roughly the same from one run to the next.

Setup

  • One brief: a landing page for a Saturday workshop in Lima, written in Spanish.
  • Seven conditions: six public design skills and a control with no skill.
  • Three runs per condition, same model and settings, fresh session each time.
  • Forty people voted blind for the best page in each of three rounds.

Results

Share of blind votes. The dot is the mean of three runs; the line runs from the worst run to the best.
ConditionMeanRangeSpread (pts)
Skill A17%9–24%15
Skill B16%11–22%11
Skill C15%6–25%19
No skill14%10–19%9
Skill D14%8–20%12
Skill E13%9–18%9
Skill F11%5–17%12

The best and worst means are 6 points apart. The average spread between runs of the same condition is 12 points. The control sits in the middle of the table.

What this means

With three runs I cannot tell these skills apart, and I cannot say that any of them beats using none. A single run would have produced a confident ranking, and a different one each time.

Limits

  • One brief and one model. A different task may separate the skills.
  • Three runs is a small sample; ten per condition would narrow the ranges.
  • Voters were acquaintances, not the audience of the page.

Reproduce it

git clone https://github.com/renatogm24/lab-003-design-skills
cd lab-003-design-skills
make run RUNS=3     # builds the 21 pages
make vote           # serves the blind voting page