When to stop searching and choose
Better decisions increase the chance of good outcomes
There are times when we want to find someone to undertake a job. We may be seeking, for example, a cleaner for our house or recruiting a team member at work. Let’s imagine we advertise a role and get 20 applicants. We would like to employ the best candidate but interviewing all 20 would take much time and effort. Let’s assume we interview candidates in random order and after each interview either offer the candidate the job or turn them down. How can we optimise the process?
37% Rule
To illustrate the challenge we face, there are two ways we could reduce the chance of selecting the best candidate:
Offer early: We miss out on potentially better candidates (that are not interviewed).
Offer late: We rejected the best candidate (interviewed earlier on).
Between these two ends of the spectrum lies the optimal solution, i.e. has the greatest chance of hiring the best person. This is known as the 37% Rule which has two components:
Evaluate 37% of options with no intention of selecting any of them.
Pick the next option that’s better than all the options sampled before.
When choosing between 20 candidates for a job, we should interview the first seven people (37% * 20 = 7.4) with no intention of hiring any of them. We then continue interviewing the remaining 13 and offer the job to the first candidate that is better than the first seven.
Optimal stopping problem variants
Determining whether a decision is good or bad means examining the quality of the beliefs informing the decision, the available options, and how the future might turn out given any choice you make. - Annie Duke
Deciding when to offer a job in an interview process is an example of an optimal stopping problem. Other examples include: finding a parking space, trading shares, house hunting and even choosing a spouse.
Michael Trick, a Mathematics graduate, applied the 37% Rule when selecting a potential wife. He reasoned that he would date between the ages of 18 and 40. Applying the 37% Rule, at age 26 he proposed marriage to his then girlfriend, who he considered was better than the previous best. However, she turned him down.
How can you tell if a Mathematician is an extrovert? He’s the one looking at your shoes, rather than his own.
The 37% Rule provides an opportunity to set a benchmark on which to base a final decision. However, as Trick found, it does not guarantee the desired result. What it does give is the best chance (which is 37%, again) of getting the best outcome.
The 37% Rule is valid if we can’t return to previous options which is often true when dating, but not hiring. If we believe there is, say, a 50% chance that past candidates will still accept our offer, i.e. they didn’t lose interest, then the 37% Rule becomes the 61% Rule. Here, we hold off offering until we interview 12 candidates (61% * 20 = 12.2). If we don’t find a better candidate in the remaining 8 then we go back and offer to the best of the 20 people interviewed.
Other resources
Algorithms to Live By talk by Brian Christian and Tom Griffiths
Better Decisions in 6 Steps post by Phil Martin
Getting Our Target Price post by Phil Martin
Mathematicians may not be the top choice for your next dinner party, but they can talk a good algorithm.
Have fun.
Phil…
Very interesting article 😀