- Using data from MidlandWingsFan74's Fan Post I estimate the probability of Detroit drafting each of nine prospects in the draft. A table of results is presented below, but don't take it too seriously.
- Every prospect who is finally picked between the third and tenth spots currently has a significant chance of landing about five spaces higher or lower.
- The League should consider arranging anonymous submission by each team of a list of its prospect ranking. The data could be used for a deterministic matchmaking process that would encourage pick-swapping and more efficient use of assets at the draft.
- In general, a matchmaking process would result in more teams getting prospects that are higher on their lists.
- The matchmaking process is only realistic for the first couple of rounds because it is doubtful that all teams could provide an ordered list ranking more than about 100 prospects.
- A defect of the matchmaking process is that the team picking first has no motivation to rank the next few prospects after the one it plans to choose. This problem could be solved by conducting the lottery after lists are submitted.
Well, somebody has to arrange the matches,
(Prospects) can't decide these things themselves.
Once upon a time I accepted a commission with the US Air Force. When it was time to go on active duty, I was allowed to request preferred assignments but there were no guarantees. Luckily, I got a research position that had been my first choice. I later learned that most of my peers thought the assignment process must involve dart boards. Few were happy about where they had been sent.
A few years later I was offered a position in a department of ophthalmology at a medical school. There I learned that the new residents were assigned to the program in a computer-optimized match process. Each year nearly a thousand young medical doctors decide they would like to be ophthalmologists and there are only enough spots in training programs for about half of them. The match process was established to provide an efficient way to help those programs fill their open positions with residents who want to be there. Although a few applicants are not happy with the results, the system seems to work reasonably well, allowing programs and residents more time to plan and prepare than the old "apply and wait" system.
What does this have to do with hockey and the Red Wings? As I slogged through daily posts about draft picks here and elsewhere, I realized that we are speculating about a matchmaking process with a twist: the person being matched has no say in his selection. The NHL established and owns the matchmaking process and as a League of self-interested profit-seekers, it established rules to satisfy each other that no member team is getting an unfair advantage. It works but it is not very efficient. In this post I will show that they could do a better job of optimizing the happiness of the teams and improve the utilization of assets. I also think there could be some latitude to allow prospects to rank the teams they would like to join.
Matchmaker, Matchmaker Make Me Match
There is less certainty about the position where a prospect will be picked than pundits and enthusiasts would like us to believe. Although scouts and managers try to do their best to measure how good each prospect is, it is not feasible to learn so much that everyone comes to the same opinion. At the draft there will be 31 different team appraisals of each prospect. Since there is no information about how other teams have ranked prospects, trading for position is potentially a very risky process for the team that moves down because they have little sense of whether they will be able to get a prospect they like at that spot. This information inefficiency leaves many teams less happy than they could be. One result is that we often are surprised by how high or low some prospects are drafted.
Let's start with the evaluation process and how it leads to a bigger spread of draft choices than you might expect. Recently MidlandWingsFan74 posted a ranking of more than fifty prospects for the 2019 draft, based on nineteen sources[i]. Luckily for me, the post included what I presume is an average rank ("consensus"), plus the highest and lowest. This was ideal for my requirements. All I needed to do is transpose the data to estimate how often the top prospects appear in ratings at each place in the draft order. Instead of knowing what the range of likely draft spots is for each prospect, we need to know how frequently each prospect shows up in estimates for each draft spot.
As amateur draft enthusiasts we must assume that the rankings we rely on are proxies for the range of rankings that teams actually use (although I'm not confident that this is true). Then we must assume that every team makes a list of "dance cards" for the draft and tosses out cards as prospects are taken by teams. We never get to see their lists so it all looks a bit random as we watch the picks unfold. It may not be obvious, but if my assumptions are true the selection process is actually deterministic[ii]: once all 31 lists are completed, before the draft takes place, the order of picking is predetermined, unless teams swap picks while the draft takes place.[iii] More on that later.
This deterministic process looks stochastic[iv]only because we lack information. Because we are making guesses, we can only assign probabilities that a particular prospect will be taken in a given spot during the draft. I'm not going to go into all the math but I took Midland's data and used them to define goals and boundary conditions in a calculation to transpose and redistribute the prospect ranking data. This is what the stacks of dance cards of the 31 teams would look like for the third through sixth picks if we could get a tally from all the teams, assuming that they assess prospects according to the data samples provided by Midland's 19 sources.
Stacks of Anonymous Team Rankings of Prospects by Rank Number
I think it is worth talking about a couple of examples to show how the transposition worked. In Midland's post, Bowen Byram ranked from 3rd to 10th with a consensus of about 4.7. Although 4.7 is nearly 5, the 5th spot is not his most common rank among teams. This is because his 4.7 average rank is the third highest among all prospects. Because Hughes and Kakko take up all the 1st and 2nd spots, Byram's rank could only be as high as 3rd and so he got a lot of those. The calculation process I use ensures that all the prospects Midland listed were distributed with the maximum, minimum and average values as close as possible to his samples, while ensuring that there are exactly 31 prospect cards at every rank. This is because each of the 31 NHL teams has to have somebody ranked 3rd, 4th, 5th and so on. I colored the cards for each prospect in this graphic, and you can see that Byram (in green) has nine 3rd ranks, seven 4th ranks, five 5th ranks and three 6th ranks. In contrast, Peyton Krebs ended up with one card at both the 3rd and 4th rank, two at the 5th and three at the 6th. This is because his highest rank was 5th, lowest was 13th and average was 8.6 - his distribution among team dance cards actually peaked at 9th and 10th (not shown here).
One of the prospects for each rank is called Wildcard. This is because a stochastic model provides an estimate without 100% certainty. I set the uncertainty level at about 3% by assuming for any draft pick that one NHL team has a prospect who wasn't ranked there by any other team. Had I written about this last year, Arizona's selection of Barrett Hayton would have been a wild card pick.
I then ran through every possible selection sequence to establish the frequency of each prospect being taken in picks three through six. The possible ordered combinations were determined using these deterministic selection rules: no prospect can be chosen twice and each team picks their highest remaining rated prospect.
The only reason we don't know before the draft the exact order in which prospects will be picked is that their dance cards are anonymous. With this set of ranking cards, there are 7,200 possible unique orderings of draft picks three through six. Some of them will happen a lot more often than others because some prospects have their names on more cards and some have fewer. To get probabilities I applied the draft rules stated above. Adding up the frequency for each prospect for the 3rd through 6th picks, here is a table of the possible outcomes:
I'm not sure whether you will think this is interesting. Certainly, Detroit's chances for picking each prospect pretty much summarizes what many people have been saying here at WIIM. There is little difference in the chances for six out of the nine named prospects. I'm sure many readers will be surprised by the fact that Cole Caufield has such low chances, but that is a consequence of the data source I relied upon. If teams actually do have a much higher opinion of him than the sample from Midland's post, then he may have a much higher chance of going in the top six. Certainly I'm not putting my money at risk with these "predictions" because I lack confidence in our information. And that's the problem: the draft process is too opaque.
Find me a Find, Catch me a Catch
NHL teams are operating in the dark almost as much as we are, unless they secretly share their rankings with each other. This is an example of an inefficient market. It is a process that follows strict rules of order and it would work much better if information about the current state of the market is available to every team. Here is a better way to manage it, subject to three new rules.
- Just before the draft, every team submits a confidential list to the NHL ranking their top 93 prospects.
- When you are on the clock for a pick, you will get the highest remaining prospect on your list or you may swap picks with another team.
- You may only swap picks for another team's next available pick but you may swap a pick at any time; you don't have to be on the clock.
- The League will hire a software partner and auditor to provide software and a user interface that displays the only possible order of prospects being picked using the 31 lists and current drafting order for picks 1 to 93.
- Teams will know the initial drafting order for the first three rounds by the time they go to the draft, i.e. they will know which prospect every team will get all the way through to the 93rd pick if no teams were to swap picks.
- Every team will always see the current information state of the draft. As swaps occur, every team will see which prospects it will be getting because of the latest swap. This is true for all teams, even if they are not involved in the swap.
- The user interface will provide "what if" capabilities. Two teams can establish a link showing possible interest in swapping picks and they will be presented with a list of information:
Each team will see which prospect it will get if no other teams between the two teams swap picks (but each team does not get to see which prospect the other team will get until after the swap is consummated)
Each team will also be presented with the names of the prospects remaining on their list with the probability of getting each of them if one, two or three teams between them are involved in swaps after the "what if" swap is completed.
- Optional processes could be incorporated, such as three-way swaps and suggested swaps. The latter would be an automated system that goes through the current draft information and alerts teams of all possible partners that would result in getting their current highest rated unpicked prospect or dropping no more than one or two prospects.
- Teams not involved in swap discussions will not know about those discussions. The only indication of a swap will be its announcement, with the subsequent automatic recalculation and availability of revised draft picks according to the new order of drafting teams and the previously submitted prospect rankings lists.
- No team gets to see any team's secret list, but every team always knows which prospect every team will currently will be getting if no other swaps occur. Nothing in the process is mysterious or probabilistic so a future audit will always confirm that the draft results were the inevitable result of final draft order and the original lists of prospect rankings.
I would expect that the NHL would decide not to let the public know the current information status of the draft except for picks that already have been completed. On the other hand, a really smart league would find a way to leverage this process for greater engagement, e.g. perhaps a betting market to let people gamble on what swaps will occur. Can you imagine the possibilities of a web site showing all picks in the first three rounds as they currently stand, before the picks have actually been made, then watching changes ripple through the list as swaps are announced? With visibility of the current information state, a reasonably smart person can see where the best chances for swaps are. But there always will be uncertainty because you don't know which prospects are ranked by each team and in what order. I think this would generate a very big buzz for the NHL.
Here is an example of how this process might unfold over the first ten picks. Assume the best 26 prospects in a draft are identified by the letters of the alphabet, with A being the highest rated and Z the lowest. I created ranking lists for the teams picking first through tenth to submit to the NHL using a probabilistic distribution that ensured the average true rankings are generally preserved but subject to some random variation. Once they submit their lists, everything is locked except the possibility of swapping picks. I've highlighted in green the only possible draft picks given these lists for these teams when taken in this order, if teams follow the rules of taking their highest remaining prospect (BPA). This is the kind of list that would appear to all teams the day before the draft but all the way out to the 93rd pick, i.e. the end of the third round (this is why every team must submit at least 93 ranked prospects).
Initial draft picks, in order, based on initial submitted ranking lists. Green= prospect picked. Hashed= previously picked.
Nobody in the public would be surprised by the first four picks: four of the top five consensus prospects would be picked. Team 5 would raise a few eyebrows among fans by taking prospect J, a guy most "experts" had about 10th. Ditto with Team 7 because prospect K was a consensus 11th. On the other hand, Team 9 gets the 6th consensus overall prospect F, because he was ranked much lower by Teams 7 and 8. Team 10 is actually quite pleased that its favorite sleeper pick T, which it rated 9th, was still available. On the other hand, Team10 notices (or the computer points it out) that its third ranked prospect K is not going to be taken until the 7th pick. This suggests the opportunity for a swap with either Team 6 or Team 7. Note that this could happen at any time, either before the draft or while earlier picks are being finalized. For this discussion I'll assume Team 10 waits until its potential partners are on the clock.
Let's start with Team 6. Team 10 pings it through the computer system to see if they are interested in swapping. The system presents Team 6 with the information that if it accepts the swap to tenth pick it will get prospect U if teams 7, 8 and 9 don't make a swap for their next picks. The computer also tells it the exact probability of getting the only possible other prospects on its list (O, N or P) if Teams 7 and/or 8 and/or 9 were to make a swap before it picks tenth. Now the manager has something to think about. All four prospects had originally been in his top 10 but he would be giving up his 4th-rated prospect for a good chance at his 6th ranked prospect and decreasing chances at somewhat lower rated prospects. In any case, he knows all the possibilities.
Table of picks if Team6 swaps with Team10. Key: Green=certain, blue= most likely, yellow through red represent decreasing likelihood. Hashed= previously picked.
Let's say Team 6 declines to accept the offer. Team 10 pings Team 7 once it is on the clock. The computer shows Team 7 that it would receive prospect L instead of prospect K, which is only a drop of one place on its list. Since Team8 and Team9 pick before the 10th pick, it could drop to prospects G or W if one or both of them swapped with another team. Again, the computer would present the exact probabilities of getting prospect G or W if another swap occurred involving those teams, based on the teams that they could still swap with according to the rules listed above. Team 7 doesn't know how high Team 8 ranked prospect H (remember, team lists are anonymous) nor does it know that Team 9 really likes prospect F so much that they ranked him 2nd, so Team 7 doesn't know there is no way Team 9 is going to swap its pick with anyone else. On the other hand, Team7 has both H and F rated much lower so he figures Teams 8 and 9 must be pretty happy for those prospects to appear as their draft picks rather than L, G or W.
Table of picks if Team7 swaps with Team10. Key: Green=certain, blue= most likely, yellow and orange represent decreasing likelihood. Hashed= previously picked.
After considering the probabilities presented by the computer, Team 7 decides to make a deal with Team 10 and they swap. The system could also allow the teams to exchange possible terms of a swap as it is requested, based on rules that they define prior to the draft, or they could do the negotiation in the old-fashioned way. Team 10 is ecstatic getting its third-ranked prospect considering it entered the draft with the 10th pick, and Teams 8 and 9 are similarly delighted that they are getting prospects they ranked higher than their own spot in the draft, so no further swaps are made in the top ten picks. But Teams 8 and 9 also are breathing a sigh of relief because when the announcement is made that Team7 swapped with Team 10, it took a moment to sink in that the new listed draft picks on the computer show that they did not drop down on their list of prospects. Everybody is very happy, including Team 7, because it ended up getting prospect L, a player that it had rated 7th (matching its original spot in the draft) plus it got an asset. Notice that the average ranking that all teams got was higher than they would have been if teams did not have visibility of which prospect was the current choice at every spot in the draft.
What I don't show here is that prospect L now disappears from somebody else's pick farther down in the draft, to be replaced according to that team's highest ranked remaining prospect on its own list. Likewise, prospect T that had originally been slated for Team 10 now pops up as somebody else's highest rated remaining player, thereby improving their current pick. In general, for every swap one team picking later will now see a lower ranked prospect as its selection and another team will now see a higher ranked prospect as its next selection. In some cases, the change will be only one or two spots down a list, in others the change could be quite large; furthermore a swap may trigger downstream ripple effects changing the picks for many teams, some getting prospects they ranked higher and some that were ranked lower. Thus, each consummated swap will require every team to check the updated selections for every position, including their own, and consider whether to propose even more swaps. The likely result is a much higher rate of pick-swapping because teams will have much more visibility of the risks of sitting pat and the opportunities for making beneficial swaps.
The example I used is presented here without modification because I wanted it to be realistic. I ran simulations a number of times and got scenarios where two teams could swap many places yet the dropping team ended up getting the same prospect it would have received anyway. The majority of the time the overall result of the process was a better matching of prospects to teams so that the average team ranking of prospects selected rises because the process improves the efficiency of matching.
I know, this sounds complicated, but it really is trivial and more importantly, unalterable, for a computer. The process is 100% deterministic and there is no room for shenanigans; the process will be auditable and easily understood after the draft is complete. What I will not show is that the system works even when neither of the two potential swap-mates are on the clock- this can be done in advance. As the team with the higher pick gets closer to being on the clock, the certainty of which prospect each team can get will grow, thereby allowing better refinement of terms and probably increasing the appropriateness of assets being received by the team moving downdraft is complete. The process should therefore improve the efficiency of asset utilization.
Although it isn't obvious, the process would also work for three-way swaps. Each team could be presented with two scenarios based on whether they get the pick of the second or third team in the group, with the exact probabilities of players to be available. It is likely that suggestions for plausible beneficial three-way combinations would need to be presented to prospective teams by the computer system because I don't expect that anyone at a draft desk has the capacity to work through all the possible combinations in real-time. The system would look for cases in which all teams are very likely to get a player that they rank as high or higher than their current likely pick and then send an alert to the teams.
While thinking about this idea I realized there are a couple of issues that need to be dealt with; I'm sure you can think of more. The first is that I doubt all NHL teams use a rigid ranking beyond a certain number of picks. I wouldn't be surprised if some teams rank prospects in the first few dozen then simply tier prospects after that, leaving the internal debate for draft day according to picks that look like they still will be available. I would hope teams would realize that my proposal eliminates much of the guessing so it would be advantageous to rank enough prospects to enable matchmaking in the first two rounds, using lists that go out to three rounds, which means 93 prospects.
I don't think this system can work without at least that much depth in the ranking lists. Even with lists of 93 prospects, the matchmaking process is only realistic for the first couple of rounds because with team rankings only available to 93, you need to allow room for teams to have information about the consequences of moving down. That probably limits the system to swaps involving at least one team at pick 62 or above. Ideally, teams would be able to submit even longer ranking lists, and maybe after some experience with the system they would agree to do it. This might become a matter of self-interest.
Another problem, and one that could be fatal, depends on whether teams often change their minds during the draft about the relative rank of prospects. This might happen, for example, if they select one prospect for a position of need, then decide they don't need another one of those. They might be motivated to re-rank prospects so a different position player is now ranked higher. My proposal requires locking in the ranking lists and accepting players you define as BPA before the draft, and I don't see a way to allow changes without destroying trust. However, teams that do change their minds can always use the "what if" scenario to see how they might "drop" down to their now newly re-ranked player on their list.
A final problem with the matchmaking process is that the team picking first under the current system has no motivation to provide an accurate ranking of prospects from two to ten since they will not be drafting them. With my proposal, they might be inclined to scramble prospects in spots right after their own, just to screw around with other teams. I haven't yet figured out an angle that leads to an advantage except it might ripple downward to make their second round pick a bit higher. However, doing so would risk getting one of those scrambled prospects with your next pick, so it would be a stupid risk to take. In any case, this is one of those things that can cause other teams to distrust the process.
The problem would be solved entirely by conducting the draft lottery after lists are submitted to the League. I can ensure you that in this case every team would make its very best list! In fact, I think it would make for a fun few days:
- Draft Day Minus Three: lists submitted
- Draft Day Minus Two: lottery to determine initial draft order
- Draft Day Minus One:Initial prospect draft order reported to teams, chance to form strategies and contact other teams about potential swaps
- Draft Day: Swap Mania.
It took me a week to finish this post so I may have forgotten some things. I hope all the editing didn't mess things up. Anyway, the last thing I can recall thinking about when noodling this idea is that prospects could also be given the opportunity to provide a list of teams they would prefer to be drafted by, ranked in order. This would need to be optional and the list could range from a single team to all thirty one. I imagine the information being presented to teams on the matchmaking system whenever a prospect's name is displayed, showing how (or whether) they were ranked by that prospect, e.g. "2nd of five teams listed" or "NR".
Find me no find, catch me no catch, unless he's a matchless match.
I suffer no delusion that the NHL would ever consider such a process. I think it could expose some teams' lack of sophistication and business acumen. I also think that Luddites inhabit some front offices. But I do enjoy fiddling around with numbers and ideas so it wasn't a waste of time. I apologize to the lyricist Sheldon Harnick for misappropriating lines from the Matchmaker, Matchmaker song in Fiddler on the Roof. Thursday is my 62nd birthday, so I'll close with a few more lines:
Hodel, oh Hodel,
Have I made a match for you!
He's handsome, he's young!
Alright, he's 62.
[i]Midland said he plans to update the post with more sources during draft week but I think the preliminary post illustrates my points quite well.
[ii]It will always produce the same output from a given starting condition or initial state
[iii]strictly speaking this is probably only true for the first few rounds because it is unlikely that all teams go to the draft with a list of hundreds of prospects ranked in order
[iv]A process that is predictable but subject to random fluctuations that limit the accuracy of future predictions.