As someone who has been writing scouting reports for several seasons now, one insight that I have gained from this experience is that assessing a prospect’s ability to think the game out is by far the most challenging facet of evaluating their game, and it is probably the facet of the game that impacts their success at the next level the most.

Whenever scouts talk about the cerebral aspect of a player’s game, any such analysis of a player will be qualitative and based on the eye test. There are no formal evaluations of this ability like you get with their physical fitness. And it’s not like there isn’t an opportunity to delve deeper into assessing these questions scientifically. They’re currently conducting the annual scouting combine as you read this.

There’s ample opportunity to do more detailed assessments, but the scouting combine focuses almost purely on the physical/athletic aspect of playing the game, with the exception of the interviews. Take a look at the assessment protocols for yourself here. The component that focuses on how these young players think the game out has remained a project that the league, and talent evaluation organizations such as NHL Central Scouting have not given nearly enough attention, especially given how much such organizations stress a player’s ability to think the game out when evaluating their probability of success at the next level.

While athleticism and physical fitness are essential to being a professional hockey player, there are aspects of the game that a player’s body composition, musculoskeletal, anaerobic, and aerobic fitness will tell you nothing about, and it’s not like there hasn’t been decades worth of development in cognitive testing that would lend itself effectively to the regimen at these combines. Such measures could provide a more complete evaluation of a prospect, and that’s data that could potentially provide insights into how the cognitive abilities of players affect their success at the next level that’s not being collected or used.

To me, the fact that they haven’t attempted to conduct any scientific evaluation of decision-making and cognitive abilities when evaluating talent at these combines is outrageous. The NHL has taken many steps forward into modernity this past decade with its embrace of analytics, but it still has so much to improve on when it comes to advancing research in sports psychology, and investing in applications of cognitive science to personnel evaluation. It’s a frontier of research that front offices across the league would benefit from pursuing, and would give organizations a more complete evaluation of players both physically and mentally.

Outside of my scouting reports at Winging It In Motown, I have a Ph.D. in cognitive/experimental psychology. My area of specialization deals with understanding mental phenomena such as memory, language, spatial reasoning, creativity, attention, perception, problem-solving, and reasoning. For those in cognitive neuroscience, this also involves studying how these processes relate to the functioning of the brain.

Much of my work has been dedicated to designing and testing experiments that allow us to better understand the content, duration, and sequencing of human cognitive operations during decision-making by using reaction time data. This involves presenting participants with a series of stimuli (e.g., words, shapes, colors, objects) that they have to make a motor response to (e.g., a button press), and recording their reaction times (in milliseconds), as measured by the interval of time between the onset of the stimulus and the execution of the response, and their accuracy rates.

Today, I am not writing as a scouting hobbyist. I am writing as an expert in my field, and will be discussing some of the research that has been conducted in my area, and how it can be applied to talent and personnel evaluation by NHL teams and scouting organizations. While the task of devising methods to evaluate the complex types of problem-solving a hockey player has to engage in can be daunting, I believe the tools we have at our disposal present us with a great opportunity to begin advancing the application of the methods and insights of the field of cognitive science to talent assessment in hockey.

The Eyes Have It

Perhaps the most obvious way that we can assess prospect decision-making abilities is in systematically studying how hockey players visually scan the ice during their shifts. Hockey is a highly visual game. Understanding where you need to be on the ice, where the puck is going to be, what play is developing, and your awareness of the positions of your teammates and opponents all come from visual attention abilities.

There is always a relationship between perception and action during the game. Being able to efficiently tune in to important cues while tuning out task-irrelevant information on the ice ensures that players can rapidly prepare and execute critical decisions that have a direct impact on their on-ice performance.

The most valuable source of information that we can use to assess how players are scanning the ice and what they’re attending to during shifts is by assessing their eye-movements as they’re playing. The eyes are more than just the window to the soul. They tell you a lot about what information a player is attending to, and how much time they are dedicating to specific events occurring on the ice.

Such measures can be used in conjunction with other measures (e.g., an electromyogram) to give us a broader understanding of a player’s ability to scan the ice, identify events unfolding around them that they need to make a decision on, and quantify the amount of time that it takes them from the onset of identifying that event to making a decision, as measured by a motor response.

The technology that allows us to track eye movements exists. The modern eye-tracking methods that are still extensively used were pioneered during the 1970s by Keith Rayner. I’ve used it in my own research to study human eye movements during reading, and it wouldn’t be the first time that the technology has been applied to sports. Think of GoPro, except now you can actually analyze what the person wearing the equipment is looking at.

It’s been used in areas of research as diverse as marketing, user experience and usability, and human factors to track and gauge participants’ visual attention during tasks such as shopping for products, navigating through websites or apps, and assessing a person’s visual attention as they’re driving a car. As I’m advocating, it’s also been used to evaluate athletic performance in sports such as soccer, tennis, golf, basketball, baseball, volleyball, and hockey.

Here’s a demonstration of the applications of eye-tracking:

Source: Tobii Pro – YouTube

Using eye-tracking, we can empirically examine whether there are differences in how elite players analyze the ice versus players with weaker situational awareness. For example, how do the fixation and gaze characteristics of a player like Connor McDavid or Sidney Crosby compare to a player like Justin Abdelkader? This is not a far-fetched idea, and there are a lot of reasons to believe that researchers could identify differences between elite players and less skilled players based on how they attend to information on the ice.

In other fields of research, for example, we see drastic differences between how expert drivers and novice drivers scan the road while they’re driving (e.g., Crundall & Underwood, 1998; Underwood, Chapman, Brocklehurst, Underwood, & Crundall, 2003; van Leeuwen, Happee, & de Winter, 2015), how chess masters and novice chess players scan the board during chess-related visual search tasks (e.g., Junior, Cesar, Rocha, & Thomaz, 2017; Reingold, Charness, Pomplun, & Stampe, 2001; Reingold & Sheridan, 2011; Sheridan & Reingold, 2014, 2017), and how expert soccer players attend to visual information during a match compared to novices (e.g., Savelsbergh, Haans, Kooijman, & van Kampen, 2010; Savelsbergh, Van der Kamp, Williams, & Ward, 2005; Savelsbergh, Williams, Van der Kamp, & Ward, 2002; see also Kredel, Vater, Klostermann, & Hossner, 2017, for a full overview of applications of eye-tracking to sports research).

The question of whether elite players and less skilled players differ in terms of their orientation of visual attention is not a new one in the realm of sports science, and it wouldn’t be the first time that this question has been assessed in hockey. Martell and Vickers (2004), for example, studied how elite and near-elite hockey players regulate their gaze during tactical plays as they defended against opponents on the ice.

However, any of the research that has been done represents only a baby step towards more comprehensive studies of visual attention control in hockey. Any studies that have been done have been smaller in scale, meaning that there is room to flesh this research out with more robust, larger-scale studies with better statistical power. Having the NHL invested in advancing the use of such tools on a larger scale is necessary for it to gain traction, and the benefits of doing so would be immense.

We already see the application of using eye-tracking when working with personnel, as can be seen with how it can be used by coaching staffs to get an idea of what their players are attending to on the ice and providing feedback. But the true potential of the application of this technology for sports science research remains untapped by professional hockey leagues.

Understanding the Role of Cognitive Functions in On-Ice Performance using Laboratory Experiments

Even outside of doing studies in the naturalistic setting of a game, there is still other work that can be done to better understand the role of cognitive abilities in on-ice performance in the laboratory setting.

One aspect of cognition that I think would be particularly important to study is working memory (WM). WM refers to a limited capacity memory system that is used to temporarily store and manipulate information for performing task-relevant behaviors in the face of distracting, task-irrelevant information. It is typically thought of as a multicomponent system that consists of a central executive that acts as a control system, and slave systems which act on and store visuo-spatial information, phonological/acoustic information, and an episodic buffer that stores and manipulates other types of information (e.g., semantic information) and acts as a link between WM and long-term memory (e.g., Baddeley, 2000, 2012; Baddeley & Hitch, 1974).

It is arguably the most important system of memory that we use on a daily basis, as it’s essential for guiding attention to relevant information and making task-appropriate actions, while suppressing irrelevant information and inappropriate actions. Name a complex form of decision-making or problem-solving that you have to perform in daily life, and WM is involved in performing that task. Playing a sport like hockey is definitely not an exception.

The important point here is that WM is a limited capacity system. You only have a finite amount of resources available in this system, and, like all cognitive abilities, people differ in terms of their WM capacity. This difference in WM capacity may also be one of the underlying correlates of many of the differences that we see in the problem-solving and decision-making abilities of hockey players. What sets players like Connor McDavid apart from his peers is his peerless vision, situational awareness, and ability to read plays as they are developing on the ice. Is this ability domain-specific (his reasoning is superior only in the domain of hockey), or are we talking about a domain-general difference in his visuo-spatial problem-solving that just happens to allow him to dominate at the sport?

Such research questions are easily testable using laboratory experiments. For the visuo-spatial component of problem solving, for example, we could have the subjects perform a dot matrix task where they have to remember the location and order of dots that are sequentially displayed in a grid on a computer screen (Alloway, 2007). As they perform the task, after answering 4 trials correctly, the task becomes increasingly more difficult, by adding an additional dot in the sequence for them to remember with each successive correct trial. The test would terminate after the subject commits 3 errors on one level of difficulty. The number of dots that they are able to sequentially retain in WM would give us an idea of the capacity of their visuo-spatial WM.

To assess the functioning of the central executive, we could have the subjects perform tasks that either require them to divide their attention between two targets or streams of stimuli, or require them to switch between tasks, such as alternating between addition and subtraction using a more demanding concurrent verbal reasoning task. In doing such tasks, of interest would be measuring the cost to performance from either dividing attention between two targets or switching.

Using such experimental methods as part of the talent evaluation process at combines, we could then study the relationship between these cognitive abilities and different outcome measures. Outcome measures could include short-term outcomes such as expert assessments of a player’s decision-making abilities (e.g., a numerical rating scale given to a panel of experts), and it could include long-term outcomes such as measures of how much the player impacts the game at the next level (e.g., be it points, goals, assists, metrics such as WAR/GAR, xG, scoring chances).

Such projects would be longitudinal in nature, meaning that we wouldn’t have a clear answer of how cognitive abilities affect longer-term outcomes until more data about these players is collected once they transition into their professional careers. The benefits of doing this, however, would be immense. Not only would it give coaches and staff a better understanding of the cognitive skills that they need to focus on developing in young players, it would also give scouts and GMs a rich source of information for projecting young talent at the next level.

Final Thoughts

Before closing out this article, there are a few things that I think are worth mentioning. First and foremost, the methods and predictors that I have discussed are by no means meant to be comprehensive, and should not be treated as such. The factors that affect player decision-making go far beyond visuo-spatial WM or the analysis of eye-movements, and this discussion has only scratched the surface of potential applications of cognitive science research to personnel evaluation.

This is an intentional oversimplification, because it is impossible to not oversimplify a topic as complex as understanding the factors that affect problem-solving and decision-making without writing an entire book about it. Even then, you are still probably oversimplifying the research.

Second, I don’t want anybody getting the impression that the only application cognitive science research has in hockey is in talent evaluation. The reason that I decided to write this article from a talent evaluation perspective is because of the practices of the scouting combine. But there are a lot of other reasons why the NHL and organizations within the NHL would want to adopt such research, and some of those reasons are far more important than talent evaluation.

The NHL is facing a serious issue when it comes to concussions and chronic traumatic encephalopathy (CTE). Along with such issues comes a serious decline in the cognitive abilities of its athletes, which can be identified early on through proactive, regular screening. If teams are not screening the cognitive health of their athletes on a regular basis using such assessments (I am not aware at this time of whether or not they do), they need to start doing so immediately.

The tools and methods are already here and at the disposal of organizations like the NHL. It’s up to the NHL to take another step forward into the 21st Century, and invest in the scientific research and make use of it. People like me can sit here all day talking about how the NHL needs to prioritize the cognitive and mental health of its athletes more, but until the NHL listens to what experts have to say, nothing will change.

Finally, below is a full list of all of the research that I cited. If you are interested in what I have discussed in this article and are interested in further readings, I strongly recommend consulting some of the excellent research studies that have been conducted in the area. Otherwise, if you have any questions, don’t hesitate to leave one in the comments.

References

Alloway, T. P. (2007). Automated working memory assessment manual. Oxford: Harcourt.

Baddeley, A. D. (2000). The episodic buffer: A new component of working memory? Trends in Cognitive Science, 4(11), 417-423. doi:10.1016/S1364-6613(00)01538-2

Baddeley, A. (2012). Working memory: Theories, models, and controversies. Annual Review of Psychology, 63, 1-29. doi:10.1146/annurev-psych-120710-100422

Baddeley, A. D., & Hitch, G. (1974). Working memory. Psychology of Learning and Motivation, 8, 47-89. doi:10.1016/S0079-7421(08)60452-1

Crundall, D. E., & Underwood, G. (1998). Effects of experience and processing demands on visual information acquisition in drivers. Ergonomics, 41, 448-458. doi:10.1080/001401398186937

Junior, L. R. S., Cesar, F. H. G., Rocha, F. T., & Thomaz, C. E. (2017). EEG and eye movement maps of chess players, presented at the 6th International Conference on Pattern Recognition Applications and Methods (ICPRAM), Porto, Portugal, February, 2017.

Kredel, R., Vater, C., Klostermann, A., & Hossner, E. -J. (2017). Eye-tracking technology and the dynamics of natural gaze behavior in sports: A systematic review of 40 years of research. Frontiers of Psychology, 8, 1845. doi:10.3389/fpsyg.2017.01845

Martell, S. G., & Vickers, J. N. (2004). Gaze characteristics of elite and near-elite athletes in ice hockey defensive tactics. Human Movements Science, 22(6), 689-712. doi:10.1016/j.humov.2004.02.004

Reingold, E. M., Charness, N., Pomplun, M., & Stampe, D. M. (2001). Visual span in expert chess players: Evidence from eye movements. Psychological Science, 12(1), 48-55. doi:10.1111/1467-9280.00309

Reingold, E. M., & Sheridan, H. (2011). Eye movements and visual expertise in chess and medicine. In S. Liversedge, I. Gilchrist, & S. Everling (Eds.), Oxford handbook of eye movements. Oxford, UK: Oxford University Press.

Savelsbergh, G. J., Haans, S. H., Kooijman, M. K., & van Kampen, P. M. (2010). A method to identify talent: Visual search and locomotion behavior in young football players. Human Movement Science, 29, 764-776. doi:10.1016/j.humov.2010.05.003

Savelsbergh, G. J., Van der Kamp, J., Williams, A. M., & Ward, P. (2005). Anticipation and visual search behaviour in expert soccer goalkeepers. Ergonomics, 48, 1686-1697. doi:10.1080/00140130500101346

Savelsbergh, G. J., Williams, A. M., Van der Kamp, J., & Ward, P. (2002). Visual search, anticipation and expertise in soccer goalkeepers. Journal of Sports Science, 20, 279-287. doi:10.1080/026404102317284826

Sheridan, H., & Reingold, E. M. (2014). Expert vs. novice differences in the detection of relevant information during a chess game: Evidence from eye movements. Frontiers of Psychology, 5. doi:10.3389/fpsyg.2014.00941

Sheridan, H., & Reingold, E. M. (2017). Chess players’ eye movements reveal rapid recognition of complex visual patterns: Evidence from a chess-related visual search task. Journal of Vision, 17(3), 4. doi:10.1167/17.3.4

Underwood, G., Chapman, P., Brocklehurst, N., Underwood, J., & Crundall, D. (2003). Visual attention while driving: Sequences of eye fixations made by experienced and novice drivers. Ergonomics, 46, 629-646. doi:10.1080/0014013031000090116

van Leeuwen, P. M., Happee, R., & de Winter, J. C. F. (2015). Changes of driving performance and gaze behavior of novice drivers during a 30-min simulator-based training. Procedia Manufacturing, 3, 3325-3332. doi:10.1016/j.promfg.2015.07.422