top of page
Ice Hockey Player

Recipe For Success 

Data Collection for Effective Player Evaluation

By: Swapneel Mehta

August 17th 2020

One of the questions I often ask myself when evaluating my client’s data needs is: Do I have enough? It’s a question where the answer is in theory simple: There is never enough data. But what types of data are most important to collect? For sporting organizations, decisions need to be made around data collection costs versus the player evaluation benefits they will reap from said data.  In this series, I will outline what I believe are the key types of data any sporting organization needs to collect in order to build an effective player evaluation function.

Physical Assessment Testing (PAT)

Based on my experiences in player evaluation analytics, the number one data type that has proven to be the most effective in prediction of player success has been physical assessment testing (PAT). While the importance varies based on the performance level of your players, the value of PAT has been clear: PAT provides a standardized evaluation mechanism, one that is not biased by the competition a player is going up against.

 

PAT can be broken down into two major sub-segments: Field of Play Testing and Off-Field Testing. Off-Field testing has been prevalent in most evaluation programs for decades. Examples of off-field testing include: Broad Jumps, Vertical Jumps, Sprints, Bench Press, Medicine Ball Toss and Fitness Testing. These drills tend to give insight into a player’s general physical fitness, strength, speed and athleticism.

Sports Practice

Physical Assessment Testing allows for a standardized evaluation that can be compared across performance levels

However, while off-field testing provides a general physical evaluation, sport-specific field of play testing provides a valuable view into how a player would perform during an actual game. These could include skating drills for a hockey player, throwing drills for a baseball player and route running for football player.

 

Predictive models that VantagePoint has built using both types of testing show clearly that field of play testing is more valuable, however models including both types of testing still generate the highest degree of predictability.

Psychological, Behavioral and Sports IQ

Prediction models in sports have long relied on data driven from a players physical abilities, whether that be gameplay data or physical assessment data. However, the next generation of models are increasingly incorporating a new vertical of sports data. Mental assessment data is an up-and-coming field that looks to understand a player’s mental ability, and how it correlates to their probability of success. Mental ability can come in different forms. Psychologists have developed sports-specific assessments that aim to understand the motivations, rationales and perspectives of a player. When a player makes the wrong decision in a game, what lead to that decision? When a player responds well after an important loss, or a missed shot, what differs about their psychological make-up from a player that does not? These are the questions that psychological and behavioral research in sports aims to answer.

 

Sports IQ has also become a developing area that performance evaluators are putting more emphasis on. How well a player understands the mechanics of a sport can greatly influence their performance during a game. Whether that be their ability to identify which play is the most likely for success given the game situation and defense showing, or their understanding of whether the foul they are trying to commit will result in free throws or a sideline inbound, Sports IQ gives evaluators a sense of how well a player knows their sport. While mental assessment testing is relatively uncommon in junior and amateur sports, it is quickly proving to be worth the cost of collection.

VantagePoint can help sporting organizations incorporate mental assessment data into their evaluation models, enhancing predictability and allowing for more effective decision making.

Test-and-Learn Approach

As new sources of data continue to show promise in the player evaluation process, the best way to ensure that your organization is collecting the most useful data is by implementing a test-and-learn approach. The goal of this approach is to build a system where new data is easily ingested, tested for predictability and then based on the results, included in the player evaluation process. Sample data can usually be collected at a minimal cost, and if proven predictive, a decision can be made as to whether it is worth the investment to collect on your entire player population.

 

VantagePoint can assist in building data collection test-and-learn functions within amateur and junior sporting organizations. Collecting the right data is the first step to making the right decision.

Gameplay Statistics

A logical data point that will add value to any performance evaluation model is gameplay statistics. If testing a player in drills that replicate gameplay are more valuable than generic drills, should not actual gameplay data be the holy grail of predictive data? The answer is, it depends. For professional sports teams, gameplay data is quite clean. Usually at the highest levels, you are using gameplay at that level to predict performance at that same level. However, with minor, junior and amateur sports, gameplay data has many limitations. Competition levels vary greatly at the sub-pro levels.

 

Let’s take hockey as an example. The goaltending in the CHL can be very different in comparison with the NCAA and USHL. Even within the CHL, it is has been shown that the WHL has a different playing style than the OHL which has a different playing style than the QMJHL. These differences end up appearing in gameplay data. A goal in one league is not worth the same as in another due to the variation in defense and goaltending. Normalization can address these issues, however the further away you move from the major leagues, the more variation exists and less powerful gameplay data becomes.

 

For junior and amateur sports, we recommend a combination of physical assessment testing and gameplay data to ensure highly predictive performance models.

Wondering what data your sporting organization should collect? Let's connect and discuss how VantagePoint can help you. 

bottom of page