Scouting During A Pandemic – Inside the draft evaluation process with Mark Yannetti

In a normal year, comparing draft-eligible players isn’t the easiest task.

Each and every NHL Draft, we see highly-drafted players not pan out, and players who fell under the radar and into the middle rounds become regular NHL’ers. We see it every year.

Now, try comparing those players when every league has played a different number of games. The WHL is playing a 20-game schedule. The European pro leagues have played a substantial sample, as did the NCAA. The OHL didn’t play at all.

That’s the job at hand of Mark Yannetti, the LA Kings Director of Scouting, in advance of the 2021 NHL Draft.

“It was hard and in some cases impossible to do this year,” Yannetti said. “We have no data at all on the OHL. We have almost a full season of data from the USHL, around 15 games from the WHL, half a season from Sweden, so the models don’t work in the state we have them in, and the state we developed them to, because the data is incomplete to none on some leagues, and the comparisons are apples to oranges anyways.”

Such is the challenge facing not just Yannetti and the Kings, but now 32 teams around the NHL approaching July’s NHL Draft. Among NHL Central Scouting’s preliminary list of players that received an “A” grading, signifying first-round potential, 10 of those players play in either the OHL or the WHL, leagues that played either no games, or a maximum of 20 games. And now, you’re potentially spending a valuable first-round selection on one of those players. That’s what Yannetti and his staff are currently facing.

Comparing a player who scored a point-per-game in one of the CHL leagues, against a player well under a point-per-game playing against men in Finland is always part of the process. It’s using the models and tools that the organization has in place to help try and bridge that gap.

“It’s balancing in the normal years,” Yannetti said. “Rob Vollman, Hayden [Speak], Jeff Solomon and I, Rob has models that we’ve expanded on for years, but in the last few years we’ve been trying to make those models better for comparison sake. It’s not easy and it’s not perfect, it’s an imperfect system, but we’ve made really great strides and we’ve got a good working model to do that.”

This season, the added wrinkle is the disparity between the leagues.

In terms of the analytical side, half of the equation, Yannetti said that the Kings have worked on taking their models and altering them to consider the differences in the data. This season, it’s a bit more caution shown, especially in the early stages, but the models still give the Kings a valuable tool when it comes to checks and balances between the analytics and the live-scouting process.

“The discussion it creates, it still works as a very good check and balance for the live scouts and vice versa,” he said. “Even if it’s painting an incomplete picture, it’s still painting a picture, which allows us to, if it’s saying something different than the live scouting, or from what we believe, it allows us to at least explore it.”

On the live scouting side, Yannetti estimated that “more than 50 – 60 percent” of evaluations have been unaffected by this season, in terms of the Kings getting an adequate number of views as they would have normally.

That is obviously very regionalized, with Yannetti citing the US National Team Development Program, the USHL, many leagues in Europe and the NCAA as leagues where viewing hasn’t been all that different. In some cases though, perhaps you can develop the same picture with less.

“There were limitations, but you have the old school mantra of seeing a guy play 20 times a year and I don’t know if that’s necessary,” he said. “You certainly don’t want to see him play once a year, but in some of these non-pandemic years, we’re having scouts see guys 15 or more times, 20 or more times. That’s great, but it’s not necessary. If you’re not getting 15 views, it doesn’t mean you’re not getting the information you need. I would say we got the information we needed, without being limited, in more than 50-60 percent of the cases this year. We made it work.”

For those other groups, the main areas were players in the three CHL leagues – The OHL, WHL and QMJHL. In Quebec, the league started off strong, but a midseason gap existed, followed by a continuation of play as of late. The WHL didn’t start up until the waning days of February, playing a limited, 20-game schedule with no postseason. The OHL recently announced it would not play at all this season due to restrictions in the province of Ontario, and an inability to safely return to play.

This certainly presents the largest gaps in the data, because seeing a prospect play, and scouting him at 16 years old is not the same player who will be drafted in July. Yannetti said that the jumps a player makes from 18 to 19, or 19 to 20, pale in comparison to the jump that a player makes from 16 to 17 or 17 to 18. You can’t simply replicate that data, and for many players who fall outside of that top group of prospects, there will be missing pieces to the puzzle when it comes to making the correct decisions.

One opportunity to help fill in some of that data, and to see some of these players, potentially for the first time this season, or to gain additional views to help make up that differential is ongoing as we speak – The IIHF World U18 Championships, which kicked on Monday in Frisco, Texas.

The Under-18 tournament is highly regarded by some organizations, and valued less so by others, with Yannetti’s viewpoint trending more towards the latter. It’s not to say he doesn’t place value on the event, which pools most of the draft’s top eligible players into one location, playing against each other at the same time. He fully acknowledges that importance, but also believes that large-scale decisions are made across the league off of a short tournament.

He is careful, within his own staff, to avoid recency bias, meaning putting too much of an end all, be all emphasis on the last time a player made a powerful impression on you, good or bad.

“Regardless of our best intentions, it’s hard not to come out of that tournament with some form of recency bias,” Yannetti explained. “It’s the best players, all together, a condensed thing at the end of the year, at the last decision point you can make. It lends itself to a recency bias, it lends itself to a little bit of added importance, and it’s hard. Some teams do it well, some teams don’t do it well, but no matter who you are, it does affect you a little bit too much sometimes, no matter what.”

For many players, those late swings in evaluations occur at the U-18 tournament, as it’s the last organized event of the season for these individuals. As a man of some incredible words, Yannetti used a pretty great metaphor to describe the pitfalls that the tournament can bring.

“I look at it like this – I have restaurants I love to go to, we all do, and I have my favorite restaurants, Yannetti said. “I can remember going to a restaurant 20-30 times, I can’t go there enough. Every time I come off the road I go there, but all it took was one bad meal for me to never go back. I’ve had 20 wonderful experiences, but one time, the food sucked, the service was brutal, it was just an all-around bad experience. I had 20 great experiences, one bad experience and I never went back. Now, my mind knows, this is a restaurant I love yet I still haven’t gone back there. I can’t get that last experience out of my mind, and I know I’m crazy, but I’ll just go somewhere else. We do that with the players too, whether it’s good or bad, it’s hard to get that last, powerful experience out of your mind.”

This year, he believes that the tournament might create even larger than usual swings, considering the circumstances facing teams around the league evaluating players from various areas.

Yannetti said wanting to watch the U-18’s and see a player play more is the wrong approach. You want to see a player as he is and confirm the work you put in scouting and analyzing during the season. Getting that true, honest view of a player at the tournament is an important thing,

“A year like this, now you’ve got to really balance it, because it’s super important,” he explained. “I could go there for a week, 10 days, and see everybody in the world that I need to see play, five or six times, and I haven’t had that opportunity this year. That, in and of itself, I can’t stress how important that is, but the thing is, now it’s even magnified even more, now I’m even more prone to these swings, more prone to recency bias, because I don’t have that body of work, I don’t have some of those specific questions, I don’t have the multiple views on multiple players, balancing it out. It’s going to be real interesting to me.”

As we eventually pass the Under-18 Tournament and progress forward, the NHL Draft is just three months away, occurring this season towards the end of July. Teams will be operating with less than usual, and there’s no avoiding it.

That being said though, this isn’t an LA Kings problem, it’s an NHL problem, meaning that all teams at base level will be on level pegging.

“We came up with solutions, some data that I think points us in the right direction, but it’s incomplete data,” Yannetti said. “It’s obviously not ideal, but it’s fair, because it’s not ideal for the other 31 teams. Each team is dealing with the exact same disadvantage and the exact same percentages, so it’s how creative you get in terms of bridging that information gap. Quite frankly, there’s just not a lot you can do on some of that.”

The Kings hold extra draft picks in both the second and third rounds this season, with five picks in total in the first three rounds. How these solutions play out for the Kings, compared to many of the other teams around the NHL, will be a process to watch play out over the next three months.