Tracking Comments on Players Entering the NFL Draft

Since I’m reading about the NFL draft, I’m constantly hearing remarks about the players from NFL draft experts. I’ve been wanting to see someone compile these commments from the experts on the high draft picks and then compare the comments to the players several years down the line. What I’m interested in identifying is how accurate these experts are in analyzing the players. Unfortunately, I have yet to find anyone who has compiled these comments. Since that’s the case, I’m wondering about a good method to do that myself.

The first thing that comes to mind is a spread sheet of some sort where I can have one column for players and a column for several experts (e.g., Mike Mayock, Mel Kyper, Todd McShay, etc.). So, for example, I might have a column for Johnny Manziel, and let’s say Mayock said that he thinks Manziel can throw well, but seems wild and undisciplined, particularly in the pocket. Now, let’s fast forward three years from now. If Manziel has trouble in the pocket, we can credit Mayock for that.

That seems simple enough, but I want to do this online so that other people can add to the spread sheet, so that I’m not the only one doing it.

Also, I think breaking each of the experts’ columns into two separate columns, for positive and negative comments, might be a good idea.

Finally, I’d want a way to rate or evaluate each of the experts. I guess we could just tally the number of times they made an accurate comment.

Any thoughts or help would be appreciated.

13 Responses to “Tracking Comments on Players Entering the NFL Draft”


  1. mitchell

    A couple of years ago, I had this same idea. I bought several pre-draft magazines at BN and stashed them away so I could look at them some time later. If I dig them up I’ll pass them along.

  2. Reid

    Cool.

    What I’m trying to do is create some form and/or procedure to be able to log down the comments every time I come across them and then be able to be able to quickly determine if there is any consensus. I would also like to be able to easily calculate who makes the most accurate judgments.

    I guess, creating the form/format wouldn’t be so hard. The difficulty would be just putting in the time to find the information and writing it down.

    Edit: What would be cool is if the form was online somewhere where you or I (or anyone else) could enter data–in a “wiki” sort of way.

  3. mitchell

    You could use a collaborative database such as Obvibase.com. I use it all the time for a few personal projects.

  4. Marc

    I think you can do this retrospectively with a little less work than you guys are proposing. I don’t know that you need to track all the pertinent pre-draft comments since a lot of pundits put themselves out there when they give teams draft grades after on the day after the draft. It’s pretty easy to google this stuff.

    From the Seahawk perspective, the 2012 draft was pretty much panned. Mel Kiper gave them a C-, si.com (sports illustrated) gave them a C, NFL network a C, Bleacher Report (a fairly well respected sports blog) gave them an F. Here’s the link just so that you can see it.

    http://bleacherreport.com/articles/1165320-2012-nfl-draft-grades-power-ranking-teams-that-failed-on-draft-day

    However, that year included four Super Bowl starters (Bruce Irvin, Bobby Wagner, Russell Wilson, JR Sweezy), some valuable reserves, and Jermaine Kearse who scored the crazy spinning touchdown. Here was Russell Wilson’s response to the F grade.

    http://www.cbssports.com/nfl/eye-on-football/24433089/russell-wilson-remembers-people-ripping-seahawks-2012-draft-class

    Of course, most experts as a consensus liked their 2010 draft where they selected their starting LT (Russell Okung) and all world safety (Earl Thomas) in the first round. But if everyone thought that their draft picks were great then exactly what expertise did these guys demonstrate? My guess is that Reid, Mitchell, and I would have been excited about those two guys as well.

    All that to say that these guys may have more information to review, more time to consider it, and be in the enviable position of being paid to speculate about sports, but in the end they probably don’t know a whole lot more than you and I do…

  5. mitchell

    Man. Where were you on February 10 when I needed you? 🙂

  6. Reid

    Marc,

    I know you can find grades, but I’m more interested in specific comments but specific comments, positive and negative, about specific players. As i was telling Mitchell, a big part of draft expertise involves assessing risks. When experts think Clowney should be the #1, this doesn’t mean they’re absolutely sure he’s going to be great. In Clowney’s case, I think this choice means that his talent is so great, that he’s worth the risk. What makes him risky? In this particular case, there are questions of his work ethic and passion for the game–as the game tape seems to suggest that he takes plays off. Some people (including myself) wouldn’t choose him as the first pick because of these concerns. Now, let’s say Clowney turns into another Bruce Smith. Does that mean that all the experts who recommended drafting him first were right and all the ones who didn’t were wrong? No, I don’t think so. Nor does this mean that the knowledge and analytic abilities of the experts and scouts are meaningless. Indeed, I think good analysis and scouting is essential to assess the risk of each player.

    Therefore, I think people should focus on the specific comments scouts and experts make on players, in addition to whether the players they recommend strongly actually succeed. Suppose Clowney plays at the level of a Bruce Smith–but then let’s say he also takes plays off at times. To some extent, this legitimizes the red flags some scouts raised. Now, let’s suppose a scout was adamant that Clowney would definitely not succeed because the scout was convinced Clowney didn’t enjoy playing. In that case, this scouts credibility would, rightly, take a significant hit.

    See what I’m getting at? I want to track some of the more specific and detailed comments, the kind that are nuanced about a player. Whether a player succeeds or not, if the expert’s analysis was mostly accurate and their risk assessment was sound, then these are the type of scouts/experts I’d pay more attention to in the future.

  7. mitchell

    Okay, off-topic, but I have to ask: I’m assuming your decision that Clowney’s not worth the risk at #1 is a football decision, right? Don’t take a guy that risky at that spot in the draft because you have your pick of everyone else, so you might as well pick someone who doesn’t offer that risk.

    How far down in the first round do you think you’d take him? Or if not in the first, how far into the draft? I understand that removing the actual context of the actual pool of available players and without the actual needs of a specific team it’s impossible to say, but you DO say you wouldn’t take him at #1. May I ask, as a general matter of principal, where in the draft you think he might be worth the risk? Or is your philosophy of team-building such that you just wouldn’t want to risk having a guy like that on your team at all?

  8. Reid

    Okay, off-topic, but I have to ask: I’m assuming your decision that Clowney’s not worth the risk at #1 is a football decision, right?

    As opposed to what? (I just don’t think he would put seats in the stadium? If so, yeah, it’s a football decision.

    How far down in the first round do you think you’d take him? Or if not in the first, how far into the draft? I understand that removing the actual context of the actual pool of available players and without the actual needs of a specific team it’s impossible to say, but you DO say you wouldn’t take him at #1. May I ask, as a general matter of principal, where in the draft you think he might be worth the risk? Or is your philosophy of team-building such that you just wouldn’t want to risk having a guy like that on your team at all?

    No, I definitely would take a risk on him. I’m not sure when , though. As you say it’s so hard to know without knowing the options and specific team. I’m guess mid to late first round?

    Here’s some more background on my position:

    1. I just don’t like choosing a guy when there’s some concerns about his work ethic or desire to play. Some have speculated that Clowney didn’t play as hard last year out of fear he might get hurt. If you can shut off your motor for that reason (a sensible one, on some level), it suggests that maybe the player is more concerned with money than loving the game. This is speculative, but I can’t shake that thought, and I wouldn’t want to choose a player with those doubts, especially since…

    2. …many say this is one of the most talented drafts in a long time, maybe ever. If that’s true, then wouldn’t it better to choose someone who was less risky? Better yet, I’d prefer trading down and get more picks–if, indeed, this draft is extremely loaded.

    3. Would you rather choose a guy that you were 90% certain would at least be a solid starter–let’s say a player who was 4th best at his position or a guy you had a 50% chance of being Bruce Smith or a 50% chance of being an average player? You can make a case either way–I’m not saying my way is the best. But given my preferences, I’d want the 90% situation.

  9. Marc

    I apologize for missing a lot of the discussion on the other thread that Mitchell started about the draft being a crapshoot. I see that he already made a bunch of points that I agree with. I also agree with a lot of Reid’s points.

    Here are my disagreements. I don’t think the draft is a total crapshoot. If it were, you would have to attribute the great Seahawk drafts in 2011 and 2012 to luck rather than to skill in player evaluation. And even though those specific drafts were very good, the Seahawks still missed on about half of the players they picked.

    I also think that the effort required to try and get at the level of knowledge that Reid is seeking is not worth it for the armchair GM/fan. He already makes the point that you have to take a lot of the pundit evaluation with a large grain of salt, especially since so many of them disagree with each other. In my view, the experts who are getting paid millions of dollars can’t even get this stuff right and they have virtually unlimited resources. I can’t imagine trying to track pre-draft commentary for a group of players and then analyzing it years later to see who was right. Maybe I just lack ambition.

    So for the most part, I have fun looking occasionally at draft commentary but over the three months between the Super Bowl and the draft I just cannot get into it very much. I think a lot of it is a media creation to feed the hungry sports fan and pay the pundit’s salaries and I ignore most of the NFL draft stuff so that I can get swept up in the media creation/information avalanche for baseball.

  10. mitchell

    As opposed to what? (I just don’t think he would put seats in the stadium? If so, yeah, it’s a football decision.

    As opposed to the business decision of paying a guy #1 or #2 money. The business decision, which teams have to keep in mind but fans don’t, seems beyond the scope of our discussion, which is why I ask.

  11. Reid

    Marc,

    In my view, the experts who are getting paid millions of dollars can’t even get this stuff right and they have virtually unlimited resources. I can’t imagine trying to track pre-draft commentary for a group of players and then analyzing it years later to see who was right. Maybe I just lack ambition.

    It would definitely take some work, and one could legitimately question the value of putting in this effort. Also, let me admit that this is a nerdy endeavor–one that would probably not interest the casual fan.

    But let me several reasons this endeavor interests me (and you guys don’t have to respond to this, I’m just letting you know where I’m coming from):

    1. We might see great differences between scouts in terms of their ability to analyze players; or we may see the differences are minimal, even negligible. If the differences are large, then we’d know who to pay more attention to. Right now, we really don’t know if some pundits are better than others or they’re all basically the same. (At the very least, wouldn’t it be likely that some experts are better at analyzing some positions versus othesr?) To me, Greg Cosell sounds like he really knows what he’s talking about. He analyzes a lot of tape, and his descriptions sound thoughtful and well-argued. But how accurate are his assessments? I have no idea. I’d like to find out. (Shouldn’t there be some accountability?)

    Also, I’d be interested in knowing what some scouts/experts better than others. As an aside, part of my interest in this also relates to my interest in work evaluations in other professions, particularly those that are highly subjective–e.g., teaching. If you can identify what makes some people better at evaluation, you might be able to train others to do this.

    2. You mention that NFL draft is partly a product of media creation. I agree, but what I’m saying can help see through that, I think. (And I suspect this is one of the reasons none of the networks have done something like what I’m proposing.)

    I don’t think the draft is a total crapshoot. If it were, you would have to attribute the great Seahawk drafts in 2011 and 2012 to luck rather than to skill in player evaluation. And even though those specific drafts were very good, the Seahawks still missed on about half of the players they picked.

    While I agree with Marc that the process isn’t a total crapshoot in the way Mitchell seems to think, I want to comment on Marc’s remarks.

    While analyzing college players is really important, I believe it is only one component in determining a player’s NFL success. Two other components come to mind: development and scheme fit–or how a team/coach uses a player. Scouts may be completely correct in believing that a player will do well in the NFL, but that player may fail because coach/team fails to develop and/or use the player properly.

    Similarly, scouts might be correct in giving a low assessment of a player, but that player may succeed because the coach/team develops the player and/or utilizes them well. This is what I think happens with a lot of Seattle’s later round picks. Seattle seems good at assessing talent (probably assessing them for their specific scheme), but they also seem very good at developing the talent and using them in an effective way. Really, besides Thomas, I don’t think the individual players are great. They’re good, to be sure, but I get teh sense that they look at lot better because of the overall system. Their defense reminds me of the Patriots offense, where they seem seemingly less talented players can be extremely productive in their system. This doesn’t mean the players aren’t good, but the system makes them look better. Seattle’s defense seems to be the same way.

    In any event, what this suggests to me is that the pundits who gave Seattle low draft marks weren’t necessarily “wrong.” I’m pretty sure they’re not evaluating the players in relation to how they would specifically fit into the Seahawks. Instead, their assessment would be more generalized–and in that sense, I think they’re probably closer to the mark. But the way Seattle develops the players to fit their schemes can make these players highly productive.

    So, to tie this back to our discussion, if we had the specific comments of the experts, we could more accurately assess the quality of their analysis, versus a letter grade system. Their observations and concerns still could be valid even if some of these players have been productive, and if that’s the case, then that wouldn’t necessarily de-legitimize their credibility.

  12. Reid

    Louis Riddick’s recent tweet (6/13/2015) on Aaron Donald, DT for the Rams:

    “Aaron Donald will be one of the all time great 3-techs that this game has ever seen when it is all said and done.”

    Pretty bold prediction. He didn’t look like it last year, but it was Donald’s first year.

    Someone asked Riddick what made Donald so special. Riddick’s repsonse:

    “Explosive get off and power, leverage, football IQ, football character. Turn on his tape. He will be exciting to watch year #2.”

  13. Reid

    Saw this on twitter from a person that collects these type of tweets:

    Is there a way to search twitter for comments by people like Mel Kiper and Todd McShay, and then collect their comments here, to analyze how right or wrong they were?

    Here’s an article about Fred Segal, the guy that runs the account. Segal mainly seems to focus on mistakes, but I would like to also focus on when the experts get it right–and then ultimately look at the ratio of “rights/wrongs.” It’s not about making fun of the experts, but gauging how accurate they are and who’s the most accurate. (What would be disappointing is if most of them end up having basically the same batting average.)

You can add images to your comment by clicking here.