Almost as soon as GridIron Dynasty came out, players found issues with the WIS player rankings. Early on, pioneers such as paranoid0, leppykahn, and ardikus, along with many others, added their own takes to recruiting classes and player rankings. Although they have retired from the ratings business, their legacy lives on through the folks currently supplying fodder for discussion every time freshmen report.
cydrych publishes ratings for the Wilkinson and Stagg worlds at First Look Recruiting. His newspaper-style format provides a great look, and his top 25 rankings gave birth to the phrase "getting a helmet." At least when it comes to GD Recruiting.
gt_deuce is famous for his Media Guides, published for a variety of worlds. What they lack in aesthetics is more than made up for by the deep content of the guides, which are always highly anticipated.
bhazlewood brings the GUESS Reports to all eleven worlds. The intros on each world's home page are a sign of a warped sense of humor, but the ratings themselves are entirely serious.
joannes3000, along with co-creator jrlenart published the World Previews before going on hiatus in December 2008. They have active plans to relaunch - eventually - with a site even bigger and better than before.
QUESTION: Thanks everyone for agreeing to participate in the Ratings Round Table. First, let's take a peek behind the curtain. Tell us a little bit about yourself.
bhazlewood - Bob Hazlewood, 48, Windows Systems Administrator from Dallas TX. Married, no kids. Went to Colorado State University where I majored in Computer Science. Currently coaching in Leahy (DIII), Warner (DII), Dobie (DIII), and Yost (DII but looking to move up). I've also played in Hayes, and in Heisman under my SgtPepper08 ID.
joannes3000 - Jon, 29. Grew up in Athens, GA - go Dawgs. I moved to the Portland Metro area 8 years ago and been here ever since. Married with a 4 y/o son. My wife and I also have a second child due in April/May. I went to Portland State, majoring in Accounting and minoring in Community Development. I currently play in Rockne (D1), Leahy (D1), and Heisman (D3). I'll probably continue to take on teams in various worlds until I run out of free time and/or burn out on GD, whichever comes first.
gt_deuce - Aaron, 31, Hardware Engineer from Atlanta, GA. Married, no kids. I went to Georgia Tech where I majored in Electrical Engineering and now I work for a large networking company. I currently have teams in Stagg, Heisman, Dobie and Yost. I've also coached in Camp, Bryant and Warner in the past.
cydrych - Gerry Wallace, 40, Software Engineer in Huntsville, AL. Married, 2 kids. Attended Purdue, graduated from Alabama-Huntsville in Computer Science. Currently coaching in Wilkinson and Stagg. I have never used alternate ids though I have helped others recruit on occasion.
QUESTION: Except for Bob, who has a boring user name, what's the story behind your ID?
bhazlewood: Hey! The sgtpepper08 ID is at least quasi-interesting!
cydrych: My other hobby (besides family time, soccer, and computer games) is medieval reenactment. My SCA persona is a 10th century Welshman named Cydrych Wallas. It is definitely not a common name in the 21st century and makes for a good username on most systems.
gt_deuce: My original "name" was gt9766b, which was my student login for the computer systems during my days at Georgia Tech. When I created an alias to get a second team in Stagg -- to take over my former D-II team when I moved up with my original ID -- I was becoming known simply as "GT", so I figured I'd just call it gt_deuce... as in the second incarnation of "GT".
joannes3000: Mine's a combination of things that came together several years ago. It's been my user name ever since.
bhazlewood: So how many times have you been mis-cast as a girl by some unsuspecting coach? LOL
joannes3000: Entirely too many times to count. Jeff(kahleb) likes to ask me why I have a girl's name every now and then.
QUESTION: What motivated you to get into the "ratings" business in the first place?
bhazlewood: I sort of fell into it by accident, to be honest. When I first started playing, Ardikus was posting ratings for every recruit class, and since he had published most of his formula, I started “pre-rating” my own recruiting classes using that. My numbers never matched up to his, however, because of his mystical “work effort bonus” that I was never quite able to get worked out. When Ardikus dropped out of sight, I figured I could put his formula into a spreadsheet and get similar results, if I could get all the players downloaded. Luckily I was able to steal, uhm I mean utilize, the code from maddiesdad’s team management spreadsheet, to which I added a loop to get all the teams and copy the stats to a table. Then I boldly published the “Not The Ardikus Ratings”. Looking back at it now, I suppose it was a bit brash, especially considering I had only been playing the game for a couple of months.
gt_deuce: When I first started playing the game, paranoid0 was doing his recruiting class rankings and a coach named Ariansa was doing some team rankings. As I am, and always have been, a bit of a stat-head, these ranking systems intrigued me. So after playing for a while and developing my own player rating system -- which included much discussion with some really great coaches such as plague and showrunner and gaheel84 -- I decided I'd like to provide a full-service ranking system for recruits, recruiting classes, players and teams.
At that time, gaheel84 had just taught himself to program for Excel and had created a nifty little spreadsheet that would download all the teams for a given world/level and put them into a spreadsheet with a summary page. It ranked teams by position and then used positional ratings to create an overall rating. I took his work and spent months expanding on that base and refining it until my Media Guides were eventually born. Over time, I tweaked and added and tinkered until we got to where I am now.
cydrych: I was hooked on Paranoid's recruiting ratings from my very first season. After that, I would get on the forums the morning after recruits reported to their teams and just keep hitting refresh until he finally posted them. I did develop my own system to rate players and I always felt that my ratings matched up reasonably well to his. When he retired from GD (no more Paranoid rankings!), I decided to get "into the business."
joannes3000: When I first started playing GD, Jeff (jrlenart) did a conference preview for the NWC. Back then, paranoid was still doing his ratings, as was leppy. When those guys stopped, I approached Jeff about expanding his previews into something much bigger. You never know how great things are until they're gone, and this was the recurring theme on the forums after we went from two spectacular rating services to none. The rest is history. I conned Jeff into taking on the massive project of coding things up, and I work as more of the promoter. I also do some testing - basically running things and complaining when they don't work right - but Jeff is the brains behind everything.
We started with just a handful of conferences and everything was done semi-manually. The ratings were originally on a free google page but we outgrew that within a few months. What we have (technically had, since we're currently down) is a nearly automated system that only takes a couple of prompts to pull the data, crunch the data, and get it ready to be uploaded to the site. When we go back online, the process will become even more automated.
bhazlewood: OK, I’ll ask it since everyone is going to want to know… if it’s all automated, why are you guys on hiatus now? Couldn't you continue to post things the "old" way while you are working on your upgrade in the background? :)
joannes3000: I think it was a perfect storm for us. Jeff and I had been going back and forth on various ways to update our previews. As you know, when the update hit, there was going to be downtime to rework the data pull.
As I've mentioned in countless sitemails and on the forums, Jeff and I have families with young kids so time is a very precious commodity for us. When it was all said and done, time ultimately became an issue. IMO, the GD community is well-served by your rankings. This made it easier to make the decision to just take things down completely.
bhazlewood: (laughs) Oh great, now everyone will say it’s my fault!
QUESTION: How often do you "tinker" with your ratings formula?
joannes3000: Hardly ever.
cydrych: I haven't tinkered with my ratings formula since I got both my teams to Division 1. I recognize that the formula should not be the same at D3 as it is at D1, mostly because cores are so much more important at D3 in comparison to the secondary skills. As you go up through the divisions, the secondary skills start taking on more significance. But I am a bit lazy and I force the rating system (at this point, designed with a focus on D1) on all divisions equally. I would like to improve this in the future by making 4 different formulas and evaluating each division with its own formula.
gt_deuce: Not often, anymore. I used to adjust it quite frequently, but now I've settled on a system that I feel works very well, so I don't mess with it.
bhazlewood: I also used to tinker with them all the time, when I first got started. There’s no such thing as the “perfect” ratings formula but that wasn’t going to stop me from trying to come up with one. I haven’t done anything to the formula in about a year now, although I started to do so earlier this year… that was just about the time QB’s suddenly became crappy and I decided to wait a bit but never went back to it. I’ve been thinking about tweaking the QB formula now to look more at WE – since I don’t have access to player potential – because you are expected to “grow” your QB’s now. I’m also beginning to think my stamina formula needs a cap, because after a certain point it doesn’t make any difference anyway. I'll probably release a tweaked system the next time Rockne rolls around. I like the idea of different formulas for each division, so I may steal that for changes in the long term.
QUESTION: Who, besides yourself of course, has had (or continues to have) a large influence on your ratings formula?
bhazlewood: Well, obviously Ardikus did at first, since they were his numbers. Now, however, there isn’t any single person – there’s a group of about six-ten coaches that I really respect who I normally sitemail when I am considering making changes. Of course, they usually give me conflicting advice. LOL And just so nobody gets upset, there’s a lot more coaches that I respect than that – these are just the guys that I have competed against in different worlds.
joannes3000: We set up a roundtable discussion and got lots of great input. My personal experience led me to value the input given by jeffkahleb and cravedogg the most.
cydrych I started out trying to mimic Paranoid's ratings so he clearly was my origin although I never had direct contact with him about my formula. Since then, I have done some tweaking (for instance, when they made stamina a bigger factor) but largely it has stayed pretty stable for the past 8 months.
gt_deuce: Probably the two biggest influences were gaheel84 and plague. Of course, gaheel84 and I pretty much co-developed the formulas and some early discussions with plague really showed me that I needed to lighten up on secondary weights, so as not to "water down" the rating.
QUESTION: Do you compensate for distribution of talent across positions (or as the statistician would call it, "normalize your results for positional discrepancies")? If so, how do you do this?
joannes3000: I believe so, but Jeff will have to give the details and/or confirm.
gt_deuce: I do normalize across positions, but it's very un-scientific, I'm embarrassed to say. For each level, I took some time to play with weights until I got a good mix of players in my Top 100. Since this is "for fun", I figured it'd be boring to see a bunch of RB and WR and TE in the Top 100, to the exclusion of other positions, so I tinkered with the positional weights until I got a nice mix.
cydrych: It used to be that who ever recruited the most linemen and RB/WRs tended to score better than everyone else because their cores usually score higher in most rating formulas. But I never liked the idea of throwing in a fudge factor for positions to match up better with other positions. That's just more ambiguity that would likely have to vary every season (and for every world) based on the talent distribution that particular year. Since I am inherently very lazy, I wanted a system that didn't rely on such "bonus" factors.
Instead, I find the best player at each position for each division and he becomes the premier guy to which I compare every other player at his position. Then every player at that position is a fraction of the premier guy. If a LB is within 10% of the best LB available, then he gets an "A". This way it doesn't matter if his scores are 10 points lower than the "A's" of the RBs. I don't have to compute a fudge factor to bring all the positions in line with all the others. The calculated fractions give me the "grade" for that player and when I put it all together, I am combining the grades from the fractions not the computed ratings so everyone is already "normalized" without having to use magic.
bhazlewood: UGH. The word “normalize” almost makes my ears bleed. I really have problems with the idea, because to me, talent is what it is. If the top players at DIII are almost all linemen, then that’s what they are. But I got so many complaints about the top 25 lists being almost exclusively OL/DL/RB that I finally gave in and started grading on a curve. Basically OL/DL get 100 points, and the other positions get between 105 and 115. I don’t really like the way it’s done, but I’m way too lazy to go through and actually statistically normalize the player populations.
QUESTION: What is your position on publishing your ratings formula, and why?
cydrych: Only one other coach has had access to my specific ratings formula. If everyone had my formula, I suppose I would have a harder time finding players to sign.
bhazlewood: If Ardikus hadn’t published his ratings formula, there wouldn’t be GUESS ratings today. So I feel like I owe it to everyone to publish the formula. I’d probably feel different about it if I had developed it completely on my own.
gt_deuce: I don't give out my formulas. The reason being, recruiting is already competitive enough... I don't need to be guiding scores of other coaches in the same direction as my own efforts by publishing my formulas and make it even more difficult to land the guys I like. I've always been more than willing to share just about all of my GD knowledge, but that's one I protect.
joannes3000: We haven't publicly given out our rating formula and I'm not sure we ever need to. I've always felt that everyone's rankings are quite similar and in fact, it's the variances that make them more worthwhile. If I get ranked lower in one set of rankings, it causes me to look at my players and see if I'm overvaluing or undervaluing certain attributes.
QUESTION: Do you use your rating system when doing your own recruiting?
cydrych As I alluded to in the previous question, I do use my rating system when I recruit. I usually know how the recruit will be graded. Still, if presented with the dilemma where I need another OL and the best I can get is a "B" recruit -or- I could recruit an "A" rated LB, I will still recruit for need (take the OL) rather than to get a good grade (by grabbing the LB). Winning games is more important to me than looking good on my own recruit rankings.
joannes3000: It probably weighs in the back of my mind, yes. But I've moved to the eyeball method years ago and haven't touched a recruiting spreadsheet since.
gt_deuce: To some extent, I do use my own formulas. I populate my watch list with the advanced search tools, then run a quick macro to collect and rate them for further analysis, before I finally make my selections. I used to spend the time to download all the recruits in the pool and browse them, but I don't do that any longer. Additionally -- as I feel that ratings are never an absolute measure of a player's worth against another -- I only use my ratings as a guideline. I often take players with lower ratings that I like better, for various reasons (growth, a particular attribute I'm looking for in that class, etc).
bhazlewood: I use a modified version of the ratings, and then only as a guide. I basically take the points away from STA and WE, since I normally won’t recruit guys below a certain level, and re-distribute them focusing on the cores. But I don’t compare ratings across positions, because of the uneven distribution we talked about earlier. Plus the ratings don’t take potential into account, so two guys might have close numbers but the lower guy might be a better player in two or three seasons.
QUESTION: Do you feel that doing the ratings has made you a better recruiter?
cydrych: A fair question and I'll give you a fair answer. If I only used my spreadsheet to publish the ratings, I don't think I would be a better recruiter for the experience. But I am most definitely a better recruiter because of the information I gather with my spreadsheet and how I use it before and during the recruiting season. Doing the ratings is just something I do to give back to the community, not something that gives me a competitive advantage or makes me a better recruiter.
joannes3000: In a very roundabout way, yes; but not in the data-pulling, number-crunching way you would probably think. Doing the world previews has led to a lot of conversations with what I consider some of the GD Elite Coaching Mafia. In turn, those conversations have helped enlighten me on the fine art of recruiting.
bhazlewood: It may have made me a better evaluator of talent, but there’s so much more that goes into recruiting than that. Actually though, I think it’s more a matter of becoming a better recruiter is making me into a better “rater”, because I look at how players will compare in the ratings formulas, and I see where I might need to adjust some.
gt_deuce: No, I really feel like recruiting/talent evaluation was my strong suit at the point where I started putting together the ratings. If I wasn't confident in their value at that time, I probably wouldn't have "gone public" with them.
QUESTION: Do you regularly visit / check out the other ratings sites, and if so, what do you find most interesting / appealing about them?
bhazlewood: I look at all the other sites. As a matter of fact, the recent changes I made to how the GUESS reports are presented are directly due to the way cydrych and joannes3000/jrlenart publish their stuff. A few of the reports I publish were inspired by gt_deuce's Media Guides - and when pittsky was doing ratings, his work gave me ideas for others.
I like comparing the different ratings of course... and in fact I've been tempted to get a team in Stagg, since that's the one world that everyone is doing their stuff for.
A side note on gt's ratings: he always ranks my teams higher than I do, so either he can't evaluate talent or I'm a lousy coach LOL. (Based on a comparison of our records I'd say it tends toward the latter.)
gt_deuce: I visit all of them -- cydrych, bhazlewood, and joannes. All of those have a presentation that far outpaces my Media Guides and includes information that I don't provide. Plus, I just love to compare my results against those of other coaches who, no doubt, have a different perspective than I do.
cydrych: I definitely like to check out the other rating sites. Especially since my rankings are derived so differently than the others, it is very interesting to see how my ratings match with the others. Sometimes I do better in someone else's system, sometimes I do worse but it’s usually reasonably close. A top 25 team in GT's rankings is usually a top 25 team in mine, for instance. I also find that the rating services tend to differ on who is #1 but I find that to be the fun part!
joannes3000: I visit GD Reports (GUESS) and gt's media guides every season. They're the rivals/scout of the recruit rankings.
QUESTION: How do you generate the ratings reports? How long does it take, from the start of the process to the posting of the ratings?
bhazlewood: I have an Excel macro that downloads all the rosters, does all the ratings, and then sorts out the data to different worksheets. Each worksheet gets saved as a CSV file, which I upload to the database (each world has its own database.) Then I need to do a little HTML editing of the world's home page to add the new season. All told, it takes about two hours - the biggest chunk of that is the data download.
cydrych: I also have an Excel macro that downloads the rosters and does all the ratings. Then the macro calculates the final grade for each team and even writes out the html files for the webpage. All of that is pretty automated and if I think about it before I go to bed, I even have my macro set to start at 4:30 when I'm asleep. Then, by the time I get up, everything has been run and all that's left is to manually changing my index page to reflect the new season and upload the files to my web server. If I don't think about it before bed, I have to kick off the macro when I get up and get my wife to upload my files to the web server for me since I will be at work by the time the macro is done. It takes 2-3 hours to run and get posted.
gt_deuce: Funny enough, I'm trying to play catch-up right now for my missed Yost and Dobie postings. Anyways, I think - including download time - it takes me about two hours. The reports are generated by Excel macros that download, parse, and sort the data, then loop through it, writing into HTML tables, which are uploaded to Bob's site. From there, a few simple edits to my main page(s), and they're ready.
joannes3000: It's about a 30 minute process from start to finish, depending on whether or not Comcrap is playing nice when it's time to pull the data. Step one grabs all of the data and dumps it into Access. This is the bulk of the time commitment and usually takes 15 minutes. Step two applies all of the ratings and then generates a folder with all of the files necessary for the website. This takes about 10 minutes. Step three is unnecessary, but I update the website from an indexing perspective. Step four uploads the files created from step two and the index file from step three. This step and the previous one take about 5 minutes combined.
bhazlewood: What data are you downloading that you can get it all in 15 minutes?
joannes3000: Team rosters, just like everyone else.
bhazlewood: I'm going to HAVE to learn how to program in Access then! LOL
QUESTION: With regards to your ratings or the way they are presented, complete the following: If I only had the time/skill/knowledge to do so, I would ________ .
gt_deuce: … spend more time on the aesthetics. I'd probably integrate helmets and give the entire site more visual appeal and organization.
cydrych: … like to move First Look Recruiting to be completely web-based. Right now, most of the work is done in Excel spreadsheet macros then moved online. If I could get all the data and calculations done via php/mysql or something similar, I could then take my recruiting service to the next level. I would offer coaches the ability to enter their own rating system, then on freshmen reporting day, everyone could see how the teams rank with their own rating system (instead of mine). Then, taking all of the coaches' rankings, I could come up with an aggregate ranking that would be like a true coaches’ poll. That, to me, would be super cool.
joannes3000: … I can't answer this because I have some ideas that I haven't even run by Jeff yet! :) I have some insane ideas that I plan on working toward.
bhazlewood: … like to find some way to link across seasons, so you can see how a player has progressed from year to year. I’m afraid of the performance hit that would entail though, with the extra reads to the database. I'd also love to bring the old style reports into the database for the new reports, but I'm not sure that the "value add" for that would be worth the amount of work it would entail.
That's it everyone. Thanks for taking the time to participate, and thanks for giving back to the GD Community!