• Welcome to OGBoards 10.0, keep in mind that we will be making LOTS of changes to smooth out the experience here and make it as close as possible functionally to the old software, but feel free to drop suggestions or requests in the Tech Support subforum!

A Statistical Analysis of CBB Recruiting Rankings as Correlated to Team Success

deacon14

Well-known member
Joined
Aug 7, 2015
Messages
7,767
Reaction score
5,619
So this is an analysis that I decided to do myself because I was curious about the correlation/ predictive power of recruiting rankings in college basketball and winning. I was particularly interested in ACC basketball from the period 2014-2019 (5 seasons), because that goes back to when Danny Manning started at Wake. I’ll try to make this Wake-centric, but really it’s more of an overall analysis. I’m not a statistician, so please don’t destroy me if something is done incorrectly. Also, this is going to be kind of long, because I ended up in several (hopefully interesting) rabbit holes.

I’ll start by defining a few terms and explaining what I did so that the way I present the data makes sense. I went back for each individual team (5 teams for each of the 15 ACC schools), and came up with a list of players that I deemed to be “contributors” on the team, for which the cut off line was > 1 PPG, > 5 MPG, AND > 10 Games Played. For this list, I went back on 247 and found the recruiting ranking of each player when leaving high school, which includes a star rating (2-5) and a composite ranking, which is a decimal lower than 1 (or in RJ Barrett’s case = to 1). Some players didn’t have a 247 recruiting profile; they were given a 1 star and 0.7 rating, as no players that had a profile were worse than that. I understand that 247 fails to rank some quality international guys (ex: Dinos Mitoglou), but this analysis is meant to test 247’s rankings anyways, so if they don’t list a guy they are considered a bad recruit.

At this point I averaged both the star rating and composite rating of all contributors on the team, and these values became the Average Star Rating (ASR) and Total Composite Rating (TCR) for each team, and there were 5 teams for each school. If I go back and play with this data more, I will try to weight the impact each player has on the overall ratings based on something like MPG, but for this analysis all contributors had equal weight towards the overall team rating. At this point I wanted to factor in wins, and used both Total Wins and ACC Wins. I came up with two more terms, which were Total Wins/ Average Star Rating and ACC Wins / Average Star Rating. (The math being done here is obvious). Essentially, these values tell you how many games a team won per the star rating of the average contributor on the team. Higher numbers would indicate overperforming recruiting expectations, and lower numbers would indicate underperforming. All 5 seasons iterations of these 4 values (ASR, TCR, Total Wins/ASR, ACC Wins/ ASR) were averaged in order to assign 4 values (Average ASR, Average TCR, etc.) for each school over the full time period (2014-2019). I understand that the average of an average is not usually a great idea, but considering that each team is independent as far as its win total, and rotation sizes are different, it made more sense to me than simply averaging all players on the 5 teams and dividing the total wins over 5 years by that.

Last couple of things that I looked at: grad transfers on each team, 1 star recruits per team, and the frequency with which a top 3 rated recruit led the team in scoring. Another thing I did for all teams 2015-2016 to 2018-2019 (4 seasons) was to come up with a percentage for how many contributors on the team are returning from being a contributor for the same school in a previous season, which is called Percent Returning. So with all of that said, let’s dive into the data and see what happened.

First, I made a simple linear regression of total wins and ACC wins with ASR and TCR (4 independent regressions). For the regressions, I’m going to mostly talk about R squared values. This is a value from -1 to 1, with -1 being a perfect negative regression (high recruiting rating guarantees low wins) and 1 being perfect positive regression (high recruiting rating guarantees high wins). In social sciences, which is basically what this is, an R squared above 0.35 suggests a meaningful correlation. I did these regressions in two ways. First way, I did it with the summary data over the whole time period. For example, how well does the fact that Wake’s average ASR is 2.8952 and average TCR is 0.8541 over 5 teams predict that Wake would win 65 games and 24 ACC games over those 5 seasons? Well, the answer is pretty damn well:
TCR and Total wins R2 = 0.7199
TCR and ACC wins R2 = 0.6579
ASR and Total wins R2 = 0.6705
ASR and ACC wins R2 = 0.5894

This suggests that TCR does a bit better job than ASR, and that both do a better job with Total Wins than with ACC wins. Both of these things I expected, as the composite ranking is 247’s attempt to fine tune star ratings, and Total Wins is just a bigger sample set, although I guess you could make the argument that simply having more talent gives you a greater advantage in an OOC game than an ACC game. Regardless, I was stunned by these values. Over the 5 year period, the recruiting rankings do a great job of predicting the number of games a school will win over that span, considering that there are plenty of other factors at play in who wins a basketball game other than the raw/perceived talent coming out of high school.

The second way I did these regressions is single-season oriented instead of school-oriented. The difference here is that the inputted data is each individual teams TCR and ASR along with its win totals, instead of a school’s average TCR or ASR and win totals over the 5 year period. So how well does a TCR of 0.8438 and ASR of 2.818 predict that the 2015-2016 Deacons will win 11 games and 2 ACC games? Not as well, but still pretty well.
TCR and Total wins R2 = 0.4263
TCR and ACC wins R2 = 0.3709
ASR and Total wins R2 = 0.3951
ASR and ACC wins R2 = 0.3384

The same trends that I discussed previously are found here, which cements to me that TCR is more correlated to winning than ASR and that both are more correlated to total wins. What’s interesting here is that the R squared values fall a good bit. I think that can be attributed to the fact that it is much more likely that a team can outperform (or underperform) its recruiting ranking in a single season vs consistently doing so over a five year period. Sample sizes as far as games involved in each data entry are five times smaller. Fluke seasons happen. A team can have a low rated recruit like John Collins carry them to 19 wins in a miracle (Get it? Like Danny the miracle?) season, but the odds of that happening again the next season are slim.

Moving on, let’s do some analysis on which schools outperform/underperform their recruiting rankings. To me, this stat is either praise or indictment on a coach, do you win more or less games than recruiting rankings would suggest? Do you get more or less out of your players than average? I mentioned these earlier, but the stats here are Total Wins/ASR and ACC Wins/ASR, which again is how many games do you win per the star rating of the average contributor on your team. I wish I could have incorporated TCR instead of ASR here, but the last two paragraphs showed me ASR is good enough, and TCR would have been tricky/confusing math with it being a value between 0 and 1. I can go back and do this if there’s interest. Anyways, here’s the rankings of each ACC school in both stats:

Average of Wins per season/Average Star Rating

Virginia 8.6373
Florida State 7.4882
UNC 7.3084
Louisville 6.9403
Duke 6.7763
Notre Dame 6.6387
Clemson 6.4473
Miami 6.229
Virginia Tech 6.0668
NC State 5.9197
Georgia Tech 5.5448
Pitt 5.5138
Syracuse 5.4300
Boston College 4.8027
Wake Forest 4.5024
Average 6.2830

Average of ACC Wins per season/Average Star Rating

Virginia 4.2649
UNC 3.2871
Louisville 3.2808
Florida State 3.2596
Clemson 2.9813
Duke 2.9096
Miami 2.7826
Notre Dame 2.7211
Virginia Tech 2.5694
Syracuse 2.4236
NC State 2.37
Georgia Tech 2.1419
Wake Forest 1.6597
Pitt 1.6433
Boston College 1.4008
Average 2.6464


Just because I’m sure you want to scroll through two more long tables, here’s the tables of the actual ASRs and TCRs (you can look up wins on your own).

Average TCR

Duke 0.9784
UNC 0.9597
Syracuse 0.9478
Virginia 0.9198
NC State 0.9154
Notre Dame 0.9152
Miami 0.9111
Louisville 0.9106
Virginia Tech 0.8959
Florida State 0.8776
Clemson 0.8603
Georgia Tech 0.8549
Wake Forest 0.8541
Pitt 0.851
Boston College 0.8262
Average 0.8985

Average Star Rating

Duke 4.4022
UNC 4.0218
Syracuse 3.8111
Miami 3.5123
Notre Dame 3.4978
NC State 3.4638
Virginia 3.42
Louisville 3.3915
Virginia Tech 3.2723
Florida State 3.0643
Clemson 2.9537
Georgia Tech 2.9345
Wake Forest 2.8952
Pitt 2.7796
Boston College 2.5906
Average 3.3340


A couple things that jump off the page:

Wake’s ratings suggest what we all know, the coaching is below average. We have bad talent, and we get even less wins out of that talent than the average ACC coach would.

WFU, Pitt, and BC are the worst in the ACC at both having talent and winning with the talent they do have. Could make an argument that bad recruits should be actually ranked lower, which would be forgiving towards the coaches (except for the recruiting part of their jobs) of these schools, but I’ll let you think on that one.

Holy shit Virginia/ Tony Bennett. I mean holy shit. They have good (but not elite) talent, but they get way more out of their talent than anyone else does.

Duke looks bad here because they have had significantly more talented rosters than anyone else but won about the same amount.

By both TCR and ASR, Cuse had the 3rd most talent, yet certainly don’t have the third most wins. Make of that what you will.

FSU and UL have both done a lot more winning than their talent would have predicted, and NC State has done a lot less. FSU’s ratings are dragged down by the large rotations that they play, which include a strange number of unranked recruits (2.2 1 star players per team).

The middle is pretty packed, so being ranked 5th and 8th in TCR really isn’t that different at all.

I was intrigued by the similarities between the talent on NC State and Notre Dame, but I’ll leave the low hanging fruit making fun of NCSU to you all.



I promise this is almost over. I’ll just throw a few Wake Forest stats at you along with the ACC averages, plus some interesting teams.

Wake has won 65 games and 24 ACC games over this period. ACC averages are 105.6 and 45, and we are 14th in each (thanks BC).

Wake returns on average 49.18% of contributors from its previous team, ACC average is 55.89%. I ran some regressions here to see if this stat impacts winning, but it’s a pretty weak positive correlation. However, 7 of the 9 ACC teams to win more than 100 games had higher than 56%, and only Clemson returned that many without winning 100 games (95 wins). Duke and Cuse are the winners that don’t return contributors, and I might go back and look into this again excluding Duke, because that one and done factory really screws this up (45.15% returners).

The frequency with which a top 3 recruit by composite ranking led the team in scoring was 56%, which was honestly much lower than I expected.

Average ACC school had 0.467 grad transfers per team. Grad transfer is designated as a guy that came to the school with only one year of eligibility remaining. Wake had 0.8 per team (4 in these 5 seasons). To me, it seemed like bad teams took more grad transfers, but I didn’t do anything to prove that because it’s really not that revolutionary of a conclusion.

Wake averaged 2.0 1 star recruits per team. This counts the same guy again for each year he is on the team. ACC average is 0.96 per team and Wake was second highest in this stat with BC. Duke, UNC, and Notre Dame all had 0. These were also the only schools to not take a grad transfer coincidentally.

The teams that most under-performed its ratings in relation to total wins include:
BC 2015-2016. ASR: 2.8 TCR: 0.8387. 7-25 (0-18)
Pitt 2017-2018. ASR: 2.3 TCR: 0.8075. 8-24 (0-18)
Wake Forest 2017-2018. ASR: 3.091 TCR: 0.875. 11-20 (4-14)

The teams that most overperformed its ratings in relation to total wins include:
UVA 2018-2019. ASR: 3.6 TCR: 0.9291. 35-3 (16-2)
Notre Dame 2014-2015. ASR: 3.4 TCR: 0.9156. 32-6 (14-4)
UVA 2017-2018. ASR: 3.3 TCR: 0.9103. 31-3 (17-1)

So there’s a LOT more I can do/talk about, but I’m going to end this post here. Ask away for any of the data I didn’t mention, I probably have it. Future things I’m going to look at include: rotation size (# of contributors), NBA draftees on a team, Tournament success, weighting the impact a player has on the team ratings based on MPG, and whatever else you all suggest. An easy thing to do would be to change the cutoff line of what a contributor is, if you guys want me to move that cutoff up or down. If anyone is interested in looking at my data, or seeing the pretty graphs I made that I can’t figure out how to embed here, shoot me a DM, and I’ll figure something out to give you view only access to the google doc (as long as you promise to not make fun of how unorganized the raw data is). If anyone takes me up on that and finds a mistake I made, please let me know. Also, please don’t be lame and take my work and repost it somewhere else without telling me. Thanks for reading this, it was a lot of fun to do/write, and let me know what you think.
 
Also if there is any interest for me to do something like this for football let me know. That might capture the interest of more Wake fans currently.
 
Do this but with football. You owe us for making us read that depressing shit.
 
Solid work.
You should send this and the rest of your data to John Currie.
He doesn't need to sit on the sidelines behind Manning and take notes...
This is all the dammed information he fucking needs.
 
Do this but with football. You owe us for making us read that depressing shit.

Solid work.
You should send this and the rest of your data to John Currie.
He doesn't need to sit on the sidelines behind Manning and take notes...
This is all the dammed information he fucking needs.

That may be true for part of the issue. However, making an excellent hire is also important.

The logic of
Any other coach > Jeff [Redacted] =Danny Manning.

Repeating that,

Any other coach > Danny Manning, year sux = ????

Clearly,

Next Wake BB Coach >>> Danny Manning is the equation for which Currie should be gathering much information.
 
I can start working on football, but it will be a while because of how much bigger team sizes are. I'll post it here whenever I get that done.

And I genuinely didn't mean for this to be something that was entirely taken to despair WFU basketball and hate on Manning. Obviously those things will come out of it here, but I was really more doing it to test the validity of using the ratings of recruiting classes as an indicator of future success, and to see who outperforms their recruiting rankings and who doesn't. I started working on this a while back when I saw the 1000th person try to make the claim that 247 recruiting rankings are inherently flawed for some reason or another, which I think I've shown is an argument that is going to need to show some evidence in order to be taken seriously.
 
I can start working on football, but it will be a while because of how much bigger team sizes are. I'll post it here whenever I get that done.

And I genuinely didn't mean for this to be something that was entirely taken to despair WFU basketball and hate on Manning. Obviously those things will come out of it here, but I was really more doing it to test the validity of using the ratings of recruiting classes as an indicator of future success, and to see who outperforms their recruiting rankings and who doesn't. I started working on this a while back when I saw the 1000th person try to make the claim that 247 recruiting rankings are inherently flawed for some reason or another, which I think I've shown is an argument that is going to need to show some evidence in order to be taken seriously.

Good stuff, deacon14. I always find these type of data interesting. I agree with your general conclusions regarding rankings, coaching, and Manning - he isn't recruiting well, isn't coaching well, and both are leading to the expected poor results.

However, I do disagree that these data strongly support the accuracy of recruiting rankings. I'm coming at this from a biomedical perspective, but correlations (R2) of 0.34 to 0.43 for individual seasons are pretty weak. Even correlations in the 0.5 to 0.65 range are still weak. In addition, instead of attributing the gap between rankings and results to coaching, I would argue that some of that gap occurs because the coach is much better at accurately identifying talent than 247. One way to potentially assess this is to give each player a grade at the end of their college career (points for being drafted, all acc etc) and then see how much that explains the win/loss results. Finally, I think you'll see even less correlation for football, as the football recruiting rankings are less accurate than basketball, in my opinion.
 
However, I do disagree that these data strongly support the accuracy of recruiting rankings. I'm coming at this from a biomedical perspective, but correlations (R2) of 0.34 to 0.43 for individual seasons are pretty weak. Even correlations in the 0.5 to 0.65 range are still weak.

That's funny, I'm also in the biomedical field, was a chemistry major at Wake. I originally thought the same thing about the R2 values, but it really varies from field to field. In social sciences, where there are infinitely more variables than a chemical reaction or even biological system (people making decisions and doing things really extrapolates things), correlations are never going to be as strong. I'm certainly not an expert on this, and maybe they are weaker than I thought, but I don't think they are particularly weak in this application.

https://condor.depaul.edu/sjost/it223/documents/correlation.htm
 
In addition, instead of attributing the gap between rankings and results to coaching, I would argue that some of that gap occurs because the coach is much better at accurately identifying talent than 247. One way to potentially assess this is to give each player a grade at the end of their college career (points for being drafted, all acc etc) and then see how much that explains the win/loss results. Finally, I think you'll see even less correlation for football, as the football recruiting rankings are less accurate than basketball, in my opinion.

I actually really like that idea, hadn't really thought about it from that angle. Either way though, it means the coach is better at his job though right? Either he is more accurately identifying talent or he is better at developing it.

I mostly agree with that assessment on football. There's just too many 2 and 3 star kids to accurately sort through for the recruiting services. Plus they get less opportunities to see the kids play. Maybe at the high end (4/5 star) they do a good job, but I'd be surprised if the correlation rivals what they do in basketball.
 
That's funny, I'm also in the biomedical field, was a chemistry major at Wake. I originally thought the same thing about the R2 values, but it really varies from field to field. In social sciences, where there are infinitely more variables than a chemical reaction or even biological system (people making decisions and doing things really extrapolates things), correlations are never going to be as strong. I'm certainly not an expert on this, and maybe they are weaker than I thought, but I don't think they are particularly weak in this application.

https://condor.depaul.edu/sjost/it223/documents/correlation.htm

Also, included that link just for the one table. Didn't mean to sound condescending and teach you what correlations are lol, I obviously am not doing that.
 
Lots of good info. Confirms conventional wisdom:

- WF is poorly coached (as is/was BC and Pitt).
- UVA is well-coached.
- Duke recruits at a ridiculous level and under-performs its talent. With the talent that Duke has had, love that fact that it has been a decade since Duke has finished 1st in the ACC regular season.
 
That's funny, I'm also in the biomedical field, was a chemistry major at Wake. I originally thought the same thing about the R2 values, but it really varies from field to field. In social sciences, where there are infinitely more variables than a chemical reaction or even biological system (people making decisions and doing things really extrapolates things), correlations are never going to be as strong. I'm certainly not an expert on this, and maybe they are weaker than I thought, but I don't think they are particularly weak in this application.

If only there was a social scientist on these boards that could provide perspective on these correlations...
 
I actually really like that idea, hadn't really thought about it from that angle. Either way though, it means the coach is better at his job though right? Either he is more accurately identifying talent or he is better at developing it.

Absolutely. I think some part of it is player development, but I think more of it is the ability to identify talent. I think most college coaches are better at identifying talent than most of the "recruiting analysts" (aka teenagers paid below minimum wage).
 
Back
Top