Results 1 to 16 of 16

Thread: A Statistical Analysis of CBB Recruiting Rankings as Correlated to Team Success

  1. #1

    A Statistical Analysis of CBB Recruiting Rankings as Correlated to Team Success

    So this is an analysis that I decided to do myself because I was curious about the correlation/ predictive power of recruiting rankings in college basketball and winning. I was particularly interested in ACC basketball from the period 2014-2019 (5 seasons), because that goes back to when Danny Manning started at Wake. Iíll try to make this Wake-centric, but really itís more of an overall analysis. Iím not a statistician, so please donít destroy me if something is done incorrectly. Also, this is going to be kind of long, because I ended up in several (hopefully interesting) rabbit holes.

    Iíll start by defining a few terms and explaining what I did so that the way I present the data makes sense. I went back for each individual team (5 teams for each of the 15 ACC schools), and came up with a list of players that I deemed to be ďcontributorsĒ on the team, for which the cut off line was > 1 PPG, > 5 MPG, AND > 10 Games Played. For this list, I went back on 247 and found the recruiting ranking of each player when leaving high school, which includes a star rating (2-5) and a composite ranking, which is a decimal lower than 1 (or in RJ Barrettís case = to 1). Some players didnít have a 247 recruiting profile; they were given a 1 star and 0.7 rating, as no players that had a profile were worse than that. I understand that 247 fails to rank some quality international guys (ex: Dinos Mitoglou), but this analysis is meant to test 247ís rankings anyways, so if they donít list a guy they are considered a bad recruit.

    At this point I averaged both the star rating and composite rating of all contributors on the team, and these values became the Average Star Rating (ASR) and Total Composite Rating (TCR) for each team, and there were 5 teams for each school. If I go back and play with this data more, I will try to weight the impact each player has on the overall ratings based on something like MPG, but for this analysis all contributors had equal weight towards the overall team rating. At this point I wanted to factor in wins, and used both Total Wins and ACC Wins. I came up with two more terms, which were Total Wins/ Average Star Rating and ACC Wins / Average Star Rating. (The math being done here is obvious). Essentially, these values tell you how many games a team won per the star rating of the average contributor on the team. Higher numbers would indicate overperforming recruiting expectations, and lower numbers would indicate underperforming. All 5 seasons iterations of these 4 values (ASR, TCR, Total Wins/ASR, ACC Wins/ ASR) were averaged in order to assign 4 values (Average ASR, Average TCR, etc.) for each school over the full time period (2014-2019). I understand that the average of an average is not usually a great idea, but considering that each team is independent as far as its win total, and rotation sizes are different, it made more sense to me than simply averaging all players on the 5 teams and dividing the total wins over 5 years by that.

    Last couple of things that I looked at: grad transfers on each team, 1 star recruits per team, and the frequency with which a top 3 rated recruit led the team in scoring. Another thing I did for all teams 2015-2016 to 2018-2019 (4 seasons) was to come up with a percentage for how many contributors on the team are returning from being a contributor for the same school in a previous season, which is called Percent Returning. So with all of that said, letís dive into the data and see what happened.

    First, I made a simple linear regression of total wins and ACC wins with ASR and TCR (4 independent regressions). For the regressions, Iím going to mostly talk about R squared values. This is a value from -1 to 1, with -1 being a perfect negative regression (high recruiting rating guarantees low wins) and 1 being perfect positive regression (high recruiting rating guarantees high wins). In social sciences, which is basically what this is, an R squared above 0.35 suggests a meaningful correlation. I did these regressions in two ways. First way, I did it with the summary data over the whole time period. For example, how well does the fact that Wakeís average ASR is 2.8952 and average TCR is 0.8541 over 5 teams predict that Wake would win 65 games and 24 ACC games over those 5 seasons? Well, the answer is pretty damn well:
    TCR and Total wins R2 = 0.7199
    TCR and ACC wins R2 = 0.6579
    ASR and Total wins R2 = 0.6705
    ASR and ACC wins R2 = 0.5894

    This suggests that TCR does a bit better job than ASR, and that both do a better job with Total Wins than with ACC wins. Both of these things I expected, as the composite ranking is 247ís attempt to fine tune star ratings, and Total Wins is just a bigger sample set, although I guess you could make the argument that simply having more talent gives you a greater advantage in an OOC game than an ACC game. Regardless, I was stunned by these values. Over the 5 year period, the recruiting rankings do a great job of predicting the number of games a school will win over that span, considering that there are plenty of other factors at play in who wins a basketball game other than the raw/perceived talent coming out of high school.

    The second way I did these regressions is single-season oriented instead of school-oriented. The difference here is that the inputted data is each individual teams TCR and ASR along with its win totals, instead of a schoolís average TCR or ASR and win totals over the 5 year period. So how well does a TCR of 0.8438 and ASR of 2.818 predict that the 2015-2016 Deacons will win 11 games and 2 ACC games? Not as well, but still pretty well.
    TCR and Total wins R2 = 0.4263
    TCR and ACC wins R2 = 0.3709
    ASR and Total wins R2 = 0.3951
    ASR and ACC wins R2 = 0.3384

    The same trends that I discussed previously are found here, which cements to me that TCR is more correlated to winning than ASR and that both are more correlated to total wins. Whatís interesting here is that the R squared values fall a good bit. I think that can be attributed to the fact that it is much more likely that a team can outperform (or underperform) its recruiting ranking in a single season vs consistently doing so over a five year period. Sample sizes as far as games involved in each data entry are five times smaller. Fluke seasons happen. A team can have a low rated recruit like John Collins carry them to 19 wins in a miracle (Get it? Like Danny the miracle?) season, but the odds of that happening again the next season are slim.

    Moving on, letís do some analysis on which schools outperform/underperform their recruiting rankings. To me, this stat is either praise or indictment on a coach, do you win more or less games than recruiting rankings would suggest? Do you get more or less out of your players than average? I mentioned these earlier, but the stats here are Total Wins/ASR and ACC Wins/ASR, which again is how many games do you win per the star rating of the average contributor on your team. I wish I could have incorporated TCR instead of ASR here, but the last two paragraphs showed me ASR is good enough, and TCR would have been tricky/confusing math with it being a value between 0 and 1. I can go back and do this if thereís interest. Anyways, hereís the rankings of each ACC school in both stats:

    Average of Wins per season/Average Star Rating

    Virginia 8.6373
    Florida State 7.4882
    UNC 7.3084
    Louisville 6.9403
    Duke 6.7763
    Notre Dame 6.6387
    Clemson 6.4473
    Miami 6.229
    Virginia Tech 6.0668
    NC State 5.9197
    Georgia Tech 5.5448
    Pitt 5.5138
    Syracuse 5.4300
    Boston College 4.8027
    Wake Forest 4.5024
    Average 6.2830

    Average of ACC Wins per season/Average Star Rating

    Virginia 4.2649
    UNC 3.2871
    Louisville 3.2808
    Florida State 3.2596
    Clemson 2.9813
    Duke 2.9096
    Miami 2.7826
    Notre Dame 2.7211
    Virginia Tech 2.5694
    Syracuse 2.4236
    NC State 2.37
    Georgia Tech 2.1419
    Wake Forest 1.6597
    Pitt 1.6433
    Boston College 1.4008
    Average 2.6464


    Just because Iím sure you want to scroll through two more long tables, hereís the tables of the actual ASRs and TCRs (you can look up wins on your own).

    Average TCR

    Duke 0.9784
    UNC 0.9597
    Syracuse 0.9478
    Virginia 0.9198
    NC State 0.9154
    Notre Dame 0.9152
    Miami 0.9111
    Louisville 0.9106
    Virginia Tech 0.8959
    Florida State 0.8776
    Clemson 0.8603
    Georgia Tech 0.8549
    Wake Forest 0.8541
    Pitt 0.851
    Boston College 0.8262
    Average 0.8985

    Average Star Rating

    Duke 4.4022
    UNC 4.0218
    Syracuse 3.8111
    Miami 3.5123
    Notre Dame 3.4978
    NC State 3.4638
    Virginia 3.42
    Louisville 3.3915
    Virginia Tech 3.2723
    Florida State 3.0643
    Clemson 2.9537
    Georgia Tech 2.9345
    Wake Forest 2.8952
    Pitt 2.7796
    Boston College 2.5906
    Average 3.3340


    A couple things that jump off the page:

    Wakeís ratings suggest what we all know, the coaching is below average. We have bad talent, and we get even less wins out of that talent than the average ACC coach would.

    WFU, Pitt, and BC are the worst in the ACC at both having talent and winning with the talent they do have. Could make an argument that bad recruits should be actually ranked lower, which would be forgiving towards the coaches (except for the recruiting part of their jobs) of these schools, but Iíll let you think on that one.

    Holy shit Virginia/ Tony Bennett. I mean holy shit. They have good (but not elite) talent, but they get way more out of their talent than anyone else does.

    Duke looks bad here because they have had significantly more talented rosters than anyone else but won about the same amount.

    By both TCR and ASR, Cuse had the 3rd most talent, yet certainly donít have the third most wins. Make of that what you will.

    FSU and UL have both done a lot more winning than their talent would have predicted, and NC State has done a lot less. FSUís ratings are dragged down by the large rotations that they play, which include a strange number of unranked recruits (2.2 1 star players per team).

    The middle is pretty packed, so being ranked 5th and 8th in TCR really isnít that different at all.

    I was intrigued by the similarities between the talent on NC State and Notre Dame, but Iíll leave the low hanging fruit making fun of NCSU to you all.



    I promise this is almost over. Iíll just throw a few Wake Forest stats at you along with the ACC averages, plus some interesting teams.

    Wake has won 65 games and 24 ACC games over this period. ACC averages are 105.6 and 45, and we are 14th in each (thanks BC).

    Wake returns on average 49.18% of contributors from its previous team, ACC average is 55.89%. I ran some regressions here to see if this stat impacts winning, but itís a pretty weak positive correlation. However, 7 of the 9 ACC teams to win more than 100 games had higher than 56%, and only Clemson returned that many without winning 100 games (95 wins). Duke and Cuse are the winners that donít return contributors, and I might go back and look into this again excluding Duke, because that one and done factory really screws this up (45.15% returners).

    The frequency with which a top 3 recruit by composite ranking led the team in scoring was 56%, which was honestly much lower than I expected.

    Average ACC school had 0.467 grad transfers per team. Grad transfer is designated as a guy that came to the school with only one year of eligibility remaining. Wake had 0.8 per team (4 in these 5 seasons). To me, it seemed like bad teams took more grad transfers, but I didnít do anything to prove that because itís really not that revolutionary of a conclusion.

    Wake averaged 2.0 1 star recruits per team. This counts the same guy again for each year he is on the team. ACC average is 0.96 per team and Wake was second highest in this stat with BC. Duke, UNC, and Notre Dame all had 0. These were also the only schools to not take a grad transfer coincidentally.

    The teams that most under-performed its ratings in relation to total wins include:
    BC 2015-2016. ASR: 2.8 TCR: 0.8387. 7-25 (0-18)
    Pitt 2017-2018. ASR: 2.3 TCR: 0.8075. 8-24 (0-18)
    Wake Forest 2017-2018. ASR: 3.091 TCR: 0.875. 11-20 (4-14)

    The teams that most overperformed its ratings in relation to total wins include:
    UVA 2018-2019. ASR: 3.6 TCR: 0.9291. 35-3 (16-2)
    Notre Dame 2014-2015. ASR: 3.4 TCR: 0.9156. 32-6 (14-4)
    UVA 2017-2018. ASR: 3.3 TCR: 0.9103. 31-3 (17-1)

    So thereís a LOT more I can do/talk about, but Iím going to end this post here. Ask away for any of the data I didnít mention, I probably have it. Future things Iím going to look at include: rotation size (# of contributors), NBA draftees on a team, Tournament success, weighting the impact a player has on the team ratings based on MPG, and whatever else you all suggest. An easy thing to do would be to change the cutoff line of what a contributor is, if you guys want me to move that cutoff up or down. If anyone is interested in looking at my data, or seeing the pretty graphs I made that I canít figure out how to embed here, shoot me a DM, and Iíll figure something out to give you view only access to the google doc (as long as you promise to not make fun of how unorganized the raw data is). If anyone takes me up on that and finds a mistake I made, please let me know. Also, please donít be lame and take my work and repost it somewhere else without telling me. Thanks for reading this, it was a lot of fun to do/write, and let me know what you think.

  2. #2
    Strong depressing work

  3. #3
    Quote Originally Posted by harmoniousmonk View Post
    Strong depressing work
    Haha, thanks. I tried to focus on not shitting too much on Wake, but can certainly add in some more Wake-focused elements to this if people here want to be sad.

  4. #4
    Also if there is any interest for me to do something like this for football let me know. That might capture the interest of more Wake fans currently.

  5. #5
    Broderick Hicks zqglass's Avatar
    Join Date
    Dec 2011
    Location
    Clemmons, NC
    Posts
    3,498
    Do this but with football. You owe us for making us read that depressing shit.
    beer and titties

  6. #6
    Do these numbers mean Manning was fired?

  7. #7
    Solid work.
    You should send this and the rest of your data to John Currie.
    He doesn't need to sit on the sidelines behind Manning and take notes...
    This is all the dammed information he fucking needs.

  8. #8
    Quote Originally Posted by zqglass View Post
    Do this but with football. You owe us for making us read that depressing shit.
    Quote Originally Posted by shogun6066 View Post
    Solid work.
    You should send this and the rest of your data to John Currie.
    He doesn't need to sit on the sidelines behind Manning and take notes...
    This is all the dammed information he fucking needs.
    That may be true for part of the issue. However, making an excellent hire is also important.

    The logic of
    Any other coach > Jeff [name redacted] =Danny Manning.

    Repeating that,

    Any other coach > Danny Manning, year sux = ????

    Clearly,

    Next Wake BB Coach >>> Danny Manning is the equation for which Currie should be gathering much information.

  9. #9
    I can start working on football, but it will be a while because of how much bigger team sizes are. I'll post it here whenever I get that done.

    And I genuinely didn't mean for this to be something that was entirely taken to despair WFU basketball and hate on Manning. Obviously those things will come out of it here, but I was really more doing it to test the validity of using the ratings of recruiting classes as an indicator of future success, and to see who outperforms their recruiting rankings and who doesn't. I started working on this a while back when I saw the 1000th person try to make the claim that 247 recruiting rankings are inherently flawed for some reason or another, which I think I've shown is an argument that is going to need to show some evidence in order to be taken seriously.

  10. #10
    Quote Originally Posted by deacon14 View Post
    I can start working on football, but it will be a while because of how much bigger team sizes are. I'll post it here whenever I get that done.

    And I genuinely didn't mean for this to be something that was entirely taken to despair WFU basketball and hate on Manning. Obviously those things will come out of it here, but I was really more doing it to test the validity of using the ratings of recruiting classes as an indicator of future success, and to see who outperforms their recruiting rankings and who doesn't. I started working on this a while back when I saw the 1000th person try to make the claim that 247 recruiting rankings are inherently flawed for some reason or another, which I think I've shown is an argument that is going to need to show some evidence in order to be taken seriously.
    Good stuff, deacon14. I always find these type of data interesting. I agree with your general conclusions regarding rankings, coaching, and Manning - he isn't recruiting well, isn't coaching well, and both are leading to the expected poor results.

    However, I do disagree that these data strongly support the accuracy of recruiting rankings. I'm coming at this from a biomedical perspective, but correlations (R2) of 0.34 to 0.43 for individual seasons are pretty weak. Even correlations in the 0.5 to 0.65 range are still weak. In addition, instead of attributing the gap between rankings and results to coaching, I would argue that some of that gap occurs because the coach is much better at accurately identifying talent than 247. One way to potentially assess this is to give each player a grade at the end of their college career (points for being drafted, all acc etc) and then see how much that explains the win/loss results. Finally, I think you'll see even less correlation for football, as the football recruiting rankings are less accurate than basketball, in my opinion.

  11. #11
    Quote Originally Posted by Rafi View Post
    However, I do disagree that these data strongly support the accuracy of recruiting rankings. I'm coming at this from a biomedical perspective, but correlations (R2) of 0.34 to 0.43 for individual seasons are pretty weak. Even correlations in the 0.5 to 0.65 range are still weak.
    That's funny, I'm also in the biomedical field, was a chemistry major at Wake. I originally thought the same thing about the R2 values, but it really varies from field to field. In social sciences, where there are infinitely more variables than a chemical reaction or even biological system (people making decisions and doing things really extrapolates things), correlations are never going to be as strong. I'm certainly not an expert on this, and maybe they are weaker than I thought, but I don't think they are particularly weak in this application.

    https://condor.depaul.edu/sjost/it22...orrelation.htm

  12. #12
    Quote Originally Posted by Rafi View Post
    In addition, instead of attributing the gap between rankings and results to coaching, I would argue that some of that gap occurs because the coach is much better at accurately identifying talent than 247. One way to potentially assess this is to give each player a grade at the end of their college career (points for being drafted, all acc etc) and then see how much that explains the win/loss results. Finally, I think you'll see even less correlation for football, as the football recruiting rankings are less accurate than basketball, in my opinion.
    I actually really like that idea, hadn't really thought about it from that angle. Either way though, it means the coach is better at his job though right? Either he is more accurately identifying talent or he is better at developing it.

    I mostly agree with that assessment on football. There's just too many 2 and 3 star kids to accurately sort through for the recruiting services. Plus they get less opportunities to see the kids play. Maybe at the high end (4/5 star) they do a good job, but I'd be surprised if the correlation rivals what they do in basketball.

  13. #13
    Quote Originally Posted by deacon14 View Post
    That's funny, I'm also in the biomedical field, was a chemistry major at Wake. I originally thought the same thing about the R2 values, but it really varies from field to field. In social sciences, where there are infinitely more variables than a chemical reaction or even biological system (people making decisions and doing things really extrapolates things), correlations are never going to be as strong. I'm certainly not an expert on this, and maybe they are weaker than I thought, but I don't think they are particularly weak in this application.

    https://condor.depaul.edu/sjost/it22...orrelation.htm
    Also, included that link just for the one table. Didn't mean to sound condescending and teach you what correlations are lol, I obviously am not doing that.

  14. #14
    Lots of good info. Confirms conventional wisdom:

    - WF is poorly coached (as is/was BC and Pitt).
    - UVA is well-coached.
    - Duke recruits at a ridiculous level and under-performs its talent. With the talent that Duke has had, love that fact that it has been a decade since Duke has finished 1st in the ACC regular season.

  15. #15
    Quote Originally Posted by deacon14 View Post
    That's funny, I'm also in the biomedical field, was a chemistry major at Wake. I originally thought the same thing about the R2 values, but it really varies from field to field. In social sciences, where there are infinitely more variables than a chemical reaction or even biological system (people making decisions and doing things really extrapolates things), correlations are never going to be as strong. I'm certainly not an expert on this, and maybe they are weaker than I thought, but I don't think they are particularly weak in this application.
    If only there was a social scientist on these boards that could provide perspective on these correlations...

  16. #16
    Quote Originally Posted by deacon14 View Post
    I actually really like that idea, hadn't really thought about it from that angle. Either way though, it means the coach is better at his job though right? Either he is more accurately identifying talent or he is better at developing it.
    Absolutely. I think some part of it is player development, but I think more of it is the ability to identify talent. I think most college coaches are better at identifying talent than most of the "recruiting analysts" (aka teenagers paid below minimum wage).

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •