Article at http://www.sbnation.com/longform/2013/3/7/4071420/sloan-sports-an*lytics-conference-recap-ssac-2013 (Read it -- well laid out SBNation #longform, per usual)
A weekend at Sloan, and the pluses and minuses of advanced an*lytics
It is 8:15 on Friday morning, and the PA system in the 3rd floor ballroom at the Boston Convention Center is stuck in 1982. A room full of 2,700 people is rocking out to Rod Stewart, The Beatles, Bob Seger, and Paul Simon. I don't know it at this point, but "Late in the Evening" and "Hollywood Nights" will be stuck in my head for the next 48 hours.
I look around a ballroom full of white males and write in my notes: "If this conference were a human, it would be wearing a blazer and bu*ton down with jeans and loafers (with no socks)." Formal enough to be taken seriously, but still loose enough to have a little fun.
You've probably already heard the MIT Sloan Sports an*lytics Conference discussed in one place or another, and read about it, too. You probably already know that it began in 2007 with Rockets GM Daryl Morey and a handful of like-minded peers gathering together at MIT's Sloan Business School to put together a series of panels for a few hundred students. You've probably also heard the conference has grown every single year since 2007.
"It seems like it doubles in size every year," a friend tells me before we get started Friday.
Outside the ballroom I'm getting coffee, and behind me I hear a college student say, "This trip was my birthday present to myself."
Somehow a conference about advanced metrics and sports business became a place everyone wants to be. It's not a ccheap birthday present. According to the website, a ticket to this event costs $132.00 for MIT students, $205.50 for students anywhere else, $494.50 for MIT faculty, and $591.90 for general admission. (I paid $132.00 to attend as media, but ESPN sponsors this event, so a majority of the media here arrived on free pa** from the Worldwide Leader.)
Inside the ballroom a few minutes later, it's time for the opening remarks, and Morey tells a crowd of 2,700, "This started as 100 people seven years ago, it's become mainstream because it works."
There are one-liners about the sequester, Wang Zhi-Zhi and Chinese hackers, the Oscars, and we see pie charts with actual pies. It's the best, dorkiest co*ktail party ever.
"Recently, we were named the Super Bowl of an*lytics by Forbes," co-founder Jessica Gelman tells the crowd.
It's hard to imagine what this conference looked like in 2007, but today this self-congratulating sentiment percolates through every panel, every presentation, and really, a whole bunch of the attendees. Which makes sense. Sloan is a victory lap for guys like Daryl Morey and even all the young people who dream of one day being the next Daryl Morey. This conference started as a blip on the periphery, and this weekend it feels something like the epicenter of the sports industry, full of outsiders who now have the inside track to running sports. Even the college kids who showed up here desperate for jobs can say they went to Sloan and it'll impress all their friends who didn't.
Anyway, I'm here as a skeptic, and I'll leave as a skeptic, but in the meantime my only goal is to figure out why exactly I'm skeptical.
So, what does it look like when 2,700 smart people talk sports? At any given time the conference features hour-long panels in three separate rooms, and four presentations each hour, two on academic research papers, and two on the "evolution of sport." In between, there are 20-minute breaks to race between rooms. All of which is to say, the conference is outrageous and overwhelming. Let's take a handful of snapshots and try to make sense of it.
“ People who make decisions or own things want to hire experts. They want to hire experts who are sure about their answers. Whereas what we're saying is, 'Boy all we can do is shift the odds.' It's sort of like conservative talk radio vs. liberal talk radio. The conservatives are very sure about [things], it makes for great radio and people argue. Whereas on the left people are like, 'Well it's sorta this, it's sorta that.' It just doesn't sell as well.”
~ Daryl Morey, Rockets general manager, SSAC co-founder
Nobody dismisses critics quite like an an*lytical genius. That talk radio an*logy takes three sentences and makes the an*lytics movement irresistible. Because if there's tension between an*lytical minds like Morey and some other faction of the basketball community, nobody wants to be on the side that reminds people of Rush Limbaugh.
For another example: There's one section in Moneyball where Michael Lewis spends a few pages on a scene from an Oakland front office meeting where they're arguing over a prospect named Jeremy Brown, a catcher from the University of Alabama. He's overweight and the scouts don't like him, but he scores high in a lot of the categories that Billy Beane and his a**istant Paul DePodesta use to predict success. An argument ensues and lasts several pages, and it ends like so:
The fat scout looks up from his giant chocolate chip cookie and seeks to find a way to get across how unimpressed he is. "Well," he says, exaggerating his natural drawl. "I mustseverely underestimated Jeremy Brown's hittin' ability."
"I just don't see it," says the vocal scout.
"That's all right," says Billy. "We're blending what we see but we aren't allowing ourselves to be victimized by what we see."
This argument is fat, slobby ignorance vs. Billy Beane, the grinning bada** who's here to shake up the game whether scouts can handle it or not.
I have no doubt that baseball was and probably still is full of backwards neanderthals who think computers are the devil. I've seen Trouble With The Curve. The problem is the a**umption that ALL skeptics are as backwards and ignorant as they seem in Moneyball. The movie or the book. Both popularized this movement more than pretty much anything else, and both give the impression that anyone who's not on board is missing the boat on the future.
So, I Googled Jeremy Brown. After the A's shocked everyone and took him in the first round, Brown played in five major league games before retiring in 2006.
Which brings us to the other side of what's incredible about this movement: an*lytics experts are never wrong. All weekend long in Boston, we hear the same refrain. It's about process, not outcome. Quantitative an*lysts ("quants") can only tell you what should work.
This ignores the obvious problem: There is so much data that if you're working to sift through it all and make decisions, there will always be anomalies. Just like if you're scouting. Maybe data gives you slightly better odds, but there's going to be a whole lot of variance regardless when you're talking about quantifying humans. This is a lot of frenetic activity to get us to ... where, exactly?
The Morey quote above comes during a panel on "The Science Of Randomess," where everyone from Nate Silver to the Cleveland Browns' new president, Alec Scheiner, weigh in on what quants can't control. "You're not giving answers," says Scheiner, "You're saying 'I think we're more likely to do well if we do it this way.'"
All weekend long, there's an echo of the same theme: Trust the process, ignore the outcome. All you can do with an*lytics is shift the odds.
If there's genius on display at Sloan, it's this: When scouts or coaches or old school GMs get something wrong, it's an example of traditional scouting methods failing. When an*lytics get something wrong, it's "randomness" that you can't control. A small part of a much bigger process, and teams and fans should trust that process until they get a better outcome. Any skeptics must be simple-minded conservative talk radio reactionaries, eating giant chocolate chip cookies.
Speaking of which, every afternoon at Sloan features a buffet of giant cookies. Chocolate chip, peanut bu*ter, and oatmeal. They are delicious.
“Team chemistry's complicated, right? It's different from sport to sport. It's been immeasurable for years. In baseball it's the locker room. Nick Swisher's a great example, right? ... Everyone talked about how he was a great locker room guy, right? He added locker room presence. Or Big Papi here in Boston. In soccer for a team like FC Barcelona, the players come up together for years and year, and they put a lot of time and energy into development of those players on the field, and off the field. They room together, they eat together, they study together. And then you see the result during games, you see them able to maintain possession longer than other squads. And then basketball. No bigger issue than the Lakers, or even before that in 2010, with LeBron James, Dwyane Wade and Chris Bosh. Everyone was wondering, could these alpha males come together and perform in a way that leads them to a championship? ”
~Daniel McCaffrey and Kevin Bickart, P.H.D., SyncStrength
I'm watching two researchers from a company called SyncStrength give a presentation called "Using Player Biology To Measure Chemistry In Real Time." On its website, SyncStrength says its "founding team has developed a proprietary an*lytics platform based on their expertise and collaborations in science, coaching and health. Using the latest research in synchrony, SyncStrength objectively an*lyzes team chemistry from physiological data like heart rate. SyncStrength's an*lytics are designed to help coaches and managers make statistically-driven inferences and decisions about team performance and team development strategies."
There are all kinds of these presentations at Sloan. Outside the main ballroom you see posterboards for studies like: Total Hockey Rating: A comprehensive statistical rating of National Hockey League forwards and defensemen based upon all on-ice events. Or, The hidden foundation of field vision in English Premier League (EPL) soccer players. Or "Sweet-Spot" Using Spatiotemporal Data to Discover and Predict Shots in Tennis. An NBA project studies Acceleration in the NBA: Towards an Algorithmic Taxonomy of Basketball Plays.
On that last one, a former ESPN stats-expert-turned-executive tweets , "Paper on acceleration in NBA is interesting ... next step is figuring out what it means." This seems like the general reaction to most of the groundbreaking research at Sloan. A collective, "Hmmmmmm."
The presentation on biology and chemistry is my favorite. The projector screen shows a clip of women's soccer players who scored a goal and exhibit similar heart rates throughout the play and subsequent celebration. "If this was a lab or we were out in nature looking at monkeys," one researcher explains, "We would code those behaviors and then try to understand the physiology. And I think that's what we're looking at here."
We study monkey behavior because they don't speak English and communicate like humans. We don't really need companies spending this much money and effort to see if Kobe Bryant, Steve Nash and Dwight Howard have chemistry together, because the answer's already obvious. Isn't that why we love watching the Lakers this year?
In any case, the SyncStrength talk is basically an advertisement, and a good example of the industry that's emerged to pair with the an*lytics movement. There are dozens of companies attending Sloan to get a piece of the consulting business. With names like SyncStrength, StatDNA, TZ Quantitative, Answers Research, IsoLynx, Pointstreak Sports Technologies, it's an industry full of brilliant people who are bad at naming companies. And they're all hoping to get paid by teams to make sense of advanced an*lytics.
The future of this industry revolves around video technology. In basketball, specifically, a company called SportVu has adapted missile-tracking technology and installed infrared cameras in roughly half the league's stadiums, and they process 25 frames-per-second (72,000 per-game), spitting out ma**ive reams of data for teams to process. Aside from being a lucrative business for SportVu--the company charges low six figures for installation--it's also created jobs for teams who need people to interpret all this. "It's a jungle of data," ESPN writes. "You need a guide to get around."
The breakout star of the weekend is Grantland writer and researcher named Kirk Goldsberry who uses the SportVu data to highlight the Bucks' Larry Sanders as one of the most underrated defenders in the NBA, while exposing David Lee as one of the worst. He communicates this with graphs like these:
If it seems like a complicated way to explain what we already know--David Lee is bad at defense, Larry Sanders is a solid young big man--it's also the first step toward quantifying a player's defensive impact, which the Sloan Conference considers pretty groundbreaking. Goldsberry will probably be consulting NBA teams by this time next March.
It's a little surreal to see biology and military technology applied on teams like the Washington Wizards, but here we are. Professional sports is a billion dollar economy--one of the only industries in America that's actually growing--where everyone is looking for an edge, and an*lytics are the newest way in. Next year's conference will probably have 10,000 people, all of whom want to be the next Kirk Goldsberry.
“ I'm not sure I would characterize that these [lockouts] were victories for one side or the other. Nobody's winning, when nobody's playing. In these recent settlements, I think the players are receiving an appropriate amount of the compensation. We certainly aren't paying our employees 50 percent of the revenue. ”
~ John Skipper, president of ESPN
“ We didn't see [the NBA lockout] as a victory at all. It was far from what we set it out to accomplish in bargaining, but I thought it was a fair compromise. ”
~ Adam Silver, deputy commissioner of the NBA
Speaking of sports business ... While most of the panels at Sloan are dedicated to debunking myth and cliches, it should be telling that the panel on sports business operates mostly in the abstract. The remarks above came during a panel called "The Changing Nature Of Ownership," during which two wealthy white men successfully conjure an alternate reality where NFL and NBA players didn't make sweeping concessions across the board, making franchises more profitable now and more valuable later.
As economist Kevin Murphy explains on a separate panel dedicated to lockouts, more valuable franchises just means that new owners will have to spend more money to buy franchises, and they'll need more money back if they expect to get the same rate of return as the other owners.
Where will they get it? Future lockouts that nobody wins, you'd think.
It's a dichotomy that's hard to miss here: Some mathematical geniuses who should otherwise be on Wall Street are at Sloan using an*lytics to learn more about sports, while other geniuses are doing exactly what they'd be doing on Wall Street--in charge of making business more opaque, gaming balance sheets to justify lockouts and make sports more profitable than ever, a cycle that will probably continue forever if nobody stops it. If we're dedicating the best and brightest minds to sports, solving this seems like a worthier cause than quantifying NBA defense.
“ What you hear at this conference ... People talking about, 'How do I communicate it to a decision-maker on a team?' We also have a different challenge, and the bigger challenge is to explain it to fans. ... So the key is, what question are you trying to answer? QBR is trying to answer who has been the best, most efficient quarterback. It's trying to reach this audience by starting with something very complicated, and putting it out on one scale that everyone can kind of grasp. It is to serve the fans. That is the goal. ”
~ Alok Pattani, an*lytics producer, ESPN
“ RGIII, he finished with great numbers on the year, obviously, but his traditional stats--pa**er rating, yards-per-attempt--everything was off the charts in his favor against Andrew Luck. Then, based on the advanced an*lytics, Luck was attempting a higher degree of difficulty, actually was a more efficient runner, hard to believe though that may be for a lot of people with the naked eye. But that was something that kind of flipped the script a little bit. ”
~ Michael Smith, commentator, ESPN
This is the sponsored panel, "ESPN's Use of an*lytics In Storytelling." It makes sense that ESPN sponsors this event, because if they can take advanced metrics mainstream, that becomes one more thing for everyone to argue about all the time, making sports just a little more inescapable.
"ESPN is the best and worst thing to ever happen to this conference," someone tells me on Friday. You could make the same argument about sports, in general.
In any case, the network's been relentless about forcing us all to adopt the Total QB Rating, the new advanced quarterback stat that nobody really asked for. It's mentioned a solid 50 times in the panel at Sloan. But they never really tell the truth about the stat: They don't need people to adopt it, they just need to co-opt the argument surrounding it.
This is what stats actually mean for fans. Total QB Rating doesn't add much clarity to the debate between Robert Griffin III and Andrew Luck, it just adds another component to trick people into thinking there's a real answer. It's more material for anyone who wants to win an argument. This is fine, but it doesn't actually make sports more fun.
Would you rather be the fan that gets excited for Miguel Cabrera's Triple Crown or the one who says he's overrated when you look at the REAL numbers? Would you rather enjoy Kobe Bryant as a psychotic crunch-time k**er or spend your days telling everyone what the REAL numbers say about Kobe in crunch time?
Sports don't have to be REAL for anyone who's not running a team.
There usually isn't a clear answer to these debates anyway, and even when there is, there's a good chance it all changes when the games happen. Randomness, variance, anomalies, etc. A growing faction of the media uses advanced stats to write mythbusting articles to make the sports conversation smarter, but it actually just makes things more pedantic. We are not scouts. Rather than couch all of our 2013 sports arguments in data that's not as conclusive as it seems, it's more fun to just have an argument.
It makes you appreciate the millions of people who don't care about any of this, because maybe they have the right idea. Look around the Sloan an*lytics Conference and you see a group of thousands of over-educated smart people, most of whom are white males, congratulating each other on expertise and hitting on all the same themes, forging this echo chamber that's supposedly rendering everyone else extinct. "This conference is the Internet," I write on Saturday afternoon.
There's a much bigger world out there, and thank God for that.
“ I don't think we're any more resistant to an*lytics stuff than a lot of the an*lytics people are to the viewpoints we have. A lot of you an*lytics people think that the game is a video game. That players will always react the way as your models say they will react. And we were talking 2-for-1. So, 2-for-1 situations at the end of the quarter. I've read the an*lytics on it, I know that's something that by the numbers we should do, and we didn't do it. Whether you agree with 2-for-1 or not, a guy races the ball up the floor at any time in the game, and jacks up some horsesh** shot. Look at the other four guys on the floor, that he's playing with. Every one of 'em's gonna be like, "What the heck?" They don't run back as hard, and now when you start the fourth quarter after the guy just did that in the third quarter, now the next guy who gets the ball says I'm jacking up the next one. One of the things in coaching is you're trying to create a style of play and a culture, that [says] 'This is how we play the game.' And every time you make an exception to that, and say, 'Okay this is how we play the game but not in this case. In this case you're allowed to throw up whatever crap you want.' Then you're breaking down your system a little bit. ... I'm not rejecting the an*lytics, I'm not even saying I'm definitely right, but there's another side to consider. ”
~ Stan Van Gundy, NBA coach, disgruntled hedgehog, American hero
It's 4 p.m. on Saturday, the conference is almost over. I just watched a panel on "Predictive an*lytics in Gambling" where a professional gambler named Haralabos spent 45-minutes picking a fight with a Vegas bookmaker. I am bleary-eyed and my mind is numb from either too much caffeine or not enough. Now, Stan Van Gundy is keeping it real on the basketball an*lytics panel, and for the first time all weekend we've got something like clarity.
In that spirit, let's be clear on a few points. an*lytics have already changed sports. To take a famous example, the 2011 NBA Finals turned when the Dallas Mavericks went to a lineup that advanced stats told them worked better against the Heat. The 2012 Finals turned when the Thunder refused to bench Kendrick Perkins, even as data suggested they were better off without him. an*lytics will continue to shape sports, and all of it is on display at Sloan.
At its heart, the an*lytics movement centers on intellectual curiosity--What can we measure better? How can we understand sports better?--and that's what spawned the Sloan Sports an*lytics Conference. Something like that is never a bad idea, and professional sports teams would be insane not to invest in the new technology and brain power that's on the market to help make them better.
But then the movement toward an*lytics gets sold as somehow more honest than what anyone else might see in sports. Like when Daryl Morey tells Michael Lewis and the New York Times, "Someone created the box score, and he should be shot."
It really doesn't matter if the box score doesn't tell you everything. No intelligent human ever thought it did. It's this smug attitude toward "traditional" understanding of sports that makes the whole movement a little unbearable sometimes. Everyone's paying such close attention to sports that they forget what made sports worth paying attention to. All weekend long at this sports conference, we hear the same vague messages that keep the crowds nodding knowingly:
• We need to find the signal in the noise, a reference to Nate Silver's best-selling book about finding meaningful signs in an ocean of data.
• Trust the process, ignore the outcome.
• The future of stats is video, but we haven't figured out what exactly we'll learn from it.
• We need to work at visually representing data for fans and players and coaches.
• an*lytics are meaningless if we can't communicate them.
It's okay if fans ignore all this, though. Focusing on the outcome isn't a sign of ignorance, and it really doesn't matter if we can't communicate an*lytics to the mainstream. Maybe it's important for the 100 people at Sloan this weekend who work for teams and have to communicate this stuff to coaches and players, but for everyone else, it just doesn't matter. The best part about sports is you never really know, and that's what keeps us watching.
When you take the misleading data that distracts teams and all the insufferable Internet arguments that distract fans and put it next to the handful of an*lytics approaches that actually add insight, the plus and minuses really might equal zero.
So maybe Stan Van Gundy understands all this better than anyone. an*lytics have value as we try to learn more, and anyone who really cares about sports will end up using them in one way or another. But in the end, all the data and process revolves around humans, and there's a big part of this that'll always be a guessing game.
The 2011 Finals may have turned when the Mavericks started playing J.J. Barea more, but it certainly helped that LeBron James suddenly stopped being the best basketball player in the world. No quantitative an*lysis can explain that, he just started being human. The 2012 Finals was the opposite. Maybe it would have helped if the Thunder had benched Kendrick Perkins, but LeBron hit another level in Game 3 and beyond. He became inhuman, and gave basketball fans a series that we'll remember forever.
Now LeBron's doing it again this year, especially the past month. He makes it all look effortless, and the best players in the world look helpless. Even his outrageous stats don't really do justice to how completely he's dominating the NBA right now. Nothing does if you aren't watching. The same is true with any great player that renders everyone else irrelevant and changes the way we understand the game all by himself. And if we're talking about what the an*lytics movement means for people like you and me, that's the point that seems missing from all the an*lytics panels about the future of sports. Now and forever, everything you can't quantify is what makes sports worth loving.