The NBA has partnered with “sports tech” start up, Second Spectrum, to display augmented graphics generated by artificial intelligence on its television broadcasts.
These graphics can display player statistics, play diagrams, or fun animations in near real-time. The league is struggling to keep the attention of younger viewers as more and more of them cut the cord. Modern broadcasts that provide better information to a viewer could help the league to gain back viewers.
For years, the viewing experience of watching the NBA has been relatively similar. While the quality of the picture on display has gotten better, broadcasts are still using the same general format, with the same camera angles providing the same information. At the same time, there is more information publicly available on the NBA than ever before.
How can stats get broadcast?
Fans are devouring advanced stats. The NBA, however, has been unable to incorporate these stats fully into its broadcasts. That is before even thinking about the implications of the latest virtual reality technology. Imagine seeing LeBron James’ famous block of Andre Iguodala during Game 7 of the NBA Finals through his eyes, or even Iguodala’s.
In March, the NBA and Second Spectrum debuted an AI-assisted broadcast feed to a national audience. For the game between the Los Angeles Lakers and Milwaukee Bucks, there was a special Second Spectrum feed on ESPN+, branded Full Court Press Second Spectrum.
The Los Angeles Clippers have been using a similar product called Clippers CourtVision, also in concert with Second Spectrum. Clippers owner Steve Ballmer is an investor in Second Spectrum, and has been heading the initiative to get the company involved with the NBA. Clippers CourtVision is available online and through a mobile app to Fox Sports Prime Ticket subscribers for Clippers games.
People everywhere are cutting the cord in increasing numbers. The NBA is feeling it just as much as everyone else. “From 2010 to 2018, among 18-to-34-year-olds—and that’s our core audience, it’s an incredibly attractive young audience—their viewership on pay TV is down almost 50 percent,” said NBA Commissioner Adam Silver. The NBA is trying lots of different things to combat this, including the Second Spectrum broadcasts. In the same talk, Silver mentioned having an option to buy individual games on demand, through Twitter. They have also tried different formats for broadcasts, including alternate gambling-themed broadcasts and an approach with “dueling analysts.” But the most intriguing use of technology are Second Spectrum’s broadcasts.
Enter AI (again)
Second Spectrum’s broadcasts use artificial intelligence to provide relevant graphics on the screen. The broadcasts offer three different options to viewers, each serving a different type of fan. Mascot Mode provides entertaining graphics like fire on the basket or a word bubble saying “Buckets” after a made basket. It is a fun, light-hearted way to watch a game.
Kevin O’Connor of The Ringer compared it to “watching a game with a Snapchat filter on.” Coach Mode diagrams plays as they are happening. It highlights which players are open on a play, and identifies play types. Coach Mode is useful for teaching fans to see a game the way a coach does.
Another feed, Player Mode, shows the shooting percentage for each offensive player from where they are on the court, adjusted for proximity to defenders. It shows a color code from green to red to detail the quality of the shot as well. This mode is nice for someone who is analytically inclined.
The broadcast on ESPN+ switched between the three modes, but in Clippers CourtVision, the viewer can choose which mode to follow. The future of broadcasts likely involve watchers choosing between these and possibly other options.
The Second Spectrum feed currently runs about two minutes behind the live game feed, but the company is working on speeding that up. It was 45 minutes in the fall of 2017, so they have already made plenty of progress. But to be a real viewing option for fans, and not just a gimmick, the time will need to be reduced even more.
Ballmer told the Los Angeles TImes that the goal was to have it be only 10-20 seconds behind, the same latency as a traditional broadcast. But since there is so much data that Second Spectrum is tracking in a game of basketball, there is an enormous amount of computing power needed to get the AI to process data that quickly.
The AI aspect is where the potential for Second Spectrum’s broadcast comes through. Augmenting sports broadcasts with graphics is not really new. The yellow first down line has been in place in football broadcasts since 1998. But using artificial intelligence to provide insightful graphics in real-time is new, if they can do it, and could have massive impacts on sports television.
Second Spectrum is a sports artificial-intelligence company and the official optical tracking partner of the NBA. Second Spectrum has installed several cameras in every NBA arena for the purpose of player tracking. Using computer vision technology, these cameras log the position of each player and the ball at every moment of the game. They translate this data into “moving dots,” and try to gain insights from them. This technique is called spatiotemporal pattern recognition, as Second Spectrum founder and CEO Rajiv Maheswaran explained in a TED talk.
This creates an enormous data set that Second Spectrum helps NBA teams analyze using machine learning. The Second Spectrum AI has learned to identify different basketball actions, from simple passes to more complex actions like pick and rolls. They are able to use this information to gain tremendous insights into the game. Second Spectrum’s analysis helps teams decide in-game strategy, scout opposing teams, and make player personnel decisions. Now it is being used to enhance broadcasts too.
The NBA is bullish on Second Spectrum’s broadcasts, awarding an NBA innovation award to the Clippers for Clippers CourtVision. Other teams are investing in the technology too. Meanwhile, Second Spectrum is looking at future innovations. Ballmer wants to move towards Virtual Reality in the future, so fans can to watch a game through a player’s eyes.
There are a lot of potential uses for augmented reality in NBA broadcasts. In addition to the modes outlined above, there could be a fantasy mode that shows player stats in a graphic next to each player.
Ballmer described a game mode called “Pick&Roll” where viewers choose players and then are awarded fantasy points for their actions. The feed keeps track of a leaderboard, and updates it in real-time. This mode highlights the potential for gambling-related applications of graphics-augmented broadcasts.
The NBA’s experimentation with the Second Spectrum broadcasts shows a league willing to embrace technology. It is exciting to see what will become of this relationship. Maheswaran said,
“This is fundamentally how everyone is going to watch sports.”
Maybe watching a game on television 10 years from now will be unlike anything we can even imagine today.
