The growing role of AI, machine learning and other emerging tech in media production

Artificial intelligence (AI) and machine learning (ML) are an increasingly popular aspect of media production.

So we at Frame 25 decided to take a closer look at how they’re being used in the industry, along with other new tech.

What is artificial intelligence (AI)?

In plain English, it’s basically the ability of a machine or software to think, act and learn like humans.

But rather than merely duplicating the way humans think, and taking on tasks that we already do, AI is perhaps best employed when assigned to tasks that we cannot – or would rather not – do.

What is machine learning (ML)?

• A technique by which a computer can ‘learn’ from data, without using a complex set of different rules
• Mainly based on training a model from datasets, and the better the quality of the datasets, the better the accuracy of the machine learning model
• A dataset is a collection of related sets of information that is composed of separate elements but can be manipulated as a unit by a computer

AI, ML and huge summer sports

So how are AI and ML being used in broadcast today?

We need look no further than a couple of massive, global sporting events that use the cutting edge of tech:

Wimbledon & AI

AI is now used to select shots, rallies and dramatic moments for highlights packages as clearly as those made by humans but in as little as 15 minutes.

Created by IBM research scientists and consultants, the system tracks players’ emotion and movement, audience noise levels and match data, enabling the Wimbledon editorial team to create highlight videos much more efficiently and devote time to lower-profile matches that might otherwise be overlooked.

IBM SlamTracker: another AI system

• An online tool that informs fans with high level overviews of matches and point-by-point analysis
• Gives fans in-depth analysis and insight into key head to head match-ups
• Reveals hidden player patterns and match dynamics in real time during critical moments in matches
• Includes the ‘momentum’ system, showing which player is in the ascendancy in a match and how the balance of power might have shifted during play
• Features a chatbot delivered on Facebook messenger, dynamic upgrades to Wimbledon.com and the creation of the official Wimbledon poster by analysing more than 300,000 archive photos

Formula 1 & machine learning

Formula 1, meanwhile, has moved the vast majority of its infrastructure from on-premises data centres to Amazon Web Services, making use of AWS’ machine learning and data analytics services to enable fans to enjoy detailed real-time stats and analysis.

The move will, say AWS:

Years of historical Formula 1 race data will also be analysed against real-time information that’s collected in every race, using AWS’s machine learning, streaming, and analytics services, uncovering new racing metrics and insights that have, until now, remained hidden.

Other emerging tech is also now commonplace in sports broadcast and TV production:

1. Video Assistant Ref (VAR) at the 2018 FIFA World Cup

Photo: Fifa

In terms of interest and media coverage, the football World Cup is the biggest single-sport event on the planet by some distance.

And, for several reasons, this summer’s tournament in Russia is widely regarded to be one of the best ever.

But one new feature that has drawn football fans’ derision (at best, and loathing at worst), particularly early in the competition, is VAR (video assistant referee).

1. 4K – 8K

Three hundred broadcasters are broadcasting the World Cup’s 64 matches live to 210 countries, using 37 cameras to shoot every game, with eight shooting in UHD/4K, which is 16 times more detailed than HD.

Latin American broadcaster Globo has been running 8K feeds in Brazil, which have been watched at Rio de Janeiro’s Museum of Tomorrow (Museu do Amanha), while the BBC has been showing games in this format on the iPlayer.

1. Virtual reality (VR)

The BBC has also been at the forefront of showing World Cup action in VR, via a dedicated app.

And in the US, the NBA has emerged ahead of the competition, first by streaming weekly live games in VR and, now, by making its entire season immersive for League Pass subscribers.

With the NextVR app, 27 live games are available via a Samsung Gear VR or Google Daydream headset, with access to the Screening Room, where viewers can pick and choose from every League Pass game in virtual reality.

The feature also enables fans to select up to 13 NBA games to livestream on a massive virtual screen.

Looking ahead: 5G and augmented reality (AR) at the Tokyo 2020 Olympic Games

Japanese public broadcaster NHK is testing technology that’ll let viewers play along with some sports during the Tokyo 2020 Olympics via remote controls, and other innovations include automated live subtitling and avatars to provide sign language, both enhancing the broadcast of unscripted events.

The predominant mobile phone operator in Japan, NTT DoCoMo, has already announced its intention to operate a live 5G network in time for the event.

The Eurosport Cube was introduced during the European broadcaster’s coverage of the Pyeongchang 2018 Olympics, an augmented reality studio where athletes were invited in to analyse their performances and those of their competitors.

By the time the Qatar 2022 World Cup rolls around, 4K channels will be standard, 5G will be everywhere and VAR’s inadequacies might have been ironed out.