The Hobbit in Fast Forward: An Exercise in Innovation

 
 
The cinema space is abuzz over The Hobbit: An Unexpected Journey, but not just over Bilbo Baggins and his ensuing adventures against the legions of Sauron. Orcs are slain and goblins lanced, yet discussion of the film has largely vectored around Peter Jackson’s unorthodox frame rate choice. While the New Zealand auteur is no stranger to innovation, his latest entry in Tolkien’s fantasy realm features one minor major technical departure: it is the first box office release both recorded and projected in 48 frames per second. Popular opinion seems to occupy one of two extremes, with its detractors the more outspoken of the bunch.

The backlash over the technical presentation did not come without warning. When Jackson debuted The Hobbit at last year’s CinemaCon in Las Vegas, the showing was met with splintered ambivalence. Some attendees cited its crisp motion compared with the legacy 24 fps format, while others dispraised its “non-cinematic” feel in concert with its rapid disintegration of the suspension of disbelief. Theater owners mostly expressed disdain over having to upgrade their 3D projection systems, a hardly trifling fee of $10,000 per screen.

In an industry which fosters innovation and traditionally greets bigger numbers with gusto, the recent friction over The Hobbit’s screening format may seem curious. After all, 48 is exactly twice the established standard, which means you are seeing double the information every second. So why is 24 fps broadly considered “cinematic” and 48 fps less so? And why are filmmakers like Jackson, Andy Serkis and James Cameron pushing for HFR (High Frame Rate) formats?

Silent Film and Beyond

With his penchant for particulars and visual flair, it’s fair to say that Jackson did not take the decision to deviate from the industry standard of more than 80 years lightly. As he details in a pre-release Facebook post, the early film era’s choice to commercialize 24 fps was one of compromise. Most people assume 24 is a magical number for film or holds some sort of psychophysical significance. Not so.

Prior to the late 1920s, there was no agreed-upon recording speed, film stock or projection speed. The first silent films were shot with variable frame rates, with some dipping as as low as 14 frame/s. Such a speed was barely enough to maintain the illusion of motion; viewers today would find the jerkiness nigh unwatchable. In the theater houses, projectionists would even alter the rate of playback to match the musical accompaniment or on-the-fly to maximize profits.

When sound film arrived, this model was no longer sustainable. Fixed image timings became necessary in order to sync picture and sound and to escape the dissonance created by mismatches in audiovisual playback. Which timing to choose hinged sensitively on the type of film stock, which is enormously expensive to buy and to process. Prior to the digital era, a film’s primary costs were a function of the width of the film stock (known as the gauge) and capture speed. All else equal, a 14 fps film was roughly 70% cheaper to shoot than a 24 fps film, while a 35mm production is roughly half as expensive as the 70mm IMAX format used in Christopher Nolan’s previous two Batman films.

As the industry convened around 35mm stock, an economic ceiling was naturally placed on capture speed. In the end it was 24 frame/s which struck a delicate equilibrium between cost-effectiveness and motion continuity. It’s likely that every movie you’ve seen at the cinema or on DVD and Blu-ray was originally recorded progressively at a rate of 24 images per second. On most playback systems, each frame is then displayed multiples times to reduce flicker.1

In a presentation on frame rate, videographer Mark Schubin tells us candidly, “There is absolutely nothing special about 24 frames per second. There is no particular psychological reason for it, no mathematical reason for it.”

The Allure of HFR

As HFR’s proponents have pointed out, none of the constraints around the 24 frame format applies to today’s world of digital ubiquity. Digital production costs do not scale dramatically with frame rate, and the latest in digital imaging allows for a variety of speeds, from 48 to 60 to 10,000 frame/s, and even speeds faster than the speed of light. Current technology is now able to capitalize on our visual system’s potential in ways that were simply not possible in the 1920s. While the human eye is unable to take in many thousands of unique images per second, 48 veers much closer to our biological limits than does 24.2

This boost in capture speed carries immediate visual improvements, namely an increase in motion resolution. If you’ve ever seen a fast-paced pan of the camera while watching a movie, you’ve likely noticed the jerkiness. This is because there are not enough frames in the source to maintain smooth motion. Filmmakers are reflexively cognizant of this (there are even tables which calculate the maximum speed of camera pans before strobing occurs) and ensure that their pans do not spill over a specified threshold. With 48 and higher frame rates, this problem goes away. With more frames to work with, camera movements are more stable, lending quick pans and the choreography of action scenes greater intelligibility.

Image via reduser.net

Jackson is well aware of this, having employed over two dozen RED EPIC cameras in his 3D production of The Hobbit. While HFR can give 2D exhibitions a more lifelike edge, its benefits are most palpable in 3D presentations, where artifacts like flicker, blur and crosstalk tend to be exacerbated. In the traditional 3D cinema experience, each eye is alternately sent 24 unique frames each second. For theaters capable of HFR playback, each eye receives 48 unique frames, resulting in a smoother, less headache-inducing visual experience.3 Jackson chose to release the HFR format for 3D exhibition only; all 2D exhibitions will be shown in the traditional 24-frame format. (Check here for a current list of theaters upgraded to 48-frame projection systems.)

For commercial purposes, 48 fps has the added benefit of being fully backward-compatible with legacy 24-frame playback systems.

The (Dis)Comfort Zone

Ultimately, the disfavor surrounding The Hobbit‘s higher frame rates stems not from an unrefined use of props, lighting and CGI, as some have suggested, but from our stiffly conditioned sensorium. We’re simply not used to viewing movies this way. Our lifelong familiarity with 24-frame cinema has etched its signature into our collective subconscious.

Though it might seem like the jump from 24 to 48 is not significant, it is massive in terms of how our optical system responds and adjusts. The added smoothness unshackles our suspension of disbelief we so intimately associate with low-motion content, ejecting us from the fantasy world of Middle Earth and onto the set, where props are revealed for what they really are. In short, a closer approximation of reality is exchanged for the filmic, almost surreal quality we identify with the cinema experience.

High-motion content is abundant outside of movies. Since the destination of broadcast material, including sports and news coverage and low-budget programming, is in the living room rather than the theater, this content is typically recorded on video at 30 frame/s.4 The visual flavor of this type of content seems every bit as natural to us as the flavor of film-based content. Our years of conditioning means that we subliminally associate different species of programming with their capture rates.

Those who own newer high-definition TVs have likely already experienced HFR movies, albeit in exaggerated and simulated form. Most flat panels today, especially 120 Hz and 240 Hz LCDs, come equipped with MCFI (motion compensation via frame interpolation), which synthesizes new frames from existing ones to enhance motion resolution.5 When this effect is applied to movies captured at 24 fps, the cinematic feel is upgraded to the feel of the high-motion programming described above. Commonly called the “soap opera effect”, this not only boosts the level of realism to uncomfortably high levels but distorts director intent by creating new frames not present in the film source. The core difference between The Hobbit and motion processing at home, of course, is that the high-motion in The Hobbit is not simulated; the frames are present in the source material.

Adapt or Go Home

As with any new technology, HFR will need to survive an incubation period. Sound, color, widescreen, digital projection, 3D; all have had their share of doomsayers, yet cinema lives on. Jackson’s Hobbit format provides us more sensory information and is actually closer to reality than what we’re used to, the same promises accompanying every other industry revision since silent film. Even so, perhaps none has had as dramatic an impact on the “feel” of the film as HFR. While The Hobbit‘s juiced frame rates and hyper-realism may be initially unnerving, most agree that it is an effect that wears off after the first 20-30 minutes. After all, our sensorium is highly adaptable and, over time, we may come to associate 48 fps with cinema the way we associate 24 fps now.

For several in the industry, HFR is long overdue, and they have Peter Jackson to thank for being the first to break down commercial barriers. As Cameron and others prepare their own high frame rate presentations for the big screen, time will tell whether the tide of opinion flows toward the new medium or recedes back to the familiar embrace of the legacy format.

Footnotes:

1. When movies are shown at your local theater, each frame is flashed twice or three times to compensate for flicker arising from the inter-frame black period. Thus while the movie itself was recorded at 24p, the movie projector runs at a 48 or 72 Hz refresh rate. The same is true for digital projection and flat-panel televisions; each film frame is repeated according to the refresh rate of the display.
   [↩]
2. The human visual system’s response rate is highly dependent on the particular stimuli. For example, our sensitivity to light in dark environments is greater than our sensitivity to dark in light surrounds. Tests with Air force pilots have shown that the human eye can identify light that is flashed only for 1/220th of a second. We are capable of taking in hundreds of light emissions per second, though we vary in our ability to make distinct sense of those images as the frequencies climb higher and the stimuli change. At the end of the day, frame-based motion is merely a simulation of reality.
   [↩]
3. The figures given are for unique frames per eye. You’ll recall that for 2D theatrical presentations, each frame is flashed twice to reduce flicker. The situation is the same for 3D exhibitions. Given a 24-frame projection system, you are processing 48 total frames per eye. For 48-frame playback, this number doubles to 96 per eye.
   [↩]
4. This frame rate is easier to reproduce at home compared with 24 frame/s. Because the standard refresh cycle of televisions (in the U.S.) is 60 Hz, only a simple 2:2 pulldown is required to render 30 frame/s material. This avoids the judder which arises when 24 frame/s material is converted to 30 frame/s for proper playback on 60 Hz playback systems. This process, called telecining, is needed to make the 24-frame standard format of film compatible with video frame rates used in television and broadcast. Additionally, the economics of higher-cost film make video capture a more popular choice for low-budget programming like daytime soaps and the rest.
   [↩]
5. MCFI’s main function is to upgrade low-motion content to appear as fluid and continuous as high-motion content. The technology algorithmically creates new frames to insert between the source frames as if they were always there, thereby artificially inflating the frame rate. The quality of the results can vary broadly by manufacturer in accordance with the processor and software fitted to the display. Most HDTVs have this feature enabled by default. To annul the video-like “soap opera effect” it is the first processing feature I disable when calibrating or bringing home a new display.
   [↩]