Aspect Ratio

Displays

1024x768 is a 4:3 (pronounced 4 by 3) resolution. What this means is that an image or display that is 1024x768 will be 1.3 (the underline signifies that the 3s go on forever) times as wide as it is tall. This can be determined by simplifying the fraction 1024/768, which gets us 4/3. Some plasmas, however, have a 1024x768 resolution but a 16:9 (1.7) shape. What’s going on?

To clear up the confusion, take a look below at the 4x3 grid:

Because the cells in the above grid are square, the 4x3 grid has a 4:3 shape. By contrast, a 16:9 4x3 grid is shown below. Notice the wider cells:

As is shown, stretching out the 4x3 grid gives it a 16:9 shape rather than a 4:3 shape. But just how much did those pixels need to be stretched? To determine this, we need to define three different types of aspect ratios. It should be noted that these three types of aspect ratio go by various names, so I chose what was easiest to follow.

The Resolution Aspect Ratio (RAR) is the aspect ratio of the numeric dimensions. Both of the 4x3 grids have a RAR of 4:3.

The Actual Aspect Ratio (AAR) is the measurement of the shape of the display. The first 4x3 grid has an AAR of 4:3 while the second 4x3 grid has an AAR of 16:9.

Finally, there is the Pixel Aspect Ratio (PAR), which is a measurement of the shape of the pixels. To determine the PAR of a display, simply divide the AAR by the RAR. Because the AAR and RAR match in the first 4x3 grid, the first 4x3 grid has a PAR of 1:1. For the second 4x3 grid, dividing 16:9 by 4:3 gets us 48:36, which simplifies to 4:3. What this means is that in order for a 4:3 RAR to gain a 16:9 AAR, the individual cells/pixels must have a 4:3 shape rather than a square shape.

Images and Videos

Images and videos can also have differing aspect ratios, although the only time this really applies to images is if you are talking about a still image from a video. Returning to the subject at hand, consider the case of a 640x480 picture. Without even seeing the picture, we know that it has a RAR of 4:3. This can be determined by simplifying 640/480. However, what about its AAR? Take a look at the 640x480 image below from the 2003 PC version of the 2001 Xbox game Halo:

This 640x480 image also has an AAR of 4:3. Because the RAR and the AAR match, everything’s fine. Now, suppose I convert the 640x480 image to 640x360 by cutting out, a.k.a. cropping, the top and bottom 60 rows. 640x360 has a RAR of 16:9, but will our resulting 640x360 video also have an AAR of 16:9? Since we are converting by cropping, then the answer is yes. This is shown below:

Now, for our last trick, let’s convert the 640x480 Halo picture to 640x360 again, but this time without cropping. The resulting image will still have an AAR of 4:3, but it will now have a RAR of 16:9 since 640x360 has a RAR of 16:9. This ends up creating a wacky-looking, too-wide, and distorted picture that is shown below:

In previous times, video and displays only had an AAR of roughly 4:3. However, this 4:3 video still came in a variety of resolutions and thus a variety of RARs. Non-HD (High-Definition) video has traditionally had a max resolution of 720x480 in America and 720x576 in Europe [1 page 14]. It is worth pointing out that neither one of these resolutions actually has a RAR of 4:3. Non-HD from computers typically is either 640x480 or 800x600, but these resolutions DO have a RAR of 4:3.

However, with the arrival of HDTVs on Aug. 6, 1998 [2], video and displays suddenly had an AAR of 16:9. The main HD video resolutions are 1280x720 (16:9 RAR and AAR), 1440x1080 (4:3 RAR and 16:9 AAR), and 1920x1080 (16:9 RAR and AAR) [1 page 14].

Film

Like video, movies come in either the non-widescreen or widescreen variety. The difference is that while non-widescreen video is 4:3 (1.3) and widescreen video is 16:9 (1.7), non-widescreen movies are either 4:3 or roughly 1.37, while widescreen movies have had several aspect ratios such as 1.66, 1.75, 1.85, 1.96, 2, 2.21, 2.35/2.39/2.4, 2.55, 2.58, and even 2.76 [3] [4] [5]. Today, the most common aspect ratios for widescreen movies are 1.85 and 2.35, with 16:9 gaining in popularity due to the arrival of HDTVs and HD video [5].

Before further explaining widescreen movies, let's first look at non-widescreen movies. Originally, movies were composed of 0.980 inch-by-0.735 inch (.980"x.735") images on film that was 35mm wide [6]. This .980"x.735" shape meant that movies originally had an aspect ratio of 4:3. However, these dimensions had to be made slightly lower for the addition of sound onto the film. Although there seems to be some disagreement over what the image dimensions of film with sound was, everyone agrees that these dimensions were lower than .980"x.735" and had a RAR/AAR of roughly 1.37 [5] [6] [7] [8]. This reduced 1.37 image and its associated sound clip fit within a larger .980"x.735" area dedicated for each frame.

1.85 movies are typically made widescreen via the use of a matte on the 1.37 non-widescreen area of film. A matte is something that covers up the top and bottom of a larger area on film to create a wider image. There are two types of mattes: a soft matte and a hard matte. The difference between a soft and hard matte lies in not what the audience sees but in what images are present on the original film [5] [9].

A soft matte is a matte that is in the projector. This means that if a soft matte is used to make a movie widescreen, the image on the original film is non-widescreen. It is similar in principle to recording a movie with the appearance of the first Halo picture but showing the audience the second Halo picture [5] [9].

On the other hand, a widescreen movie filmed with a hard matte is filmed with black bars on the top and bottom put in place by the camera. This means that the widescreen picture that the theater audience is seeing is the same image present on the original film, although the widescreen picture on the film will have black bars on the top and bottom [5] [9].

2.35 movies are made a few different ways. One way is to use anamorphic film. Although the images on film do not have a set resolution like in video, these images do have set dimensions, with .980"x.735" being one such an example. It is easiest to think of the aspect ratio of these dimensions as the film's RAR. The 4:3, 1.37, and 1.85 film methods discussed so far all have images on film that have a matching RAR and AAR. However, the images on anamorphic film do NOT have a matching RAR and AAR. Instead, the image dimensions (RAR) have a narrower aspect ratio than the aspect ratio the image is meant to be viewed at (AAR). This is corrected for by the projector, which takes the initially too-narrow picture and stretches it to give it the correct shape. The benefit of this method is that all of the image space on film is used rather than only part of the image. This in turn can create a better looking image when it is projected [6].

The next method to create a 2.35 movie is to use 2-perf film, a.k.a. the Techniscope process. In order for film to go through a projector, it has holes on its sides for the projector to latch onto, and these holes are called perforations. Standard 35mm film has 4 perforations on each side of each image in the film. However, if each image in the film is the height of only 2 of these perforations, you can get an image that has an aspect ratio of roughly 2.35. Thus, 2-perf film is similar in principle to a matte widescreen movie, with the main differences being the wider aspect ratio and the lack of unseen areas being stored on the film. 2-perf film also allows a filmmaker to not have to buy as much film since more images can be stored in the same amount of film. For theater exhibition, the images on the 2-perf film are converted to anamorphic 4-perf film images [6].

Yet another method to produce a 2.35 film is to use the Super 35 process. The Super 35 process is similar to the soft matte process, but with two important exceptions. The first is that a 2.35 portion rather than a 1.85 portion is shown to the audience. The next, and most important, is that sound is not recorded onto the film but is recorded elsewhere, allowing the recorded image to take up the entire .980"x.735" area. The fact that the image takes up this bigger area helps to make up for the fact that so much is being cropped from the top and bottom. After the movie is filmed in Super 35, the film images along with the sound recorded elsewhere are transferred to anamorphic 4-perf film images for theater exhibition [10].

Finally, 16:9 movies are typically recorded one of two ways. One way is to simply record the movie using 1920x1080 video. Another method is to use 3-perf Super 35, which ends up having an aspect ratio of roughly 16:9. Something to keep in mind, however, is that a movie filmed on 3-perf Super 35 may be shown with a 2.35 aspect ratio in theaters.

Converting Film to 4:3 Video

Converting non-widescreen film to 4:3 video is relatively simple process, with a couple of caveats. When converting 4:3 film to 4:3 video, there are two possible choices to make, but the end results end up being pretty similar. The issue here is that while video is a rectangular shape and therefore has sharp edges, the images on film have slightly rounded edges [6]. Therefore, if the entire 4:3 film image was transferred to 4:3 video, you'd end up having curved black edges in the corner of the video. Now, the people in charge of transferring this 4:3 film to 4:3 video might be OK with that. If they're not, however, they'd have to exclude a small amount from each side of the film in the film-to-video transfer. 1.37 film not only has curved edges but also a slightly wider aspect ratio, meaning that for 1.37 films there is also a choice between transferring all vs. nearly all of the film for 4:3 video release.

For soft matte widescreen movies, three basic options are available for transferring the movie to 4:3 video. The simplest way to transfer a soft matte widescreen movie to 4:3 video is to use an open matte transfer, which is when nearly all of the 4:3/1.37 image area present on the original film is transferred to 4:3 video. Letterboxing, on the other hand, takes the widescreen image that was shown in theaters and adds black bars to prevent the picture from getting distorted. Finally, pan-and-scan chops off the sides of the widescreen movie to get it to fill the 4:3 area. This 4:3 area can "pan" to the left and right to include as much important information as possible, hence the name pan-and-scan. For other types of widescreen movies, the open matte method and letterbox methods are essentially equivalent since the only extra information to show on the top and bottom is black bars. Open matte, letterboxing, and pan-and-scan are illustrated below:

4:3 Video Transfer Options for Soft Matte Widescreen Movies

Open matte - nearly all of the image present on the original film is transferred to video.	Letterboxing - the movie remains widescreen by replacing the top and bottom with black bars.	Pan-and-scan - the sides of the widescreen picture are chopped off.

Sometimes, the 4:3 version of a widescreen movie will end up being at a midway point between an open matte and pan-and-scan release. These "hybrid" transfers will add some image information to the top and/or bottom while still taking away a noticeable amount from the sides. An example of this is the movie Terminator 2: Judgment Day, sometimes abbreviated as T2.

T2 was shot in the Super 35 format. An image from T2 is shown below [11]:

The red section is the section that was shown in the theater and has an aspect ratio of 2.35. The blue section is the part used for the 4:3 video transfer and thus has an aspect ratio of 4:3. As you can see, the 4:3 video transfer adds quite a bit to the bottom but also takes a noticeable amount from the sides.

What's odd though about T2 is that the overlay of the red and blue sections has a resolution of 164x100 and therefore an aspect ratio of 1.64. This would suggest that the image present on the original film has an aspect ratio of around 1.64. However, this doesn't quite make sense since T2 is a Super 35 film and should therefore have 4:3 or 16:9 images on film, not 1.64 images. Even if you take into account the whole curved edges issue, the curved edges on 4:3 film images would certainly not force this overlay to go from a 4:3 to a 1.64 aspect ratio. The gray bars I added show the larger 4:3 area.

What exactly is going on in the gray areas of the T2 picture remains a mystery. The site that the picture is from claims that for T2 the 4:3 version not only pans left and right but zooms in and out [11]. Someone at another site says that the cameras used for T2 had a hard matte and gave the images on the film an aspect ratio of 1.56 [12]. Of course, 1.56 is still not 1.64, so it is possible that T2 was both filmed with a hard matte to 1.56 and has a 4:3 transfer that zooms in and out.

Converting Film to 16:9 Video

Although transferring a widescreen movie to 4:3 video with letterboxes is an option, a much better option – assuming the open matte option is not desired or is not available – is to transfer the movie to 16:9 video. This allows the black bars placed in the picture to be smaller or even eliminated if the movie was filmed in 16:9. For example, a 4:3 720x480 video that stores a 2.35 movie will only use a 720x272 area for the actual picture (480 x 4/3 = 640, 640/2.35 ≈ 272). By contrast, a 16:9 720x480 video that stores a 2.35 movie will use a 720x363 area for the picture. This greater use of picture area will create a sharper picture when upscaling. Of course, the best option would be to record the 2.35 movie in 1920x1080, which would result in a picture area of 1920x817. Assuming you are watching this 1920x1080 video on a 1920x1080 display, no upscaling would be required.

If you go back to the end of the "Images and Video" section, you'll notice that the available HD resolutions – 1280x720, 1440x1080, and 1920x1080 – all have an AAR of 16:9. This means that any film-to-HD transfer will be a transfer to video with an AAR of 16:9. HD's sole AAR of 16:9 raises an important question: how is non-widescreen film transferred to HD?

Before answering this question, it is important to point out that there are plenty of movies and even TV shows, such as Seinfeld, shot on non-widescreen film [13]. As previously explained, soft matte widescreen movies are also shot on non-widescreen film. In fact, before 1953, movies only came in the non-widescreen variety [5].

Now that importance of non-widescreen film has been established, it can be explained how non-widescreen film is transferred to HD, preferably to 1920x1080. The first option is to pillarbox the non-widescreen picture. This is similar to letterboxing, only the black bars are placed at the sides rather than the top and bottom. This allows the entire original film picture to be viewed. The other option is to crop out the top and bottom of the non-widescreen picture to fill up the 16:9 area. This second option eliminates the pillarboxes but it also means you are not seeing all of the original picture. If you are the type of person that really hates black bars, keep in mind that most HDTVs will have a zoom mode to zoom in on the central 16:9 portion of pillarboxed picture. This means that just because you are dealing with pillarboxed video doesn't mean you can't fill up the whole screen.

DVDs

A Blu-ray Disc (BD) allows for the storage of HD video at up to 1920x1080 [1 page 14]. A Digital Versatile Disc (DVD) [14], the predecessor to BD, stores video at a max of 720x480i59.94 or 720x576i50 [15]. In order to allow for smaller or nonexistent black bars and thus improved video quality, video at these resolutions can have an AAR of 16:9 [16].

To be more specific, DVD supports anamorphic video. As already mentioned, anamorphic film is any film where the AAR is wider than the RAR. However, this definition of anamorphic does not apply to video. Instead, anamorphic video applies strictly to video with an AAR of 16:9 but a RAR of 3:2 (1.5) or less. Thus, 4:3 720x576 video would NOT be considered anamorphic video despite the fact that the AAR, 4:3, is wider than the RAR, which is 5:4 (1.25). Only 16:9 720x576 would be considered anamorphic video [16].

Although anamorphic video lets you reduce or even eliminate black bars, its existence raises a fundamental question – how does a display know how to show video correctly? For example, suppose a 1920x1080 HDTV receives a 720x480 signal. This signal could be a 4:3 signal, in which case it should be pillarboxed. On the other hand, it could be a 16:9 signal, in which case it should be stretched to cover up the whole screen.

Luckily, there are a few different solutions to this problem. Before discussing these solutions, let's take a step back and look at how DVDs are displayed correctly on old-fashioned CRTs. We'll use 720x480 DVD video as an example.

As discussed in the Resolution, Scaling, & Progressive vs. Interlaced article, CRTs don't have a native resolution but instead a max resolution. This allows a CRT to not only display lower resolutions without scaling them but to also take an image and squish or stretch it to a different aspect ratio without actually scaling it. So, even though 720x480 has a RAR of 3:2, a CRT television will squish this down to a 4:3 aspect ratio.

What about anamorphic video – how is that shown correctly on a standard CRT television? To be shown correctly, anamorphic video needs to be given a 16:9 shape. However, simply outputting an anamorphic image to a conventional CRT television would give the image a 4:3 shape and everything would be too skinny. This is because the CRT television would be configured from the factory to show 720x480 content as 4:3 and not 16:9.

In order to deal with this problem, DVD players typically offer a 4:3 output and a 16:9 output option. Here's how it works. 4:3 video is output using the entire 720x480 resolution regardless of whether the DVD player is set to 4:3 or 16:9 output. Anamorphic video, however, is output differently depending on what you've set your DVD player to. If you've set your DVD player to 16:9 output, then the anamorphic video is output using the whole 720x480 resolution. If, on the other hand, you've set your DVD player to 4:3 output, then the anamorphic video is downscaled to 720x360 and black bars are placed on the top and bottom to fill in the remaining 720x480 area. Thus, the anamorphic video gains a widescreen shape within a 4:3 picture. Anamorphic video on a DVD is indicated as anamorphic via the use of a widescreen "flag" that is read by the DVD player [17].

Now that it has been established how anamorphic DVDs are shown on a conventional CRT television, we can discuss some of the other ways to show anamorphic video. After all, the whole point of anamorphic video is to be able to use the entire 720x480 area for video, not have it downscaled to 720x360.

The most basic way to display anamorphic video without downscaling is to use the manual zoom selection method. Let's return to the original example of a 1920x1080 HDTV getting a 720x480 signal. When displaying non-HD content, HDTVs will typically have a 4:3 zoom mode that will pillarbox the non-HD content and a 16:9 zoom mode that will stretch the non-HD content. This allows the video to be displayed with the correct AAR. Even some 4:3 CRT televisions have a 16:9 zoom mode that takes the CRT's 720x480 max resolution and squeezes it down to a 16:9 shape.

The main problem with the manual zoom selection method is that it’s a hassle. Many anamorphic DVD movies, for instance, still have 4:3 bonus feature videos. This means that while watching the movie 16:9 would be the proper zoom mode, but while watching the bonus feature video 4:3 would be the proper zoom mode. To make matters worse, you'd have to go through the hassle of switching the DVD output between 4:3 and 16:9 to actually determine the aspect ratio of the bonus features.

The next method to deal with anamorphic content is for the video player to inform the display if the video is anamorphic. This is known as widescreen signaling. So, if widescreen signaling were used, the player outputting the 720x480 video would send a message to the TV saying the video has an AAR of 16:9, and the TV would automatically switch to its 16:9 zoom mode. Widescreen signaling is sometimes possible over an S-Video, component, or HDMI connection [18] [19] [20] [21] [22]. However, for widescreen signaling to work, it has to be supported both by the player and the display over a common connection.

Finally, the last method is to get an upscaling player. As already mentioned, the widescreen flag present in anamorphic video alerts a DVD player to add letterboxes to the video when the player is set to 4:3 output. Without this flag, anamorphic DVD wouldn't be able to be shown correctly on a conventional CRT television. Therefore, it is crucial that DVD players read this flag. If the player is an upscaling player, this means that it has 1920x1080 output. The player can use this flag to properly pillarbox or stretch the DVD video to 1920x1080 and then simply send this video to a TV. This eliminates the need to inform the TV what AAR the video has because the TV is no longer doing any scaling.

Non-HD Video Games

For DVD video, the main benefit of anamorphic video is that it allows more of the resolution to be used for the actual picture rather than black bars. However, with the exception of short movies contained within video games called cutscenes, video games don't need letterboxes. This is because, unless you are talking about a cutscene, there are no static widescreen images to crop; you, the player, generate the images. If you want to see to the left or right of your character, just move your character!

Rather, the main benefit from anamorphic video games is that it allows the video to be stretched over an entire 16:9 display without distorting the picture. Non-anamorphic non-HD video games must have pillarboxes on the sides to be viewed correctly on a 16:9 display.

The three main non-portable video game systems today are Microsoft's Xbox 360, Sony's PlayStation 3 (PS3), and Nintendo's Wii. Each system handles the aspect ratio of non-HD games differently.

Games for the Xbox 360 are in HD, so for these games aspect ratio is not an issue. However, the Xbox 360 is also capable of playing back some Xbox games [23], and most of these games don't support HD. However, when set to HD output, the Xbox 360 automatically adds pillarboxes to Xbox games that don't support widescreen [24]. This allows the Xbox 360 to have the most hassle-free way of playing back non-HD games.

Games for the PS3 are also in HD, so for PS3 games aspect ratio is also not an issue. PlayStation 1 (PS1) games, however, do not support HD, and neither do most PlayStation 2 (PS2) games. Initial PS3 models could play back nearly all PS2 games, later PS3 models could play back some PS2 games, and current PS3 models cannot play back any PS2 games [25]. However, when the playback of a PS2 game is supported, the PS3 is able to upscale PS2 games to 1280x720 or 1920x1080 [26] and pillarbox non-widescreen games or stretch widescreen games. All current PS3 models play back most PS1 games and take the same approach for PS1 games as they do for PS2 games [25].

Unfortunately, the PS3 is not able to pillarbox or stretch PS1 or PS2 games correctly automatically. For the few PS1 games that supported anamorphic output [27], these games had a widescreen option that the user selected in the game's menu. PS2 games take a similar approach, although Burnout 3 simply looks at if the system is set to 16:9 output and omits an in-game option [28]. Returning to the PS3, the "Normal" option in "PS/PS2 Upscaler" section puts out an upscaled picture with pillarboxes while the "Full Screen" option puts out an upscaled picture that is stretched to widescreen [25]. The fact that an "Auto" option is missing presumably means that the PS3 is unable to detect what aspect ratio a PS1 or PS2 game is running at, although some PS2 games, such as Burnout 3, are still able to detect what aspect ratio the system is set at.

Finally, we have the Wii. In addition to playing Wii games, the Wii can also play games from the previous Nintendo system, GameCube (GC) [29 page 0], as well as downloadable versions of games from even older systems in what is known as the Virtual Console (VC) service [30]. However, Wii, GC, and VC games are all non-HD. Additionally, the Wii cannot output HD video, meaning it can't do any game upscaling. Not only that, but the Wii doesn't even support the use of a widescreen flag [31] [32] [33] [34]. This leaves the manual zoom method as the only option for displaying widescreen games.

When playing back GC or VC games on the Wii, any available widescreen mode is set within the game. When playing Wii games on the Wii, whether or not a game goes into widescreen is determined by if the Wii is set to 4:3 or 16:9 output. This would imply that, for Wii games, all you have to do is set the Wii to 16:9 and then set your TV's zoom mode to 16:9 and you're good to go. However, not all Wii games support widescreen [35], and for these games the proper thing to do would be to set your TV's zoom mode to 4:3. The Wii can be left at 16:9 output.

The 720p Video and Display Discrepancy

720p video is video that is 1280x720. However, most of the current 720p displays today have a native resolution of 1366x768, not 1280x720. At first, this may seem like a bad idea. After all, for a display with a native resolution of 1366x768 to show 1280x720 content, it would either have to center the 1280x720 content or scale it to 1366x768, and as demonstrated by the grids in the Resolution, Scaling, & Progressive vs. Interlaced article, both of these would be problematic.

Part of the reason that 720p displays typically have a native resolution of 1366x768 is to fit 1024x768 video content without downscaling. 1280x720 has a central 4:3 area of 960x720, and this is obviously less than 1024x768. The other part of the reason is to avoid having black bars for 720p video. If all you want to do is have a display that is capable of displaying both 1280x720 and 1024x768 content without downscaling, then a display with a native resolution of 1280x768 is sufficient. However, a 1280x768 display would display black bars on the top and bottom for 1280x720 content. The solution to this issue is to take 1280x768, which has an aspect ratio of 5:3, and add enough columns to make an aspect ratio of 16:9. Unfortunately, with 768 rows, this is impossible. This is because if you divide 768 by 9/16, you get 1365.3. The next best thing then is a resolution of 1366x768, although this ends up having a slightly too wide aspect ratio of 683:384 and as already mentioned worsens the 720p image if scaled to the entire display.

Sources

8. 35 mm film. Wikipedia.

11. Moving Pictures. Home Cinema City.

12. Thread started by Odeex on Dec. 14, 2003 2:21 AM ET. Dumb question about super 35. Lockergnome.

18. S-Video. Wikipedia.

20. Thread started by kojiro on May 15, 2005 9:47 PM GMT -5 and ended by Foxbat121 on July 31, 2005 12:34 PM GMT -5. Finally, auto-Pillarbox via Component on PWD7UY!!!. AVS FORUM ARCHIVE 2. ©1995-2007 AVS Forum, Inc.

27. HDTV Arcade Playstation One database. HDTV Arcade.

28. Thread started by Kal on Apr. 18, 2005 12:18 AM. How does 16:9 work with a PS2 and an Xbox?. HDTV Arcade.

32. Thread started by DracIsBack on May 25, 2008 7:55 AM. Wii, HDTV and Virtual Console Games. AtariAge.

35. High Definition Game Database. HDGAMES.NET.