Subtitles

Some programs and online software allow automatic captions, mainly using speech-to-text features.

For example, on YouTube, automatic captions are available in English, Dutch, French, German, Italian, Japanese, Korean, Portuguese, Russian, Indonesian, Spanish, Turkish, Ukrainian and Vietnamese. If automatic captions are available for the language, they will automatically be published on the video.[1][2]

The "CC in a TV" symbol Jack Foley created, while senior graphic designer at Boston public broadcaster WGBH that invented captioning for television, is public domain so that anyone who captions TV programs can use it.

Closed captioning is the American term for closed subtitles specifically intended for people who are deaf or hard-of-hearing. These are a transcription rather than a translation, and usually also contain lyrics and descriptions of important non-dialogue audio such as (SIGHS), (WIND HOWLING), ("SONG TITLE" PLAYING), (KISSES), (THUNDER RUMBLING) and (DOOR CREAKING). From the expression "closed captions", the word "caption" has in recent years come to mean a subtitle intended for the deaf or hard-of-hearing, be it "open" or "closed". In British English, "subtitles" usually refers to subtitles for the deaf or hard-of-hearing (SDH); however, the term "SDH" is sometimes used when there is a need to make a distinction between the two.

Programs such as news bulletins, current affairs programs, sports, some talk shows, and political and special events utilize real time or online captioning.[3] Live captioning is increasingly common, especially in the United Kingdom and the United States, as a result of regulations that stipulate that virtually all TV eventually must be accessible for people who are deaf and hard-of-hearing.[4] In practice, however, these "real time" subtitles will typically lag the audio by several seconds due to the inherent delay in transcribing, encoding, and transmitting the subtitles. Real time subtitles are also challenged by typographic errors or mishearing of the spoken words, with no time available to correct before transmission.

Some programs may be prepared in their entirety several hours before broadcast, but with insufficient time to prepare a timecoded caption file for automatic play-out. Pre-prepared captions look similar to offline captions, although the accuracy of cueing may be compromised slightly as the captions are not locked to program timecode.[3]

Newsroom captioning involves the automatic transfer of text from the newsroom computer system to a device which outputs it as captions. It does work, but its suitability as an exclusive system would only apply to programs which had been scripted in their entirety on the newsroom computer system, such as short interstitial updates.[3]

In the United States and Canada, some broadcasters have used it exclusively and simply left uncaptioned sections of the bulletin for which a script was unavailable.[3] Newsroom captioning limits captions to pre-scripted materials and, therefore, does not cover 100% of the news, weather and sports segments of a typical local news broadcast which are typically not pre-scripted. This includes last-second breaking news or changes to the scripts, ad-lib conversations of the broadcasters, and emergency or other live remote broadcasts by reporters in-the-field. By failing to cover items such as these, newsroom style captioning (or use of the teleprompter for captioning) typically results in coverage of less than 30% of a local news broadcast.[5]

Communication access real-time translation (CART) stenographers, who use a computer with using either stenotype or Velotype keyboards to transcribe stenographic input for presentation as captions within two or three seconds of the representing audio, must caption anything which is purely live and unscripted;[3] however, more recent developments include operators using speech recognition software and re-voicing the dialogue. Speech recognition technology has advanced so quickly in the United States that about 50% of all live captioning was through speech recognition as of 2005. Real-time captions look different from offline captions, as they are presented as a continuous flow of text as people speak.[3]

Stenography is a system of rendering words phonetically, and English, with its multitude of homophones (e.g., there, their, they're), is particularly unsuited to easy transcriptions. Stenographers working in courts and inquiries usually have 24 hours in which to deliver their transcripts. Consequently, they may enter the same phonetic stenographic codes for a variety of homophones, and fix up the spelling later. Real-time stenographers must deliver their transcriptions accurately and immediately. They must therefore develop techniques for keying homophones differently, and be unswayed by the pressures of delivering accurate product on immediate demand.[3]

Submissions to recent captioning-related inquiries have revealed concerns from broadcasters about captioning sports. Captioning sports may also affect many different people because of the weather outside of it. In much sport captioning's absence, the Australian Caption Centre submitted to the National Working Party on Captioning (NWPC), in November 1998, three examples of sport captioning, each performed on tennis, rugby league and swimming programs:

• Heavily reduced: Captioners ignore commentary and provide only scores and essential information such as "try" or "out".
• Significantly reduced: Captioners use QWERTY input to type summary captions yielding the essence of what the commentators are saying, delayed due to the limitations of QWERTY input.
• Comprehensive realtime: Captioners use stenography to caption the commentary in its entirety.[3]

The NWPC concluded that the standard they accept is the comprehensive real-time method, which gives them access to the commentary in its entirety. Also, not all sports are live. Many events are pre-recorded hours before they are broadcast, allowing a captioner to caption them using offline methods.[3]

Because different programs are produced under different conditions, a case-by-case basis must consequently determine captioning methodology. Some bulletins may have a high incidence of truly live material, or insufficient access to video feeds and scripts may be provided to the captioning facility, making stenography unavoidable. Other bulletins may be pre-recorded just before going to air, making pre-prepared text preferable.[3]

News captioning applications currently available are designed to accept text from a variety of inputs: stenography, Velotype, QWERTY, ASCII import, and the newsroom computer. This allows one facility to handle a variety of online captioning requirements and to ensure that captioners properly caption all programs.[3]

Current affairs programs usually require stenographic assistance. Even though the segments which comprise a current affairs program may be produced in advance, they are usually done so just before on-air time and their duration makes QWERTY input of text unfeasible.[3]

News bulletins, on the other hand, can often be captioned without stenographic input (unless there are live crosses or ad-libbing by the presenters). This is because:

• Most items are scripted on the newsroom computer system and this text can be electronically imported into the captioning system.
• Individual news stories are of short duration, so even if they are made available only just prior to broadcast, there is still time to use QWERTY in text.[3]

For non-live, or pre-recorded programs, television program providers can choose offline captioning. Captioners gear offline captioning toward the high-end television industry, providing highly customized captioning features, such as pop-on style captions, specialized screen placement, speaker identifications, italics, special characters, and sound effects.[6]

Offline captioning involves a five-step design and editing process, and does much more than simply display the text of a program. Offline captioning helps the viewer follow a story line, become aware of mood and feeling, and allows them to fully enjoy the entire viewing experience. Offline captioning is the preferred presentation style for entertainment-type programming.[6]

Subtitles for the deaf or hard-of-hearing (SDH) is an American term introduced by the DVD industry.[7] It refers to regular subtitles in the original language where important non-dialogue information has been added, as well as speaker identification, which may be useful when the viewer cannot otherwise visually tell who is saying what.

The only significant difference for the user between SDH subtitles and closed captions is their appearance: SDH subtitles usually are displayed with the same proportional font used for the translation subtitles on the DVD; however, closed captions are displayed as white text on a black band, which blocks a large portion of the view. Closed captioning is falling out of favor as many users have no difficulty reading SDH subtitles, which are text with contrast outline. In addition, DVD subtitles can specify many colors on the same character: primary, outline, shadow, and background. This allows subtitlers to display subtitles on a usually translucent band for easier reading; however, this is rare, since most subtitles use an outline and shadow instead, in order to block a smaller portion of the picture. Closed captions may still supersede DVD subtitles, since many SDH subtitles present all of the text centered (an example of this is DVDs and Blu-ray Discs manufactured by Warner Bros.), while closed captions usually specify position on the screen: centered, left align, right align, top, etc. This is helpful for speaker identification and overlapping conversation. Some SDH subtitles (such as the subtitles of newer Universal Studios DVDs/Blu-ray Discs and most 20th Century Fox Blu-ray Discs, and some Columbia Pictures DVDs) do have positioning, but it is not as common.

DVDs for the U.S. market now sometimes have three forms of English subtitles: SDH subtitles; English subtitles, helpful for viewers who may not be hearing impaired but whose first language may not be English (although they are usually an exact transcript and not simplified); and closed caption data that is decoded by the end-user's closed caption decoder. Most anime releases in the U.S. only include translations of the original material as subtitles; therefore, SDH subtitles of English dubs ("dubtitles") are uncommon. [8][9]

High-definition disc media (HD DVD, Blu-ray Disc) uses SDH subtitles as the sole method because technical specifications do not require HD to support line 21 closed captions. Some Blu-ray Discs, however, are said to carry a closed caption stream that only displays through standard-definition connections. Many HDTVs allow the end-user to customize the captions, including the ability to remove the black band.

• 
  Example of speaker ID in SDH (voices are off-screen)
• 
  Example of non-speech information in SDH (source is off-screen)
• 
  Example of non-speech information in SDH (two sources: off-screen and on-screen)
• 
  Example of music description in SDH
• 
  Example of lyrics transcription in SDH

Song lyrics are not always captioned, as additional copyright permissions may be required to reproduce the lyrics on-screen as part of the subtitle track. In October 2015, major studios and Netflix were sued over this practice, citing claims of false advertising (as the work is henceforth not completely subtitled) and civil rights violations (under California's Unruh Civil Rights Act, guaranteeing equal rights for people with disabilities). Judge Stephen Victor Wilson dismissed the suit in September 2016, ruling that allegations of civil rights violations did not present evidence of intentional discrimination against viewers with disabilities, and that allegations over misrepresenting the extent of subtitles "fall far short of demonstrating that reasonable consumers would actually be deceived as to the amount of subtitled content provided, as there are no representations whatsoever that all song lyrics would be captioned, or even that the content would be 'fully' captioned."[10][11]

Although same-language subtitles and captions are produced primarily with the deaf and hard-of-hearing in mind, many others use them for convenience. Subtitles are increasingly popular among younger viewers for improved understanding and faster comprehension. Subtitles allow viewers to understand dialogue that is poorly enunciated, delivered quietly, in unfamiliar dialects, or spoken by background characters. A 2021 UK survey found that 80% of viewers between 18 and 25 regularly used subtitles, while less than a quarter of those between 56 and 75 did.[12][13][14]

Same language subtitling (SLS) is the use of synchronized captioning of musical lyrics (or any text with an audio/video source) as a repeated reading activity. The basic reading activity involves students viewing a short subtitled presentation projected onscreen, while completing a response worksheet. To be really effective, the subtitling should have high quality synchronization of audio and text, and better yet, subtitling should change color in syllabic synchronization to audio model, and the text should be at a level to challenge students' language abilities.[15][16] Studies (including those by the University of Nottingham and the What Works Clearinghouse of the United States Department of Education) have found that use of subtitles can help promote reading comprehension in school-aged children.[17] Same-language captioning can improve literacy and reading growth across a broad range of reading abilities.[18][19] It is used for this purpose by national television broadcasters in China and in India such as Doordarshan.[18][20]

In some Asian television programming, captioning is considered a part of the genre, and has evolved beyond simply capturing what is being said. The captions are used artistically; it is common to see the words appear one by one as they are spoken, in a multitude of fonts, colors, and sizes that capture the spirit of what is being said. Languages like Japanese also have a rich vocabulary of onomatopoeia which is used in captioning.

In some East Asian countries, especially Chinese-speaking ones, subtitling is common in all taped television programs and films. In these countries, written text remains mostly uniform while regional dialects in the spoken form can be mutually unintelligible. Therefore, subtitling offers a distinct advantage to aid comprehension. With subtitles, programs in Mandarin or any dialect can be understood by viewers unfamiliar with it.

According to HK Magazine, the practice to caption in Standard Chinese was pioneered in Hong Kong during the 1960s by Run Run Shaw of Shaw Brothers Studio. In a bid to reach the largest audience possible, Shaw had already recorded his films in Mandarin, reasoning it would be most universal variety of Chinese. However, this did not guarantee that the films could be understood by non-Mandarin-speaking audiences, and dubbing into different varieties was seen as too costly. The decision was thus made to include Standard Chinese subtitles in all Shaw Brothers films. As the films were made in British-ruled Hong Kong, Shaw also decided to include English subtitles to reach English speakers in Hong Kong and allow for exports outside Asia.[21]

On-screen subtitles as seen in Japanese variety and other reality television shows are more for decorative purpose, something that is not seen in television in Europe and the Americas. Some shows even place sound effects over those subtitles. This practice of subtitling has been spread to neighbouring countries including South Korea and Taiwan. ATV in Hong Kong once practiced this style of decorative subtitles on its variety shows while it was owned by Want Want Holdings in Taiwan (which also owns CTV and CTI) during 2009.

Real-time translation subtitling usually involves an interpreter and a stenographer working concurrently, whereby the former quickly translates the dialogue while the latter types; this form of subtitling is rare. The unavoidable delay, typing errors, lack of editing, and high cost mean that real-time translation subtitling is in low demand. Allowing the interpreter to directly speak to the viewers is usually both cheaper and quicker; however, the translation is not accessible to people who are deaf and hard-of-hearing.

Some subtitlers purposely provide edited subtitles or captions to match the needs of their audience, for learners of the spoken dialogue as a second or foreign language, visual learners, beginning readers who are deaf or hard of hearing and for people with learning and/or mental disabilities. For example, for many of its films and television programs, PBS displays standard captions representing speech from the program audio, word-for-word, if the viewer selects "CC1" by using the television remote control or on-screen menu; however, they also provide edited captions to present simplified sentences at a slower rate, if the viewer selects "CC2". Programs with a diverse audience also often have captions in another language. This is common with popular Latin American soap operas in Spanish. Since CC1 and CC2 share bandwidth, the U.S. Federal Communications Commission (FCC) recommends translation subtitles be placed in CC3. CC4, which shares bandwidth with CC3, is also available, but programs seldom use it.

The two alternative methods of 'translating' films in a foreign language are dubbing, in which other actors record over the voices of the original actors in a different language, and lectoring, a form of voice-over for fictional material where a narrator tells the audience what the actors are saying while their voices can be heard in the background. Lectoring is common for television in Russia, Poland, and a few other East European countries, while cinemas in these countries commonly show films dubbed or subtitled.

The preference for dubbing or subtitling in various countries is largely based on decisions made in the late 1920s and early 1930s. With the arrival of sound film, the film importers in Germany, Italy, France, Switzerland, Luxembourg, Austria, San Marino, Liechtenstein, Monaco, Czech Republic, Slovakia, Hungary, Belarus, Ukraine, Russia, Andorra, Spain and United Kingdom decided to dub the foreign voices, while the rest of Europe elected to display the dialogue as translated subtitles. The choice was largely due to financial reasons (subtitling is more economical and quicker than dubbing), but during the 1930s it also became a political preference in Germany, Italy and Spain; an expedient form of censorship that ensured that foreign views and ideas could be stopped from reaching the local audience, as dubbing makes it possible to create a dialogue which is totally different from the original. In larger German cities a few "special cinemas" use subtitling instead of dubbing.

Dubbing is still the norm and favored form in these four countries, but the proportion of subtitling is slowly growing, mainly to save cost and turnaround-time, but also due to a growing acceptance among younger generations, who are better readers and increasingly have a basic knowledge of English (the dominant language in film and TV) and thus prefer to hear the original dialogue.

Nevertheless, in Spain, for example, only public TV channels show subtitled foreign films, usually at late night. It is extremely rare that any Spanish TV channel shows subtitled versions of TV programs, series or documentaries. With the advent of digital land broadcast TV, it has become common practice in Spain to provide optional audio and subtitle streams that allow watching dubbed programs with the original audio and subtitles. In addition, only a small proportion of cinemas show subtitled films. Films with dialogue in Galician, Catalan or Basque are always dubbed, not subtitled, when they are shown in the rest of the country. Some non-Spanish-speaking TV stations subtitle interviews in Spanish; others do not.

In many Latin American countries, local network television will show dubbed versions of English-language programs and movies, while cable stations (often international) more commonly broadcast subtitled material. Preference for subtitles or dubbing varies according to individual taste and reading ability, and theaters may order two prints of the most popular films, allowing moviegoers to choose between dubbing or subtitles. Animation and children's programming, however, is nearly universally dubbed, as in other regions.

Since the introduction of the DVD and, later, the Blu-ray Disc, some high budget films include the simultaneous option of both subtitles and/or dubbing. Often in such cases, the translations are made separately, rather than the subtitles being a verbatim transcript of the dubbed scenes of the film. While this allows for the smoothest possible flow of the subtitles, it can be frustrating for someone attempting to learn a foreign language.

In the traditional subtitling countries, dubbing is generally regarded as something strange and unnatural and is only used for animated films and TV programs intended for pre-school children. As animated films are "dubbed" even in their original language and ambient noise and effects are usually recorded on a separate sound track, dubbing a low quality production into a second language produces little or no noticeable effect on the viewing experience. In dubbed live-action television or film, however, viewers are often distracted by the fact that the audio does not match the actors' lip movements. Furthermore, the dubbed voices may seem detached, inappropriate for the character, or overly expressive, and some ambient sounds may not be transferred to the dubbed track, creating a less enjoyable viewing experience.

A map of Europe showing which audiovisual translation methods are preferred in each country.

  Subtitles: Countries in Europe where dubbing is only used for children's programs and family films; otherwise, subtitles are solely used.

  Mixed areas: Countries in Europe occasionally using either multi-voice voice-over translations (Bulgaria) or dubbing (Turkey, Northern Cyprus); otherwise, subtitles are solely used, with the exception of children's programs and family films.

  Voice-over: Countries in Europe usually using voice-over translations which feature one or two voices, while lowering the volume of the original soundtrack; examples are Poland, Russia and Lithuania. This method is mainly used in television broadcasts and home media, but these countries generally also use dubbing or subtitles as well in other contexts.

  General dubbing: Countries in Europe where dubbing is used for most foreign-language films and TV series.

  Countries which occasionally produce their own dubbings, but generally instead use dubbing versions from neighboring countries produced in mutually intelligible languages instead (Belarus and Slovakia).

  Belgium: The Dutch-speaking region occasionally produces its own dubbing versions (but usually uses the same ones as the Netherlands), otherwise solely subtitles. The French-speaking region of Wallonia and the German-speaking region of East Belgium use exclusively full-cast dubbing for both films and TV series.

In several countries or regions nearly all foreign language TV programs are subtitled, instead of dubbed, such as:

It is also common that television services in minority languages subtitle their programs in the dominant language as well. Examples include the Welsh S4C and Irish TG4 who subtitle in English and the Swedish Yle Fem in Finland who subtitle in the majority language Finnish.

In Wallonia (Belgium) films are usually dubbed, but sometimes they are played on two channels at the same time: one dubbed (on La Une) and the other subtitled (on La Deux), but this is no longer done as frequently due to low ratings.

In Australia, one FTA network, SBS airs its foreign-language shows subtitled in English.

There are still many more uncommon formats. Most of them are text-based and have the extension .txt.

For cinema movies shown in a theatre:

• Cinema
• D-Cinema: digital projection of movie in DCP format

For movies on DVD Video:

• DVD-Video subtitles (related to VobSub)
• Blu-ray Disc subtitles (related to PGS)

For TV broadcast:

• Teletext
• DVB Subtitles (DVB-SUB)
• Philips Overlay Graphics Text
• Imitext

Subtitles created for TV broadcast are stored in a variety of file formats. The majority of these formats are proprietary to the vendors of subtitle insertion systems.

Broadcast subtitle formats include: .ESY, .XIF, .X32, .PAC, .RAC, .CHK, .AYA, .890, .CIP, .CAP, .ULT, .USF, .CIN, .L32, .ST4, .ST7, .TIT, .STL

The EBU format defined by Technical Reference 3264-E[31] is an 'open' format intended for subtitle exchange between broadcasters. Files in this format have the extension .stl (not to be mixed up with text "Spruce subtitle format" mentioned above, which also has extension .stl)

For internet delivery:

• SMIL
• TTML/DFXP
  • SMPTE-TT/CFF-TT (for UltraViolet-compatible players)
  • EBU-TT-D

The Timed Text format currently a "Candidate Recommendation" of the W3C (called DFXP[32]) is also proposed as an 'open' format for subtitle exchange and distribution to media players, such as Microsoft Silverlight.