Technology for subtitling: a 360-degree turn*
Tecnología para la subtitulación: un giro de 360
grados
BELÉN AGULLÓ
INVESTIGADORA INDEPENDIENTE / TRANSMEDIA CATALONIA
Dirección de correo electrónico: belen.agullo.garcia@gmail.com
ORCID: https://orcid.org/0000-0003-3270-1840
Recibido: 28/3/2019. Aceptado: 3/7/2019.
Cómo citar: Agulló, Belén, «Tecnología para la subtitulación: un giro de 360 grados»,
Hermēneus. Revista de Traducción e Interpretación, 22 (2020): 11-40.
DOI: https://doi.org/10.24197/her.22.2020.11-40
Abstract: Subtitling has become one of the most important audiovisual translation modes and
cannot be understood outside the context of the technology that makes it possible. New
audiovisual media are appearing, such as 360o videos, and the necessity of subtitling this type of
content to make it accessible is emerging. In this article, an updated review of current subtitling
technology is presented to contextualise the study. Then, a review of main immersive
environments (3D, augmented reality and virtual reality) and their implications for subtitling has
also been introduced. The focus of the study is on virtual reality and, therefore, the main
challenges of subtitling 360o content are presented. To respond to the needs of subtitling this
type of content, a prototype version of a subtitle editor has been developed and presented to
twenty-seven professional subtitlers who have tested the tool and reported the correspondent
feedback on usability and preferences. This study has proven the importance of carrying out
usability tests with end users when developing specific software. Finally, the challenges faced by
subtitlers in new audiovisual media such as 360o content are presented.
Keywords: Subtitling, subtitling technology, 360o content, reception study, usability.
Resumen: La subtitulación se ha convertido en uno de los modos de la traducción audiovisual
más importantes y no puede estudiarse fuera del contexto tecnológico que la hace posible.
Asimismo, nuevos medios audiovisuales, como los vídeos de 360o, están emergiendo y la
*
ImAc has received funding from the European Union’s Horizon 2020 research and
innovation programme under grant agreement No.761974. The author is a member of
TransMedia Catalonia, a research group funded by the Secretaria d’Universitats i
Recerca del Departament d’Empresa i Coneixement de la Generalitat de Catalunya,
under the SGR funding scheme (ref. code 2017SGR113). This article reflects only the
author’s views and the funding institutions hold no responsibility for any use that may
be made of the information it contains. This article is part of Belén Agulló’s PhD in
Translation and Intercultural Studies at the Department of Translation, Interpreting and
East Asian Studies (Departament de Traducció i d’Interpretació i d’Estudis de l’Àsia
Oriental) of the Universitat Autònoma de Barcelona.
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
12
Belén Agullo García
necesidad de subtitular este tipo de contenidos para hacerlos accesibles es evidente. En este
artículo se presenta una revisión de la tecnología de subtitulación existente para contextualizar
el estudio. A continuación, se presenta una revisión de los principales entornos inmersivos (3D,
realidad aumentada y realidad virtual) y sus implicaciones para la subtitulación. El estudio se
centra en la realidad virtual y, por lo tanto, se presentan los principales retos de la subtitulación
de contenidos en 360o. Para responder a las necesidades de subtitulación de este tipo de vídeos,
se ha desarrollado una versión prototípica de un editor de subtítulos que se ha presentado a
veintisiete subtituladores profesionales que han probado la herramienta y aportado sus
opiniones y sugerencias. En este estudio se ha demostrado la importancia de realizar pruebas de
usabilidad con los usuarios finales a la hora de desarrollar software específico. Finalmente, se
presentan los retos a los que se enfrentan los subtituladores en nuevos medios audiovisuales
como los contenidos 360o.
Palabras clave: Subtitulación, tecnología de la subtitulación, contenido en 360o, estudio de
recepción, usabilidad.
Summary: 1. Introduction; 2. Subtitling technology: overview; 3. Immersive media and subtitling;
4. ImAc web editor; 5. Evaluation, 5.1. Evaluation setup, 5.2. Evaluation methodology, 5.3.
Participants, 5.4. Evaluation results; 6. Conclusions; Acknowledgements; References.
Sumario: 1. Introducción; 2. Tecnología de la subtitulación; 3. Medios inmersivos y subtitulación;
4. Editor web de ImAc; 5. Evaluación, 5.1. Configuración, 5.2. Metodología, 5.3. Participantes,
5.4. Resultados; 6. Conclusiones; Agradecimientos; Referencias bibliográficas.
1. INTRODUCTION
We are experiencing a subtitling revolution. The distinction between
subtitling and dubbing countries is becoming more and more outdated.
Nowadays, on certain platforms the choice between subtitled or dubbed
versions is made individually by each viewer and not imposed by
distribution companies. The key aspect of this revolution is the new
distribution model for audiovisual content based on Video on Demand
(VoD) platforms such as Netflix, Prime Video, HBO, Hulu and the ones
to come, such as Disney+ or Apple TV+. In these platforms, the viewers
can access content from all over the world, in different languages and
with different audiovisual translation modes available, mainly subtitling
and dubbing. Subtitling has become such an important element in
audiovisual communication that it has recently been the focus of a heated
debate. Netflix decided to include Castilian Spanish subtitles for the film
Roma (2018), which was filmed in Mexican Spanish, by the Mexican
award-winning film director Alfonso Cuarón. Some viewers and linguists
disagreed with this approach, and Cuarón himself considered the subtitles
“provincial, ignorant and offensive to Spaniards themselves” (Cuarón
quoted in Morales et al., 2019: online). Due to social pressure and the
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
Technology for subtitling: a 360-degree turn
13
comments from the director, Netflix ended up removing the Castilian
Spanish subtitles for this film.
Changes are also being introduced in the way audiovisual stories are
created. Recently, the first interactive series episode was launched by
Netflix, accessible via computers, smartphones or Smart TV. The episode
called “Bandersnatch” is part of the science fiction series Black Mirror.
The main character of the episode set in the 80s is an aspiring video
game programmer who is working on coding a game based on a chooseyour-own-adventure novel called Bandersnatch. In this episode, the
viewer has the control over the main character’s decisions through the
story, leading to a myriad of different endings (Harris et al., 2019). Also,
new technology such as virtual reality (VR) or augmented reality (AR)
are inspiring content creators and storytellers to develop engaging stories.
The European Broadcasting Union (EBU) issued a report on VR / AR
(EBU, 2017) stating that 49% of its members are developing or planning
to develop immersive content. Members believe that VR offers new
opportunities to tell stories from a different perspective and could be
more engaging for the audience. In a more recent survey on the industry
insights of VR and AR, results show that the interest in VR / AR
applications for films and series has dropped since previous surveys
(Perkins Coie LLP, 2018). According to the report, this could be due to
the belief that shorter experiences work better for AR / VR. However, in
the same report, they state that video games sector is still leading the list
of the industries with more investment in VR / AR. This seems
contradictory, because video games are long audiovisual experiences.
Therefore, the reasons behind the drop in investments for VR / AR in
audiovisual content other than video games might be different (for
example, poor quality of user experience, lack of engaging content or
hardware high prices). All in all, these new formats and technologies
applied to audiovisual content creation may have an impact on how
content is subtitled, and technology should be prepared to face the new
challenges.
Academic studies on subtitling technology are limited with a few
exceptions (Georgakopoulou, 2012; Díaz-Cintas, 2013, 2014). However,
the importance of technology in the practice of subtitling is paramount,
as technology is necessary for its existence and further development.
Subtitling software has rapidly evolved for the past few years, giving
response to this new audiovisual and digital society, which has led to a
“technology turn” in audiovisual translation (AVT) and, specifically, in
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
14
Belén Agullo García
subtitling (Díaz-Cintas, 2013). Many are the innovations introduced in
subtitling software, mainly focused on automation to increase
productivity and satisfy the ever-growing need for subtitles in different
contexts and situations, as it will be detailed in the next section.
Moreover, new audiovisual formats, such as VR or AR, introduce new
dimensions and challenges for AVT that need to be tackled by the
development of the appropriate technology. To that end, a prototype of a
subtitle web editor for 360º videos has been developed in the framework
of the European-funded project Immersive Accessibility (ImAc). 1 The
main goal of ImAc is to make immersive content accessible to all types
of users. To do so, a first step was to develop different technologies to
produce the access services for this new medium including subtitles,
because according to our research no commercial solutions were
available at this point. A first version of a subtitle editor for 360º content
has been developed. This prototype has been tested with twenty-seven
professional users, with the aim of gathering their feedback regarding the
technological needs of subtitlers working with immersive content.
In this article, an overview of current subtitling technology will be
offered. Then, the challenges posed by immersive content in terms of
subtitling are reviewed. In section 4, the prototype of the ImAc subtitle
web editor will be presented and discussed. In section 5, the methodology
and the results will be explained. Finally, some conclusions will be
drawn.
2. SUBTITLING TECHNOLOGY: OVERVIEW
The change of analogue signal to digital broadcasting, the
appearance of DVD and Blu-Ray and the universal access to internet
have all contributed to the growth of subtitling and subtitling technology
(Díaz-Cintas, 2014). Nowadays, both intralingual and interlingual
subtitles are available at the touch of a button for the audience, which has
caused what Díaz-Cintas labelled the “commoditisation of subtitling”
(Díaz-Cintas, 2013, 2014). The consolidation of digital television and the
proliferation of VoD platforms as a mainstream way of consuming
audiovisual content and the accessibility approach taken by them,
offering subtitles, subtitles for the deaf and hard-of-hearing (SDH),
dubbing and audio description in several languages, has given a new
1
www.imac-project.eu/
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
Technology for subtitling: a 360-degree turn
15
boost to audiovisual translation. The demand for digital video content
continues to increase and forecasts point to a growth also in media
localisation that will become a USD 2.5 billion industry by the end of
2020 (Estopace, 2017). The needs generated by this exponential increase
in content and localisation demand will only be met with the support of
technology that is able to adapt to the new challenges.
In previous studies, Díaz-Cintas (2013, 2014) offered an extensive
review of subtitling technology advances up to that date. From the
analogue subtitling practices from the 70s to the digital revolution that
brought new advancements to this field, the practice of subtitling has
evolved. Nowadays, with the professional software available, subtitlers
are able “to create the timecodes, to respect shot changes, to control the
reading speed of their subtitles, to translate and spell check their texts, to
simulate their subtitles on screen, etc.” (Díaz-Cintas, 2013: p. 124). Some
examples of professional subtitling software are: EZTitles, FAB, Screen
Subtitling, Swift, Spot or WinCAPS. However, the high prices of
professional tools favoured the appearance of free software such as
Subtitle Workshop, Aegisub or Subtitle Edit. The latter are mostly used
by fansubbers to create their own subtitles for the fan community. DíazCintas (2014) distinguishes between fansubs and crowdsourced
subtitling. One of the main differences is that fansubbers produce
subtitles without the consent of the content owners, and crowdsourced
subtitles are distributed with the consent of the interested parties, without
copyright infringements. Some platforms that encourage volunteers to
create crowdsourced subtitles are Amara, TED or even YouTube. These
platforms offer cloud-based subtitling platforms for the volunteers to
create subtitles, but with very limited options compared to professional
tools. Nonetheless, the trend of cloud-based professional subtitling tools
to streamline the creation and distribution of subtitles is noticeable. For
example, Zoosubs, imediatrans or OOONA offer an integrated, cloudbased workflow for localising audiovisual content.
The increase of audiovisual content makes subtitling a commodity.
Therefore, media companies are continuously looking for solutions to
streamline their processes and be able to cope with the growing demand.
Subtitling technology includes new features to respond to industry
expectations, mainly based on automation. According to Díaz-Cintas
(2014), some professional subtitling tools include an automatic detection
of shot changes, facilitating the spotting task. Also, most programmes
include an audio wave indicator that shows the audio behaviour in the
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
16
Belén Agullo García
video. Subtitlers can then skip the parts of the video where the dialogue
is missing and go directly to the interest points, especially during quality
assurance stage. Also, it is helpful for spotting, because subtitlers can
visually see where the subtitle should start and end (Díaz-Cintas, 2014).
Speech alignment technology can facilitate the task even more,
automatically synchronising the transcription with the soundtrack and the
video. Another step forward in automation comes from automatic speech
recognition (ASR) software. Some subtitling software, such as
Omniscien, can transcribe the audio, extract the dialogue and create a
subtitle draft, even in the target language with machine translation (MT)
technology (Dranch, 2018).
Several studies have been carried out with the aim of creating live
subtitles automatically (Garcia et al., 2009; Álvarez et al., 2010; Mathur
et al., 2015). Some technologies that are key to the generation of
automatic subtitles are: voice activity detection, ASR, discourse segment
detection and speaker diarisation (Álvarez et al., 2010). When this
technology is put together in a single system, the creation of machine
generated live subtitles is possible. A study on the reception of automatic
generated and translated subtitles was carried out by Matamala et al.
(2015). In the main experiment, thirty participants with different levels of
English watched three randomised clips in English: one with no subtitles,
one with automatic generated intralingual subtitles and another with
automatic generated interlingual subtitles into Spanish. The conclusions
of the study showed that intralingual subtitles can be beneficial to
increase language comprehension for viewers with a medium level of
English (B2). However, interlingual subtitles had a negative impact on
C2 level viewers, probably due to a distracting effect.
The necessity of making live events, such as conferences, talks or
even lessons, accessible to audiences with hearing impairments has
promoted the implementation of ASR systems in several contexts. For
example, about 1,500 deaf and hard-of-hearing (HoH) students are an
integral part of the campus at the Rochester Institute of Technology.
Apart from American Sign Language (ASL) interpreters, they have
implemented Microsoft Translate, a communication technology that uses
Artificial Intelligence technology to generate subtitles (Roach, 2018).
This system uses advanced ASR technology to convert speech into
fluent, correctly punctuated text, which can be automatically translated
into the 60 languages that the system supports (with the quality that MT
currently offers). The deaf and HoH students agreed that this was a useful
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
Technology for subtitling: a 360-degree turn
17
tool to complement the ASL interpretation during lessons. Also,
professors at Rochester Institute of Technology recognised the potential
of this tool for education, because all students (deaf or hearing) used the
transcripts as a learning and course material (Roach, 2018). Similar
initiatives are being reproduced in other universities, such as the app
poliSubs developed by the Universitat Politècnica de València. 2 Previous
attempts have also been made at a European level, such as the EU-funded
project transLectures 3 (2011-2014), which aimed at developing tools for
automatic transcription and translation for online educational videos.
Open and free solutions to generate automatic subtitles with ASR
technology can be found in YouTube 4 and the app Web Captioner. 5
YouTube integrates an option to generate automatic subtitles in English,
Dutch, French, German, Italian, Japanese, Korean, Portuguese, Russian,
and Spanish. When a video is uploaded to YouTube, it can be edited, and
transcriptions and translations can be added. Automatic transcriptions
with timings are generated and can then be postedited through a basic
interface. In the interface, the subtitler can add and delete subtitles,
modify the time codes, modify subtitle content, preview subtitles in the
video and use the audio wave for a more accurate spotting. A list of
keyboard shortcuts is also provided to streamline the process. The
transcribed subtitles can be then translated to as many languages as
desired and be added to the video. Subtitles generated in YouTube can
also be downloaded in different formats (.vtt, .srt, .sbv). Although the
quality of the automatic subtitles varies depending on the type of content,
the quality of sound, background noises, different accents, etc., this could
be a powerful tool to increase accessibility. Web Captioner is a free realtime subtitling app for browsers based on speech-to-text transcription
technology. It supports over forty languages and dialects and only a
computer with internet connection and a microphone is necessary.
Subtitles can be then downloaded in a plain text format .txt or .doc
(without time codes).
The use of computer-aided translation (CAT) tools, even if present in
many translation fields such as technical or medical translation, is limited
in the field of AVT. It is argued that CAT tools are not suitable for AVT
2
https://www.elmundo.es/comunidad-valenciana/2018/12/26/5c235fdefc6c83 c3018b4
56e.html
3
https://www.mllp.upv.es/projects/translectures/
4
https://support.google.com/youtube/answer/6373554?hl=en
5
https://webcaptioner.com/
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
18
Belén Agullo García
due to the culturally-bound and creative types of content. However, CAT
tools are widely used in game localisation (O’Hagan and Mangiron,
2013), which also includes dialogue text to be dubbed and subtitled and
creativity and cultural references are involved in the translation process.
CAT tools were introduced in game localisation to boost productivity due
to the large number of words and the limited time to translate the content.
They were also important to maintain consistency across projects where
teams of several translators work simultaneously thanks to translation
memories (TM) and glossaries integrated in the tools. Consistency and
intertextuality are also common and necessary in films and series.
Different translators can work in different seasons of the same series
without having the reference material. So far, the only way to ensure
consistency is to watch previous episodes or look for previous subtitle
files, but this could be very time-consuming. Having all the translations
in a TM would significantly ease the translator’s work. Anticipating
industry’s needs, some CAT tool companies such as memoQ have
introduced some features to facilitate the task of the subtitlers. For
example, the tool can process specific subtitle formats such as .srt. Also,
the translator can preview the video with the translated subtitles in the
same tool with a simple plugin, as can be seen in Image 1.
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
Technology for subtitling: a 360-degree turn
19
Image 1. memoQ translation panel with video preview.
The next step in automation would be to introduce (MT) in the subtitling
workflow. Several funded projects aimed at achieving automatic
subtitling with automatic translation have been developed in the last
years such as MUSA 6 (2004-2007) and SUMAT 7 (2011-2014). The
former’s ambitious goal was to create a whole automatic system that
converted audio into text, then generated the time-coded subtitles and,
finally, translated the subtitles into other languages. However, no
tangible results ever materialised (Díaz-Cintas, 2014). The latter’s main
goal was “to use the archives of subtitle files owned by several subtitling
companies to build a large corpus of aligned subtitles that will be used to
train SMT [statistical machine translation] systems in various language
pairs” (Bywood et al., 2013: p. 596). An evaluation with trained
subtitlers was carried out to test the quality of the MT-generated
subtitles. Although the study has some limitations, according to their
results, “subtitlers considered 56.79% of the subtitles [27,565 MT
subtitles] they rated to be of quality 4 or 5, meaning that they required
little to no post-editing to be of a publishable standard” (Bywood,
Georgakopoulou and Etchegoyhen, 2017: p. 497). Even if results are far
from optimal, they should not be disregarded. It is worth noticing that
when SUMAT project was developed, the MT system was based on
statistical MT and, nowadays, research on the MT field is focused on
neural MT, which is the new paradigm for MT (Forcada, 2017).
Applying neural MT might result in different results and further research
is necessary to clarify the future of MT in AVT.
3. IMMERSIVE MEDIA AND SUBTITLING
Immersive media aim at making the audience feel immersed in a
story, as if they were transported to another reality. One example of
immersive medium is stereoscopic 3D, which has been present in homes
and cinemas in the past few decades. The blockbuster Avatar (2009) by
James Cameron that was projected in cinemas in stereoscopic 3D all over
the world caused a comeback of this technology that was already popular
during the 50s (Agulló and Orero, 2017). However, accessibility was
6
7
http://sifnos.ilsp.gr/musa/
http://www.fp7-sumat-project.eu/
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
20
Belén Agullo García
considered as an afterthought in this medium, and the implementation of
subtitles was part of the postproduction stage. Due to that and the lack of
specialised subtitling technology for stereoscopic 3D, some issues raised
when integrating subtitles. Superimposing 2D subtitles on a 3D image
provoked undesirable effects such as ghosting, hindering the readability
of subtitles and causing fatigue and eye strain (Agulló and Matamala,
2019).
Other immersive medium still under research and development is
AR, which combines real world objects with superimposed virtual
objects (Agulló and Matamala, 2019). The applications of this
technology are very promising in the field of accessibility, especially in
theatres and cinemas. A project led by the researcher Romero-Fresco to
use smart glasses to display live subtitling in theatre has been developed
by the GALMA Research Group 8 together with the National Theatre in
London. 9 They use AR and speech recognition technology to implement
live subtitles for their plays. Glasses are adaptable and subtitles
customisable. 10 This is an ongoing project and evaluation is still not
available, according to our research. If this technology is fine-tuned and
mainstreamed, it might become a revolution in accessibility.
Finally, the immersive medium under consideration in this study is
VR, specifically 360º videos. This type of content can be watched in
head-mounted displays (HMD), giving the viewer the freedom to look
around, usually with 180º of freedom in the field of view (FoV). This
new format poses different challenges for implementing subtitles. Firstly,
unlike in traditional TV / cinema content, the frame for each scene
depends on where the viewers decide to look at any moment, so there is
no way to guarantee that a specific area of the 360º video is seen by
them. Also, the background image where the subtitles will be displayed
cannot be foreseen either, which can cause contrast issues and have a
negative impact on readability if the subtitles are not well produced.
Therefore, one of the main issues that needs to be solved is where to
locate the subtitles. Secondly, if the speaker is outside the FoV, this
needs to be indicated somehow, to make the content accessible for deaf
and HoH viewers as well. And thirdly, the implementation of subtitles
must not disrupt the VR experience, that is, subtitles should not break
8
http://galmaobservatory.eu/projects/captioning-glasses-for-the-national-theatre/
https://www.nationaltheatre.org.uk/your-visit/access/caption-glasses
10
https://www.youtube.com/watch?v=Hdtf4qUWos4
9
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
Technology for subtitling: a 360-degree turn
21
immersion and should not worsen the VR sickness effect (i.e., viewers
feeling dizziness, headache or eye fatigue because of the consumption of
VR content).
Some studies have been already carried out regarding implementing
subtitles in 360º content (Agulló et al., 2018; Brown et al., 2018; Rothe
et al., 2018; Agulló and Matamala, 2019). Some preliminary studies were
carried out in order to gather feedback from end-users on how to
implement subtitles in 360º contents (Agulló et al., 2018; Agulló and
Matamala, 2019). Results from a focus group in Spain showed that endusers would like to receive subtitles as similar as possible as shown in
TV content. For example, participants suggested using the current
Spanish standard for SDH (AENOR, 2003). They also suggested locating
subtitles in a fixed position in relation to the FoV and highlighted the
importance of using a black background box in order to avoid readability
issues. Regarding providing directions to enhance accessibility,
participants suggested including icons (arrows) or text in brackets (to the
left, to the right), as well as the possibility to include a compass or radar,
in order to indicate where the speaker is located in the 360º video (Agulló
and Matamala, 2019). In a different study, a prototype was tested with a
limited number of participants in order to gather information on the
comfortable field of view for reading subtitles, and the speaker location
identification. Also, a new methodological approach for accessibility
studies is proposed, based on capabilities instead of disabilities (Agulló et
al., 2018). Both in Brown et al. and Rothe et al., they compared different
subtitling modes in order to evaluate which was the preferred solution. In
the first study by the BBC team (Brown et al., 2018), they compared four
different modes: (1) Evenly spaced: subtitles equally spaced by 120º in a
fixed position below the eye line; (2) Follow head immediately: subtitles
follow the viewer as they look around, displayed always in front of them;
(3) Follow with lag: the subtitle appears directly in front of the viewer
and it remains there until the viewers look somewhere else; then, the
subtitle rotates smoothly to the new position in front of the viewer; and
(4) Appear in front, then fixed: subtitles appearing in front of viewers,
and then fixed until they disappear (in this case, the subtitles do not
follow the viewer if they look around). In their study conducted with
twenty-four participants, they concluded that the (2) option was the
preferred one. In the study by Rothe et al. (2018), they tested two types
of subtitles with thirty-four participants: static subtitles (that is, subtitles
that are always visible in the viewer’s FoV) and dynamic subtitles (that
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
22
Belén Agullo García
is, subtitles that are in a fixed position close to the speaker). Even if the
participants did not state a preference in the comparison part of the
questionnaire, dynamic subtitles seemed to have better results in the
questions about presence, sickness and workload.
Even if some previous studies have been carried out about the
implementation of subtitles in 360º videos, to the best of our knowledge
there is currently no specific subtitling software for this type of content,
and subtitles so far have been created manually. Therefore, there is a
need to develop a subtitling software that responds to the current
challenges that have been discussed and that is usable and accessible to
professional subtitlers. For that reason, it was decided in the ImAc
project to develop a prototype of a subtitle web editor that will be
discussed in the following section.
4. IMAC WEB EDITOR
The prototype of the ImAc subtitle web editor has been developed
with the aim of producing accessibility services, specifically subtitles, in
audiovisual content in 360º. The prototype version of the editor was
created following the feedback from professional subtitlers that was
gathered in a previous focus group (Agulló and Matamala, 2019). The
tool has been developed in a collaboration between the different partners
of the ImAc project 11 that includes broadcasters, researchers and
technological companies. Specifically, the company Anglatècnic, experts
in the engineering and development of software systems for the broadcast
and IT sectors, was in charge of the technical development. End-users
had a key role in the development, providing valuable feedback from the
beginning of the project. It was decided to first develop a light version in
web format as a prototype, and a desktop version would be created at a
later stage of the project. The prototype editor, which is accessible via
browser, supports 360º videos that can be uploaded and then previewed
for subtitling. Navigation in the 360º video with the mouse or the
keyboard is possible. Most of the options available in the editor are
similar to those in other commercial subtitle editors. In Image 2, main
options and sections can be seen: video controls (play, stop, pause,
forward / backward options, etc.), subtitle controls (style, position of the
subtitles, alignment options, colour coding for character identification,
11
http://www.imac-project.eu/consortium/
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
Technology for subtitling: a 360-degree turn
23
actions to navigate the subtitles, buttons for time code in and out, etc.),
video preview, subtitle text box, subtitle list, among others.
Image 2. ImAc web subtitle editor interface. 12
The prototype editor also includes a reading speed thermometer for
character limitation. The thermometer is a visual guide to avoid
exceeding the permitted words per minute. The default parameter is 120
words per minute. The thermometer becomes redder as the subtitles
approach the limit. Another option is the pre-set regions. That option
allows the subtitler to set different regions for the subtitles (for example,
up, down, left, right, etc.). The regions are highlighted with a white
rectangle as can be seen in Image 2, so that they can be spotted easily.
Moreover, the prototype editor includes an option that had to be
designed due to the nature of 360º content. This option is called “Set
current angle”. As it has been explained in previous sections, the
speakers are not always positioned within the viewers’ FoV, because they
can move in the 360º space. Therefore, the position of the speaker for
each subtitle needs to be indicated when creating the subtitles. To do so,
the subtitler can navigate in the video and select the angle where the
speaker is positioned. Then, when the option “Set current angle” is
activated, the selected subtitle is anchored to the angle where the speaker
is positioned. This information will then be translated into a metadata in
12
The sample video in this image belongs to the video Holy Land (episode 5) created by
RYOT.
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
24
Belén Agullo García
the subtitle file that will be processed by the 360º video player and, when
the speaker is out of the viewers’ FoV, a guiding mechanism will appear,
for example, an arrow or a radar, to indicate to the viewers where to find
the speaker.
Finally, it is worth mentioning that the prototype editor includes
three preview modes: (1) Edit mode: for editing the subtitles; (2) Forced
preview: for previewing the subtitles (edit options are blocked) with the
system forcing the selected angle for each subtitle; and (3) Free preview:
for previewing the subtitles (edit options are blocked), being free to
navigate the video as desired.
5. EVALUATION
An experiment to test the prototype version of the ImAc web subtitle
editor was conducted. The goal of this experiment was to test the
usability of the editor and, most importantly, to gather participants’
feedback on the tool and on their needs when subtitling 360º content.
Before carrying out the present test, a pilot was conducted with three
users (one practitioner, one university lecturer and one engineer) to verify
that the methodology worked properly. Pilot participants took the test
online, timed themselves and provided feedback to the researcher. This
section describes the evaluation methodology and presents the obtained
results.
5.1. Evaluation setup
The test was carried out online and each participant took the test
with their personal computers / laptops. The only technical requirements
were to have a stable, high speed internet connection and to access the
web editor with Google Chrome (recommended) or Firefox. The web
editor includes a content management module that was used to set up the
entire test. In the content management tool, users with the role of
subtitler (for example, P1, P2, P3, etc.) were created and assigned to each
participant. Also, each participant was assigned an individual task for
subtitling the same sample video. The video and the task were copied as
per the number of participants, so they had access to an individual task
only accessible by each of them. The videos were uploaded in low
resolution to avoid overloading the server, causing poor performance.
The login information was provided by email to participants, together
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
Technology for subtitling: a 360-degree turn
25
with the instructions. Questionnaires were administered using Google
Forms. The test was designed in English, including the video to be
subtitled, the instructions and the questionnaires.
5.2. Evaluation methodology
The test was sent to participants via email. It was divided into
three parts and participants were required to complete it in just one
session:
(1) Introduction, ethical clearance and demographic questionnaire
In the first part of the test, context information about the ImAc
project, the goal of the test, the estimated duration (30 minutes) and the
participant code was provided in the email to participants. Then, they
were asked to give their consent to participate in the test via digital form,
as requested by the ethics committee at Universitat Autònoma de
Barcelona. Finally, they needed to fill in a demographic questionnaire in
order to gather data about their profile, as will be reported in section 5.4.
(2) Technical information and instructions
In this second part of the test, the participants were asked to first
read a quick user guide that was created bespoke for the test. It was
decided that a user guide would be provided, instead of letting them
figure out how the editor worked, because it was assumed that most
participants were not familiar with 360º videos, nor with the potential
issues of subtitling this type of content.
In the user guide, an emphasis was placed on the “Set the current
angle” option, because this was the newest feature and the most difficult
to understand, compared to current subtitling software. Also, giving
participants more information about the tool would speed up the tasks
and elicit more comprehensive and valuable replies in the open questions
part of the test.
Then, instructions on how to access the web editor, as well as login
information were provided. A transcription for the video was also
provided to speed up the process. The tasks to be carried out were clearly
written and shared with the participants as well, and were as follows:
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
26
Belén Agullo García
The tasks that we kindly ask you to perform are:
1. Go to the subtitle editor using the login information. To access the
subtitle web editor, you have to go to this address: XXXX and enter the login
info that has been provided to you in the email.
2. Open the video that has been assigned to you for subtitling.
3. Subtitle the video into your native language from 00:00:00 to
00:01:11.
a. Add subtitles with the correct timecodes.
b. Assign the different colours to the different characters in the
video.
c. Set the angle for each subtitle.
d. Set a second region for subtitles and apply it to one subtitle.
e. Change the alignment to the left for one subtitle.
f. Insert a subtitle between two existing subtitles.
g. Delete two subtitles.
h. Look for a subtitle by content.
4. Preview the video with the forced mode.
5. Save the subtitles and go back to the main window.
6. Open the video again.
7. Preview the video with free mode.
8. Save the subtitles and go back to the main window.
(3) Evaluation questionnaires
After performing all the tasks, participants were asked to reply to an
online questionnaire that was divided into two parts: System Usability
Scale (SUS) questionnaire and open questions.
Testing user experience is a widely researched field of study
(Goodman et al., 2012; Tullis and Albert, 2013; Pannafino and McNeil,
2017). There are several methods to test user experience in relation to
effectiveness, efficiency and satisfaction. For example, cognitive
walkthroughs, card sorting, icon usability testing, contextual inquiry or
online surveys. In this test, the main focus is on usability as defined by
Brooke (1996: 1): “Usability is not a quality that exists in any real or
absolute sense. Perhaps it can be best summed up as being a general
quality of the appropriateness to a purpose of any particular artefact.” In
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
Technology for subtitling: a 360-degree turn
27
this sense, the appropriateness of the prototype of ImAc web editor for
subtitling 360º content was tested. For that purpose, one of the widest
known and used scales, namely the System Usability Scale (SUS), was
administered to participants. SUS is a ten-item Likert scale. Each item
needs to be assessed from 0 to 5. This scale was chosen because it is easy
to administer, provides reliable results with small sample sizes, and has
been validated and used in many studies as a standard self-reported
metrics (Brooke, 2013), becoming one of the most popular measurements
for usability testing (Lewis, 2018).
Image 3. System Usability Scale (Brooke, 1996).
To complement the quantitative data from SUS, open questions were also
included in the post-questionnaire. The questions were aimed at
gathering qualitative information on participants’ feedback and
impressions on the prototype and its specific functionalities. The
questions were:
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
28
Belén Agullo García
1. What did you like most about the subtitle editor?
2. What did you like least about the subtitle editor?
3. What do you think could be improved, and how?
4. Did you miss any functionality? If yes, can you tell us which?
5. Do you find the feature for setting the angle for the subtitle easy to use?
Explain why.
6. Were the preview modes useful for you? Explain why.
7. Do you think it will take you longer to subtitle videos in 360º? Why?
8. Do you think 360º videos will impact your work as a subtitler?
9. Other comments.
5.3. Participants
Twenty-seven participants took part in the current study. The
recruitment criterion was that participants had to be subtitlers who
professionally subtitle audiovisual content. They were recruited via
personal contacts and also by email and public posts on Twitter. The test
was designed in English so that professionals from different countries
could participate without language being a major limitation. The source
language of the video was English. Participants could choose the
language into which they preferred to create the subtitles. The quality of
the subtitles was not observed; therefore, the participants did not need to
master the target languages.
5.4. Evaluation results
The results from the different questionnaires are reported in this
subsection.
Demographic questionnaire
Twenty-seven participants took part in the test (twenty females and
seven males), with ages ranging 24-48 (mean=35.6, sd=6.9). Their main
languages (that is, mother tongue) were Spanish (19), Catalan (3),
English (3), Polish (2), Basque (1), Croatian (1) and Romanian (1) (two
participants were bilingual Spanish / Catalan and one participant was
bilingual Spanish / Basque). They describe their main jobs as AVT
translators, subtitlers for different kind of products, university lecturers
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
Technology for subtitling: a 360-degree turn
29
and researchers. Only one participant has subtitled a 360º video before.
They presented a varying experience in the field of subtitling, varying
from 1 month to 20 years (mean=8.1; sd=6.1). sixteen participants have
produced more than 300 hours of subtitled content, three participants
have produced between 151 and 300 hours of subtitled content, four
participants have produced between 51 and 150 hours and four
participants have produced less than 50 hours. Participants usually
subtitle in Spanish (21), English (10), Catalan (4), Polish (2), Basque (1),
Croatian (1), French (1), Italian (1) or Romanian (1). Participants
declared using different subtitling software (FAB, WinCAPS, Aegisub,
VisualSubSync, Subtitle Workshop, EZTitles, Swift, Subtitle Edit, TED,
Amara, YouTube, Spot, VICOM, Jayex, proprietary software from
clients, among others). Twenty-six participants have studies of university
level and one participant has further education. Eighteen participants
have a degree or master’s degree in translation and interpreting studies
(or languages degrees), eight of them have PhD studies and nine of them
specialise in Audiovisual Translation. Twenty-four participants have
received specialised training on subtitling in MAs, specialised courses or
training.
When asked about which devices they used on a daily basis, all
participants indicated that they use mobile phones; twenty-three
participants use laptops; twenty-one participants use TVs, seventeen
participants use PCs; and nine of them use tablets. When asked about
how often they watch virtual reality content, none of the participants
have watched VR content on a tablet, twenty-three participants have
never watched VR content on a smartphone connected to HMD or in
HMD; some (fourteen) occasionally watch VR content on a smartphone,
twelve participants on a PC, four on a smartphone connected to HMD
and three in HMD; one participant watches VR content on a PC at least
once a month, and one participant on a HMD; finally, one participant
watches VR content on a smartphone at least once a week. When asked
to explain why they have never used virtual reality content such as 360º
videos or only occasionally, three participants replied that they are not
interested, four participants replied that it is not accessible, sixteen
participants replied that they have not had the chance to use it, and others
gave other reasons regarding the expensive price, difficulties to use the
technology or the lack of appealing content. When asked to state their
level of agreement with the statement “I am interested in virtual reality
content (such as 360º videos)”, three participants replied that they
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
30
Belén Agullo García
strongly agreed, thirteen replied that they agreed, seven that they neither
agreed nor disagreed and four of them disagreed. Finally, when asked if
they own any device to access virtual reality content, fifteen participants
replied that they do not own any, five replied that they do not know or
prefer not to reply and seven replied that they do (including BOBVR Z4,
HTC Vive, PC, laptop, smartphone and PlayStation VR).
SUS
SUS scoring system is standardised and defined by its creators. It is
considered that a score above 68 is above average and below 68 would be
below average. The raw SUS score for this test was 59.5. The letter grade
is D+, and the score corresponds to the percentile rank: 29-30% (Sauro
and Lewis 2016: pp. 203-204). The result could also be due to the lack of
experience subtitling 360º content. However, this was a first prototype,
so the most important part of the present study was the qualitative
feedback that will be detailed below and that provides valuable input on
how to improve this first version of the 360º video subtitle editor.
SUS statements
1. I think that I
would like to use
this system
frequently
2. I found the system
unnecessarily
complex
3. I thought the
system was easy to
use
4. I think that I
would need the
support of a
technical person to
be able to use this
system
5. I found the
various functions in
1
(strongly
disagree)
2
3
4
5
(strongly
agree)
3
(11.1%)
4
(14.8%)
11
(40.8%)
8
(29.6%)
1
(3.7%)
5
(18.5%)
10
(37.1%)
7
(25.9%)
4
(14.8%)
1
(3.7%)
1
(3.7%)
6
(22.2%)
4
(14.8%)
14
(51.9%)
2
(7.4%)
8
(29.6%)
12
(44.5%)
5
(18.5%)
0
(0%)
2
(7.4%)
0
(0%)
6
(22.2%)
9
(33.3%)
10
(37.1%)
2
(7.4%)
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
Technology for subtitling: a 360-degree turn
this system were
well integrated
6. I thought there
was too much
inconsistency in this
system
7. I would imagine
that most people
would learn to use
this system very
quickly
8. I found the system
very cumbersome to
use
9. I felt very
confident using the
system
10. I needed to learn
a lot of things before
I could get going
with this system
31
4
(14.8%)
10
(37.1%)
11
(40.7%)
2
(7.4%)
0
(0%)
2
(7.4%)
2
(7.4%)
7
(30%)
8
(29.6%)
8
(29.6%)
2
(7.4%)
5
(18.5%)
8
(29.6%)
10
(37.1%)
2
(7.4%)
0
(0%)
8
(29.6%)
8
(29.6%)
11
(40.8%)
0
(0%)
9
(33.3%)
3
(11.1%)
10
(37.1%)
4
(14.8%)
1
(3.7%)
Table 1. SUS replies from participants.
Open questions
The analysis of replies to open questions follows a qualitative
approach. Participants were asked to reply with their own words to the
questions specified in section 5.2. All replies were thoroughly revised
and tagged. Some of the questions were generic aimed at gathering
general feedback on the ImAc web editor and others were more specific
about functionalities. Finally, some questions were aimed at gathering
feedback on how subtitlers perceived that the subtitling task could be
impacted by immersive environments. The analysis allowed to define
different areas that work well and others that need to be improved in the
current ImAc editor, how the new functionalities were received and
subtitlers’ feedback on the future practice of subtitling immersive
content. The most relevant aspects are described below.
(1) General feedback on web editor
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
32
Belén Agullo García
As stated before, the open questions for the general feedback on the
prototype were focused on what participants liked about the tool, what
they did not like and what they would improve. After analysing the
results, it can be stated that the prototype of the 360º subtitle editor was
well received due to several reasons. Participants stated that what they
liked the most was that the system and some of its features were easy to
use (the word “easy” was used in nine of the replies). Also, adjectives
such as intuitive (2), user-friendly (1) and straightforward (1) were used
to describe the tool, as well as characteristics such as simplicity (1),
practicality (1) and versatility (1). The interface was referred to by some
respondents (4) as a positive part of the web editor, considered clear and
well designed. Some of the functions that were more praised and are
related to the specific task of subtitling 360º content were: navigation and
subtitling of 360º videos (5), set the angle option (4), assigning pre-set
regions (4), assigning colours to different speakers (3) and reading speed
thermometer (2). Two respondents stated that they liked the tool was
cloud-based. Therefore, the main innovations introduced in this prototype
such as preview and navigation of 360º content and setting the angle for
the speakers can be considered usable.
As for improvable features, two main blocks were detected: (a)
general subtitling features and (b) specific 360º content subtitling
features. The first block is less relevant for the current study, because the
tested version was a prototype and general features were planned to be
improved in future versions. The second block, however, is important to
redefine the needs of subtitlers regarding 360º videos that might have not
been considered when developing the tool. Regarding the first block,
several improvements were suggested: customisable shortcuts; improved
and clearer time coding options; freedom to break lines as desired (at the
time the test was carried out, an automatic segmentation based on a
default character limitation was implemented, not allowing
customisation); transparency in the reading speed thermometer and
clearer information about characters per second (cps) or words per
minute (wpm); sound wave to ease spotting; more editing options for
subtitles (including bold, italics, different colours); more quality
assurance options such as spellcheck; among other minor suggestions.
In the second block, some interesting comments for improvement
were spotted for the 360º content subtitling features. Firstly, two
participants suggested to integrate editing and preview modes. Instead of
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
Technology for subtitling: a 360-degree turn
33
changing from edit to preview mode each time the subtitler needs to
preview their work, participants would prefer to have it integrated in a
unique mode as in other 2D subtitling editors. This suggestion could ease
the spotting process and quality assurance. Secondly, regarding the set
the angle option, one participant suggested that it would be easier to right
click on the video with the mouse to indicate where the speaker is or
select with the mouse the area where the speaker is, instead of navigating
with the mouse or arrows. This suggestion could improve accuracy for
this feature. Finally, two respondents suggested to include an automatic
shot detection system. However, this suggestion is not compatible with
360º videos. As stated before, the viewer is in control of the camera for
this type of content. Therefore, the changes of shots depend on viewers
and are unpredictable. Automatic shot change options are not relevant in
subtitling for immersive content.
(2) Specific functionalities
The second part of the open questionnaire was addressed to specific
functionalities for 360º content: the “set the angle” option and the
preview modes. Regarding the “set the angle” option, most participants
(20) considered it was easy to use, describing the option as easy or very
easy, relatively simple, straightforward, not complex, intuitive and
logical. Five users, though, found it difficult. No correlation was found
between previous experience or knowledge of immersive environments
for this negative reply. Participants also made some recommendations to
improve this option. Three of them suggested to have an option to apply
the same angle to consecutive subtitles. This would definitely ease the
task of assigning an angle for each character and scene. Also, one
subtitler raised a concern about off-screen speakers (for example, a
narrator), in which case no angle should be chosen. This was not
considered when developing the tool, as it was assumed that the speaker
was always on screen, but it is not always the case. Therefore, an option
for indicating off-screen voices without angle needs to be implemented.
As far as the preview modes, most participants (23) considered these
modes useful. Three participants stated that forced mode was useful to
check the subtitling work and the free mode was useful to observe the
subtitles as a final user would do. Four participants considered it
necessary to integrate edit and preview modes, so that they can edit
subtitles while previewing the video.
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
34
Belén Agullo García
(3) Impact of subtitling 360º content
The last part of the open questionnaire was aimed at gathering
general feedback about the professional subtitlers’ impressions on this
new medium and subtitling practice. When asked whether subtitling 360º
video will take longer, most participants (22) responded affirmatively,
and most of them (16) considered that having to set the angle would be
more time-consuming. Two participants also stated that subtitling 360º
would take longer, because the subtitler needs to check the whole 360º
video in case there are some texts on screen or inserts that need to be
translated in the subtitle. Three participants considered that it should not
take longer as long as subtitlers have access to specific software for it.
Finally, one participant was worried about shot changes and how they
would work in 360º. As it has been stated before, this is not relevant in
360º videos.
When participants were asked about the impact of subtitling 360º
videos on the job of a subtitler, different opinions were expressed. Three
participants believed that 360º video will have no impact on the subtitler
profession and five were not sure about it. Six participants thought that
this will have an immediate impact on the profession and most of them
(9) considered that 360º videos will impact subtitling, but in the future, if
this medium is mainstreamed and the demand increases.
7. CONCLUSIONS
An updated review of current subtitling technology has been
presented in this article in order to contextualise the study. Different
innovative solutions are being developed to cope with current subtitling
demands as the audiovisual translation industry grows, due to the
digitalisation of communication and proliferation of VoD platforms.
Efforts are put into automating and streamlining the workflow of
subtitling, with technologies such as ASR, CAT tools or MT. However,
new challenges for subtitling are being posed not only by the increasing
demand but also by the appearance of new media, such as immersive
environments. A review of main immersive environments (3D, AR and
VR) and their implications for subtitling has also been introduced in this
article. The focus of the study is on VR and, therefore, the main
challenges of subtitling 360º content have been presented. Professional
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
Technology for subtitling: a 360-degree turn
35
subtitlers now need to consider new aspects such as where to locate the
subtitles for an optimal reading or how to indicate when a speaker is
outside of the FoV of the viewer. To that end, a prototype of the ImAc
web editor has been developed and presented to twenty-seven
professional subtitlers who have tested the tool and reported the
correspondent feedback on usability and preferences.
The test has shed some light on the possibilities of subtitling 360º
content as well as the most important characteristics that a subtitling
software should include to be usable for that task. New features designed
considering the nature of 360º videos such as setting the angle, pre-set
regions or preview modes have been well received by most participants.
According to their feedback, these features are usable and suitable to
generate subtitles for this kind of content, although some improvements
have been suggested and will be implemented in future versions of the
editor. For example, the integration of the edit and preview mode or
easing the task of setting the angle or including an option for off-screen
characters. Also, it is important to remark that traditional subtitling
considerations such as shot changes do not apply in this new medium.
Therefore, automatic shot changes options will not need to be
implemented in this type of subtitle editors. As reported in the results
section, most participants believe that this new medium will have an
impact on the subtitling practice, mainly because of the new dimension
brought by it: directions. Having to set the angle for the different
speakers and having a 360º sphere to look around seems to concern
subtitlers. Therefore, the technology provided for this task needs to focus
on simplifying and automating these additional tasks to make the
subtitling process easier.
Some limitations were identified in this test. The estimated duration
of the test was not accurate. When the pilot for the test was carried out,
pilot participants informed that the test would take around 30 minutes,
but then this time was considered insufficient by some of the participants.
This does not mean that participants had to stop the test, but some of
them were frustrated that the estimation was not accurate enough. For
future tests of this type, the duration should be longer, or participants
should perform a previous session to get use to the tool before carrying
out the test. Another limitation was the profile of subtitlers. Some
subtitlers (especially for interlingual subtitles) are used to translating
templates, not generating subtitles from scratch. Therefore, some of them
might not be used to the process of setting timings, which was considered
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
36
Belén Agullo García
time-consuming and difficult by some participants. However, the
participants of this test were familiar with subtitling software, because
they replied to the question “What software do you normally use?” as
reported in the demographic results section. For future tests, if possible,
all participants should be accustomed to generating subtitles from scratch
or at least a question in the demographic questionnaire should be
included in this regard in order to find whether this has a correlation with
the results or not.
This study has proven the importance of carrying out usability tests
with end users when developing specific software. Subtitlers’ feedback
has been essential to define the future of the ImAc editor and to develop
a tool that would respond to professional needs and market demands.
Future versions should be again tested and polished until professional
subtitlers are satisfied with the results. Subtitling is a powerful tool for
accessing information and is becoming an intrinsic part of the
audiovisual communication. Therefore, the necessity to have appropriate
software and technology to respond to the increasing demands is clear. A
subtitling revolution will never be achieved without a revolutionary
technology that complements it and subtitlers are the cornerstone of this
revolution.
ACKNOWLEDGEMENTS
I would like to thank Anglatècnic, and especially Enric Torres and Àlex
Soler, who worked on the development of the web editor. And, most of
all, to all professional subtitlers who took part in the tests, for their
valuable feedback.
REFERENCIAS BIBLOGRÁFICAS
Agulló, Belén and Anna Matamala (2019), “The challenge of subtitling
for the deaf and hard-of-hearing in immersive environments: results
from a focus group”, The Journal of Specialised Translation, 32, pp.
217-235, on https://jostrans.org/issue32/art_agullo.pdf (consulted
22/12/2020).
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
Technology for subtitling: a 360-degree turn
37
Agulló, Belén and Pilar Orero (2017), “3D Movie Subtitling: Searching
for the best viewing experience”, CoMe-Studi di Comunicazione e
Mediazione linguistica e culturale, 2, pp. 91-101.
Agulló, Belén, Anna Matamala, and Pilar Orero (2018), “From
disabilities to capabilities: testing subtitles in immersive
environments with end users”, HIKMA, 17, pp. 195-220. DOI:
https://doi.org/10.21071/hikma.v17i0.11167.
Álvarez, Aitor, Arantza del Pozo and, Andoni Arruti (2010), “APyCA:
Towards the automatic subtitling of television content in Spanish”,
en Maria Ganzha, Marcin Paprzycki (eds.) Proceedings of the
International Multiconference on Computer Science and Information
Technology, Danvers, MA, IEEE, pp. 567-574, on https://
ieeexplore.ieee.org/document/5680055 (source consulted 22/12/
2020) . DOI: https://doi.org/10.1109/IMCSIT.2010.5680055.
Asociación Española de Normalización y Certificación (AENOR)
(2003), Norma UNE 153010: Subtitulado para personas sordas y
personas con discapacidad auditiva. Subtitulado a través del
teletexto. Madrid: Asociación Española de Normalización y
Certificación.
Brooke, John (1996), “SUS-A quick and dirty usability scale”, in Patrick
W. Jordan et al. (eds.), Usability evaluation in industry, London,
Taylor and Francis, pp. 189-194.
Brooke, John (2013), “SUS: A Retrospective”, Journal of Usability
Studies, 8(2), pp. 29-40.
Brown, Andy, Jayson Turner, Jake Patterson, Anastasia Schmitz, Mike
Armstrong, and Maxine Glancy (2018), “Exploring Subtitle
Behaviour for 360° Video.” White Paper WHP 330, BBC Research
& Development, on https://www.bbc.co.uk/rd/publications/white
paper330 (consulted 26/3/2019).
Bywood, Lindsay, Martin Volk, Mark Fishel, and Panayota
Georgakopoulou (2013), “Parallel subtitle corpora and their
applications in machine translation and translatology”, Perspectives,
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
38
Belén Agullo García
21(4), pp. 595-610. DOI: https://doi.org/10.1080/0907676X.201
3.831920.
Bywood, Lindsey, Panayota Georgakopoulou and Thierry Etchegoyhen
(2017), “Embracing the threat: machine translation as a solution for
subtitling”, Perspectives, 25(3), pp. 492-508. DOI: https://doi.org/
10.1080/0907676X.2017.1291695.
Díaz-Cintas, Jorge (2013), “The technology turn in subtitling”, in Marcel
Thelen and Barbara LewandowskaTomaszczyk (eds.), Translation
and Meaning: Part 9, Maastricht, Zuyd University of Applied
Sciences, pp. 119-132.
Díaz-Cintas, Jorge (2014), “Technological Strides in Subtitling”, in Chan
Sin-wai (ed.), The Routledge Encyclopedia of Translation
Technology Routledge, Abingdon, UK, Routledge, pp. 632-643.
Dranch, Konstantin (2018), “Translation tech goes media: Subtitling and
dubbing won’t stay uncontested for long”, Nimdzi Blog, on
https://www.nimdzi.com/translation-tech-goes-media/ (consulted 18/
1/2019).
Estopace, Eden (2017), “Audiovisual Translation Hits a Sweet Spot as
Subscription Video On-Demand Skyrockets”, Slator, on
https://slator.com/features/audiovisual-translation-hits-sweet-spotsubscription-video-on-demand-skyrockets/ (consulted 15/1/2019).
European Broadcasting Union (EBU) (2017), “Virtual Reality: How are
public broadcasters using it?”, on https://www.ebu.ch/publications/
virtual-reality-how-are-public-broadcasters-using-it (consulted 11/1/
2019).
Forcada, Mikel L. (2017), “Making sense of neural machine translation”,
Translation Spaces, 6(2), pp. 291–309. DOI: https://doi.org/10.1075
/ts.6.2.06for.
Garcia, Jose Enrique, Alfonso Ortega, Eduardo Lleida, Tomas Lozano,
Emiliano Bernues, and Daniel Sanchez (2009), “Audio and text
synchronization for TV news subtitling based on Automatic Speech
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
Technology for subtitling: a 360-degree turn
39
Recognition”, 2009 IEEE International Symposium on Broadband
Multimedia Systems and Broadcasting, Piscataway, NJ, IEEE, pp. 16. DOI: https://doi.org/10.1109/ISBMSB.2009.5133758.
Georgakopoulou, Panayota (2012), “Challenges for the audiovisual
industry in the digital age: the everchanging needs of subtitle
production”, The Journal of Specialised Translation, 17, pp. 78-103.
Goodman, Elizabeth, Mike Kuniavsky and Andrea Moed (2012),
Observing the User Experience: A Practitioner's Guide to User
Research, San Francisco, CA, Morgan Kaufmann.
Harris, Aisha, Margaret Lyons, and Maureen Ryan (2019),
“‘Bandersnatch’ Has Many Paths, but Do Any of Them Add Up to
Anything?” The New York Times, Jan. 4, on https://www.
nytimes.com/2019/01/04/arts/television/bandersnatch-black-mirrornetflix.html (consulted 11/1/2019).
Lewis, James R. (2018), “The System Usability Scale: Past, Present, and
Future”, International Journal of Human-Computer Interaction,
34(7), pp. 577-590. DOI: https://doi.org/10.1080/10447318.2018.14
55307.
Matamala, Anna, Andreu Oliver Moreno, Aitor Álvarez Muniain and
Andoni Azpeitia Zaldua (2015), “The reception of intralingual and
interlingual subtitling: an exploratory study within the HBB4ALL
project”, Proceedings of the 37th Conference Translating and the
Computer, pp. 12-17, in https://ddd.uab.cat/record/144868
(consulted 11/1/2019).
Mathur, Abhinav, Tanya Saxena, and Rajalakshmi Krishnamurthi (2015),
“Generating Subtitles Automatically Using Audio Extraction and
Speech Recognition”, 2015 IEEE International Conference on
Computational Intelligence & Communication Technology,
Ghaziabad, India, IEEE, pp. 621-626, on http://www.proce
edings.com/25890.html (consulted 22/12/2020). DOI: https://doi.org
/10.1109/CICT.2015.46.
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X
40
Belén Agullo García
Morales, Manuel, Tommaso Koch, and Luis Pablo Beauregard (2019),
“Netflix removes Castilian Spanish subtitles from Alfonso Cuarón’s
‘Roma’”, El País, October 1st, https://elpais.com/elpais/2019/01/
10/inenglish/1547125380_758985.html (consulted 11/1/2019).
O’Hagan, Minako and Carme Mangiron (2013), Game Localization:
Translating for the Global Digital Entertainment Industry.
Philadelphia, John Benjamins. DOI: https://doi.org/10.1075/btl.106.
Pannafino, James and Patrick McNeil (2017), UX Methods: A Quick
Guide to User Experience Research Methods, CDUXP LLC.
Perkins Coie LLP (2018), “2018 Augmented and Virtual Reality survey
report: Insights into the future of AR/VR”, on https://www.p
erkinscoie.com/images/content/1/8/v2/187785/2018-VR-AR-SurveyDigital.pdf (consulted on 11/1/2019).
Roach, Joan (2018). “AI technology helps students who are deaf learn”,
The AI Blog, on https://blogs.microsoft.com/ai/ai-powered-caption
ing/ (consulted on 29/1/2019).
Rothe, Sylvia, Kim Tran and Heinrich Hussmann (2018), “Dynamic
Subtitles in Cinematic Virtual Reality”, Interactive Experiences for
Television and Online Video (ACM TVX) 2018, ACM, on
https://tvx.acm.org/2018/ (consulted on 22/12/2020). DOI: https://
doi.org/10.1145/3210825.3213556.
Sauro, Jeff and James R. Lewis (2016), Quantifying the User Experience:
Practical Statistics for User Research. Amsterdam, Morgan
Kaufmann. DOI: https://doi.org/10.1016/B978-0-12-802308-2.0000
2-3.
Tullis, Thomas and William Albert (2013), Measuring the User
Experience: Collecting, Analyzing, and Presenting Usability
Metrics, Burlingtom MA, Morgan Kaufmann.
HERMĒNEUS, 22 (2020): págs. 11-40
ISSN: 2530-609X