ZČU
On-line titulkování přenosů jednání Poslanecké sněmovny ČR vysílaných Českou televizí
Abstract
This result was obtained after analysis aimed on the possibility of real-time subtitling of some of the broadcasted programmes of the Czech Television. On the basis of requirements needed to be fulfilled by the subtitling system (85% - 90% accuracy, on-line transcription with minimal audio/subtitle delay) a several suitable programmes were identified. After consultations with the Management Board of Czech Television, the decision to develop an experimental system for subtitling of the Meetings of the House of Parliament of the Czech Republic has been made. After several months of extensive research the system fulfilling the formulated demands was developed. The Management Board of Czech Television suggested to begin with pilot plant of the online subtitling service. The experimental service of closed captioning of the Meetings of the House of Parliament of the Czech Republic started nighttimes on 25.11 - 26.11.2008 and three experimental sessions were successfully made in 2008 in total (25.11 - 26.11.2008, 2.12 - 3.12 and 17.12 - 18.12). In 2009, we continue with this service.
Technical parameters
A brief description of the system of automatic subtitle creation based on processing of the accompanying acoustic track follows.
The complete system consists of several modules. The most important modules are: the automatic speech recognition module, the module dealing with recognized text processing and subtitle preparation, the communication module (receiving data and dispatching the resulting subtitles) and supporting auxiliary modules for actualization of vocabulary and the language model from news sites on the Internet.
- The automatic speech recognition module uses an acoustic model based on PLP parametrization and was trained on approx. 200 hours of carefully annotated recordings of the parliamentary meetings; the acoustic space is modeled by 96k gaussians; the language model was trained on all of the transcriptions of previous parliamentary meetings available on web site of the House of Parliament together with other relevant text data obtained by traversing the internet web sites. The primary vocabulary contains approx. 220k words.
- The module for subtitle correction and adjustment was specifically designed to reach a high degree of understandability while minimizing the audio/subtitle delay. The preparation of multi-line subtitles must be performed with a number of limitations on mind. These limitations include limitations induced by Czech grammar, general rules for closed captions preparation and specific needs of the target watchers. Reflecting these limitations, the module converts spoken numbers into numerals, generally known abbreviations are used, the resulting text is enhanced by punctuation, etc.
- For communication between CT and UWB an ISDN BRI line is used. Both ends are equipped with a specialized communication device - codec - provided by CT. Because the bandwidth of the ISDN BRI line is limited to 128kb/s, we use audio signal compression. After extensive testing, the algorithm most suitable for the target application was identified (MPEG-1 Layer 3, 32 kHz, 128 kb/s). Overall, this interconnection design guarantees audio track quality sufficient for speech recognition, low network latency and a high degree of security.
Innovational Aspects
Usage of automatic speech recognition techniques for closed captioning of live television programmes is a very young application. Besides BBC, that started automatic subtitling programmes several years ago, the automatic subtitling in other European countries is mostly in stage of preparation or planning. The task of automatic live subtitling is mostly approached using a "shadow" speaker, who listens the original audio track and re-narrates the track. The recognition system's acoustic as well as language model is highly adapted to the shadow speaker. This is the original target of this project, too. One of the innovational aspects of this project consists in successful construction of the automatic subtitling system able to use the sole original audio track, i.e. without the intermediary shadow speaker.
The fundamental difference between these two approaches lies in increased complexity of the task of recognition of general audio track. The system must be able to recognize speech of any speaker (200 members of parliament, ministers and guests), must cope with lower SNR, background sounds a variable acoustic background, must be able to reflect the non-continuous speech of speaker and usage of neologisms and non-standard words. To reach the needed recognition accuracy (and consequently, accuracy of the subtitles) the system makes use of several auxiliary procedures and supporting tools to improve the acoustic model as well as the language model including supplementation of the vocabulary with words on actual topic and automatic postprocessing of the resulting subtitles.
We are not aware of any other broadcasting corporation approached the subtitling problem from this view. Moreover, the system construction supports distributed layout. The complete recognition system is located at UWB site in Pilsen and the audio track and the resulting subtitles are sent from Prague through ISDN network.
Examples and Results
Opinion Poll
The team with cooperation with the web site www.kochlear.cz arranged an anonymous opinion poll in the group of targeted watcher. Results of the opinion poll (in Czech only, sorry)
Photographs and Examples of Subtitling Session
Flash Player
Video: Subtitled recording of Meeting of House of Parliament broadcasted on CT24 channel. Captured using DVB-T USB dongle.
Contributions and Benefits
Live subtitling of the Meetings of the House of Parliament of the Czech Republic is a additional advanced service targeted on hearing-impaired watchers. Partial benefit is the fact, that the "shadow speaker" person is not needed for this service; the subtitles are prepared from the real acoustic track.
Licence
This experimental pilot is operated and evaluated within the scope of the project "Elimination of the language barriers faced by the handicapped watchers of the Czech Television", reg. number 2C06020.
Confirmation of usage
The aforementioned technology operated as pilot plant is used by the Czech Television for on-line subtitling (closed captioning) of the Meetings of the House of Parliament of Czech Republic programm. The programme is broadcasted on the CT24 channel, in days of parliamentary meeting, nighttimes. The closed captions are available as teletext subtitles on teletext page 888.
For further information about this pilot plant, please contact prof. Josef Psutka, Department of Cybernetics, Faculty of Applied Sciences, West Bohemia in Pilsen, tel. +420 377 632 500, email: psutka _at_ kky.zcu.cz
Potvrzení o užívání ke stažení zde.
