Scientific and technical results

Speech prosody.

Emotion recognition.

Multilingual speech-controlled telematics framework for personalised automotive services.

NKFP072-TELEAUTO (2008-2010)

Partners:
Ygomi Europe Kft., as the leader of the consortium is responsible for the elaboration of the whole project. The company takes an active part in the work of international standardisation associations. Ygomi participates in the work of Ertico, Europe's ITS (intelligent traffic systems) organization, which finances researches and defines industrial standards in the field of intelligent transportation.
ROC Development Hungary Kft. has had many years of experience in developing call centres.
Connexis Kft. has great experience in the field of automotive software development, telecommunication, especially the development of automotive communication protocols.
BME Laboratory of Speech Technology

Child-Computer Interaction Techniques: Exploring the Prosody.

TéT - Balaton project (2008-2009)

One of the primary aims of the project is to improve computer assisted speech learning methods for handicapped children (hard of hearing and autistic children) by the introduction of prosody models. These models will be firstly used for the development of the prosody of the speakers. Prosody plays an important role in communication. Prosodic cues help the human to understand the messages through a conversation, thus the correct pronunciation of the different prosodic cues are very important. Moreover, we will use prosody for the determination of the emotional state of the speaker which can be take into account in the interface.

Partners:
The Institute of Intelligent Systems and Robotics ISIR CNRS FRE2507 (France) has wide experience in different classification technics in speech technology. Moreover, the institute is leading several projects involving normal and autistic children.

Cross-Modal Analysis of Verbal and Non-verbal Communication

COST Action 2102 (2006-2011)

The main objective of the Action is to develop an advanced acoustical, perceptual and psychological analysis of verbal and non-verbal communication signals originating in spontaneous face-to-face interaction, in order to identify algorithms and automatic procedures capable of identifying the human emotional states. Several key aspects will be considered, such as the integration of the developed algorithms and procedures for application in telecommunication, and for the recognition of emotional states, gestures, speech and facial expressions, in anticipation of the implementation of intelligent avatars and interactive dialogue systems that could be exploited to improve user access to future telecommunication services.
Link to COST homepage

Our Contribution

Partners: countries from Europe

Continuous speech recognition:

-  A development tool (MKBF 1.0) for constructing continuous speech recognizers has been created under Windows XP. The system is based on a statistical approach (HMM phoneme models, and bi-gram language models with non linear smoothing) and works in real time. The tool is able to construct a middle sized speech recognizer with a vocabulary of 1000-20000 words. New solutions have been developed for the acoustical pre-processing, for the statistical model building of phonemes, and in syntactic level [ ](IKTA, OTKA). (2004- )
For details in Hungarian, click here! or click on the picture below

A prosodic recogniser has been developed.A cross-lingual study for agglutinative, fixed stressed languages, like Hungarian and Finnish was prepared by this recognizer, about the segmentation of continuous speech on word level by examination of supra-segmental parameters.

We have developed different algorithms based either on a rule based or a data-driven approach. The best results were obtained by data-driven algorithms (HMM-based methods) using the time series of fundamental frequency and energy together. Word boundaries were marked with acceptable accuracy, even if we were unable to find all of them. On the base of this study a word level segmentationer has been developed which can indicate the word boundaries with acceptable precision for both languages

IKTA-00056/2003 project

For details in Hungarian, click here!

Statistical examination of languages:

- Search of optimal unit of CSR system.
- Construction of optimal sized teaching and testing material on the base of ture of the language concerned [19] (1999- )

-Examination of the Pronunciation Variation from hand-labelled corpora

Data driven method has been used for the examination of the pronunciation variation rules.

Systematic statistical analysis was prepared and the obtained results demonstrate how the pronunciation variation depends on the position and the connection of the sound. Pronunciation matrixes were constructed, presenting the probability of the occurences. Both word-internal and cross-word rules were examined separately and compared.

The examination was prepared based on the Hungarian speech databases, but the method is adaptable for other languages.[B3], [B4] (2003-,[](ITKA, OMFB)

-Rule based and statistical language model for Hungarian speech recognition (ITKA, OMFB)[],[](2004-

Database collection:

- Hungarian Reference Speech Database - Read speech database collection in office like environments for training and testing continuous speech recognition programs running in PC-s [ ](IKTA). (2004-2005)

- SpeechDat(E) - European project (1999- ) Speech database collection through telephone lines. A realistic base noth for the training and testing of the present-day teleservices and the training of real speaker independent recognizers. [D6][E6[E7]]
- BABEL - A multilingual speech database collection (1995-99) INCO-COPERNICUS project. Clear, read speech for general speech processing purposes. [B6],[15],[E1][E8],[E9]
- CHILDREN SPEECH data-base collection [D9], (1999- ). Part of the SPECO Copernicus program.

-MTBA, BESZTEL-Speech database collection through telephone lines and mobil telephones.[C3],[C16] (2000-2003)

Speech perception:

- The most relevant acoustical parameters for the perception of different speech-sounds were determined;
- the role of the forward and backward masking in speech perception were examined.
[1], [2], [3], [4], [5] (1977-1983).

Speech analysis:

- A simplified auditory model (Bark scale, masking, loudness involved) were developed, and speech sounds were analyzed by this model [7], [8]. (1983-1985)
- The bark scale representation, FFT (Hamming window, 512 point), and LPC (autocorrelation, order 24) were compared. (1986-1988)

Speech processing for speech handicapped:

- The "Speech Corrector" an audio-visual speech-pronunciation teaching system has been developed for speech handicapped people. The database of Corrector System for Hungarian and German language is finished. [16], [17], [20]. The system uses up the knowledge of speech recognition results. Demo of the program can be reached.
- We lead an international European Copernicus program, titled SPECO - A Multilingual pronunciation teaching and training method and a software system for hearing and speech handicapped children. [C17],[D4],[D5],[D7] [D8],[Đ10](1986-)

- Interactive hearing and speech perception theraphy through the internet (item 39)

The proposed hearing and speech perception training program will help the education of hearing impaired children by didactically well-constructed and playful exercises using specially constructed databases. Poor hearing abilities are stimulated by different environmental sounds and special speech examples. In this way we develop the ability of the perception and distinction of different sounds and moreover we train the speech understanding and combination skills of children. The program will help the rehabilitation of children with cochlear implant, too. The program will be available through the Internet for everybody free of charge.[[D1][E2](2003)

Speech enhancement:

Isolated word recognition:

Dialogues systems

Annotation and segmentation:

- Automatic speech segmentation on phonetic, sub-phonetic level of continuous speech, automatic labelling (1997- )
- A language independent automatic segmentation and labelling technique is done for training speech recognizers, respectively to collect data-base  [D11,22]. (1990- )