hex23.gif
hex38.gif
hex32.gif
hex26.gif
hex35.gif
hex29.gif

Budapest University of Technology and Economics Department of Telecommunications and Media Informatics Home send e-mail
Back to the first page
 


 

 
 

A Multimedia Multilingual Teaching and Training System for Speech Handicapped Children
(INCO-Copernicus 977126-SPECO)

 
Project coordinator: Klara Vicsi
Budapest University of Technology and Economics
 

Summary

The aim of the project is to develop a new audio-visual pronunciation teaching and training method and a software system for hearing and speech-handicapped persons to help them to control their speech production. A teaching method is drawn up for progression from the individual sound preparation to practice of the sounds in sentences. The software system is planned to be built from different modules. The basic part is a general language-independent measuring system and database editor. This database editor makes it possible to construct modules for all participant languages and for different speech disabilities. Two modules are planned for its construction in all languages, one of them being for teaching and training vowels for hearing-impaired children, the other one is for correction of misarticulated fricative sounds. These are the most common forms of articulation disorders.

The system helps patients to discover how to control their speech organs by simultaneously comparing the visual patterns (speech pictures) of the normal acoustic speech signal with the defective one. In the system the patient can see and hear the speech pattern at the same time.

We aim to follow the traditional steps of speech therapy. These are sound preparation, sound development, practice in words and automation. A picture symbol belongs to all phonemes for those children who do not know letters, together with the sounds, syllables and words in written form, and the speech-sound patterns. In this way the system helps children to learn letters too.

For the project, there is a need for the cooperation of scientists who represent different fields, such as digital speech processing, speech acoustics, expertise in linguistics on different levels, in speech therapy, and knowledge of the newest technological facilities. With the cooperation of different experts we will develop the system for all participant languages.

We plan to create a system with a general language-independent database editor and measuring system, which makes it possible to adapt the method to any European language. The construction of such a method would help experts dealing with speech therapy and make speech education and training more efficient.

Description of the Project

THE OBJECTIVES AND RESULTS OF THE PROJECT:

The aim of the project is to develop a new audio-visual pronunciation teaching and training method and a software system for hearing and speech-handicapped persons to help them to control their speech production. A teaching method is drawn up for progression from the individual sound preparation to practice of the sounds in sentences. The system helps patients to discover how to control their speech organs by simultaneously comparing the visual patterns (speech pictures) of the normal acoustic speech signal with the defective one. In the system the patient can see and hear the speech pattern at the same time. We aim to follow the traditional steps of speech therapy. These are sound preparation, sound development, practice in words and automation. A picture symbol corresponds to all phonemes for those children who do not know letters, together with the sounds, syllables and words in written form, and the speech-sound patterns. In this way the system helps children to learn letters too. The software system is planned to be built from different modules. The basic part is a general language-independent measuring system and database editor. This database editor makes it possible to construct modules for all participant languages and for different speech disabilities. Two modules are planned for their construction in all languages, one of them being for teaching and training vowels for hearing-impaired children, the other one is for correction of misarticulated fricative and affricate sounds. These are the most common forms of articulation disorders.

Specific, measurable objectives and yardsticks:

  1. For appropriate speech processing, we want to use the newest techniques in this field, making use of current technical facilities. For spectral analysis we use a special filter bank to imitate the filtering characteristics of the inner ear. So our speech picture can be much more similar to the perceived one than a simple row of traditional filters, or FFT spectra. For pitch contours we plan to use an AMDF algorithm.

  2. For time alignment and distance a computation different non-linear time warping techniques and Euclidean distance computations will be used.
    The processing method is language-independent, and will be built into the database editor.
    The system evaluates the parameters which are important in speech quality and visualises them in an illustrative and entertaining way. Our complex system enables improved efficiency of speech therapy in the planned languages.
  3. A general language-independent database editor and measuring system will be constructed.

  4. This makes it possible to construct modules for all participating languages, and for different speech disabilities.
  5. Before collecting databases it is necessary to plan the text of the speech and picture database for all participating languages.
    • for vowels
    • for fricatives and affricates
  6. Pick up speech-database for statistical examination and for the speech perception test for all participating languages.
    • for vowels
    • for fricatives and affricates
  7. Compile speech database for the speech therapy.
    • for vowels
    • for fricatives and affricates
  8. In this project the main aim is to develop an audio-visual articulation training and teaching system for all participating languages, these being English, Swedish, Slovenian and Hungarian. That system are composed by a general language independent frame program and the language dependent database files. Two databases are planed for all participating languages.
    • one is for teaching and training vowels for hearing-impaired children,-named: VOWEL CORRECTOR
    • the other one is for correction of misarticulated fricative and affricate sounds, named: FRICATIVE CORRECTOR
The software system will run on a Pentium PC, under "Windows 95" using a special microphone and two commercial loudspeakers.

Demos and Documentation : http://www.rcs.hu/sc.htm


List of Partners

In the project the following short name of the partner institutes will be used:

BUTE - Budapest University of Technical and Economics

RDG - University of Reading

KTH - Kungl. Tekniska Högskolan

UMB - University of Maribor

RCS - Robot Control Software BT.

The key persons within the organisations are the followings:

Peter Roach - RDG: p.j.roach@reading.ac.uk

Anne-Marie Öster - KTH: annemarie@speech.kth.se

Zdravko Kacic - UMB: kacic@uni-mb.si

Peter Barczikay - RCS: pbarczi@rcs.hu

Klara Vicsi - TUB: vicsi@tmit.bme.hu

Documents

Description of the language independent editor and measuring system

Text of the reference database :
                                              English
                                              Hungarian
                                              Slovenian
                                              Swedish

Papers


Budapest, 02. March 2001.