Workplan

Objective and strategy overview

The project structure has been designed to ensure that the consortium will maximise the likelihood of project objectives achievements.

• Objective 1 : Improve significantly robustness of speech recognition against noise

• Objective 2 : Improve the robustness of speech recognition to users' voice specificities and interaction abilities

• Objective 3 : Evaluation of the potential impact on applications

The main principles that we have adopted to maximise the probability of the achievement of these objectives are:

Generic approach of the robustness problematic. Our purpose is not to tailor a speech recognition to a given application by corpus collection or fine-tuning of parameters but to investigate new tracks in speech processing so as to set the basis for a more reliable and robust voice recognition. It is expected that this is the only alternative capable of providing suitable level of performances and dependability for both project applications. Moreover, this approach will ensure that the work conducted within HIWIRE will be of general interest.

Consortium with a scientific colour. In our consortium, several partners are research labs involved in multi-modality and speech sciences. This is deliberate, and one of the points that could lead to significant benefits in the achievement of our ambitious objectives. It will give the opportunity to deal with complex problems, like speech non linear representation, complex acoustic modelling, under all its angles, implement competitive approaches, and identify all potential synergies between conceptual approaches.

Project with clear outputs. According to project objectives, we are committed to evaluate the gain of the conceptual investigations performed in the project on two types of application that are representative of very different contexts.

• Aeronautic. We will evaluate, on realistic dialog scenarios, how the improved spoken dialog system is able to handle non-native speakers and reach the required level of reliability. This evaluation will be performed thanks to the coupling of the spoken dialog module to an advanced desktop simulator.

• Handheld devices. Among all potential uses of this device, we will focus on a professional equipment devoted to maintenance in the aeronautic application.

Evaluation. The last target of the project is to investigate how cutting edge investigations into embedded complex dialog interfaces could improve the efficiency and the user-friendliness of interactive systems. This will enlarge the scope of the project and provide human feed back on the acceptability of ambitious dialog with large vocabulary speech input.

General structure

In the project structure two type of activities can be distinguished :

• Research oriented activities that will mobilize most of the project resources and that will be oriented toward the achievement of the objectives 1 and 2.

• Application related activities that aim at the achievement of the third project objective and that will allow to issue a first feed back on the potential of the enhanced spoken dialog. Within this activity the evaluation will not be only directed toward performance analysis of the speech recognition but also on the analysis of more subjective features like workload reduction and interaction naturalness.

The main milestones for the WP1 and WP2 are related to three steps of evaluation performed on public databases (AURORA) and on a project database collected for the purpose of evaluation on non-native speech.

• M1 m10 Completion of baseline experiments on environment and sensor robustness. These experiments will be the first running of the project evaluation platform on public databases and project database. It will be the starting event for WP3 activities (integration of robust spoken dialog on platforms).

• M2 m21 Completion of phase1 experiments on environment and sensor robustness. This second step of evaluation will allow to identify best suited processing units that shall be integrated in both platforms.

• M4 m33 Completion of phase2 experiments on environment and sensor robustness. This is the last step of these evaluations on databases. In consequence, at this stage the best results are gained and a definitive view of the project achievement in regards with the technical objectives can be drawn.

Concerning the Application side of the project there are two main milestones:

• M3 m24 Completion of the integration of a first version of the spoken dialog to both fixed and mobile platforms. At this stage a functional validation of the interface can begin. The configuration of the speech recognition system will be derived from the results of the first milestone. The results of the phase1 experiments will be integrated in the platforms after.

• M5 m33 Completion of evaluation on mobile and fixed platform, the evaluation on platforms are completed and data are ready to be analysed in order to produce qualitative and quantitative outputs on spoken dialog performances for both considered application.

Overview of the workplan

Research Activities

An analysis of the project structure reveals the main mechanisms that have been implemented in order to create a project entity. The main aspect is the use of a common evaluation framework for all research activities of HIWIRE. This will ensure:

• A consistent evaluation of all concepts developed within this project.

• An evaluation of synergies between approaches, and thus the capability to identify best suited system for both application.

• The capability to have an efficient monitoring of the research activities at the project level by providing a yearly status on results.

In terms of work plan this approach lead to the following planing for research activities:

YEAR 1 : Development of the evaluation environment:

• Development of the generic HTK evaluation platform.

• Definition and diffusion of software interfaces of the HTK platform to consortium partners.

• Database inventory and collection, training of acoustic models.

Research activities:

• The innovative concepts of speech processing and modelling are investigated (see WP1 and 2 descriptions).

• Realisation of the baseline evaluation on the HTK platform.

• Exploitation of the results in order to orient research plan and identify synergy for the second year.

YEAR 2 :

• Research activities according to baseline evaluation outputs.

• Realisation of phase1 experiments.

• Exploitation of the results in order to orient research plan and identify synergy for third year.

• Dissemination of results and participation to public evaluation.

YEAR 3 :

• Research activities according to phase1 evaluation outputs.

• Realisation of phase2 experiments.

• Exploitation of the results.

• Dissemination of results and participation to public evaluation.

Application related activities

This side of the project covers two major activities : Integration of robust vocal dialog on two applicative platforms (fixed platform: cockpit simulator; mobile platform: handheld device for aeronautic maintenance). The target is to take into account limitations proper to each environment and define the best suited recognition functional architecture. Of course this will rely on the evaluation conducted on the HTK platforms. In terms of planing the objective is to have a first integration realised on both platforms after two years of project.

Evaluation in real context. The objective here is to complement evaluation of algorithm by evaluation with human in the loop on both platforms. We will assess if the performance of the recognition are degraded in a real interactive context, and have a first feed back on the expected operational improvements due to vocal interaction (reduction of cognitive load, speed of the interaction, naturalness)

HIWIRE's Project Workpackage list

Workpackage No.	Workpackage title	Lead Contractor
0	Technical managment, review and assessement of progress toward objectives	THAV
1	Environment and sensor robustness	TSI-TUC
2	User Robustness	LORIA
3	Integration	LOQ
4	Evaluation	TRT
5	Exploitation and dissemination	THAV
6	Administrative and financial managment	THAV

Work package descriptions

If we consider RTD activities, the HIWIRE project contains four main technical work packages. Two are focused on advanced research issues:

• WP1 : Environment and sensor robustness

• WP2 : Users robustness

while two others are related to evaluation of robust speech recognition in the frame of two applications involved in the project :

• WP3 : System integration

• WP4 : Evaluation