Reducing language barriers for tourists using handwriting recognition enabled mobile application

Abstract

Modern mobile devices and networks permit to develop applications that reduce the language barrier for tourists and visitors in foreign countries. This paper describes a mobile application built on a distributed architecture that allows tourists to obtain additional information about location names and menu entries in Arabic language. By simply taking a photo, translation and additional information are displayed to users in their preferred language. While several mobile applications exist to translate the texts into photos, the present application offers the advantage of translating both handwritten and printed texts. The work was focused on two important aspects: the recognition system and the mobile application. The major challenge was to build a recognition system able to recognize handwritten or printed texts of various writing styles or fonts. Therefore, we propose an original approach consisting of recognizing handwritten and printed words using the Balamand-ENST handwriting recognition system, trained on handwritten texts. Experiments conducted on a collected database have shown that the recognition system trained on handwritten text provides excellent performance when used in recognizing printed text. This recognition system and its performance in both modalities are briefly described in this paper. In addition, a full description on how the mobile application was designed and its functionality is presented.