Blog post written by Griet Depoorter, Darja Fišer, Jakob Lenardič, Ineke Schuurman and Leen Sevens
Text2Picto and Picto2Text are two complementary translation tools aimed at enhancing communication for people with reading disabilities. Both tools have been developed by the Centre for Computational Linguistics at the University of Leuven. Text2Picto translates sentences into pictographs – that is, graphic symbols that serve as stand-ins for verbal communication – while Picto2Text does the reverse by allowing users to select the pictographs that they want to translate into written text. Two different sets of pictographs are available for both tools – the Beta set and the Sclera set. The symbols in each set are designed to be very concrete and easy to interpret, so their use reduces the cognitive complexity of reading e-mails, web pages, chats and work documents. Currently, the tools are available for Dutch, Spanish and English, but other languages can be used as well if there is a Wordnet available for them. Picture 1 shows the use of Text2Pico on the basis of the Beta set and Picture 2 on the basis of the Sclera set.
Currently, work is being done by Leen Sevens from the University of Leuven to add aditional features to the tools. Demos of these features are already available for Dutch:
- Spelling Correction, which is a crucially important feature since the users of Text2Picto often spell phonetically;
- Word-Sense Disambiguation, which identifies the correct sense of polysemous words and retrieves the correct pictograph for that sense; and
- Text2Picto + Simplification & Temporal Detection for Dutch (2017), which adds pictographs depicting temporal relations between the other symbols and simplifies syntactic structure (Picture 3).
The tools have generated a lot of interest and are already being used by their target audience. They were awarded the prestigious Language Industry Award in 2016 and have been implemented into the WAI-NOT website, which allows people with mental disabilities to use the Internet within an accessible online environment. By using the pictographs, they can play games and chat even if they aren’t able to read. There is a YouTube video available in Dutch that demonstrates the implementation in the WAI-NOT website. Additionally, the software is also part of the ABLE social services app, which is available through Google Play.
|
|
Picture 1: Translation of the sentence My friends and I go to school every day into the Beta pictograph set.
|
|
|
Picture 2: Translation of the same sentence into the Sclera set
|
|
|
Picture 3: Translation of the Dutch sentence Ik zal de rode wijn drinken die mijn moeder gekocht heeft (“I will drink the wine that my mother has bought”) into the Sclera set. Note the second pictograph in the upper row indicating future time reference corresponding to zal (“will”) and the middle pictograph in the lower row indicating the past time reference of buying the wine.
|
Click here to read more about Tour de CLARIN