Programme CLARIN Annual Conference 2022

Abstract: We present a hybrid lemmatizer and POS-tagger for Akkadian, the language of the ancient Assyrians and Babylonians, documented from 2350 BCE to 100 CE. In our approach the text is first POS-tagged and lemmatized with TurkuNLP trained with human-verified labels, and then post-corrected with dictionary-based methods to improve the lemmatization quality. The postcorrection also assigns labels with confidence scores to flag the most suspicious lemmatizations for manual validation. We demonstrate that the presented tool achieves a Lemma+POS labelling accuracy of 94%, and a lemmatization accuracy of 95% in a held-out test set.

Abstract: WebLicht is a workflow engine that gives researchers access to a well-inhabited space of natural language processing tools that can be combined into tool chains to perform complex natural language analyses. In this paper, we present WebLicht-Batch, a web-based interface to WebLicht’s chainer back-end. WebLicht-Batch helps users to automatically feed large input data, or input data of multiple files into WebLicht. It disassembles large input into smaller, more digestible sizes, feeds the resulting parts into WebLicht’s pipelining and execution engine, and then assembles the results of such processing into files that preserve the usual input-output dichotomy.

Abstract: The paper reports on the system for creating dictionaries within the CLaDA-BG infrastructure. At the heart of the system lies the BTB-Wordnet around which all other language resources are organised. These are various dictionaries, ontologies, corpora. The specific features and functionalities are outlined. Also, the rationale behind the construction of such a system is given.

Abstract: This paper presents our work in progress on building a flexible workflow engine. The architecture of the engine is based on the message queue programming model and is implemented in Python. NLP tools are exposed as remotely executable tasks and wrapped in standard abstractions. The main contribution is to provide a uniform API for defining various kinds of tasks and workflows. The use case of the library is building a text analytics platform for digital humanities, providing a usage-friendly interface to predefined workflows.

Abstract: This paper presents a design of a web-based application for the analysis and visualisation of relations between terms, named entities, and topics. The goal of this project is to create, in close cooperation between the literary research community and the IT professionals, a comprehensive workflow tailored to the specificity of literary studies at large, and, the current debates and trends in humanities research. The application brings together the already existing tools offered by CLARIN-PL and the resources and tools developed at Dariah.lab. It consists of three components: the named entity relationship analysis component, the terminology extraction component and the topic modelling component. The whole system is not only based on an automatic operation of the components but is constructed to allow user intervention in individual sub-processes of the entire processing pipeline. A strong emphasis is put on the use of metadata of the analysed texts (e.g., for filtering and grouping documents) and the visualisation of results.

Abstract: This contribution presents a system of independent software components meant to support the creation of ecosystems of interrelated language data (i.e. lexica linked to textual testimonies, concepts, metadata, bibliographic references, and other external lexical resources) according to the current state-of-the-art representational models for the semantic web. The system is implemented as a set of autonomous servers exposing Restful APIs that in principle can serve different frontend applications and use cases. In this work they serve the EpiLexO GUI application designed and geared to support scholars of ancient languages of fragmentary attestation in their studies. The development of both the back-ends and the front-end is still work-in progress, but a first version is ready for use.

Abstract: Human variation in labeling is typically considered noise. Annotation projects in computer vision and natural language processing typically aim at minimizing human label variation, in order to maximize data quality and in turn optimize and maximize machine learning metrics. However, is  human variation in labeling noise, or can we turn such information into signal for machine learning? In this talk, I will first illustrate the problem and then discuss approaches to tackle this fundamental issue.

Bio: Barbara Plank’s background is in computer science and language technologies. She is an expert in Natural Language Processing (NLP) and the chair (full professor) of AI and Computational Linguistics at LMU Munich, where she leads the MaiNLP research lab at the Center for Information and Language Processing (CIS). In her view, NLP should be able to handle any language and any domain, which is not easily achieved. Her work thus focuses on making NLP models more robust so that they can learn under circumstances of selection or annotation bias, or in cases of limited data availability, for example. Plank plays a pivotal role in several NLP research projects (DFF Sapere Aude, Amazon Research Award) and is involved in teaching and supervision activities for NLP scholars. She recently received an ERC Consolidator grant to work on natural language understanding for non-standard languages and dialects.

Abstract: The Data Governance Act was proposed in late 2020 as part of the European Strategy for Data, adopted on 30 May 2022 (as Regulation 2022/868). It will enter into application on 24 September 2023. The Data governance Act is a major development in the legal framework affecting CLARIN and the whole language community. With its new rules on the re-use of data held by the public sector bodies and on the provision of data sharing services, its new limits on international transfers of non-personal data, and especially its encouragement of data altruism, the Data Governance Act creates new opportunities and new challenges for CLARIN ERIC. This abstracts briefly analyses the provisions of the Data Governance Act, and aims at initiating the debate on how they will impact CLARIN and the whole language community.

Abstract: Language Representation Models based on the Transformer architecture have obtained state-of-the-art results on a wide variety of natural language processing (NLP) tasks. The Transformer is far more scalable than previous neural network architectures, allowing larger and more powerful models to be trained than ever before. The size of the largest Transformer-based model has grown from 110 million parameters in 2018 (GPT) to 1.6 trillion parameters in 2021 (Switch Transformer). However, this growth comes at the cost of significantly higher computational requirements. Current state-of-the-art models can require several weeks to train, using thousands of high-end graphical processing units (GPUs) or tensor processing units (TPUs). Furthermore, there has been an exponential growth in the amount of text these models are trained on. Transformer-based language models are typically first pre-trained on large amounts of text on an unsupervised task, such as predicting the next word given a preceding text sequence. Once pre-trained, the model can be fine-tuned on more practical tasks (known as downstream tasks in this context), such as question answering and automatic text summarisation. It has been shown that increasing the size of pre-training corpora can significantly improve downstream performance. As a result, the size of the largest pre-training dataset for English has grown from 800 million words in 2018 (GPT) to 1.4 trillion words in 2022 (Chinchilla). For research on low and medium-resource languages, it becomes increasingly important to investigate how the Transformer architecture can best be utilised in settings where pre-training data is scarce and access to computational resources is limited. We will evaluate the data efficiency of various pre-training methods, experiment with how small monolingual pre-training corpora can best be augmented with multilingual or machine-translated text and perform a thorough evaluation of commonly used text filtering techniques. Furthermore, we will consider how different subword tokenisation algorithms and vocabulary sizes affect downstream performance under low and medium-resource settings.

Abstract: Text is inherently infused with historic linguistic properties and descriptions of traditions, cultural practices and social structures – making it an exceptional source for linguists and historians to study and quantify past and present trends. Text analysis is a core component of both Natural Language Processing (NLP) and the (Digital) Humanities, but there exist vast differences in user culture, end goals and the level of technical knowledge between these two research fields. While NLP researchers adhere to rigid workflows to cater to linguistics-centred questions and effectuate language model improvement, DH scholars seek to answer meta-textual questions in a heuristic framework. Partly due to this divergent perspective on text as data, a cross-pollination of methodologies between these two research fields remains rather limited. Specifically Named Entity Recognition (NER) and Sentiment Analysis (SA) are heavily exploited information extraction tools in NLP applications. Despite having shown great potential in humanities research settings and cultural heritage institutions, NER and SA are often overlooked as a research tool by literary scholars and historians. A lack of familiarity with NLP tools, along with the known technical challenges that come with literary-historical text processing, are often enough to deter digital humanists from applying NER or SA in their workflows, possibly leaving a wealth of valuable insights untapped. Answering the urgent calls for a methodology-driven agenda in the Digital Humanities, this research seeks to develop transparent, reproducible and durable step-wise workflows with the integration of NER and SA for application in heuristic literary-historical research settings. For this purpose, a large multilingual corpus of travelogues in English, Dutch, German and French, ranging from the 15th to the 20th century is being collected, as the exceptional characteristics of travelogues as highly idiosyncratic lenses into the past account for a wide range of linguistic and historical variance. A set of relevant literary-historical research questions will be investigated with the support of NER and SA. Rather than focusing on resolving the research questions, our research takes a step back and aims to develop over-arching workflows regarding open-source tool selection and evaluation, tool adaptation and mitigating the challenges inherent to literary-historical and multilingual corpus processing. Finally, the workflows and code used for this purpose will be made open-source to encourage methodological transparency and reproducibility in similar future research. Our research aims to generate much-needed insights regarding the potential and limitations of NER and SA in literary-historical research and intends to foster a tool- and data-critical attitude among digital humanists through the development of clear-cut NLP-workflows, benefiting the CLARIN-infrastructure and DH-community alike.

Abstract: The Dutch language contains several strong verbs, which are divided into seven different verb classes based on the ablaut. Over time, some of these verb classes have undergone language change. This happened, for example, to the verbs from verb class III, as can be illustrated with the verb zingen, ‘to sing’. This strong verb displayed a three-part ABC-paradigm in Middle Dutch: zing(en) ‘sing’ – zang, zongen ‘sang’ – gezongen ‘sung’, with zang being used for preterite singular (‘I, you (sg.), he or she sang’) and zongen for preterite plural (‘we, you (pl.) or they sang’; De Smet, 2021: 17; Van der Sijs, 2021: 453). Nowadays, Standard Dutch has lost the preterite singular variant in these verbs and uses the same form for singular and plural preterite, thus displaying an ABB-paradigm: zing(en) ‘sing’ – zong(en) ‘sang’ – gezongen ‘sung’. In a large corpus study on strong verbs in Dutch, De Smet (2021) has shown that this change solidified in the 17th century – a century that is also known as the formative period in the standardisation of Dutch (Van der Wal, 1995; Van der Sijs, 2021). Even though De Smet acknowledges that standardisation and concomitant codification of verb paradigms in the late 16th and early 17th century may have played a role here, she focuses her analysis on possible language internal factors influencing this change (2021: 108-109). The language external factor has therefore not been studied in detail yet, leaving the following question unanswered: could standardisation have played a part in this language change? In my poster presentation, I will present the results of a study carried out to try and answer this question, following the precept versus practice method (as detailed in e.g. Anderwald, 2011; 2012, for strong verbs in English). Firstly, I performed a comprehensive study of the norms for 41 17th-century class III-verbs by mapping out all stances on these verbs encountered in the 10 normative grammars on Standard Dutch written between 1550 and 1650. Secondly, I investigated the actual use of these verbs in the new CourantenCorpus, a corpus containing Dutch newspapers from 1618 to 1700 (ca. 19 million words in total). The second study especially provided a use case for CLARIAH tools such as Blacklab (e.g. De Does et al., 2017), the GaLAHaD platform for linguistic annotation of historical Dutch (Brouwer et al., 2022), and the PIE-based framework for tagging and lemmatising historical text (Creten et al., 2020). The focus of this poster presentation will therefore be on this second study and its use of the corpus and tools, addressing questions such as: were the search possibilities sufficient to find the relevant data? What was the impact of the inaccuracies of automatic linguistic annotation? To what extent can we still draw valid conclusions from noisy material? What can be done to circumvent these issues?

Abstract: Lithuanian is a synthetic language with numerous forms. Particularly verbs have a variety of forms – they can be conjugated, inflected, and uninflected. Inflected verb forms are participles, for example, valgantis vaikas 'an eating child', nupirktas maistas 'food purchased'. Even though a participle is regarded as a verb form, it has the properties of both verbs and adjectives. It has the meaning of a verb, categories of voice and tense, and valency similar to that of verbs (vakar nupirktas maistas 'food that has been bought yesterday'); it can also be inflected like an adjective, have degrees, pronominal forms, and attributive as well as predicative functions. Some Lithuanian participles lose their verbal properties and become adjectivised, for example, įprastas ('usual'; verb įprasti 'get used to'); nevykęs ('lame'; verb nevykti 'not to happen'); užimtas žmogus ('busy person'; verb užimti 'to occupy'); valgomos uogos ('edible, not poisonous berries'; verb valgyti 'to eat'). They are used as adjectives and are often separated from verbal meaning; moreover, they lose the categories of tense and voice. Usually, these words are regarded as adjectives in dictionaries. Sometimes it is difficult to differentiate between adjectivised and simple participles. It is problematic to distinguish when participles should have separate entries in dictionaries; also, part-of-speech tagging poses some problems. The aim of the thesis is to identify criteria to differentiate between adjectives and participles that have a similar function. The research is based on Lithuanian corpora. Initially, a list of criteria to identify adjectivised participles was created. These criteria are semantic, grammatical, derivational, and statistical. Some of them are more subjective, for example, the synonyms and antonyms of participles and adjectives, or the changed lexical meaning of the participle. The other criteria are more objective, for example, the derivational property of the participle to form an adverb (papildomas 'additional', papildomai 'additionally', verb papildyti 'to replenish'), or the tendency for participles to combine with adverbs of measure/degree (labai mokytas 'highly-educated', verb mokyti 'to teach'). The list of criteria was created by analysing theoretical works and conducting a pilot study. During the pilot study, the 160 most frequent verbs and 43 participial adjectives from the Lexical Database of Lithuanian Language Usage (2021, https://kalbu.vdu.lt/mokymosi-priemones/leksikonas/) were analysed. It is planned that this database will soon be available in the CLARIN repository. This database is based on the Pedagogic Corpus of Lithuanian (669 000 words, https://clarin.vdu.lt/xmlui/handle/20.500.11821/50). The selected words were analysed in this corpus and, when needed, in the Corpus of Contemporary Lithuanian Language (https://clarin.vdu.lt/xmlui/handle/20.500.11821/16). Now it is planned to analyse more words and to classify participles by the degree of adjectivisation. Also, it is planned to use statistical information (for example, ratios of participles to other verbal forms and the frequency of participles) for automatic analysis. One of the possible results of the analysis could be word lists in which participles would be classified by the degree of adjectivisation. They would be useful for morphological and syntactic analysis, lexicographic works, etc. It is planned that they would appear in the CLARIN repository.

Abstract: The aim of the thesis is to present a model for incorporating prepositions in the structure of BulTreeBank WordNet (BTB-WN). Prepositions are viewed as a necessary part of speech in a lexical resource such as wordnet, because their integration can be beneficial for several NLP tasks (such as semantic annotation, word sense disambiguation, machine translation, parsing, knowledge extraction, word embeddings, text analysis and generation, etc.). A semantic classification of Bulgarian prepositions is done and a model for preposition synsets and relations in BTB-WN will be presented.

Abstract:In my paper, I will introduce the research project underlying my PhD thesis, the process of creating a corpus and the preliminary results of my analysis. My study is situated at the intersection of legal and medical sociology, sociolinguistics and corpus linguistics. I aim to reconstruct discursive strategies adopted by forensic psychiatrists in response to the challenges and contradictions related to their role in the criminal justice system. When assessing the defendant’s sanity, forensic psychiatrists are required to draw a line to distinguish crime and insanity. This involves bringing scientific reliability in line with the logic of the law and also to reconcile two discourses: legal and medical. The forensic psychiatrists do so through linguistic means: by formulating a narrative about the perpetrator of a crime with the use of categories taken from the social and medical sciences. The process—although appearing to be rooted in neutrality—is not independent of the broader social context including gender stereotypes, particularly stereotypes of femininity. To date, this issue has remained on the margins of interest in the social science. I intend to fill this gap and explore how the meaning of normality is developed at the interface of psychiatry and law. The study will answer the question of how medical categories are translated into legal categories and, more broadly, how an agreement between representatives of (radically) different disciplines can be reached. I will also capture discursive mechanisms of constructing gender representations in documents issued by expert psychiatrists. The study is based on the mixed-methods approach. I combine methods that are extremely rarely employed in analysis conducted at the intersection of law, psychiatry and language: computational text analysis and discourse analysis developed in the sociology of science and discursive psychology. As the core research material, I compiled a corpus consisting of 225 psychiatric reports issued by Polish forensic experts with a volume of about 1.5 million tokens. Employing tools provided by CLARIN-PL I identify the singularities of the language of forensic psychiatry and forensic psychiatric reports as a genre. I compare, on the one hand, forensic psychiatric reports with strictly medical and legal texts, and, on the other hand, reports concerning men with reports on women. I analyse lexical, grammatical and narrative features, and, further on, identify the interpretative repertoires used by forensic experts. I conduct, i.a., stylometric analysis using the Open Stylometric System WebSty and semantic analysis using plWordNet. Written psychiatric reports as texts are still under-researched. My research is the first case of applying computational text analysis methods to explore these types of documents. The methodological framework I have created and the results of my analysis can serve as new showcases for the CLARIN infrastructure aimed at researchers in the fields of law, medicine and social sciences. They can also provide a starting point for comparative research carried out within other legal systems and expanding CLARIN legal corpora family.

Abstract: A metaphor is an expression that uses a comparison with another concept for rhetorical effect. For example, instead of saying ‘his words were offensive’, we might say ‘his words cut deeper than a knife’. In this case, words did not physically cause a cut; instead, the words caused a pain that is analogous to a cut. Metaphors are ubiquitous in language and add colour to our conversations. Because of this, having the ability to detect and form metaphors has potential applications in machine translation and automatic creative writing, such as news headline generation or rephrasal. In addition, the detection of metaphors allows us to analyse discourse and see how language evolves. In our work, we present initial experiments on the automatic token-level detection of metaphors for the Slovene language. To do so, we leverage several resources from the CLARIN resource repository: - The KOMET and G-KOMET corpora. KOMET is a corpus containing 259 839 tokens across 13 963 sentences and covers journalistic, fiction, and online text. G-KOMET is smaller, contains 52 955 tokens across 5 695 sentences, and covers speech transcripts. Both corpora contain metaphor signals, direct, indirect, and borderline metaphors. In our work, we consider direct and indirect metaphors as positive examples (metaphors), and the rest as negative examples (not metaphors). - The SloBERTa and CroSloEngual BERT language models. SloBERTa is a large transformer model, pre-trained on a large collection of Slovene text. Similarly, CroSloEngual BERT is a large transformer model, but it is pre-trained on text from two languages in addition to Slovene: Croatian and English. We additionally experiment with a multilingual model XLM-RoBERTa, which is not part of the repository and is trained on data from 100 languages. To apply the models to our problem, we take existing pre-trained weights and fine-tune them for metaphor detection on the two corpora. Specifically, we fine-tune the models by minimising the cross-entropy between the predicted metaphoricity tags and the annotated metaphoricity tags. We evaluate our models quantitatively using the token-level F1 score and qualitatively by analysing examples of correct and incorrect predictions. We perform our experiments on two versions of each corpus: the original (unmodified) corpus and the corpus with only noun and verb metaphors. We find that the models reach a reasonably high detection accuracy on the first version (up to 0.60 F1 score), but perform less well on the second version (up to 0.41 F1 score), which contains harder-to-detect but semantically more interesting metaphors. The best detection accuracy is consistently achieved by the monolingual SloBERTa, followed by the trilingual CroSloEngual BERT and the multilingual XLM-RoBERTa. Additionally, we test whether combining the training data of both corpora can improve the metaphor detection accuracy and find that it can lead to small improvements.