The first task on my list of 10 open problems in computational diversity linguistics deals with morphemes, that is, the minimal meaning-bearing parts in a language. A morpheme can be a word, but it does not have to be a word, since words may consist of more than one morpheme, and — depending on the language in question — may do so almost by default.
Examples of morphemes in English include clear-cut cases of compounding, where two words are joined to form a new word. Often, this is not even readily reflected in spelling, and, as a result, speakers may at times think that a word like "primary school" is not a single word, although it is easy to determine from its semantics that the word is indeed pointing to one uniform concept. Other examples include grammatical markers, such as the ending -s for most English plurals, or to mark the third person singular of verbs. When confronted with a word form like walks, linguists will analyze this word as consisting of two morphemes, illustrating it by adding a dash as a boundary marker: walk-s.
The problem
The task of automatic morpheme segmentation is thus a pretty straightforward one: given a list of words, potentially along with additional information, such as their meaning, or their frequency in the given language, try to identify all morpheme boundaries, and mark this by adding dash symbols where a boundary has been identified.
One may ask why automatic identification of morphemes should be a problem — and some people commenting on my presentation of the 10 open problems last month did ask this. The problem is not unrecognized in the field of Natural Language Processing, and solutions have been discussed from the 1950s onwards (Harris 1955, Benden 2005, Bordag 2008, Hammarström 2006, see also the overview by Goldsmith 2017).
Roughly speaking, all approaches build on statistics about n-grams, i.e., recurring symbol sequences of arbitrary length. Assuming that n-grams representing meaning-building units should be distributed more frequently across the lexicon of a language, they assemble these statistics from the data, trying to infer the ones which "matter". With Morfessor (Creutz and Lagus 2005, there is also a popular family of algorithms available in form of a very stable and easy-to-use Python library (Virpioja et al. 2013). Applying and testing methods for automatic morpheme segmentation is thus very straightforward nowadays.
The issue with all of these approaches and ideas is that they require a very large amount of data for training, while our actual datasets are small and sparse, by nature. As a result, all currently available algorithms fail graciously when it comes to determining the morphemes in datasets of less of 1,000 words.
Interestingly, even when having been trained on large datasets, the algorithms still commit surprising errors, as can be easily seen when testing the online demo of the Morfessor software for German (https://asr.aalto.fi/morfessordemo/). When testing words like auftürmen "pile up", for example, the algorithm yields the segmentation auf-türme-n, which is probably understandable from the fact that the word Türme "towers" is quite frequent in the German lexicon, thus confusing the algorithm; but for a German speaker, who knows that verbs end in -en in their infinitive, it is clear that the auftürmen can only be segmented as auf-türm-en.
If I understand the information on the website correctly, the Morfessor algorithm offered online was trained with more than 1 million different word forms in German. Given that in our linguistic approaches we can usually dispose of 1,000 words, if not less, per language, it is clear that the algorithms won't provide help in finding the morphemes in our data.
To illustrate this, I ran a small test on the Morfessor software, using two datasets for training, one big dataset with about 50000 words from Baayen et al. (1995), and one smaller dataset of about 600 words which I used as a cognate detection benchmark when writing my dissertation (List 2014). I then used these two datasets to train the Morfessor software and then applied the trained models to segment a list of 10 German words (see the GitHub.Gist here.
The results for the two models (small data and big data) as well as the segmentations proposed by the online application (online) are given in the table below (with my own judgments on morphemes given in the column word).
What can be seen clearly from the table, where all forms deviating from my analysis are marked in red font, is that none of the models makes a convincing job in
segmenting my ten test words. More importantly, however, we can clearly see
that the algorithm's problems increase drastically when dealing with small training data. Since the segmentations proposed in the Small data column are clearly the
worst, splitting words in a seemingly random fashion into letters.
What is interesting in this context is that trained linguists would rarely fail
at this task, even when all they were given is the small data list for training. That they do not
fail is shown by the numerous studies where linguistic fieldworkers
have investigated so far under-investigated languages, and quickly figured out
how the morphology works.
Why is it so difficult to find morpheme boundaries?
What makes the detection of morpheme boundaries so difficult, also for humans, is that they are inherently ambiguous. A final -s can mark the plural in German, especially on borrowings, as in Job-s, but it can likewise mark a short variant of es "it", where the vowel is deleted, as in ist's "it's", and in many other cases, it can just mark nothing, but instead be part of a larger morpheme, like Haus "house". Whether or not a certain substring of sounds in a language can function as a morpheme depends on the meaning of the word, not on the substring itself. We can — once more — see one of the great differences between sequences in biology and sequences in linguistics here: linguistic sequences derive their "function" (ie. their meaning) from the context in which they are used, not from their structure alone.
If speakers are no longer able to clearly understand the morphological structure of a given word, they may even start to change it, in order to make it more "transparent" in its denotation. Examples for this are the numerous cases of folk etymology, where speakers re-interpret the morphemes in a word, with English ham-burger as a prominent example, since the word originally seems to derive from the city Hamburg, which has nothing to do with ham.
How do humans find morphemes?
The
reasons why human linguists can relatively easy find morphemes in sparse data, while machines cannot, is still not
entirely clear to me (ie. humans are good at pattern recognition and
machines are not). However, I do have some basic ideas about why humans
largely
outperform machines when it comes to morpheme segmentation; and I think
that future approaches that try to take these ideas into account might
drastically improve the performance of automatic morpheme segmentation
methods.
As a first point, given the importance of meaning in order to determine morphemic structure, it seems almost absurd to me to try to identify morphemes in a given language corpus based on a pure analysis of the sequences, without taking their meaning into account. If we are confronted with two words
like Spanish hermano "brother" and hermana "sister", it is clear — if we know what
they mean — that the -o vs. -a most likely denotes a distinction of
gender. While the machines compare potential similarities inside the words
independent of semantics, humans will always start from those pairs where
they think that they could expect to find interesting alternations. As long as
the meanings are supplied, a human linguist — even when not familiar with a
given language — can easily propose a more or less convincing segmentation of
a list of only 500 words.
A second point that is disregarded in current automatic approaches is the fact that morphological structures vary drastically among languages. In Chinese and many South-East Asian languages, for example, it is almost a rule that every syllable represents one morpheme (with minimal exceptions being attested and discussed in the literature). Since syllables are again easy to find in these languages, since words can often only end in a specific number of sounds, an algorithm to detect words in those languages would not need any n-gram statistics, but just a theory on syllable structures. Instead of global strategies, we may rather have to use for local strategies of morpheme segmentation, in which we identify different types of languages for which a given algorithm seems suitable.
This brings us to a third point. A peculiarity of linguistic sequences in spoken languages is that they are built by specific phonotactic rules that govern their overall structure. Whether or not a language tolerates more than three consonants in the beginning of a word depends on its phonotactics, its set of rules by which the inventory of sounds is combined to form morphemes and words. Phonotactics itself can also give hints on morpheme boundaries, since they may prohibit combinations of sounds within morphemes which can occur when morphemes are joined to form words. German Ur-instinkt "basic instinct", for example, is pronounced with a glottal stop after the Ur-, which can only occur in the beginning of German words and morphemes, thus marking the word clearly as a compound (otherwise the word could be parsed as Urin-stinkt "urine smells".
A fourth point that is also generally disregarded in current approaches to
automatic morpheme segmentation is that of cross-linguistic evidence. In
many cases, the speakers of a given language may themselves no longer be aware
of the original morphological segmentation of some of their words, while the
comparison with closely related languages can still reveal it. If we have a
potentially multi-morphemic word in one language, for example, and only one of
the two potential morphemes reflected as a normal word in the other language,
this is clear evidence that the potentially multi-morphemic word does, indeed,
consist of multiple morphemes.
Suggestions
Linguists regularly use multiple types of evidence when trying to understand
the
morphological composition of the words in a given language. If we want
to
advance the field of automatic morpheme segmentation, it seems to me
indispensable that we give up the idea of detecting the morphology of a language just by looking at the distribution of letters across word forms. Instead, we
should
make use of semantic, phonotactic, and comparative information. We should further give up the idea of designing universal morpheme segmentation algorithms, but rather study which approach works best on which morphological type. How
these aspects can
be combined in a unified framework, however, is still not entirely clear
to me; and this is also the reason why I list automatic morpheme
segmentation as the first of my ten open problems in computational
diversity linguistics.
Even more important than the strategies for the solutions of the problem, however, is that we start to work on extensive datasets for testing and training of new algorithms that seek to identify morpheme boundaries on sparse data. As of now, no such datasets exist. Approaches like Morfessor were designed to identify morpheme boundaries in written languages, they barely work with phonetic transcriptions. But if we had the datasets for testing and training available, be it only some 20 or 40 languages from different language families, manually annotated by experts, segmented both with respect to the phonetics and to the morphemes, this would allow us to investigate both existing and new approaches much more profoundly, and I expect it could give a real boost to our discipline and greatly help us to develop advanced solutions for the problem.
References
Baayen, R. H. and Piepenbrock, R. and Gulikers, L. (eds.) (1995) The CELEX Lexical Database. Version 2. Philadelphia.
Benden, Christoph (2005) Automated detection of morphemes using
distributional measurements. In: Claus Weihs and Wolfgang Gaul (eds.):
Classification -- the Ubiquitous Challenge. Berlin and
Heidelberg:Springer. pp 490-497.
Bordag, Stefan (2008) Unsupervised and knowledge-free morpheme
segmentation and analysis. In: Carol Peters, Valentin Jijkoun,
Thomas Mandl, Henning Müller, Douglas W. Oard, Anselmo Peñas, Vivien Petras and Diana Santos (eds.): Advances in Multilingual and Multimodal Information Retrieval. Berlin and Heidelberg:Springer, pp
881-891.
Creutz, M. and Lagus, K. (2005) Unsupervised morpheme segmentation
and morphology induction from text corpora using Morfessor 1.0.
Technical Report. Helsinki University of Technology.
Goldsmith, John A. and Lee, Jackson L. and Xanthos, Aris (2017) Computational learning of morphology. Annual Review of Linguistics 3.1: 85-106.
Hammarström, Harald (2006) A Naive Theory of Affixation and an
Algorithm for Extraction. In: Proceedings of the Eighth Meeting of the
ACL Special Interest Group on Computational Phonology and Morphology at
HLT-NAACL 2006 pp. 79-88.
Harris, Zellig S. (1955) From phoneme to morpheme. Language 31.2: 190-222.
List, Johann-Mattis (2014) Sequence Comparison in Historical Linguistics. Düsseldorf:Düsseldorf University Press.
Virpioja, Sami, Smit, Peter, Grönroos, Stig-Arne and Kurimo,
Mikko (2013) Morfessor 2.0: Python Implementation and Extensions for
Morfessor Baseline. Helsinki:Aalto University.
|
|---|
No comments:
Post a Comment