This was not my first time working with Python to figure out the tokens of sentences via computational means. However, my last time working with Python (in a linguistics BA seminar with Prof. Kevin Tang) has been some time back, so I have appreciated that working our example sentences through it has been made very easy.<\/p>\n\n\n\n
Running the three example sentences given to us, I noticed that punctuations marking speech are often interpreted as part of words or compounds and are tagged as POS, which makes it difficult to check which tokens refer to which words.<\/p>\n\n\n\n
Additionally, foreign words were always interpreted as proper nouns \u2013 even in the Spanish phrase clarita del huevo<\/em>, in which del<\/em> is clearly not a proper noun. I would have thought that maybe the Spanish language would be easier to interpret or perhaps translate than the Swahili sentence as it might be more commonly known, but Python does not do any translating and so struggles with anything that is not English. Dependencies can thus not be correctly determined: in the sentence T\u00edas called me blanca, palida, clarita del huevo<\/em> the last three parts (blanca, palida, clarita del huevo<\/em>) are a listing, all nouns or noun phrases are on equal standing and not dependent on each other, but Python marks huevo<\/em> as a dependent of palida<\/em>.<\/p>\n\n\n\n The given example with Swahili words in it cannot be properly tagged at all as \u2013 according to Python – \u2018Ayaaaana! Haki ya Mungu … aieee!\u2019 <\/em>The threat-drenched contralto came from the bushes to the left of the mangroves. \u2018Aii, mwanangu, mbona wanitesa?\u2019 <\/em>is made up entirely of nouns or proper nouns. Only the English part could be identified correctly.<\/p>\n\n\n\n The English-Chinese example provides similar issues:<\/p>\n\n\n\n In this example, the Chinese hanzi<\/em> are not computed as proper nouns. Python interprets \u5bb6 as a noun once and an adjective another time, marking it \u2013 quite nonsensically \u2013 as a dependent of legs<\/em>.<\/p>\n\n\n\n Now looking at the novel I have been reading \u2013 A Concise Chinese-English Dictionary for Lovers<\/em> by Xiaolu Guo \u2013 I looked for similar multilingual sentences to test in the programme. I couldn\u2019t find many such sentences, but tested the ones I did find:<\/p>\n\n\n\n \u77e5\u8bc6 is here correctly interpreted as a noun, although it is interpreted as one singular noun rather than a noun phrase consisting of a verb (\u77e5 \u2013 to know) and a noun (\u8bc6 \u2013 knowledge).<\/p>\n\n\n\n Similar goes for\u5206\u5b50, interpreted as one noun rather than a noun phrase consisting of a verb (\u5206 \u2013 divide) and a noun (\u5b50 \u2013 son, child).<\/p>\n\n\n\n The word \u201cmean\u201d in this sentence is the verb \u201cto mean\u201d, but not conjugated correctly because the narrator is not fluent in English yet and struggles with English grammar. Python thus tags it as an adjective. Accordingly, the dependencies turn out incorrect as well, as the mean<\/em> should function as the head of its sentence. Instead, knowledge<\/em> becomes that head upon which all other words depend.<\/p>\n\n\n\n In the 3rd<\/sup> sentence \u201crepresent\u201d is interpreted as a dependent of \u201cis\u201d \u2013 again, likely due to the improper grammar the narrator uses. The hanzi are not computed at all (\u5c41), marked as a proper noun (\u5c38) or a noun (\u6bd4).<\/p>\n\n\n\n This sentence starts out sort of elliptical. The phrase very important to us Chinese<\/em> is tagged and interpreted correctly, but Python struggles with everything that comes before it, marking Chi (<\/em>\u6c14<\/em>)<\/em> as a dependent of is<\/em>. Evidently, Python cannot compute and interpret punctuation correctly, which in this case should indicated that the first word is separate from the sentence after the comma.<\/p>\n\n\n\n In the next two examples I wanted to test how Python deals with incorrect English grammar but without the interference of non-English words, hypothesising that situations like the above mean<\/em> would also occur here.<\/p>\n\n\n\n In this case, coming from previous errors Python made due to incorrect grammar and conjugation, I thought that feeling<\/em> might be interpreted as a noun because the auxiliary \u201cam\u201d of the progressive form is missing. Surprisingly, Python has no problem recognising feeling<\/em> as the verb it is indeed supposed to be, marking it correctly as the head\/root of the entire sentence<\/p>\n\n\n\n Similar as above, I wanted to test how Python deals with these grammatical errors (seems<\/em> instead of seem<\/em>; directing<\/em> instead of direction<\/em>) \u2013 again, surprisingly, all tokens were tagged and interpreted properly with all their dependencies. Even directing<\/em> has been correctly identified as a noun instead of a progressive verb.<\/p>\n\n\n\n Evidently, annotating multi-lingual sentences correctly is not a possibility \u2013 at least not with the Python code we have been given. While the programme has no problem interpreting English sentences with incorrect grammar, it is thrown for a loop as soon as non-English words are introduced, which was a very interesting observation to make.<\/p>\n","protected":false},"excerpt":{"rendered":" This was not my first time working with Python to figure out the tokens of sentences via computational means. However, my last time working with Python (in a linguistics BA seminar with Prof. Kevin Tang) has been some time back, … Weiterlesen \n
\n
\n
\n
\n
\n