The general chairman of this year’s meeting of the European Chapter in Association for Computational Linguistics (EACL) is Alessandro Moschitti, a main scientist of the Aixa AI organization, and the conference comes at a peculiar time in the field’s history. With the remark recently results of large language models (LLMS), Moschitti says: “Most of the key problems we had in NLP [natural-language processing] The fact that we considered part of AI – for example, syntactic parsing, semantic parsing, kereferens resolution or sense of analysis – is basically resolved. You can send some inquiries to chatgpt and it can check syntax in a piece of text. “
At the same time, the way most researchers interact with LLMs is through an application programming interface; They cannot access the inner work of the models. The result is a recent concentration on quick technique or cites input into LLMS that induces the desired output.
“If you want to do a more basic study – produce different architectures – and you are not affiliated with owners of LLMS, there is no way you can do this kind of research,” says Moschitti. “Alternatively, you still need a proxy a smaller model – which by the way is more less the same architecture, just less.”
Consequently, Moschitti explains that a number of papers at EACL still relate to the previous generation of language models, such as Bert, Roberta, Electra and T5. LLMs are based on the same architecture, but with size orders several parameters and trained in similar text finishing tasks, but over much larger data sets. So the results of experiment involving the smaller models will often have consequences for LLMs.
Of race, the highly reported disadvantage of LLMs is their tendency to “hallucinate” or make plausible but false impact stands. Remedy of hallucinations is already an active research area, Moschitti explains, and it will probably only grow in popularity.
“You want to look at EACL more papers that try to add this restriction of transformation -based models,” says Moschitti. “There are mostly two methods. One method is to train the model and then analyze output and build up some classifiers that can detect whether the output is a hallucination or not.
“The second approach, which is technically cleaner, is to train the model in such a way that we hallucinate less. In the past, we used limited decoding, which means we are some limitation that we are decorating input so that we diverges now, a topic that is pervasive is a so -called group.”
Responsibility
As at most AI-related conferences in the last few years, responsible AI is also a major topic at EACL, says Moschitti. Some types of bias in NLP models are straightforward to measure: for example, cases where a model connects certain sex (eg woman) with certain professions (eg Nurse). These types of bias can be addressed using general machine learning that applies across disciplines.
But other types of bias are more subtle, such as facilitated different tone in the way members of different demographic groups are described. To eradicate these types of bias requires NLP scientists’ special skills.
“It’s definitely an important topic that tracks the NLP community because it’s a matter of style about how you say things,” says Moschitti. “So the meaning and also the pragmatics.”
Ethical concerns are the basis of another recent shift on EACL, says Moschitti – but this is a procedural shift, not a shift in paper topics.
“We have three or four most important NLP conferences every year,” says Moschitti. “Before EACL we had Emnlp [the Conference on Empirical Methods in Natural-Language Processing]And they started a new tendency to require each paper submission to have a section on restrictions. It must describe the context of the applicability of the proposed solution. Becuse typically people tend to write as if their approaches offered improvement everywhere. Objective usal has an approach restrictions. It can be efficiency; It can be the utility of language security; It can be set; Or it may require certain resources. So Emnlp started this idea of ​​adding this limitation section, and in our papers in EACL we also have this requirement. “