Abstract
End-to-end neural models are the mainstream in current natural language processing. However, in order to support information analysis and causal relationship analysis, which essentially require trial and error, it is essential to recognize entities, events, and their relationships. linguistic analysis is still important.
Until now, analyzers have been developed for each task, and there was a problem that the cost was high to connect them and use them. In this research, we design and construct a highly accurate integrated Japanese language analyzer KWJA based on a general-purpose language model.
This analyzer integrates input error correction, punctuation, word normalization, morphological analysis, named entity recognition, linguistic feature assignment, syntactic analysis, predicate-argument structure analysis, bridging anaphora analysis, coreference analysis, and discourse relation analysis. It is realized by architecture.
The constructed analyzer is published at https://github.com/ku-nlp/kwja.