ISSN 2071-8594

Russian academy of sciences

Editor-in-Chief

Gennady Osipov

N.V. Loukachevitch, I.I. Chetviorkin. Open evaluating Sentiment Analysis Systems in Russian

Abstract.

In this paper we describe our experience in conducting the first open sentiment analysis evaluations in Russian within ROMIP 2011-2012. Several train collections were created for such tasks as sentiment classification in blogs and newswire, opinion retrieval. The paper describes the state of the art in sentiment analysis in Russian, collection characteristics, track tasks and evaluation metrics.

Keywords:

sentiment analysis, opinion mining, sentiment classification, ROMIP.

PP. 25-33.

Full version of the article in pdf.

REFERENCES

1. Abdul-Mageed M., Diab M., Korayem M. Subjectivity and Sentiment Analysis of Modern Standard Arabic. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, 2011. pp. 587-591.
2. Amigo E., Corujo A., Gonzalo J., Meij E., Rijke Md. Overview of replab 2012: Evaluating online reputation management systems. In Proceedings of the CLEF 2012 Labs and Workshop Notebook Papers. 2012. pp. 1–24.
3. Awadallah R., Ramanath M., Weikum G. PolariCQ: Polarity Classification of Political Quotations In Proceedings of CIKM-2012, 2012. pp. 1945-1949.
4. Balasubramanyan R., Cohen W., Pierce D., Redlawsk D. Modeling polarizing topics: When do different political communities respond differently to the same news? Proceedings of ICWSM. 2012.
5. Blinov P., Klekovkina M., Kotelnikov E, Pestov O. Research of lexical approach and machine learning methods for sentiment analysis. In Proceedings of Dialog, volume 2, 2013. pp. 51-61.
6. Chetviorkin I., Braslavskiy P., Loukachevich N. Sentiment Analysis Track at ROMIP 2011. In Proceedings of International Conference Dialog-2012, volume 2, 2012. pp. 1-14.
7. Chetviorkin I., Loukachevitch N. Extraction of Russian Sentiment Lexicon for Product Meta-Domain In Proceedings of COLING 2012, 2012. pp. 593-610.
8. Chetvirokin I., Loukachevitch N. Sentiment Analysis Track at ROMIP 2012. In Proceedings of International Conference Dialog-2013, volume 2, 2013. pp. 40-50.
9. Choi Y., Cardie C. Adapting a polarity lexicon using integer linear programming for domain-specific sentiment classification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2009. pp. 590-598.
10. Yermakov A.Ye. Izvlechenie znaniy iz teksta i ikh obrabotka: sostoyanie i perspektivy.// Informatsionnye tekhnologii, № 7, 2009. C. 50-55.
11. Kotelnikov Ye.V., Klekovkina, M.V. Avtomaticheskiy analiz tonalnosti tekstov na osnove metodov mashinnogo obucheniya. Kompyuternaya lingvistika i intellektualnye tekhnologii: po materialam ezhegodnoy mezhdunarodnoy konferentsii Dialog. T. 2, 2012. C. 27-36.
12. Kuznetsova E.S., Loukachevitch N.V., Chetviorkin I.I. Testing rules for sentiment analysis system. Computational Linguistics and Intellectual Technologies. Proc. of International Conference Dialog-2013, vol. 2, 2013. pp. 71-80.
13. Macdonald C., Ounis I., Soboroff I. Overview of the TREC 2007 blog track. In Proceedings of TREC-2007. Gaithersburg, USA, 2008.
14. Macdonald C., Ounis I., Soboroff I. Overview of the TREC 2009 blog track. In Proceedings of TREC-2009. Gaithersburg, USA, 2010.
15. Manning C. D., Raghavan P., Sch?tze H. Introduction to information retrieval. – Cambridge: Cambridge University Press. 2008.
16. Mihalcea R., Banea C., Wiebe J. Learning multilingual subjective language via crosslingual projections. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, Prague, Czech Republic, 2007. pp. 976-983.
17. Morante R., Blanco E. *SEM 2012 shared task: Resolving the scope and focus of negation. In Proceedings of the First Joint Conference on Lexical and Computational Semantics, Montreal,. 2012. pp. 265-274
18. Pak A., Paroubek P. Language independent approach to sentiment analysis (LIMSI Participation in ROMIP’11) Computational Linguistics and Intellectual Technologies. Proc. of International Conference Dialog-2012, vol. 2, 2012. pp. 37-50.
19. Pan S. J., Ni X., Sun J-T, Yang Q., Chen Z. Cross-Domain Sentiment Classification via Spectral Feature Alignment. In Proceedings of the World Wide Web Conference, 2010. pp. 751-760.
20. Pang B., Lee L. Opinion mining and sentiment analysis. Foundations and Trends® in Information Retrieval. Now Publishers, 2008.
21. Panicheva P. Sistema sentimentnogo analiza ATEX, osnovannaya na pravilakh, pri obrabotke tekstov raz-lichnykh tematik. Kompyuternaya lingvistika i intel-lektualnye tekhnologii: po materialam ezhegodnoy mezhdunarodnoy konferentsii Dialog. T. 2, 2013. S.101-113.
22. Pazelskaya A. G., Solovev A. N. Metod opredeleniya emotsiy v tekstakh na russkom yazyke. Kompyuternaya lingvistika i intellektualnye tekhnologii: po materialam ezhegodnoy mezhdunarodnoy konferentsii Dialog, 2011. C. 510-522.
23. Perez-Rosas V., Banea C., Mihalcea R. Learning Sentiment Lexicons in Spanish. In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC'12). 2012.
24. Polyakov, P.Yu., Kalinina, M.V., Pleshko, V.V. Issledovanie primenimosti metodov tematicheskoy klassifikatsii v zadache klassifikatsii otzyvov o knigakh Kompyuternaya lingvistika i intellektualnye tekhnologii: po materialam ezhegodnoy mezhdunarodnoy konferentsii Dialog T. 2, 2012. S. 51-59.
25. Pestian J., Matykiewicz.P., Linn-Gust M. Sentiment analysis of suicide notes: A shared task. Biomedical Informatics Insights. 2012;5 (Suppl. 1), 2012. pp. 3-16.
26. Ounis I., de Rijke M., Macdonald C., Mishne G., Soboroff I. Overview of TREC-2006 Blog track. In Proceedings of TREC-2006, Gaithersburg, USA, 2007.
27. Ounis I., Macdonald C., Soboroff I. Overview of the TREC 2008 blog track. In Proceedings of TREC-2008. Gaithersburg, USA, 2009.
28. Ounis I., Macdonald C., Soboroff I. Overview of the TREC 2010 blog track. In Proceedings of TREC-2010. Gaithersburg, USA, 2011.
29. Seki Y., Evans D., Ku L., Chen H., Kando N., Lin C. Overview of opinion analysis pilot task at NTCIR-6. In Proceedings of NTCIR-6 Workshop Meeting. 2007. pp. 265-278.
30. Steinberger J., Lenkova P., Ebrahim M., Ehrmann M., Hurriyetogly A., Kabadjov M., Steinberger R., Tanev H., Zavarella V. and Vazquez S. Creating Sentiment Dictionaries via Triangulation. In Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis, ACL-HLT, 2011. pp. 28-36.
31. Taboada M., Brooke J., Tofiloski M., Voll K., Stede M. Lexicon-based methods for Sentiment Analysis. Computational linguistics, 37(2), 2011. pp. 267-307.
32. Wu Y., Jin P. Semeval-2010 task 18: Disambiguating sentiment ambiguous adjectives. In Proceedings of the 5th International Workshop on Semantic Evaluation. 2010. pp. 81-85.
33. Zagibalov T., Belyatskaya K., Carroll J. Comparable English-Russian Book Review Corpora for Sentiment Analysis. In Proceedings of the 1st Workshop on Computational Approaches to Subjectivity and Sentiment Analysis WASSA, 2010. pp. 67-72.