Naive Bayes Classifier for Dynamic Chaining Approach in Multi-label Learning

AUTHOR(S):

TITLE

Naive Bayes Classifier for Dynamic Chaining Approach in Multi-label Learning

ABSTRACT

In this paper, we addressed an issue of building dynamic classifier chain ensembles for multi-label classification. We built a classifier that allows us to change label order of the chain without rebuilding the entire model. Such a model allows anticipating the instance-specific chain order without a significant increase in computational burden. The proposed chain model is built using the Naive Bayes classifier as a base single-label classifier. Additionally, we proposed a simple heuristic that allows the system to find relatively good label order. That is, the heuristic tries to minimise the phenomenon of error propagation in the chain. The experimental results showed that the proposed model based on Naive Bayes classifier the above-mentioned heuristic is an efficient tool for building dynamic chain classifiers.

KEYWORDS

multi-label, classifier-chains, naive bayes, dynamic chains

REFERENCES

[1] E. Alvares Cherman, J. Metz and M. C. Monard. A Simple Approach to Incorporate Label Dependency in Multi-label Classification. In Advances in Soft Computing. Springer Berlin Heidelberg, 2010. pp. 33–43. doi:10.1007/ 978-3-642-16773-7 3. [1] E. Alvares Cherman, J. Metz and M. C. Monard. A Simple Approach to Incorporate Label Dependency in Multi-label Classification. In Advances in Soft Computing. Springer Berlin Heidelberg, 2010. pp. 33–43. doi:10.1007/ 978-3-642-16773-7 3.

[2] F. Charte, A. Rivera, M. J. del Jesus and F. Herrera. Concurrence among Imbalanced Labels and Its Influence on Multilabel Resampling Algorithms. In Lecture Notes in Computer Science. Springer International Publishing, 2014. pp. 110–121. doi:10.1007/978-3-319-07617-1 10.

[3] F. Charte, A. J. Rivera, M. J. del Jesus and F. Herrera. QUINTA: A question tagging assistant to improve the answering ratio in electronic forums. In IEEE EUROCON 2015 - International Conference on Computer as a Tool (EUROCON). IEEE. doi:10.1109/eurocon.2015. 7313677.

[4] P. N. da Silva, E. C. Gonc¸alves, A. Plastino and A. A. Freitas. Distinct Chains for Different Instances: An Effective Strategy for Multi-label Classifier Chains. In Machine Learning and Knowledge Discovery in Databases. Springer Berlin Heidelberg, 2014. pp. 453–468. doi: 10.1007/978-3-662-44851-9 29.

[5] J. D´ıez, O. Luaces, J. J. del Coz and A. Bahamonde. Optimizing different loss functions in multilabel classifications. Progress in Artificial Intelligence, 3(2), (2014), pp. 107–118. doi: 10.1007/s13748-014-0060-7.

[6] J. Demsar. Statistical comparisons of classifiers ˇ over multiple data sets. The Journal of Machine Learning Research, 7, (2006), pp. 1–30.

[7] M. Dhar. On Cardinality of Fuzzy Sets. International Journal of Intelligent Systems and Applications, 5(6), (2013), pp. 47–52. doi:10.5815/ ijisa.2013.06.06.

[8] M. Friedman. A Comparison of Alternative Tests of Significance for the Problem of m Rankings. The Annals of Mathematical Statistics, 11(1), (1940), pp. 86–92. doi:10.1214/aoms/ 1177731944.

[9] V. Garc´ıa, J. Sanchez and R. Mollineda. On ´ the effectiveness of preprocessing methods when dealing with different levels of class imbalance. Knowledge-Based Systems, 25(1), (2012), pp. 13–21. doi:10.1016/j.knosys.2011.06.013.

[10] E. Gibaja and S. Ventura. Multi-label learning: a review of the state of the art and ongoing research. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 4(6), (2014), pp. 411–444. doi:10.1002/widm.1139.

[11] E. C. Gonc¸alves, A. Plastino and A. A. Freitas. Simpler is Better. In Proceedings of the 2015 on Genetic and Evolutionary Computation Conference - GECCO ’15. ACM Press. doi: 10.1145/2739480.2754650.

[12] E. C. Goncalves, A. Plastino and A. A. Freitas. A Genetic Algorithm for Optimizing the Label Ordering in Multi-label Classifier Chains. In 2013 IEEE 25th International Conference on Tools with Artificial Intelligence. IEEE. doi: 10.1109/ictai.2013.76.

[13] M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann and I. H. Witten. The WEKA data mining software. ACM SIGKDD Explorations Newsletter, 11(1), (2009), p. 10. doi: 10.1145/1656274.1656278.

[14] M. A. Hall. Correlation-based feature selection for machine learning. Ph.D. thesis, The University of Waikato, 1999.

[15] D. J. Hand and K. Yu. Idiot’s Bayes: Not So Stupid after All? International Statistical Review / Revue Internationale de Statistique, 69(3), (2001), p. 385. doi:10.2307/1403452.

[16] S. Holm. A Simple Sequentially Rejective Multiple Test Procedure. Scandinavian Journal of Statistics, 6(2), (1979), pp. 65–70. ISSN 03036898. doi:10.2307/4615733.

[17] J.-Y. Jiang, S.-C. Tsai and S.-J. Lee. FSKNN: Multi-label text categorization based on fuzzy similarity and k nearest neighbors. Expert Systems with Applications, 39(3), (2012), pp. 2813– 2821. doi:10.1016/j.eswa.2011.08.141.

[18] T. Joachims. Text Categorization with Suport Vector Machines: Learning with Many Relevant Features. In Proc. 10th European Conference on Machine Learning. pp. 137–142.

[19] O. Luaces, J. D´ıez, J. Barranquero, J. J. del Coz and A. Bahamonde. Binary relevance efficacy for multilabel classification. Progress in Artifi- cial Intelligence, 1(4), (2012), pp. 303–313. doi: 10.1007/s13748-012-0030-x.

[20] J. Read, L. Martino and D. Luengo. Effi- cient monte carlo methods for multi-dimensional learning with classifier chains. Pattern Recognition, 47(3), (2014), pp. 1535–1546. doi:10.1016/ j.patcog.2013.10.006.

[21] J. Read and R. Peter. Meka:http://meka.sourceforge.net/, 2017. URL http://meka.sourceforge.net/.

[22] J. Read, B. Pfahringer, G. Holmes and E. Frank. Classifier Chains for Multi-label Classification. In Machine Learning and Knowledge Discovery in Databases. Springer Berlin Heidelberg, 2009. pp. 254–269. doi:10.1007/978-3-642-04174-7 17.

[23] J. Read, B. Pfahringer, G. Holmes and E. Frank. Classifier chains for multi-label classification. Machine Learning, 85(3), (2011), pp. 333–359. doi:10.1007/s10994-011-5256-5.

[24] C. Sanden and J. Z. Zhang. Enhancing multilabel music genre classification through ensemble techniques. In Proceedings of the 34th international ACM SIGIR conference on Research and development in Information - SIGIR ’11. ACM Press. doi:10.1145/2009916.2010011.

[25] R. Senge, J. J. del Coz and E. Hullermeier. ¨ On the Problem of Error Propagation in Classifier Chains for Multi-label Classification. In Studies in Classification, Data Analysis, and Knowledge Organization. Springer International Publishing, 2013. pp. 163–170. doi:10.1007/ 978-3-319-01595-8 18.

[26] E. Spyromitros-Xioufis, G. Tsoumakas, W. Groves and I. Vlahavas. Multi-target regression via input space expansion: treating targets as inputs. Machine Learning, 104(1), (2016), pp. 55–98. doi:10.1007/s10994-016-5546-z.

[27] E. Spyromitros-Xioufis, G. Tsoumakas, W. Groves and I. Vlahavas. Multi-target regression via input space expansion: treating targets as inputs. Machine Learning, 104(1), (2016), pp. 55–98. doi:10.1007/s10994-016-5546-z.

[28] J. T. Tomas, N. Spola ´ or, E. A. Cherman and ˆ M. C. Monard. A Framework to Generate Synthetic Multi-label Datasets. Electronic Notes in Theoretical Computer Science, 302, (2014), pp. 155–176. doi:10.1016/j.entcs.2014.01.025.

[29] P. Trajdos and M. Kurzynski. PermutationBased Diversity Measure for Classifier-Chain Approach. In Advances in Intelligent Systems and Computing. Springer International Publishing, 2017. pp. 412–422. doi:10.1007/ 978-3-319-59162-9 43.

[30] G. Tsoumakas, I. Katakis and I. Vlahavas. Effective and efficient multilabel classification in domains with large number of labels, 2008. p. 30–44.

[31] Y. Wei, W. Xia, M. Lin, J. Huang, B. Ni, J. Dong, Y. Zhao and S. Yan. HCP: A Flexible CNN Framework for Multi-Label Image Classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(9), (2016), pp. 1901– 1907. doi:10.1109/tpami.2015.2491929.

[32] F. Wilcoxon. Individual Comparisons by Ranking Methods. Biometrics Bulletin, 1(6), (1945), p. 80. doi:10.2307/3001968.

[33] J.-S. Wu, S.-J. Huang and Z.-H. Zhou. Genome-Wide Protein Function Prediction through Multi-Instance Multi-Label Learning. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 11(5), (2014), pp. 891–902. doi:10.1109/tcbb.2014.2323058.

[34] J. Xu. Fast multi-label core vector machine. Pattern Recognition, 46(3), (2013), pp. 885–898. doi:10.1016/j.patcog.2012.09.003.

[35] L. Zadeh. Fuzzy sets. Information and Control, 8(3), (1965), pp. 338–353. doi:10.1016/ s0019-9958(65)90241-x.

[36] P. Zhang, Y. Yang and X. Zhu. Approaching Multi-dimensional Classification by Using Bayesian Network Chain Classifiers. In 2014 Sixth International Conference on Intelligent Human-Machine Systems and Cybernetics. IEEE. doi:10.1109/ihmsc.2014.129.

[37] Z.-H. Zhou, M.-L. Zhang, S.-J. Huang and Y.-F. Li. Multi-instance multi-label learning. Artifi- cial Intelligence, 176(1), (2012), pp. 2291–2320. doi:10.1016/j.artint.2011.10.002.

Cite this paper

Pawel Trajdos, Marek Kurzynski. (2017) Naive Bayes Classifier for Dynamic Chaining Approach in Multi-label Learning. International Journal of Education and Learning Systems, 2, 133-142