OntoSum

"A key factor which makes a particular discipline or approach scientific is the ability to evaluate and compare the ideas within the area"

In this thesis our contribution to the evaluation of ontology learning methods, is through a framework and a set of evaluation measures in order to assess learned ontologies against gold standard ontologies.

The proposed methodology transforms the ontology concepts and properties into a vector space representation to avoid the common string matching of concepts and properties at the lexical layer. The proposed evaluation measures exploit this vectorial representation and calculate the similarity of the two ontologies (learned and gold).

For now, further information can be found on: Zavitsanos, E. et al. A Distributional Approach to Evaluating Ontology Learning Methods Using a Gold Standard. In OLP3, ECAI, 2008.

• Brewster, C et al. Data driven ontology evaluation. Proceedings of Int. Conf, on Language Resources and Evaluation, Lisbon, 2004.
• Burton-Jones,A et al. A semiotic metrics suite for assessing the quality of ontologies. Data and Knowledge Engineering, 2004.
• Ding, L et al. Swoogle: A search and metadata engine for the semantic web. Proc. CIKM 2004, pp. 652-659.
• Ehri, M et al. Similarity for ontologies - a comprehensive framework. Proc. Eur. C.Inf. Sys. 2005.
• Fox, M S et al. An organization ontology for enterprise modelling. In M. Prietula et al. Simulating organizations, MIT Press, 1998.
• Gomez-Perez, A. Some ideas and examples to evaluate ontologies. Knowledge Systems Laboratory, Stanford University, 1994.
• Gomez-Perez, A. Towards a framewok to verify knowledge sharing technology. Expert Systems with Applications, 11(4):519-529, 1996.
• Guarino, N, Welty, C. Evaluating ontological decisions with OntoClean. Comm. of the ACM, 45(2):61-65, February 2002.
• Lozano-Tello, A, Gomez-Perez, A. Ontometric: A methd to choose the appropriate ontology. J. Datab. Mgmt., 15(2)1-18, 2004.
• Maedche, A, Staab, S. Measuring similarity between ontologies. Proc. CIKM 2002. LNAI vol. 2473.
• Patel, C et al. OntoKhoj: A semantic web portal for ontology searching, ranking and classification. ACM Web Inf. & Data Mgmt., 2004.
• Porzel, R, Malaka, R. A task-based approach for ontology evaluation, ECAI 2004 Workshop Ont. Learning and Population.
• Spyns, P. EvaLexon: Assessing triples mined from texts. Technical Report 09, STAR Lab, Brussels, Belgium, 2005.
• Supekar, K. A peer-review approach for ontology evaluation. Proc. 8th Intl. Protege Conference, Madrid, Spain, July 18-21, 2005.
• Velardi, P et al. Evaluation of OntoLear, a methodology for automatic learning of domain ontologies. In Ont. Learning from Text: Methods, Evaluation and Applications, IOS Press, 2005.
• Lozano-Tello, A, Gomez-Perez, A, Sosa, E. Selection ofontologies for the semantic web. J.M. Cueva Lovelle et al. (Eds.): ICWE 2003, LNCS 2722, pp. 413-416, 2003.
• Jones, A.B. et al. A Semiotic Suite for Assessing the Quality of Ontologies. In Data and Knowledge Engineering, 55(1):84-102, 2004.
• Brank, J. et al. Gold Standard Based Ontology Evaluation Using Instance Asignment. Proc. of the EON Workshop, 2006.
• Dellschaft, K. and Staab, S. On How to Perform a Gold Standard Based Evaluation of Ontology Learning. Proc. of the 5th Int. Conf. on Semantic Web, 2006.
• Maynard, D. et al. Metrics for Evaluation of Ontology-based Information Extraction. Proc. of the EON Workshop, 2006.
• Zavitsanos, E. et al. A Distributional Approach to Evaluating Ontology Learning Methods Using a Gold Standard. In OLP3, ECAI, 2008.
• Ehrig, M. et al. Similarity for Ontologies - A Comprehensive Framework. Proc. of the European Conf. in Information Systems, 2005.