The “living” ontologies that will furnish the Semantic Web are lacking. The problem is that in ontology engineering practice, the underlying methodological and organisational principles to involve the community are mostly ignored. Each of the involved activities in the community-based ontology evolution methodology require certain skills and tools which domain experts usually lack. Finding a social arrangement of roles and responsibilities that must supervise the consistent implementation of methods and tools is a wicked problem. Based on three technology-independent problem dimensions of ontology construction, we propose a set of social performance indicators (SPIs) to bring insights in the social arrangement evolving the ontology, and how it should be adapted to the changing needs of the community. We illustrate the SPIs on data from a realistic experiment in the domain of competency-centric HRM.
The illustration here is a sneak preview of what’s to come. It is the analysis of an SPI that observes the balance between the human resources spent on the respective parts (computational\formal vs. substantial\informal parts) of the representation of individual concept types through time. This may indicate the need to adapt the social arrangement accordingly. The actions are grouped per part of the ontology: G0 for the discussion part; G1 for the formal part; and G2 for the informal part. G3, 4 and 5 resp. for creating, deleting and moving concept pages. The graph shows three moments (i.e., 3/26; 4/2; and 4/23) where all groups peak. These moments indicate (i) an intermediary deadline for a new ontology version to be accepted, and (ii) and consequently a point where the domain is rescoped for another iteration of the ontology evolution cycle, resulting in a temporarily higher production. The initial scoping peak is the largest, while the following two peaks become gradually smaller. This indicates the ontology reaches a ﬁxpoint as the ﬁnal deadline approaches, as more concepts covering the domain become mature. There are two isolated peaks of actions on the formal parts in the second iteration: 29 actions on 2009-04-09 and 22 on 2009-04-16. This shift of balance between formal and informal actions is the result of a general request by the core domain expert to spent more resources on formalisation of core concept types.
The full experiment will be presented at and published by the International Semantic Web Conference (ISWC) 2009.
Full article: De Leenheer, P., Debruyne, C., Peeter, J. (2009) Towards Social Performance Indicators for Community-based Ontology Evolution. In Proc. of ISWC Workshop on Collaborative Construction, Management and Linking of Structured Knowledge (CK2008)