Abstract
Local community participation is a cornerstone of World Heritage Site (WHS) management, yet the actual impact of this participation often remains unverified. This paper presents "Audit Design," a semi-automated method utilizing Natural Language Processing (NLP) and Knowledge Representation (KR) to measure the transfer of local knowledge into management plan actions. Using the Diyarbakir Fortress and Hevsel Gardens Cultural Landscape as a case study, the research demonstrates how structured management plans can be audited to ensure inclusivity and transparency in heritage conservation. The study validates a five-level scale of knowledge transfer, providing a reliable tool for monitoring the development of WHS management systems.
Introduction
UNESCO mandates that local communities be involved in the planning and monitoring of World Heritage Sites. However, while "participation" is ubiquitous in rhetoric, it is rarely audited in practice. There is a significant "monitoring gap": while we know meetings occur, we often lack the tools to determine if the ideas raised by residents actually influence final management documents. Currently, assessing this "knowledge transfer" requires intensive manual qualitative analysis. Because WHS management plans are highly structured and prescriptive, they are prime candidates for automation. The "Audit Design" method aims to move beyond simply counting meetings toward measuring the actual impact of community voices on policy.
The Audit Design Framework
The authors developed a semi-automated pipeline that integrates Information Extraction (IE) via NLP and Knowledge Representation (KR). The process begins with a database of unstructured text, such as focus group reports and official management plans, and moves through a qualitative research framework to create a structured knowledge base.
The system utilizes the GATE (General Architecture for Text Engineering) framework to process text. It employs ANNIE for entity recognition and JAPE (Java Annotation Patterns Engine) to label sentences based on specific categories (e.g., space, society, heritage) and types (e.g., suggestions, requests, policies).
Semantic Mapping and Knowledge Representation
A core innovation of Audit Design is its ability to map "issue assertions" from the community to "management plan actions" using a rule-based approach. This system doesn't just look for keywords; it identifies logical connections. For example, if a community requests "transportation planning" (Request) and the plan includes "preparing circulation plans for cyclists" (Intervention), the system recognizes this as a successful knowledge transfer based on the shared "Space" category.
One of the most significant findings was that a simple binary scale (transferred vs. not transferred) was insufficient. The automated system helped identify nuanced "in-between" states, such as p-full (nearly full) and n-partial (nearly none). This allows auditors to see where a policy almost met a community need but fell short on specific details like budget or assigned actors.
Results and Validation
The study used the Diyarbakir Fortress and Hevsel Gardens Cultural Landscape WHS in Turkey as a case study, analyzing 91 community issue assertions and 189 management plan actions. To validate the results, the authors used WEKA (Waikato Environment for Knowledge Analysis), applying canopy clustering and the J48 tree classifier to verify the five-level scale of knowledge transfer.
The "Audit Design" method proved highly reliable when compared to manual human auditing:
- Type Identification: The system achieved 96% precision for identifying issue types and 95% precision for management plan action types.
- Domain Identification: Identifying knowledge domains (categories like "Agriculture" or "Communication") reached 81.6% to 83.5% precision.
- Inference Accuracy: The rule-based system correctly matched community issues to plan actions with 87.5% accuracy.
Conclusion
Audit Design reduces the burden of monitoring community inclusion "enormously," allowing for real-time auditing during the development of management plans. While the current model requires translation for non-English documents and can struggle with multi-clause sentence complexity, it represents a major step toward transparent heritage governance. Future work involves partnering with international bodies like UNESCO to create standardized protocols, allowing Audit Design to be used globally to benchmark community inclusion in conservation policy.
Summary updated