The Competency Question Gap

If you don’t explicitly ask about something during knowledge elicitation, it won’t appear in the resulting structure, regardless of how central it is to the domain.

Researchers created nine competency questions to validate their ontologies: “What proficiencies does a Fighter have?” “Which spells require material components?” None of the interview-derived ontologies could answer them. Only the manually-constructed base truth, built from comprehensive documentation, had the coverage.

The interviews weren’t guided by these questions, so the information never surfaced. Open-ended elicitation captures what experts volunteer, not what they know. Structured questions are required to ensure coverage of specific knowledge areas.

This has implications for any knowledge capture process: the questions you ask define the boundaries of what you’ll learn.

Related: [None yet]