Representation Learning for Taxonomy Maintenance

Traditional taxonomy maintenance is labor-intensive: experts manually review relationships, map new terms, identify inconsistencies. ESCO demonstrates how representation learning can assist this process.

The approach: Train a multilingual language model (XLM-RoBERTa) on labor market data, job advertisements, qualifications databases, existing taxonomies. The model learns to embed occupations, skills, and job titles in a shared vector space where semantic similarity correlates with spatial proximity.

What this enables:

• Mapping assistance: Suggest likely ESCO matches for terms from national classifications
• Outlier detection: Flag concepts whose embeddings don’t cluster with their assigned category
• Relationship discovery: Surface potential occupation-skill connections the experts missed
• Quality validation: Compare expert-built hierarchy against bottom-up patterns in real labor market data

Key insight: The ML model doesn’t replace expert judgment, it surfaces cases that warrant expert attention. Flagging potential issues is cheaper than reviewing everything manually.

Results: 75-94% of expert-validated mappings appear in the model’s top-5 suggestions, depending on data quality.

Related: 01-atom—human-in-the-loop