Vehicle vs. Content Distinction

In representation theory, there’s a fundamental distinction between the vehicle and the content of a representation:

Representational vehicle: The internal symbol, signal, or aspect of activity whose function is to encode content. In neural networks, this might be a neuron, an attention head, or a direction in activation space. Vehicles enter into causal-computational relations with other vehicles.

Representational content: The task-relevant external condition (object, property, relation, proposition) that is represented by a vehicle. This is what makes sense of the cascade of causal interactions between vehicles.

The term “feature” in ML conflates these two senses — sometimes referring to an internal component (vehicle), sometimes to a property of input (content).

Keeping this distinction clear enables better research questions:

• “What contents do models learn to represent?” (e.g., does the model represent causal information or just syntactic patterns?)
• “What are the vehicles of content?” (e.g., when do models develop representational vehicles aligned with non-basis directions vs. individual neurons?)

Related: [None yet]