semantic-naming-and-machine-readable-contracts

Semantic naming and machine-readable contracts for AI-ready design systems

Status: Research synthesis and evidence record. Production design-system recommendations are intentionally deferred pending direct enum-value evaluation.

Document type: Research synthesis and evidence record (v1.0, reconciled)
Scope: Empirical and adjacent evidence (2023–2026) on how LLMs select components, properties, and enum values from schemas; constraint representation; structured generation; design-system and standards practice; and a proposed controlled Button evaluation focused on a single semantic axis (action hierarchy).

This record reconciles the prior canonical draft with additional deep-research reports on naming, decision boundaries, retrieval, constraints, and design-system case studies. Earlier overclaims (especially treating tool/parameter naming as enum-value evidence) remain corrected.

---

1. Research purpose

The underlying design-system problem is not merely “adding AI features” to an existing component library. It is making the system explicit enough that both people and machines can understand:

• intent;
• rules;
• behaviour;
• limits;
• component selection;
• property selection;
• variant (enum-value) selection;
• valid composition.

When an LLM agent is given a design-system schema and a user intent, it must choose the correct component, fill the correct properties, and emit a valid instance. Failures take several forms: wrong component, wrong property interpretation, wrong enum value, invented values, invalid combinations, and high-confidence errors under name–description conflict.

Two major research areas structure this record.

1.1 Semantic naming

How names attached to components, properties, parameters, and enum values affect an LLM’s ability to choose correctly. The central unresolved question is whether (and under what conditions) meaningful labels outperform opaque identifiers, and whether familiar but locally misleading labels are worse than opaque ones—especially when behavioural descriptions are held fixed.

1.2 Machine-readable contracts

How structured semantics, constraints, descriptions, types, examples, and deterministic validation affect generation and composition. The term is not reduced to a serialization format (JSON vs YAML vs prose). It is decomposed into information classes: names, descriptions, types, enumerated choices, structural hierarchy, constraints, examples, and validation feedback.

A third related concern—decision-boundary metadata and retrieval units—appears throughout the wider research trail: when options overlap, name + definition alone is often insufficient for reliable selection; contrastive boundaries, alternatives, and decision-local retrieval become relevant. These topics are adjacent to naming and are treated as such unless controlled evidence directly tests them.

---

2. Research questions

Naming

• Do component/tool names affect selection when descriptions stay fixed?
• Do property/parameter names (including JSON schema keys) affect interpretation?
• Do enum-value names affect selection when descriptions stay fixed?
• Are meaningful labels better than opaque identifiers?
• Is a familiar but misleading name worse than an opaque identifier?
• Can descriptions override a misleading pretrained association?
• Does the property name change interpretation of the same enum values?
• Do effects vary by model size or family?
• Can models reliably learn arbitrary local labels from definitions?
• Do models continue to infer meaning from labels even when instructed otherwise?

Machine-readable contracts and selection

• What information actually needs to be machine-readable?
• What do names, descriptions, types, enums, constraints, examples, and validation each contribute?
• What should guide selection before generation versus be enforced after generation?
• Do hard constraints hurt semantic correctness?
• Does delayed validation or constrained packaging perform better?
• Is a concise schema + validator preferable to a large ontology in context?
• How should rules such as requires, conflictsWith, exactlyOneOf, onlyValidInside, and mustAccompany be represented?
• What is the smallest useful retrieval unit for component/variant decisions?
• When are useWhen / avoidWhen boundaries needed beyond name + definition?

---

3. Terminology and levels of semantics

These identifier classes and alignment forms are kept separate throughout. Evidence about one class is never treated as direct evidence about another.

3.1 Component / tool name

Examples: Button, create_calendar_event, BOOK /reservations.

The discrete named action or component the model selects from a catalog.

3.2 Property / parameter name (including schema keys)

Examples: variant, intent, priority, tone; also JSON keys such as answer vs instructional key wording.

The slot name under which values are filled. Changing the property name or schema key may change how the same value set is interpreted.

3.3 Enum-value name

Examples: primary, secondary, destructive, v1, v2.

The concrete value chosen for a property. This is the class for which direct controlled evidence remains missing.

3.4 Enum-value description

Example: primary: "Use for the single main action on the page."

Natural-language text attached to a specific value. Distinct from property-level description and from the value’s name.

3.5 Human-semantic alignment

A design-system practitioner judges that the name accurately describes the concept’s behaviour.

3.6 Model-semantic alignment

The name’s likely pretrained meaning (elicited zero-context from the model) agrees with the behaviour required by the local schema.

3.7 Local-system alignment

The identifier agrees with the meaning assigned by this specific design system’s documentation and rules.

Why these three may conflict.
A local system might use priority="low" to mean visually subtle, while models interpret priority as business urgency. Or a value named secondary might be locally bound to a transaction-commit action (as in some product-specific APIs), while pretrained priors treat secondary as “supporting / lower priority.” Human designers may approve the local definition while the model’s pretrained associations pull toward a different behaviour. Carbon’s secondary button (paired negative action such as Cancel/Back) differs from Shopify’s secondary (medium-emphasis supporting action), illustrating cross-system lexical conflict for the same string [DS-Carbon][DS-Shopify].

---

4. Initial practitioner evidence

Practitioner claims on X and related channels motivated deeper empirical investigation. They are recorded as leads, not controlled evidence. Where direct post URLs were not preserved, provenance is marked unresolved.

Person	Handle	Claim (summary)	Evidence class	Provenance
Cédric	@cedric_design	Name layers for the agent; design system is the real prompt	Practitioner observation	URL unresolved; X ~Aug 2026
Gavin Elliott	@gavinelliott	Machine-readable constraints vs human-only docs	Practitioner observation	URL unresolved; X 2026
Hemant Agarwal	@sudohemant	AI-native systems need prop semantics, override rules, edge cases	Practitioner observation	URL unresolved; X ~Aug 2026
John Rodrigues	@john_rodrigues_	Component contracts; JSON more reliable than YAML in informal tests	Practitioner experiment / lead	URL unresolved; X 2026
Amitanshu Sahu	@amitanshusahu	GeckoFX agent component selection demo	Demo	https://github.com/geckostacklab/GeckoFx
Julie Loves Tech	@JulieLovesTech	Figma agent on mature system; token accuracy when names specified	Practitioner experiment	URL unresolved; details not re-verified
Adham Dannaway	@AdhamDannaway	Figma agent documentation demo	Demo	URL unresolved
Anand Chowdhary	@AnandChowdhary	Component libraries as part of the model prompt	Practitioner observation	URL unresolved
Siarhei Tarasenka	@starasenka	AI as third consumer of design systems	Practitioner observation	URL unresolved
cammy	@caomicc	Naming contract to stop agent breakage	Practitioner observation	URL unresolved

Patterns: semantic naming repeatedly claimed helpful; system maturity associated with better agent outcomes; richer machine-readable contracts preferred; measured quantitative evidence for variant selection sparse.

---

5. Evidence standard

1. Peer-reviewed controlled experiments
2. arXiv / preprint controlled experiments with clear methodology
3. Reproducible benchmark papers
4. Detailed technical reports (e.g. SSRN + open harness)
5. Standards / specifications
6. Vendor documentation and production design-system practice
7. Practitioner experiments
8. Anecdotal claims and demos

Demonstrated: Direct controlled evidence for the stated claim.
Adjacent: Relevant evidence from a related task or representation.
Unestablished: No adequate direct evidence found.

Adjacent evidence is never promoted to direct evidence. In particular: tool/component name ≠ enum-value; parameter name ≠ value selection; structural validity ≠ semantic correctness.

---

6. Empirical evidence table

Verified numbers prefer primary sources. Identifier class is stated explicitly.

#	Source	Year	Status	Authors	Models	Task	Manipulated	Identifier class	Fixed	Descs?	Metric	Verified result	Comp.	Prop.	Enum val.	Class
[1]	PA-Tool	2025–26	Accepted ACL 2026 / arXiv:2510.07248	Lee et al.	SLMs + closed	Tool sel. + param ID	Rename tool + param names only	Tool + parameter names	Descs, examples, tasks	Yes	Acc.; misalignment	Up to ~17% tool-sel gains; misalignment ↓80.0%	High	Medium	None direct	Demonstrated (names)
[2]	RoTBench	2024	EMNLP 2024 / arXiv:2401.08326	Ye et al.	6 incl. GPT-4	Tool sel., param ID, fill	Name noise levels	Tool + parameter names	Functionality	Yes	Acc. by phase	GPT-4 Clean 80.00 → 58.10 under noise	High	Medium	None direct	Demonstrated
[3]	Hood method naming	2026	SSRN + author report	Chris Hood	Claude, Grok, GPT-4o, Llama-3.2-3B	Endpoint selection	Agentic vs CRUD names; desc-swap	Method / tool names	Catalog, tasks	Yes / swapped	Acc.; confidence	Frontier +10–29 pp; z=3.77, p<0.001; Llama-3.2 0 pp; swap −39 to −43 pp; conf. ~87–88% at ~30% acc	High	Low	None	Strong tech report; non-peer-reviewed
[4]	ToolTweak	2025	arXiv:2510.02554	per arXiv	Qwen, DeepSeek, etc.	Tool selection	Name + description jointly	Tool name (+ desc)	Baseline share	Both edited	Selection rate	~20% → 46–82%	High	Low	None	Adjacent (joint IV)
[5]	When Names Disappear	2025	arXiv:2510.03178	per arXiv	GPT-4o, DeepSeek, etc.	Code sum. + exec.	Identifier obfuscation	Code identifiers	Behaviour	N/A	Pass@1	e.g. 85.7→76.1; 85.4→71.2	Low	Low	Adjacent	Adjacent
[6]	Semantic Anchors	2025	arXiv:2511.21038	Krishna Kumar	8 models, 1–12B	Classification	Inverted label mappings	Classification labels	Inputs	N/A	Override rate	Override rate exactly 0% (320 conditions)	Adjacent	Adjacent	Adjacent	Adjacent (not schema enums)
[7]	Constraint Tax	2026	arXiv:2605.26128	Jaideep Ray	Qwen2.5-0.5/1.5B, SmolLM2-1.7B	Structured answer; calendar tool	Hard schema vs prompt JSON	Structure / enum enforcement	Instances fixed	Schema present	Validity; answer/exec	Validity 61.5→100%; answer 19.7→11.0%; calendar exec 91.5→48.0%	Medium	Medium	Validity only	Demonstrated (tax)
[8]	Hammer	2024	Preprint / related	Lin et al. (per reports)	Qwen, DeepSeek sizes	Tool selection + args	Mask/replace function & param names in training	Tool + parameter names	Descs emphasized	Yes	Robustness	Misleading names identified as major generalization issue; randomizing names shifts reliance to descriptions	High	Medium	None direct	Adjacent (training-time)
[9]	Schema-Key Wording	2026	Preprint	Le (per reports)	Qwen2.5-7B, Llama-3.2-3B, others	Structured gen. under constrained decoding	JSON key wording only	Property / schema keys	Model, task, field order, types, shape, decoder	Keys vary	Accuracy	Qwen2.5-7B 79.61→86.50 on GSM8K key-only; Llama-3.2-3B 53.15→37.38 (opposite direction)	Low	High	None	Demonstrated (keys)
[10]	Live API Bench	2025	Per reports	—	Various	API selection + slots	Opaque FUNC_0, ARG_1 vs natural names	Tool + slot names	Descriptive info retained	Yes	Selection / slot	Severe drops under obfuscation on some tasks	High	Medium	None direct	Adjacent / demonstrated for opacity
[11]	ACEBench	2025	Per reports	—	Multiple	Tool + value matching	Separates enum param class among types	Evaluates enum values as task	N/A (no name ablation)	—	Exact value match	Enum filling is a distinct subtask; large model/family differences; does not alter enum names	Medium	Medium	Task analogue only	Direct task; weak causal
[12]	IFEval-FC	2025	Per reports	—	GPT, Claude, GigaChat families	Parameter-value generation	Verifiable rules in parameter descriptions	Description channel	Tool choice done	Yes (rules in desc)	Adherence	No model >80% overall; large cross-family variation; enums intentionally excluded	Low	High (desc)	None	Adjacent (desc obedience)
[13]	EASYTOOL	2025	NAACL 2025	—	ChatGPT, GPT-4, etc.	Tool retrieval + use	Long inconsistent docs → concise standardized instructions	Documentation form	Tool set	Transformed	NDCG; param error; success	Token use −70.43%; Ada NDCG@1/@5 45.7/38.8→76.7/85.6; ChatGPT param error 21%→6% with one example	High (docs)	Medium	Low	Demonstrated (doc form)
[14]	JSONSchemaBench	2025	Per reports	—	Constrained-decoding systems	Schema adherence	Grammar/schema constraints	Structure	Schemas	—	Adherence; downstream	Constrained decoding improves adherence; downstream gains modest (~up to ~4 pp in reported settings)	Medium	Medium	Validity	Demonstrated (validity)
[15]	D3ST	—	Per reports	Zhao et al.	—	Schema-guided dialogue	Intent/slot names vs natural-language descriptions	Intent/slot labels	Task	Descriptions vs names	Understanding; transfer	Names may fail to convey semantics; description-driven representation improved robustness and transfer	Adjacent	Adjacent	Adjacent	Adjacent
[16]	Chen et al. relevance	2025	NAACL 2025	Chen et al.	—	Function-call relevance / abstention	Tool-role separation; non-call training cases	Presentation + negative evidence	—	—	Relevance detection	Tool-role format 39.58%→49.58%; +non-call cases up to 57.50–65.42%	Adjacent	—	—	Adjacent (boundaries)
[17]	Tool Documentation	2023	Per reports	—	—	Zero-shot tool use	Documentation vs few-shot demos	Descriptions	—	Yes	Tool-use performance	Docs alone comparable to few-shot; more valuable with large API sets	Adjacent	Adjacent	—	Adjacent
[18]	MFTR / multi-field retrieval	2025–26	Per reports	—	—	Tool retrieval	Field masking; multi-field scoring	Documentation fields	—	—	NDCG; Recall	Fields contribute unequally; joint multi-field > single field; full-doc baseline weaker	Adjacent	—	—	Adjacent (retrieval)
[19]	SchemaRAG	—	Per reports	—	—	IE / structured	Retrieve schema rows not full schema	Schema portions	—	—	Task metrics	Reduced schema + examples can outperform full schema dump	Adjacent	Adjacent	Adjacent	Adjacent
[20]	Production skill / ToolBench rewrite	2026	Per reports	—	—	Skill routing	FP/FN-driven description rewrite	Description boundaries	—	—	F1	Full / FP / FN feedback ~+4.5 F1; single rewrite often enough	Adjacent	—	—	Adjacent
[21]	Description-edit sensitivity	2025	EMNLP 2025 (per reports)	—	17 models incl. GPT-4.1, Qwen2.5-7B	Tool routing	Controlled description edits	Descriptions	—	Edited	Tool use rate	>10× change in tool use for some models	Adjacent	—	—	Adjacent (desc fragility)

Note: Delayed constrained-decoding / draft-then-constrain claims remain provisionally supported where exact primary titles were not fully preserved across research passes; treat as adjacent architecture evidence until re-verified.

---

7. Source-by-source analysis (core causal sources)

7.1 PA-Tool [1]

Renames only tool and parameter names; descriptions, examples, and instructions unchanged. Up to ~17% tool-selection gains; schema-misalignment errors ↓80.0%. Establishes causal effect of tool/parameter names. Does not isolate enum-value labels.

7.2 RoTBench [2]

Name noise on tools and parameters. GPT-4 tool selection 80.00 → 58.10 under union noise. Establishes name robustness as causal. Not enum-value isolation.

7.3 Hood [3]

Intent-aligned method names vs CRUD; description-swap. Frontier +10–29 pp; ~3B null; name–description conflict −39 to −43 pp with high confidence at low accuracy. Strong technical report, not peer-reviewed journal equivalent. Not enum values.

7.4 Hammer [8]

Training-time masking of function/parameter names to force description reliance. Identifies misleading familiar naming as a generalization problem. Adjacent to pretrained-prior issues; intervention is training-time, not inference-time enum ablation.

7.5 Schema-Key Wording [9]

Only JSON key wording changed under constrained decoding. Accuracy moved substantially and in opposite directions across model families (Qwen up, Llama down). Direct evidence that property/schema keys are an instruction channel. Not enum-value names; task was structured maths-style generation, not design-system value selection.

7.6 Live API Bench [10]

Opaque FUNC_0 / ARG_1 vs natural names with documentation retained. Natural names supply useful signal. Tool and slot names changed together.

7.7 ACEBench [11]

Evaluates enum-valued parameters as a distinct atom task. Confirms value selection is a real stage. Does not experimentally change enum labels.

7.8 IFEval-FC [12]

Rules embedded only in parameter descriptions; adherence imperfect (<80% overall). Descriptions influence values but are not reliably obeyed. Enum fields excluded.

7.9 EASYTOOL [13]

Concise standardized instructions cut tokens ~70% while improving retrieval and use; one example sharply reduced parameter error. Documentation form matters; not a naming ablation.

7.10 Semantic Anchors [6]

Inverted classification labels; override rate 0% at 1–12B. Mapping mismatch: classification labels ≠ schema enum values; few-shot inversion ≠ schema descriptions; not frontier-only.

7.11 Constraint Tax [7]

Hard schema constraints: validity up, semantic/executable accuracy down on SLMs (e.g. calendar exec 91.5%→48.0%). Validity ≠ correctness.

7.12 ToolTweak [4]

Joint name+description optimization. Cannot isolate name-only effects.

---

8. Findings: component and tool naming

Demonstrated

• Tool/component names causally affect selection with descriptions fixed [1][2][3].
• Opaque/noisy/anti-semantic names reduce performance [2][10].
• Intent-aligned method names can outperform conventional names at frontier scale [3].
• Name–description conflict can cause large errors and overconfidence [3].
• Schema misalignment (plausible non-schema names) reduced by name alignment [1].

Adjacent

• Training-time name masking improves robustness by shifting reliance to descriptions [8].
• Joint name+description optimization can dominate selection rates [4].
• Code identifier obfuscation hurts understanding [5].

Unestablished

• Nothing major for the existence of tool/component name causality; magnitudes depend on catalog, model, and docs.

---

9. Findings: property and parameter naming

Demonstrated

• Parameter names affect parameter identification [1][2].
• Schema-key wording alone can change structured-generation accuracy, with model-family-dependent direction [9].

Adjacent

• Live API Bench slot-name opacity [10]; Hammer parameter-name priors [8]; D3ST slot/intent names [15].

Unestablished

• Whether design-system property names (variant vs tone vs intent vs opaque) change interpretation of an identical enum-value set and descriptions. Schema-key evidence is strong adjacent support that field names are not inert, but not the exact DS selection experiment.

---

10. Findings: enum-value naming

Plain statement: No verified controlled experiment in this research directly isolates enum-value names inside a fixed schema while keeping property name, descriptions, behaviour, task, and available values constant.

Demonstrated

None for schema enum-value selection under isolation criteria.

Adjacent

• Semantic Anchors: pretrained labels resist override [6].
• Tool/method naming causality [1][2][3].
• ACEBench: enum filling is a real distinct subtask [11].
• Hammer / RoTBench: familiar-wrong names disruptive in tool settings [8][2].
• Practitioner DS claims (§4).

Unestablished

• Semantic vs opaque enum labels with fixed descriptions.
• Misleading familiar vs opaque enum labels.
• Description override of misleading value names.
• Property-name × enum-value-name interaction.
• Enum-set-size effects; model-size effects specific to enum values.

This is the central naming evidence gap identified in this research.

---

11. Semantic conflict and pretrained meaning

Type	Definition
Aligned	Model prior matches local behaviour
Opaque	Little useful prior
Misleading	Model prior conflicts with local behaviour

Adjacent findings: override rate 0% under inverted ICL labels [6]; name–description conflict produces overconfident errors [3]; shuffled real names can be more disruptive than nonsense in some RoTBench conditions [2]; Hammer treats familiar training associations as a misalignment source [8].

Limit: Does not prove misleading enum values are worse than opaque enum values in a design-system schema. That is hypothesis H2 (§22).

---

12. Can descriptions override names?

Tool/method names: Names can dominate weak descriptions; weak names do not survive strong mismatched descriptions [3]; name alignment helps with descriptions fixed [1].

Parameter descriptions: Can carry executable rules but imperfectly obeyed [12]; description edits can change tool use by >10× [21].

Enum values: Unknown—no controlled isolation of per-value descriptions against misleading vs aligned vs opaque value names.

Boundary metadata (adjacent): Concise normalized docs help [13]; collision-focused rewriting helps routing [20]; useWhen/avoidWhen are standard platform guidance but lack isolated field A/B for design-system variant selection.

---

13. Model-size and model-family effects

Finding	Source
Method-name effect at frontier, absent at ~3B	Hood [3]
SLMs benefit from name alignment	PA-Tool [1]
Schema-key intervention can help one family and hurt another	Schema-Key [9]
Semantic override fails 1–12B ICL	Semantic Anchors [6]
Constraint tax severe on sub-3B	Constraint Tax [7]

Effects may not scale monotonically. Family and post-training appear at least as important as size [9][1]. Enum-name × capability interaction is exploratory.

---

14. Machine-readable contracts: information classes

Class	Role	Evidence in this record
Names	Identify/select concepts	Causal for tools/methods/params/keys [1][2][3][9]; untested for enum values
Descriptions	Boundaries, filling	Strong channel [12][13][17]; fragile [21]; untested as enum override
Types	Shape	Validity support [14]
Enumerated choices	Restrict values	When enforced, block inventions [7][14]; can impose semantic cost [7]
Structural hierarchy	Composition	Expressiveness high; agent-selection benefit under-tested
Constraints	Valid combinations	Expressible in many formalisms (§15); showing vs enforcing under-tested for DS
Examples	Grounding	Help execution more than retrieval in some studies [13]; mixed retrieval value
Validation feedback	Repair	Common practice; DS-specific effect not established here

---

15. Constraint representation

Adjacent systems (expressiveness ≠ proven agent-selection benefit):

• JSON Schema: enum, const, dependentRequired, if/then/else, oneOf/anyOf/allOf/not
• SHACL: graph shapes, paths, severity, validation reports
• Feature/configuration models: requires/excludes, cardinality, SAT analysis
• Discriminated unions / type systems: local illegal states unrepresentable
• Policy-as-code (e.g. Cedar/OPA): context-dependent allow/deny
• Planning preconditions/effects: workflow-stage gates
• Lint / static analysis: post-hoc detection

Critical separation: A formalism can express a rule without that rule improving LLM selection when shown in-prompt. Hard validity should be evaluated as generation + deterministic validation. Constrained decoding prevents illegal tokens; it does not choose the correct legal value [7][14].

Candidate architecture (hypothesis): canonical rule model → context-specific selection projection → structured generation → full deterministic validation → runtime gate for live state.

---

16. Structured generation and the constraint tax

• Hard constraints can raise validity to 100% while lowering semantic/executable accuracy on SLMs [7].
• Wrong-but-valid outputs can rise sharply [7].
• JSONSchemaBench: adherence up; downstream gains modest [14].
• Schema-valid ≠ semantically correct (also reflected in transaction/structured benchmarks in the wider trail).

Generating a valid structure is not the same as choosing the semantically correct values.

---

17. Guidance before generation vs validation after generation

Candidate architecture informed by current evidence (not a proven optimum):

1. Expose selection-relevant semantics (names, clear boundaries, small relevant schema).
2. Let the model make semantic choices.
3. Validate deterministic rules separately.
4. Repair with structured feedback when needed.

Adjacent support: constraint tax [7]; IFEval-FC imperfect description obedience [12]; UI-linting hybrid practice; configuration systems that prune then validate.

Untested head-to-head for design systems: large ontology in prompt vs concise semantic schema vs hard constrained generation vs schema + validator vs examples-only vs source code.

---

18. Decision boundaries and retrieval (adjacent synthesis)

From tool-routing and retrieval research (not DS-specific causal proof):

• Name + definition is a useful discovery primitive but a weak selection contract when candidates overlap [13][16][18].
• Collision-focused boundaries (useWhen / avoidWhen / alternatives) are standard in platform guidance and mature DS docs (Carbon, Fluent, Polaris); isolated runtime field A/B for agents is sparse.
• Smallest useful retrieval unit is unlikely to be a full page or an isolated value; stronger hypothesis is a decision-complete packet (item + parent context + closest rivals + hard applicable constraints) [18][19].
• Progressive disclosure: pre-selection boundaries; post-selection implementation detail.
• Position effects (“lost in the middle”) and hard negatives in RAG argue against dumping full docs [adjacent long-context work].

These points inform evaluation design; they are not demonstrated design-system agent results.

---

19. Standards and design-system case studies

Separate from causal LLM experiments.

Standards / specifications

Standard	Relevant principle	Role here
DTCG Design Tokens	Explicit $type; do not infer type/purpose from arbitrary groups or value inspection	Machine semantics should be declared, not reverse-engineered from names/hierarchy
WAI-ARIA	Role / state / property separation	Precedent for factoring interaction semantics
MCP Tool schema	name, title, description, JSON Schema, behavioral annotations (destructiveHint, etc.)	Operational behaviours deserve explicit fields
SKOS	Concept ≠ label; preferred/alt/hidden labels	Stable identity with multiple surface forms
OpenAPI	Structured operation/parameter metadata	Structured semantics for dual human/machine consumption
JSON Schema	Assertions vs annotations; enum/const/conditionals	Hard validity vs guidance

Design-system practice (illustrative)

System	Pattern	Implication for agents
Shopify Polaris Button	variant = visual hierarchy; tone = impact; separate behaviour props	Factorization reduces axis mixing
Carbon Button	secondary = paired negative action; ghost can be functional primary in context	Local meaning can diverge from common prior
Primer / Atlassian / Fluent / Material	Mixed hierarchy, appearance, severity in appearance systems to varying degrees	Compact human labels often compress multiple dimensions

These do not prove which enum strings maximize LLM accuracy. They show that production systems already struggle with overloaded variant namespaces and that factorization is a recurring structural response.

---

20. Evidence maturity matrix

Row	Direct causal	Adjacent	Unestablished
Component/tool name	[1][2][3][10]	[4][8]	—
Property/parameter/schema key	[1][2][9]	[8][10][15]	DS property × same values
Enum-value name	None	[6][11][2][8]	Aligned/opaque/misleading; override
Enum-value description	None isolated	[12][3]	Per-value override
Types / enums as constraints	Validity [7][14]	—	—
Structural constraints	Expressiveness §15	—	Selection benefit of showing rules
Validation	Practice	—	Optimal DS architecture
Examples	[13] mixed	—	Coverage limits
Decision-boundary fields	—	[13][16][20] platform guidance	Structured vs equivalent prose
Retrieval unit	—	[18][19]	DS decision packet

---

21. Mapping to AI-ready design systems

Target task: Given user intent and a Button schema, select correct property values and emit a valid instance.

Evidence	Mapping	Unsafe inference
Tool/method naming [1][2][3]	High for component choice	“Therefore primary beats v1”
Schema keys [9]	High for property naming	Exact ranking of variant/tone/intent
Semantic Anchors [6]	Adjacent to misleading values	Proof that misleading < opaque in DS
Constraint tax [7]	Validity half of task	Hard constraints always help agents
DS case studies §19	Motivation / factorization patterns	Production naming policy without measurement
Practitioner §4	Leads	Controlled effect sizes

---

22. Corrected conclusions

Supported

• Component/tool naming can causally affect selection with descriptions fixed [1][2][3].
• Parameter naming and schema-key wording can causally affect identification/generation [1][2][9].
• Name–description conflict can create severe errors and overconfidence [3].
• Hard structural constraints can improve validity [7][14].
• Hard constrained decoding can reduce semantic correctness in some small-model settings [7].
• Concise normalized documentation can improve tool retrieval/use and cut tokens [13].
• Description-embedded rules are imperfectly obeyed [12].

Provisionally supported / adjacent

• Delayed constraint application may recover structured accuracy (sources not fully pinned in this trail).
• Boundary-focused documentation and decision-local retrieval likely help when options overlap (§18).
• Familiar-wrong names may be worse than opaque in some tool settings [2][8]—not established for enum values.

Not established

• Semantic enum names are better than opaque enum names.
• Misleading familiar enum names are worse than opaque IDs.
• Strong per-value descriptions reliably override misleading enum names.
• variant / intent / tone are predictably different for value selection when values and descriptions are fixed.
• A specific machine-readable contract architecture is optimal for design-system agents.
• JSON is inherently better than YAML/XML/prose once information content is controlled.
• Production systems must rename primary/secondary for AI (risk is treating compact labels as complete machine semantics, not the words themselves).

---

23. Research gaps

1. Largest naming gap: Enum-value naming not isolated in a Button-like schema-selection task.
2. Property-name semantics with fixed value sets and descriptions.
3. Misleading vs opaque value names.
4. Description override of conflicting value names.
5. Model-size/family interaction for enum values (exploratory).
6. Structured useWhen/avoidWhen fields vs equivalent prose at matched information content.
7. Optimal retrieval unit for component variants.
8. Constraint guidance in-prompt vs deterministic post-validation for DS composition.
9. Confidence calibration under naming conflict.

---

24. Proposed Button evaluation

Status: Proposed experiment. Not established evidence.

Scope of first study: Single semantic axis — Button action hierarchy — to avoid confounding hierarchy with impact, navigation, form behaviour, or visual style.

Research question

When an LLM must select Button hierarchy values from a fixed component schema, how do semantic alignment, opacity, and semantic conflict in property and enum names affect selection accuracy?

Hypotheses

• H0: Enum-value name type has no effect when behavioural descriptions are identical.
• H1: Aligned names differ from opaque and/or misleading names.
• H2: Misleading familiar names yield lower accuracy than opaque identifiers under identical descriptions.
• H3: Richer descriptions reduce (may not eliminate) name-type gaps.
• H4: Property name affects value selection with values and descriptions fixed.
• H5: Enum-name effects interact with model capability and family; direction/magnitude exploratory.
• H6: Under misleading names, a measurable share of errors are prior-capture.

Canonical concepts (hierarchy only)

ID	Definition
C1	Main action in the current surface; at most one per view region
C2	Supporting affirmative action; not the sole primary goal
C3	Optional / low-priority action

Destructive, navigation, cancel, submit axes: future replications.

Factors

• E: aligned / opaque / misleading enum labels (descriptions identical; labels set via pre-treatment elicitation).
• D: none / concise / full + when/avoid.
• P: property name block (aligned / opaque / conflicting) at fixed D.
• Optional constraint delivery block.

Controls and metrics

Randomize order; ban label leakage in user tasks; no few-shots in main run; primary metric = exact concept-ID match; secondary = validity, inventions, invalid combos, confidence, consistency, tokens.

Failure taxonomy

P1 prior capture; P2 description ignore; P3 property misread; P4 invented; P5 valid wrong combo; P6 lexical match to user; P7 schema misunderstanding; P8 residual.

What patterns would establish

Pattern	Interpretation
E1 > E2 > E3	Misleading worse than opaque (H2)
E1 ≈ E2 > E3	Only conflict hurts
Gaps vanish at D2	Descriptions largely override
Gaps persist at D2	Names remain independent channel
High P1 under E3	Prior-driven mechanism

No production PDS naming recommendation follows until results exist.

---

25. Implications for future research

Can inform now: component/parameter/schema-key names are causal levers; name–description conflict is dangerous; validity and semantic correctness must be measured separately; concise structured docs can outperform noisy long docs for tool use.

Must still test: enum-value naming under fixed descriptions; property-name effects on identical value sets; description override; guidance+validator vs hard constraints for DS composition; decision-packet retrieval for variants.

Frame next work as evaluation questions, not production decisions. Candidate approaches (factorized semantics, decision cards, resolvers, legacy aliases) are hypotheses to test, not demonstrated requirements.

---

26. References

Empirical / technical

[1] Jonggeun Lee, Woojung Song, Jongwook Han, Haesung Pyun, Yohan Jo. Don’t Adapt Small Language Models for Tools; Adapt Tool Schemas to the Models. Accepted ACL 2026. arXiv:2510.07248. https://arxiv.org/abs/2510.07248

[2] Junjie Ye et al. RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning. EMNLP 2024. arXiv:2401.08326. https://arxiv.org/abs/2401.08326

[3] Chris Hood. Semantic method naming / agentic vs CRUD controlled benchmark. SSRN 2026. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6460766 ; https://chrishood.com/grammar-over-vocabulary-why-crud-fails-apis-for-agents/

[4] ToolTweak: An Attack on Tool Selection in LLM-based Agents. arXiv:2510.02554. https://arxiv.org/abs/2510.02554

[5] When Names Disappear: Revealing What LLMs Actually Understand About Code. arXiv:2510.03178. https://arxiv.org/abs/2510.03178

[6] Anantha Padmanaban Krishna Kumar. Semantic Anchors in In-Context Learning: Why Small LLMs Cannot Flip Their Labels. arXiv:2511.21038. https://arxiv.org/abs/2511.21038

[7] Jaideep Ray. The Constraint Tax: Measuring Validity-Correctness Tradeoffs in Structured Outputs for Small Language Models. arXiv:2605.26128. https://arxiv.org/abs/2605.26128

[8] Hammer (function/parameter name masking for tool-calling robustness). 2024; full bibliographic details as listed on primary preprint when re-verified.

[9] Schema-Key Wording study (Le, 2026 preprint per research trail). Key-only wording under constrained decoding; re-verify exact title/arXiv on archival use.

[10] Live API Bench (2025 per research trail). Opaque API/slot names; re-verify exact citation for archival use.

[11] ACEBench (2025 per research trail). Atom tasks including enum parameters; re-verify exact citation.

[12] IFEval-FC (2025 per research trail). Verifiable constraints in parameter descriptions; re-verify exact citation.

[13] EASYTOOL. NAACL 2025. Concise tool instructions; token and accuracy results as reported in research trail.

[14] JSONSchemaBench (2025 per research trail). Constrained decoding over real JSON schemas. Related: arXiv:2501.10868 discussed in trail.

[15] D3ST (Zhao et al.). Description-driven schema-guided dialogue; intent/slot names vs descriptions.

[16] Chen et al. NAACL 2025. Function-call relevance detection / abstention.

[17] Tool Documentation study (2023 per research trail). Zero-shot tool use with documentation.

[18] MFTR / multi-field tool retrieval work (2025–26 per research trail).

[19] SchemaRAG (per research trail). Schema-row retrieval.

[20] Production skill description optimization / ToolBench open-world subset (2026 per research trail).

[21] EMNLP 2025 description-edit / tool-use sensitivity study (per research trail; 17 models).

Standards and design systems

[DS-DTCG] Design Tokens Community Group. Design Tokens Format Module (e.g. 2025.10 materials in trail). Explicit $type; no inference from arbitrary groups.

[DS-ARIA] WAI-ARIA. Roles, states, and properties.

[DS-MCP] Model Context Protocol Tool specification (2026 materials in trail).

[DS-SKOS] SKOS Simple Knowledge Organization System.

[DS-Shopify] Shopify Polaris Button API: variant hierarchy vs tone impact.

[DS-Carbon] IBM Carbon Button guidance: secondary as paired negative action; ghost contextual primary.

[DS-Primer] GitHub Primer Button variants.

[DS-Atlassian] Atlassian button appearance / token grammar materials in trail.

[DS-Fluent] Microsoft Fluent Button / Checkbox guidance.

[DS-Material] Material Design 3 button styles.

Practitioner

See §4; most X URLs unresolved in this trail. GeckoFX: https://github.com/geckostacklab/GeckoFx

---

27. Research provenance and correction log

Issue	Original risk	Correction	Reason
PA-Tool / RoTBench as enum-value evidence	Over-generalization	Restricted to tool + parameter names	Primary methods [1][2]
ToolTweak as name-only	Isolation failure	Joint name+description IV [4]	Method
Hood as peer-reviewed equivalent	Status inflation	Strong controlled technical report; non-peer-reviewed [3]	Venue
Architecture as proven optimum	Recommendation creep	Candidate hypothesis §17	No DS head-to-head
Validity = correctness	Metric collapse	Constraint tax + structured benches [7][14]	Results
Multi-axis Button concepts	Confounded first experiment	Single hierarchy axis C1–C3 §24	Design review
H5 directional size claim	Overfit to Hood	Exploratory interaction; PA-Tool shows SLM gains	[1][3]
“Central gap for AI-ready DS”	Field-wide claim	Central naming gap in this research	Scope
DCCD as supported	Weak provenance	Provisionally supported pending source pin	Trail
Uploaded reports recommending production rename/registry	Prescriptive overreach	Retained as candidate approaches to test	Evidence standard
Carbon/Shopify secondary as LLM experiment	Case study ≠ causal	Standards/case §19 only	Class
Schema-key result as enum-value result	Class collapse	Property/key class only [9]	IV
Semantic Anchors as schema proof	Task mismatch	Adjacent; mapping mismatch documented §7.10	[6]

---

End of research record.