Preference datasets¶
Preference is a Language type for alignment-style training: each row needs a Prompt, a Chosen response, and a Rejected response.
Create¶
Same modal as other language types: Language → Preference, then upload CSV or Hugging Face on Files.
The type switcher notes that preference data can sometimes feed SFT-style work, but the types stay separate in Arena. If you only have prompt and target text, use SFT instead.
Map columns on Data¶
On Data, the page title is Select Prompt, Chosen, and Rejected Columns.
Assign:
Prompt
Chosen
Rejected
Use the preview table to verify rows. Reset a mapping by clicking its badge again.
Preprocessing¶
Preference datasets do not get a Preprocessing tab. That tab is only for tabular and non-tabular supervised data.
Experiments¶
Advanced Training projects only. The wizard’s dataset step mirrors reasoning: column mapping first, then agent and training steps for algorithms such as DPO. You only need a plan that includes Advanced Training; tabular and non-tabular types additionally require an Enterprise plan.