PANPersonality-conscientious

Prepared for gender-fairness (TPR-gap) evaluation of fine-tuned language models.

Schema

column	type	meaning
text	str	input text
label	str	task class name (Y), used as the generation target in prompted fine-tuning
label_id	int	contiguous class id (Y)
gender	str	protected attribute (Z), female or male
group_id	int	0 = female, 1 = male

Splits: train, dev (aliased as validation), test.

Key statistics

• Examples: 2,736
• Classes: 2
• Test P(female|Y) range: 0.392 - 0.615
• Test Cramer's V (Y vs Z): 0.223
• Smallest test Y x Z cell: 103

Full per-class Y x Z tables are in dataset_stats.json.

Provenance

• corpus_root: data/pan15
• languages: ['test', 'train']
• citation: Rangel, Celli, Rosso, Potthast, Stein, Daelemans (2015). Overview of the 3rd Author Profiling Task at PAN 2015. CLEF.
• labels_source: conscientious from truth.txt, binarized (median)
• gender_source: self-reported by the author in the online BFI-10 test
• trait_scale: BFI-10 self-assessment, scores in [-0.5, 0.5]
• unit: chunk
• distinct_authors: 294
• sample_size_caveat: PAN 2015 has 152 English / 110 Spanish / 38 Italian / 34 Dutch training authors. Splits are author-disjoint, so the effective sample size is the author count, not the tweet count.
• cleaning: {'input_rows': 2761, 'dropped_unusable_gender': 0, 'dropped_too_short': 25, 'dropped_duplicate_text': 0, 'output_rows': 2736}

Gender is a binary proxy attribute used only to measure disparate model performance. It is not an endorsement of a binary model of gender, and for some sources it is inferred rather than self-reported (see provenance above).