---
base_model: allenai/OLMo-3-1025-7B
library_name: transformers
tags:
  - grpo
  - rlhf
  - instruction-following
  - olmo
license: apache-2.0
---

# OLMo-3 7B Instruct-Only (GRPO)

Fine-tuned from [allenai/OLMo-3-1025-7B](https://huggingface.co/allenai/OLMo-3-1025-7B) using GRPO (Group Relative Policy Optimization) on instruction-following tasks.

## Training Details

- **Base model**: allenai/OLMo-3-1025-7B
- **Training method**: GRPO with RL-Zero (no supervised warmup)
- **Dataset**: [allenai/Dolci-RLZero-IF-7B](https://huggingface.co/datasets/allenai/Dolci-RLZero-IF-7B) (IFEval instruction-following)
- **Config**: `if_valley_thinker` — valley length penalty (512–4096 token sweet spot) + think token reward shaping
- **Chat template**: OLMo thinker (prefills `<think>` tag for chain-of-thought reasoning)
- **Precision**: bfloat16
- **Hardware**: 2 nodes × 4 NVIDIA GH200 120GB GPUs (Isambard-AI)

## Reward Components

| Component | Description |
|-----------|-------------|
| IFEval verifiable reward | Binary per-constraint score for instruction-following |
| Valley length penalty | Penalizes responses <512 or >4096 tokens (coeff: -0.001) |
| Think tag reward | +0.125 for correct `</think>` closure |
| Think length penalty | -0.1 if thinking block <10 words |

## Performance (late-stage averages)

| Metric | Value |
|--------|-------|
| IFEval correct rate | 0.88 |
| Training reward | 6.36 |
| Think word count | ~886 words |
| Sequence length | ~1353 tokens |

## Checkpoints

Each training checkpoint is available as a separate branch/revision:
- `main` — step 3800 (latest)
- `step_600` through `step_3600` — intermediate checkpoints (every 200 steps)

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load latest
model = AutoModelForCausalLM.from_pretrained("camgeodesic/olmo3-7b-instruct-only")

# Load specific checkpoint
model = AutoModelForCausalLM.from_pretrained("camgeodesic/olmo3-7b-instruct-only", revision="step_2000")
```