Felladrin/TinyMistral-248M-Chat-v4
Text Generation • 0.2B • Updated • 89 • 32
Dataset Viewer
The dataset viewer is not available for this dataset.
Unexpected token '<', "<html>
<h"... is not valid JSON
Need help to make the dataset viewer work? Make sure to review how to configure the dataset viewer, and open a discussion for direct support.
CohereForAI/aya_dataset in ChatML format, ready to use in HuggingFace TRL's SFT Trainer.
Python code used for conversion:
from datasets import load_dataset
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("Felladrin/Llama-160M-Chat-v1")
dataset = load_dataset("CohereForAI/aya_dataset", split="train")
def format(columns):
messages = [
{
"role": "user",
"content": columns["inputs"].strip(),
},
{
"role": "assistant",
"content": columns["targets"].strip(),
},
]
return { "text": tokenizer.apply_chat_template(messages, tokenize=False) }
dataset.map(format).select_columns(['text', 'language', 'language_code', 'annotation_type', 'user_id']).to_parquet("train.parquet")
- Downloads last month
- 31
Models trained or fine-tuned on Felladrin/ChatML-aya_dataset
