Tool
Split JSONL into train, val, and test
A seed and FNV-1a place each group in a 0–9999 bucket. The same instruction or first user turn stays in one split. Lines never leave this tab.
Splitting runs in this tab. GearDel does not receive the JSONL.
How a line is assigned
Each kept object is hashed with FNV-1a 32-bit over seed + NUL + group key, then taken modulo 10,000. train% maps to the first train% × 100 buckets, val% to the next val% × 100, and the rest to test. The same seed and key always land in the same split.
The default group key is a prompt: Alpaca instruction plus input, or the first user/human turn in OpenAI messages or ShareGPT conversations. Rows that share that string go together so a prompt does not leak across splits.
If you group by whole JSON line, JSON.stringify of the object is the key. Two objects with the same prompt but different outputs can then land in different splits.
NFC, when on, runs String.normalize('NFC') on the key before hashing. Hangul that looks the same under NFD and NFC then shares a bucket.
Optional exact-object drop keeps the first JSON.stringify and skips later copies. Blank lines are counted and skipped. A leading JSON array is rejected; this page is JSONL.
What this page does not do
- It does not stratify by a label field or guarantee class balance. The hash is not a statistical sample of the file.
- It does not shuffle in RAM and then slice. Percentages are bucket widths, so a tiny file can miss val or test even when those percents are non-zero.
- It does not talk to Hugging Face datasets.splits. Download the three JSONL files from this tab if you need files.
- At most 20,000 lines or about two million characters.