TOON Documentation

TOON (Token-Oriented Object Notation) is a compact data notation format created for AI and LLM-driven systems. It focuses on reducing token usage, improving readability for models, and simplifying structured data exchange.

Why TOON Exists

Traditional formats like JSON include punctuation and repeated keys that increase token count. When using LLMs, every token matters. TOON minimizes unnecessary characters while preserving structure, making it efficient for AI workflows.

Core Idea

TOON separates field definitions from data entries. You define column names once, then list data rows beneath them, reducing repetition.

Example Comparison

JSON:

{
  "users": [
    {"id": 1, "name": "Alice", "role": "admin"},
    {"id": 2, "name": "Bob", "role": "user"}
  ]
}

TOON:

users[2]{id,name,role}:
  1,Alice,admin
  2,Bob,user

TOON removes quotes, braces, and repeating field names, lowering token cost.

Benefits

1. Lower Token Usage

Fewer characters mean fewer tokens. This reduces API cost and allows more data per prompt.

2. Better LLM Parsing

LLMs handle structured tables efficiently. TOON presents arrays as clean row-based data.

3. Compact Structure

Clear separation of schema and data simplifies transformation and consumption.

4. Faster Prompting

TOON reduces verbosity, speeding up processing inside LLM pipelines.

Syntax Breakdown

Structure

<identifier>[<rowCount>]{<field1>,<field2>,...}:
  <value1>,<value2>,...
  <value1>,<value2>,...

Components

• identifier: dataset name
• rowCount: number of entries
• fields: column names
• rows: comma-separated values matching fields

When TOON Works Best

• Large arrays of repeated objects
• LLMs requiring fast, cost-efficient input
• Embedded agent systems
• Conversational retrieval systems using structured data

When JSON Might Still Be Better

• Deeply nested data
• APIs expecting strict JSON schema
• Systems requiring standard validation tools

Converting JSON to TOON

1. Extract keys from the first object
2. Count entries
3. Build TOON header
4. Map object values into CSV-like rows

Example

JSON:

[{"id":1,"score":9},{"id":2,"score":7}]

TOON:

scores[2]{id,score}:
  1,9
  2,7

Libraries for TOON

• JS/TS: @toon-format/toon
• Python: (community tools emerging)

Summary

TOON offers a lightweight, token-efficient alternative to JSON for AI-driven tasks. It simplifies repetitive structures and optimizes data for LLM consumption, making it an excellent choice for prompt engineering and data-heavy AI workflows.