Format Overview

TONL syntax reference with concrete examples. See Documentation for full API reference.

Data Model

TONL models data the same way as JSON:

  • Primitives: strings, numbers, booleans, and null
  • Objects: mappings from string keys to values
  • Arrays: ordered sequences of values

Root Forms

A TONL document can represent different root forms:

  • Root object (most common): Fields appear at depth 0 with no parent key
  • Root array: Begins with key[N]{fields}: at depth 0
  • Root primitive: A single primitive value (string, number, boolean, or null)

Document Headers

TONL documents can include optional metadata headers at the top.

Version Header

#version 1.0

Specifies the TONL format version. Defaults to 1.0 if omitted.

Delimiter Header

#delimiter "|"

Declares the column separator for tabular data. Supported delimiters:

Delimiter Declaration Use Case
Comma (default) Most compact, standard data
Pipe #delimiter "|" Data containing commas
Tab #delimiter "\t" Visual alignment, spreadsheet-like
Semicolon #delimiter ";" CSV-style data with embedded commas

Complete Header Example

#version 1.0
#delimiter "|"

users[2]{id|name|role}:
  1|Alice|admin
  2|Bob|user

Objects

Objects use indentation-based syntax. One space follows the colon.

Simple Objects

Objects with primitive values use key: value syntax:

id: 123
name: Ada
active: true

Objects with Type Hints

Type hints are optional and appear in the header:

user{id:u32,name:str,active:bool}:
  id: 123
  name: Ada
  active: true

The header user{id:u32,name:str,active:bool}: declares object name and field types.

Nested Objects

Nested objects add one indentation level (default: 2 spaces):

user{id:u32,name:str,contact:obj}:
  id: 1
  name: Alice
  contact{email:str,phone:str}:
    email: alice@example.com
    phone: +123456789

Empty Objects

A nested empty object is key: alone, with no children:

user{id:u32,metadata:obj}:
  id: 1
  metadata:

Arrays

TONL detects array structure and chooses the most efficient representation. Arrays declare their length in brackets: [N].

Primitive Arrays (Inline)

Arrays of primitives are rendered inline:

tags[3]: admin, ops, dev
scores[4]: 95, 87, 92, 100
flags[3]: true, false, true

Arrays of Objects (Tabular)

When all objects share the same primitive-valued keys, TONL uses tabular format:

users[3]{id:u32,name:str,role:str}:
  1, Alice, admin
  2, Bob, user
  3, Carol, editor

Note: Tabular format requires identical field sets across all objects and primitive values only (no nested arrays/objects).

With Quoted Values

Strings containing the delimiter must be quoted:

users[2]{id:u32,name:str,role:str}:
  1, Alice Admin, admin
  2, "Bob, Jr.", user

Empty Arrays

items[0]:

The header declares length zero, with no elements following.

Tip: The array length [N] helps LLMs validate structure. If you ask a model to generate TONL output, explicit lengths let you detect truncation or malformed data.

Type Hints

Type hints are optional annotations that provide schema information in the header.

Syntax

key{field:type,field:type,...}:

Available Types

Type Description Example
str String name:str
u32 Unsigned 32-bit integer id:u32
i32 Signed 32-bit integer count:i32
f64 64-bit float price:f64
bool Boolean active:bool
null Null value deleted:null
obj Object contact:obj
list Array tags:list

Type Inference

When type hints are omitted, the decoder infers types automatically:

Token Inferred Type
Unquoted number numeric (u32, i32, or f64)
Quoted number string
true / false boolean
null null
Everything else string

Type Coercion

In strict mode, values are coerced to match type hints:

user{age:u32,name:str,active:bool}:
  age: "25"           # Coerced to number 25
  name: alice         # Coerced to string "alice"
  active: "true"      # Coerced to boolean true

Quoting and Types

TONL quotes strings only when necessary to maximize token efficiency.

When Strings Need Quotes

A string must be quoted if:

  • It's empty ("")
  • It has leading or trailing whitespace
  • It equals true, false, or null
  • It looks like a number (e.g., "42", "-3.14")
  • It contains special characters: : , # {} [] " \
  • It contains the active delimiter character

Otherwise, strings can be unquoted. Unicode, emoji, and internal spaces are safe:

message: Hello world
note: This has inner spaces
emoji: Check this out

Escape Sequences

Character Escape
Backslash (\) \\
Double quote (") "" or \"
Newline \n
Carriage return \r
Tab \t

Multiline Strings

Triple quotes (""") are used for multiline content:

description: """This is a
multi-line string
with preserved formatting"""

Type Conversions

Input Output
Finite number Canonical decimal
NaN, Infinity null
BigInt (safe range) Number
BigInt (out of range) Quoted decimal string
Date ISO string in quotes
undefined, function, symbol null

Key Handling

Rules for object keys and special character handling.

Valid Identifiers

  • Characters: Letters, numbers, underscore (_)
  • First character: Must be a letter or underscore
  • Case sensitivity: Case-sensitive
valid_name: value
isValid123: value
_privateField: value

Quoted Keys (Special Characters)

Keys containing special characters must be quoted:

"field-with-dash": value
"key with spaces": value
"@type": value
"#comment": value

Dual-Mode System (v2.0.6+)

TONL offers two modes for handling problematic keys:

Default Mode (Quoting)

Perfect round-trip fidelity

""[1]:
  empty-value
"#"[1]:
  hash-value

Preprocessing Mode

Clean, readable keys (--preprocess)

empty[1]:
  empty-value
comment[1]:
  hash-value
Original Key Preprocessed Key
"" (empty) empty
"#" comment
"@type" type
"key with spaces" key_with_spaces

Complete Examples

Real-world TONL document examples.

E-commerce Order

@tonl v1

order{orderId,status,placedAt}:
  orderId: ORD-2025-11-001234
  status: processing
  placedAt: 2025-11-03T14:30:00Z

customer{id:u32,name:str,email:str,tier:str}:
  id: 5678
  name: "John Smith"
  email: john.smith@example.com
  tier: premium

items[3]{sku,name,quantity:u32,unitPrice:f64,discount:f64}:
  WIDGET-001, "Premium Widget", 2, 29.99, 0.10
  GADGET-042, "Smart Gadget Pro", 1, 149.99, 0.00
  ACCESSORY-123, "Universal Adapter", 3, 12.99, 0.15

summary{subtotal:f64,tax:f64,shipping:f64,total:f64}:
  subtotal: 248.93
  tax: 21.16
  shipping: 9.99
  total: 266.63

notes[2]: "Gift wrap requested", "Leave package at front door"

User Profile with Nested Data

#version 1.0

user{id:u32,name:str,email:str,preferences:obj}:
  id: 1
  name: Ahmet Yilmaz
  email: ahmet@example.com
  preferences{language:str,notifications:bool,theme:str}:
    language: tr
    notifications: true
    theme: dark

stats{loginCount:u32,sessionDuration:f64}:
  loginCount: 156
  sessionDuration: 45.2

roles[3]: admin, editor, viewer

JSON Comparison

See how TONL compares to JSON in terms of size and token efficiency.

JSON (245 bytes, ~89 tokens)

{
  "users": [
    { "id": 1, "name": "Alice", "role": "admin" },
    { "id": 2, "name": "Bob, Jr.", "role": "user" },
    { "id": 3, "name": "Carol", "role": "editor" }
  ]
}

TONL (158 bytes, ~49 tokens - 45% reduction)

#version 1.0
users[3]{id:u32,name:str,role:str}:
  1, Alice, admin
  2, "Bob, Jr.", user
  3, Carol, editor

Key Differences

No redundant field names

Field names declared once in header, not repeated per row

No structural punctuation

No braces or commas around objects

Tabular data format

Values separated by delimiter, not key-value pairs

Type information

Optional type hints for validation and documentation

Performance Summary

Metric JSON TONL Improvement
Size (bytes) 245 158 36% smaller
Tokens (GPT-5) 89 49 45% fewer
Encoding Speed 1.0x 12-15x 12-15x faster
Decoding Speed 1.0x 10x 10x faster