Content is loading…
Split long text into token-bounded chunks with sentence, paragraph, or hard token boundaries.
chunk_text#chunking#rag#embeddings#tokens#split textSplits text into segments based on a maxTokens budget and optional overlap. Accepts text, maxTokens, encoding (cl100k_base/o200k_base), and splitOn (sentence, paragraph, or token) parameters. Returns a list of text chunks with their respective token counts for RAG or context window management.
| Name | Type | Required | Description |
|---|---|---|---|
text | string | required | The text to split into chunks |
maxTokens | number | optional | Max tokens per chunk. Default 512. |
overlap | number | optional | Token overlap between consecutive chunks. Default 0. |
encoding | enum ("cl100k_base" | "o200k_base") | optional | Tokenizer. Default cl100k_base. |
splitOn | enum ("token" | "sentence" | "paragraph") | optional | Boundary preference: hard token cut, sentence-aware, or paragraph-aware packing. Default sentence. |
{
"text": "Long article text…",
"maxTokens": 512,
"overlap": 64,
"splitOn": "sentence"
}curl -X POST https://toolsy.my/api/mcp \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-H "Accept: application/json, text/event-stream" \
-d '{
"jsonrpc": "2.0",
"id": 1,
"method": "tools/call",
"params": {
"name": "chunk_text",
"arguments": {
"text": "Long article text…",
"maxTokens": 512,
"overlap": 64,
"splitOn": "sentence"
}
}
}'Generate a free API key or set up an OAuth connector for Claude.ai.