Why Document Formatting Dictates AI Accuracy
When you upload a document to train an AI chatbot, the computer does not "read" your PDF like a human eye. Instead, an optical character recognition (OCR) or document parser extracts raw strings of text, strips away layout geometry, and splits the stream into discrete semantic chunks.
If your source document contains conflicting return policies, unlabelled tables, or fragmented footers repeating across every page, those text artifacts contaminate your vector index. Clean source data produces precise, confident AI answers; noisy source data produces confused or evasive responses.
PDF vs. DOCX vs. TXT: Preparing File Types for Ingestion
#, ##) to clearly delineate topics and separate FAQ pairs with clean line breaks.Data Sanitization: Pre-Flight Document Checklist
Before uploading company manuals or help desk exports into your chatbot studio, run through this pre-flight audit:
- Remove Confidential PII: Strip internal employee phone numbers, private banking details, and internal API keys.
- Resolve Policy Contradictions: If an older document states "30-day returns" and a newer memo states "14-day returns", delete the outdated version to prevent vector retrieval conflict.
- Format Atomic FAQs: Where possible, structure complex policies into clear question-and-answer pairs:
### Q: Can customers exchange opened software packages?
A: No. Software packages with broken seals may only be returned for exchange if defective, within 14 days of delivery with original proof of purchase.
Understanding Chunk Sizes and Token Overlap
ChatCraft AI automatically segments your files into semantic windows. Understanding this mechanism helps you write better reference material:
- Target Chunk Size: 750 tokens (~550 English words). This represents the sweet spot between capturing complete logical explanations and avoiding irrelevant background context.
- Chunk Overlap: 100 tokens (~75 words). By preserving the last two sentences of Chunk A at the beginning of Chunk B, sentences positioned right at the cut-off boundary are never severed from their context.
Step-by-Step: Uploading & Training Your Bot in ChatCraft Studio
Training your chatbot on business documents requires four simple steps:
- Create or Open Your Bot: Log in to your ChatCraft dashboard and navigate to the Bot Studio.
- Access the Knowledge Sources Tab: Select Files & Documents from the knowledge manager.
- Drag & Drop Your Files: Upload your prepared PDFs, Word documents, or text files. The engine parses, sanitizes, and vectors your text in under 60 seconds.
- Inspect Vector Chunks: Click on any uploaded file to inspect the generated text chunks and confirm proper formatting.
Auditing & Testing Accuracy in the Live Simulator
Before embedding your bot on a live production storefront, open the interactive simulator in the Studio:
- Test Direct Questions: Ask exact phrasing found in your documents ("What are your shipping rates to Canada?").
- Test Semantic Variations: Ask conversational queries ("How much does it cost to get stuff sent to Toronto?").
- Test Negative Proofs: Ask an out-of-scope question ("Do you sell airline tickets?"). Verify that the bot politely declines instead of hallucinating.