I fed Microsoft's free MarkItDown to 6 files today, and one of them broke my R… · @ReignSwords · TwKit

@ReignSwords

I fed Microsoft's free MarkItDown to 6 files today, and one of them broke my RAG pipeline in the quietest way possible. Here's what actually happened: 1. Install trap first. Plain pip install markitdown gave me version 0.1.9, but PDFs failed outright. It needs the extras: pip install 'markitdown[pdf,docx,xlsx]'. The error message tells you this, at least. 2. Word and Excel mostly worked. My test DOCX (heading + list + 3x3 table) came out clean in 2.4 seconds, except for one ghost empty row at the top of the table. My XLSX came out in 1.8 seconds, but every blank cell turned into the literal word NaN. Feed that to an LLM and it will happily quote NaN as a fact. 3. The real killer: a scanned PDF. One page, one line of text as an image, no text layer. MarkItDown finished in 1.1 seconds, exit code 0, and produced exactly 0 characters. No warning. No error. Just silence. 4. A real printed PDF from my own browser did worse in a different way: 1,098 characters of table soup, 96 pipe characters, navigation menus mashed into the body text. So the verdict: a great free converter for text files you already control, and a dangerous blind pipeline step. Run it like this: markitdown report.pdf -o report.md My new checklist before anything goes into an index: - Reject any output under 10 characters, even on exit 0 - Search for NaN and ghost rows before chunking - OCR scanned PDFs first, never trust the converter to warn you Copy this and run it on your worst file first, not your cleanest one. If you build RAG stuff, follow along — I test the unsexy plumbing so you don't have to. Have you ever shipped a RAG answer that traced back to a converter silently eating a document, or am I the only one who found out this way? 🔵 Fellow Premium members, let's connect! I post about AI & tech — follow for more 👥

曝光 60 · 评论 6 · 点赞 1 · 书签 0