Vision-Centric Financial Swarm
传统 RAG 把 PDF 当文本拆，最容易丢掉图表、表格和页面布局。这个项目直接把财报页面当视觉对象处理：用 ColPali 做多向量检索，再通过 ROI 裁剪把关键区域送给视觉模型，最后由 agent 做跨页推理和数值校验。
Traditional RAG pipelines struggle with annual reports and investor decks because the critical evidence often lives in tables, figures, and page regions that text chunking destroys.
I built a multi-modal RAG prototype around ColPali-style visual retrieval, ROI cropping, and agentic verification over financial document evidence.
This is the strongest candidate to add as a research-system project: it connects multi-modal retrieval, financial reasoning, and agent orchestration in one coherent case.
- 01Modeled the document pipeline around page-level visual retrieval rather than text-only chunks.
- 02Designed the agent loop for research, inspection, and cross-page verification.
- 03Kept the presentation grounded in evidence paths and bounded claims rather than generic AI chat output.
Financial documents are visual artifacts. Tables, charts, page layout, and multi-page references are part of the answer, so treating PDFs as plain text throws away signal before retrieval begins.
Document pages are embedded as visual evidence, retrieved with multi-vector representations, cropped into regions of interest, and passed to downstream reasoning steps. The agent layer separates retrieval, inspection, synthesis, and verification so the final answer can point back to visual evidence.
- 01Visual retrieval keeps layout signal that text chunking often discards.
- 02ROI cropping is framed as a cost-control and evidence-focus mechanism for vision model calls.
- 03This project complements Structure-Aware RAG by moving from document structure to visual document understanding.