Knowledge-Guided Multimodal Reasoning
Developed a knowledge-guided multimodal classification framework that combines vision-language model predictions with botanical traits retrieved from a Neo4j knowledge graph through an LLM-based reasoning layer.
The system improved top-1 classification accuracy from 60% to 86% on an iNaturalist-derived benchmark. Controlled ablations using correct, shuffled, and incorrect graph relations isolated the contribution of external knowledge; refining ontology relations produced an additional 6% improvement.
Built a reproducible evaluation framework covering more than 100 species and five biological traits, including automated benchmark runs, per-class metrics, and systematic error categorization.
project page / paper / code — coming later