Medicinal Plant Identification Using Hierarchical Multimodal Feature Fusion with Deep Vision Transformers
Keywords:
Medicinal plant identification, Vision Transformers, deep learning ensemble methods, multi-modal feature fusion, Grad-CAM interpretability, hierarchical classification, botanical feature learning, transfer learning, field deployment.Abstract
Accurate identification of medicinal plants is critical for pharmaceutical quality assurance, supply chain integrity, traditional medicine practice, and global biodiversity conservation. This paper presents a novel hierarchical multi-modal feature fusion system combining Vision Transformers (ViT) with five state-of-the-art deep convolutional neural network architectures (DenseNet201, ResNet50V2, EfficientNet-B3, InceptionV3) seamlessly integrated with hand-crafted morphological, textural, and spectral features. Comprehensive evaluations on the Flavia benchmark dataset (1,907 images, 32 European plant species) achieved 99.34% accuracy under controlled laboratory conditions, while the larger Medicinal Plant Collection dataset (5,878 images, 30 Ayurvedic/traditional medicine species) achieved 98.91% accuracy under realistic field deployment conditions, demonstrating exceptional generalization with only 0.43% accuracy degradation. Multi-modal feature fusion contributed +1.78% cumulative accuracy improvement through demonstrated synergistic complementarity of feature types. Grad-CAM interpretability analysis validated that learned features align precisely with botanical identification principles. Independent statistical significance testing (p<0.01 for major comparisons) confirmed all reported improvements are genuine rather than random variation. The system operates at sub-300 millisecond inference time on standard CPU hardware and under 50 milliseconds on GPU, enabling practical smartphone deployment for healthcare practitioners, traditional medicine practitioners, and conservation biologists worldwide.