Distinct from circular dichroism (CD) spectroscopy related to the spin angular momentum (SAM), the vortex dichroism (VD) induced by orbital angular momentum (OAM) can provide a new methodology for chiral detecting and sensing. Despite relative high VD can be observed in stereometamaterial by accumulation interaction along the propagation direction of the vortex beam, VD responses for individual nano-structures are still not pronounced. And the underlying physical mechanism for enhanced VD is also not clear. In this study, we unveil a competitive approach for realizing giant VD through high-order toroidal multipolar resonance engineering. By employing the established multipole expansion theory, we demonstrate that magnetic toroidal quadrupoles can make the major contribution to the VD. Through precise spectral and geometric design, the proposed pinwheel structure can achieve a record VD of 1.67. The `blade' length and rotational symmetry allow for flexible VD modulation with a broader spectrum while maintaining the peak values. Our findings open a new horizon for understanding the interaction between OAM and chiral structure, which can find potential applications in new-generation integrated chiral sensors and vortex optical devices.