The ideal indoor visible light communication (VLC) channel transmission model is typically unsuitable for an actual irregular indoor environment. To address this issue, this study investigates the irregular reflection of visible light from irregular scenes and establishes a more realistic VLC channel model; further, based on this model, it develops a genetic algorithm-optimized back propagation (BP) neural network positioning (GA-BP) algorithm to overcome the poor performance of the BP neural network in handling nonlinear systems. Herein, by simulating the reflection element normal vector information of the actual channel model in irregular scenes, the direction of light reflection can be determined, enabling the receiver to collect more accurate optical power values. In an irregular indoor environment with dimensions of 5 m × 5 m × 5 m, the simulations reveal that the total optical power received by the system fluctuates within a range of 0.0141-0.0639 W; additionally, compared with the BP neural network, the GA-BP algorithm achieves a significantly reduced positioning error of 2.32 cm, along with an average positioning time of 0.0625 s.