Diverse applications and varying resource demands result in nonuniform traffic distribution and imbalanced resource utilization in data center networks (DCNs). We propose a fine-grained baseband function reallocation scheme in heterogeneous optical switching-based data center networks (HOS-DCNs). A deep reinforcement learning (DRL) based functional split and resource mapping approach (DRL-BFM) is proposed to maximize throughput in high-load server racks through implementing load balancing in DCNs. The results demonstrate that DRL-BFM improves throughput of 20.8%, 22.8% and 29.8% on average compared to existing algorithms under different computational, bandwidth capacities and latency conditions, respectively.