In the visible spectrum range, the accurate recognition of target fruit is the fundamental guarantee for achieving orchard yield measurement and machine automatic picking. However, this task is susceptible to many interferences, such as the complex unstructured orchard environment, the close color between green apples and background leaves, etc., which significantly restrict the detection accuracy of target fruits and bring great challenges to recognition of machine vision. It targeted the different illumination environments and fruit postures under the complex orchard environment. An optimized convolution and one-stage (FCOS) fully neural network model for green apple recognition is proposed in this study. Firstly, the new model combines the feature extraction ability of convolutional neural network (CNN) based on FCOS, eliminates the dependence of previous detectors on anchor boxes, and switches to a novel manner of one-stage, full convolution and anchor-free for predicting the fruit confidence and boxes offsets, which greatly improves the recognition speed of the model while ensuring the detection accuracy simultaneously. Secondly, the bottom-up feature fusion architecture is embedded after the feature pyramid to provide more accurate positioning information for high -levels and thus further optimize the detection effect of green apple. Finally, the overall loss function is designed to complete the iterative training given three output branches of FCOS. To simulate the real orchard environment as possible, we collected green apple images in various environments with different lighting environments, illumination angle, occlusion type, camera distance for data sets generation and model training, and then evaluated the optimal model on validation set containing different scenes. The experimental results show that our proposed model's average precision (AP) is 85.6%, which is 0.9, 10.5, 2.5 and 1.9 percentage points higher than the state-of-the-art detection models Faster- R-CNN, SSD, RetinaNet and FSAF, respectively. In the aspect of model design, the model parameters of FCOS and the calculation of the whole detection process are 32.0 M and 47.5 GFLOPs (billion floating-point operations), respectively, which are 9.5 M and 12.5 GFLOPs lower than those of Faster R-CNN. Comparisons of experimental results show that the new model has higher detection accuracy and recognition efficiency in the visible spectrum, which can provide theoretical and technical support for orchard yield measurement and automatic picking. In addition, the new model can also provide theoretical references for other kinds of fruits and vegetables.