Optical remote sensing is extensively used in military^［1］， ecological^［2］， agriculture^［3］， aerospace^［4］， disaster reduction^［5］， and daily life applications^［6］. The overall design of the optical remote sensing system is the most important task according to application requirements （field-of-view， meteorological and environmental conditions， characteristics of target and background， effective distance， etc.）. For land-based low-light-level and infrared detection systems， there are effectiveness evaluation methods based on Minimum Resolvable Contrast（MRC）^［7-8］ and MRTD^［8-9］. However， there is still a lack of effective means for the evaluation of the effectiveness of optical remote sensing systems. Ground Resolution Distance （GSD）， Modulation Transfer Function （MTF）， and the Signal-to-Noise Ratio （SNR）， cannot evaluate the detection effectiveness scientifically and/or intuitively that is not conducive to system design and user use. At present， image interpretation is commonly used to evaluate the effectiveness of the detection system. The United States has studied the National Imagery Interpretability Rating Scale （NIIRS）^［10］ since the early 1970s. Based on these studies， the image quality can be described quantitatively. According to the interpretation task， it can be divided into different levels （0~9）， and can be used as a reference to evaluate the ability of detection of different targets. However， the calculation of the NIIRS level needs an image acquired by the system for evaluation. NIIRS is the post-evaluation method and cannot provide reference for the system demonstration and design. Moreover， the NIIRS level is not specific， and the quantification is not adequately precise. The Johnson criteria is a method used for the prediction of the probability of target discrimination. When the human eye searches for the target in a certain background or on the display， the continuous response of the eye can be divided into three stages： detection， recognition， and identification. The different stages correspond to different discrimination measures. Discrimination measures combine the effectiveness of system with human vision. Discrimination measures need to be quantified using visual psychological experiments. Johnson combined target detection with equivalent bar detection based on experiments^［11-12］. It is possible to evaluate the detection ability of the imaging system with target equivalent bar resolution without considerations of the nature of the target and image defects. Johnson validated the fact that the equivalent bar resolution can be used to predict the probability of target discrimination， and determined the criteria pertaining to the cycles of the equivalent bar for different discrimination measures. Johnson criteria has been extensively used worldwide.

Therefore， on the basis of considering target and background contrast， atmospheric transmission， detectors and human eyes， and Noise Equivalent Contrast （NEC）^［13］， this study establishes a new effectiveness evaluation method of an optical imaging system based on the Johnson criteria and MRC that was designed and implemented for a space-based optical imaging system. This method can evaluate the effectiveness quantitatively and intuitively， which is close to the actual needs. Additionally， our method can calculate the probability of detection， recognition， and identification of targets in typical backgrounds based on given parameters in the demonstration stage to guide the selection of the remote sensing detector. It is of great significance to the design of a space-based imaging system and the evaluation of on orbit satellite detection capability.

To facilitate the establishment of an MRC model， we introduce the concept of NEC. In the photoelectric imaging system， when the output signal of the reference electronic filter equals the root-mean-square of the noise of the system itself， the contrast between the target and the background is the noise equivalent contrast.

The SNR from the reference electronic filter can be expressed as

$\mathrm{S}\mathrm{N}\mathrm{R}=\frac{S}{N}=\frac{\mathrm{\Delta }\Phi \left(\lambda \right)R\left(\lambda \right)}{\sqrt[]{{\int }_{\mathrm{0}}^{\mathrm{\infty }}{s}^{\text{'}}\left(f\right)\mathrm{M}\mathrm{T}{\mathrm{F}}_{\mathrm{e}}^{\mathrm{2}}\left(f\right)\mathrm{d}f}}$

where is radiation flux difference between the target and background， R is the responsivity of the detector， s'（f） is the noise power spectrum of the detector，MTF_e（f） is the MTF of the electronic filter.

$\mathrm{\Delta }\Phi \left(\lambda \right)=\frac{\mathrm{1}}{\mathrm{4}}\mathrm{\Delta }E\left(\lambda \right)A_{\mathrm{d}}{\tau }_{\mathrm{0}}\left(\lambda \right){\left(\frac{D}{{f_{\mathrm{0}}}^{\mathrm{\text{'}}}}\right)}^{\mathrm{2}}/\left[\mathrm{1}+\frac{\mathrm{1}}{\mathrm{4}}{\left(\frac{D}{{f_{\mathrm{0}}}^{\mathrm{\text{'}}}}\right)}^{\mathrm{2}}\right]$

where is the irradiance difference between the reflected light of target E_t and background E_b at the entrance pupil， is the pixel area of the detector， τ is the transmittance of the optical system， D is the aperture， and f₀' is the focal length.

According to the definition of NEC， Eq. （1） is assumed to be equal to unity，

$\mathrm{\Delta }E\left(\lambda \right)=\frac{\mathrm{4}{F}^{\mathrm{2}}\left(\mathrm{1}+\frac{\mathrm{1}}{\mathrm{4}{F}^{\mathrm{2}}}\right)\sqrt[]{\mathrm{\Delta }{f}_{\mathrm{n}}}}{R\left(\lambda \right)A_{\mathrm{d}}{\tau }_{\mathrm{0}}\left(\lambda \right)}$

$\mathrm{\Delta }{f}_{\mathrm{n}}={\displaystyle \underset{\mathrm{0}}{\overset{\mathrm{\infty }}{\int }}}{s}^{\text{'}}\left(f\right)\mathrm{M}\mathrm{T}{\mathrm{F}}_{\mathrm{e}}^{\mathrm{2}}\left(f\right)\mathrm{d}f$

$\mathrm{N}\mathrm{E}\mathrm{C}=\frac{\mathrm{\Delta }E\left(\lambda \right)}{E_{\mathrm{t}}\left(\lambda \right)+E_{\mathrm{b}}\left(\lambda \right)}$

where Δf_nis the noise equivalent bandwidth，and F=D/f₀'. In fact， the nature of NEC is to measure noise based on the intensity of the input signal. In the case of Charge-Coupled Device （CCD） Complementary Metal-Oxide Semiconductor （CMOS） detectors， in addition to the inherent shot noise that is related to the number of charges produced by photoelectric effect， the readout noise should also be considered.

On the basis of the NEC model， the MRC model is also deduced as described in the next subsection.

MRC is defined as the contrast between the target and the background when the observer can barely distinguish the bars at a specific spatial frequency. It is an evaluation parameter that can describe quantitatively the threshold contrast of the visible photoelectric imaging system. It includes the factors of sensitivity， noise， spatial frequency， and human eyes. It can fully reflect the detection ability of the photoelectric imaging system.

The process of MRC derivation is as follows. The SNR received by the system can be described as

$\mathrm{S}\mathrm{N}{\mathrm{R}}_{\mathrm{0}}=\frac{C}{\mathrm{N}\mathrm{E}\mathrm{C}}$

where C is the contrast between the target and the background. The SNR of the image on the display is

$\mathrm{S}\mathrm{N}{\mathrm{R}}_i=\mathrm{S}\mathrm{N}{\mathrm{R}}_{\mathrm{0}}\cdot R\left(f\right){\left[\mathrm{\Delta }{f}_{\mathrm{n}}/{\int }_{\mathrm{0}}^{\mathrm{\infty }}s\left(f\right)\mathrm{M}\mathrm{T}{\mathrm{F}}_{\mathrm{m}}^{\mathrm{2}}\left(f\right)\mathrm{d}f\right]}^{\mathrm{1}/\mathrm{2}}$

where R（f） is the square wave response of the system， MTF_m（f） is the modulation transfer function of the section behind the readout circuit， and s（f） is the noise power spectrum. According to the relationship between R（f） and MTF_s（f）， and based on the approximation of the first term，

$R\left(f\right)\approx \frac{\mathrm{4}}{\mathrm{\pi }}\mathrm{M}\mathrm{T}{\mathrm{F}}_{\mathrm{s}}(f)$

When the observer observes the target， the human eye will modify the SNR according to the following four aspects^［14-15］

1） The eye extracts the strip pattern. When the signal is distinguishable， the high-order harmonic is filtered out and the first-order harmonic is maintained. The peak value of the signal is then attenuated as follows

$\frac{\mathrm{2}}{\mathrm{\pi }}R\left(f\right)=\frac{\mathrm{8}}{{\mathrm{\pi }}^{\mathrm{2}}}\mathrm{M}\mathrm{T}{\mathrm{F}}_{\mathrm{s}}(f)$

2） Because of the time integration， the signal will be sampled independently according to the integral time of the human eye （t_e= 0.2 s）， while noise will be superimposed at the same time. Therefore， the SNR will be improved （t_ef_p）^1/2， whereby f_p is the frame rate of display.

3） In the vertical direction， the eye will integrate the signal in the spatial domain， and will use the instantaneous vertical field-of-view β as the correlation length of the noise. Thus， the SNR will be improved according to

${\left(\frac{L}{\beta }\right)}^{\mathrm{1}/\mathrm{2}}={\left(\frac{{\epsilon }_{\mathrm{0}}W}{\beta }\right)}^{\mathrm{1}/\mathrm{2}}={\left(\frac{{\epsilon }_{\mathrm{0}}}{\mathrm{2}{f}_{\mathrm{T}}\beta }\right)}^{\mathrm{1}/\mathrm{2}}$

where L and W are the angle length and angle width of the strips， respectively， and ε₀ = L/W， f_T=1/（2W）.

4） For the periodic rectangular strip target with frequency f_T， the narrow-band spatial filtering effect of the human eye is equivalent to the matched filter whose transfer properties can be expressed as sinc²（π/2⋅f/f_T）⋅MTF²_s（f）， where f represents the spatial frequency. Therefore， the integral response of the human eye can be converted into the noise bandwidth Δf_eye considering the human eye matching filter according to the actual system bandwidth

$\mathrm{\Delta }{f}_{\mathrm{e}\mathrm{y}\mathrm{e}}={\displaystyle \underset{\mathrm{0}}{\overset{\mathrm{\infty }}{\int }}}s\left(f\right)\mathrm{M}\mathrm{T}{\mathrm{F}}_{\mathrm{s}}^{\mathrm{2}}(f)\mathrm{M}\mathrm{T}{\mathrm{F}}_{\mathrm{e}\mathrm{y}\mathrm{e}}^{\mathrm{2}}(f)\mathrm{s}\mathrm{i}\mathrm{n}{\mathrm{c}}^{\mathrm{2}}\left(\frac{\mathrm{\pi }}{\mathrm{2}}\frac{f}{f_{\mathrm{T}}}\right)\mathrm{d}f$

where MTF_eye（f） is the modulation transfer function of the human eye， and sinc（·） is the sine basis function. As f_T goes to zero， Δf_eye approaches the target frequency f_T （assuming white noise）^［9］.

Combining the four aforementioned corrections with the SNR_i， the visual SNR can be obtained as

$\mathrm{S}\mathrm{N}{\mathrm{R}}_{\mathrm{v}}=\frac{\mathrm{8}}{{\mathrm{\pi }}^{\mathrm{2}}}\mathrm{M}\mathrm{T}{\mathrm{F}}_{\mathrm{s}}(f)\mathrm{M}\mathrm{T}{\mathrm{F}}_{\mathrm{e}\mathrm{y}\mathrm{e}}(f)\frac{C{\left(t_{\mathrm{e}}{f}_{\mathrm{p}}\right)}^{\mathrm{1}/\mathrm{2}}}{\mathrm{N}\mathrm{E}\mathrm{C}}{\left(\frac{{\epsilon }_{\mathrm{0}}}{\mathrm{2}{f}_{\mathrm{T}}\beta }\right)}^{\mathrm{1}/\mathrm{2}}{\left(\frac{\mathrm{\Delta }{f}_{\mathrm{n}}}{\mathrm{\Delta }{f}_{\mathrm{e}\mathrm{y}\mathrm{e}}}\right)}^{\mathrm{1}/\mathrm{2}}$

Let SNR_v be equal to the threshold SNR such that the human eye can distinguish the stripSNR_DT. At this time， C is the MRC.

$\mathrm{M}\mathrm{R}\mathrm{C}=\frac{{\mathrm{\pi }}^{\mathrm{2}}}{\mathrm{8}}\frac{\mathrm{S}\mathrm{N}{\mathrm{R}}_{\mathrm{D}\mathrm{T}}\times \mathrm{N}\mathrm{E}\mathrm{C}\cdot f_{\mathrm{T}}}{\mathrm{M}\mathrm{T}{\mathrm{F}}_{\mathrm{s}}(f)\mathrm{M}\mathrm{T}{\mathrm{F}}_{\mathrm{e}\mathrm{y}\mathrm{e}}(f)}{\left(\frac{\mathrm{2}\beta }{{\epsilon }_{\mathrm{0}}{t}_{\mathrm{e}}{f}_{\mathrm{p}}}\right)}^{\mathrm{1}/\mathrm{2}}{\left(\frac{\mathrm{1}}{\mathrm{\Delta }{f}_{\mathrm{n}}}\right)}^{\mathrm{1}/\mathrm{2}}$

Based on practical applications， the MRC model needs to be modified：

The ratio of length to width of the common strips in laboratory is 7∶1 （or 5∶1）. However， ε₀ is not only related to the target itself， but it is also related to the number of cycles for 50% detection according to the Johnson criteria. Therefore， the ratio of the length-to-width after the correction ε is，

$\epsilon =\left(\mathrm{2}nL/W\right)/\mathrm{7}$

where n is the number of cycles for 50% detection， as shown in Table 1.

In practice， only the dark current noise of the detector is considered， while the dark current noise has a fixed value， that is， NEC is only related to the irradiance of the target and background， but it is also affected by the integration time in practice. Therefore， it needs to be corrected. Herein， we used the multiple t_n of integral time to improve SNR instead of Δf_n.

The final MRC can be derived as

$\mathrm{M}\mathrm{R}\mathrm{C}=\frac{{\mathrm{\pi }}^{\mathrm{2}}}{\mathrm{8}}\frac{\mathrm{S}\mathrm{N}{\mathrm{R}}_{\mathrm{D}\mathrm{T}}\times \mathrm{N}\mathrm{E}\mathrm{C}\cdot f_{\mathrm{T}}}{\mathrm{M}\mathrm{T}{\mathrm{F}}_{\mathrm{s}}(f)\mathrm{M}\mathrm{T}{\mathrm{F}}_{\mathrm{e}\mathrm{y}\mathrm{e}}(f)}{\left(\frac{\mathrm{2}\beta }{\epsilon t_{\mathrm{e}}{f}_{\mathrm{p}}}\right)}^{\mathrm{1}/\mathrm{2}}{\left(\frac{\mathrm{1}}{t_{\mathrm{n}}}\right)}^{\mathrm{1}/\mathrm{2}}$

By substituting MRC and the contrast between the target and background C₀ into the target transfer probability function^［12］， the contrast limited probability P_C can be obtained as

$P_{\mathrm{C}}=\frac{{\left(C_{\mathrm{0}}/\mathrm{M}\mathrm{R}\mathrm{C}\right)}^{E_{\mathrm{C}}}}{\mathrm{1}+{\left(C_{\mathrm{0}}/\mathrm{M}\mathrm{R}\mathrm{C}\right)}^{E_{\mathrm{C}}}}$

where E_C= 2.7+0.7（C₀/MRC）.

By substituting the resolvable cycles N across the target and n into the target transfer probability function， the resolution limited probability P_J is obtained.

$P_{\mathrm{J}}=\frac{{\left(N/n\right)}^{E_{\mathrm{J}}}}{\mathrm{1}+{\left(N/n\right)}^{E_{\mathrm{J}}}}$

where E_J= 2.7+0.7（N/n）.

Finally， the probability of detection is

$P=P_{\mathrm{C}}\cdot P_{\mathrm{J}}$

The flow diagram of the proposed method is shown in Fig.1.

An aviation detection experiment was carried out in Jingmen， Hubei Province on May 20， 2019. The weather was clear on that day， the visibility was 23 km， the time was 9：30 a.m.， and the solar altitude angle was 40°. The relevant parameters of the detector for the aviation detection experiment are shown in Table 2. The radiance values of the target （small vehicle） and background （cement pavement） obtained with the atmospheric transmission simulation software were L_t = 0.003 72 W/cm²/sr and L_b= 0.001 56 W/cm²/sr， respectively. The NEC was 0.003 2. The width and length of the target （a car is shown in Fig.2） were 4.8 m and 1.7 m， respectively. We used 2-D criteria in Table 1， n=［0.75 3 6］， where the three numbers represented the levels of detection， recognition， and identification， respectively. We obtained the MRC=［0.0097 0.019 0.027］ according to Eq. 15. According to Eqs. （16）~（17）， we obtained P_C=［1.00 1.00 1.00］ and P_J=［1.00 0.46 0.10］. Finally， P=［1.00 0.46 0.10］， therefore， the probabilities of detection， recognition， and identification， were 100%， 46%， and 10%， respectively. It was worth noting that the contrast-limited probability P_C was ［1.00 1.00 1.00］. This means that the contrast was adequate to visualize the target at this time. Accordingly， the contrast was not the main factor that affected the probability. Obviously， cars can be easily detected in the aerial image shown in Fig. 2 （the probability of detection was 100%）. Furthermore， we can recognize it as a car （the probability of recognition was 46%） but we cannot identify the specific type of the car （the probability of identification was10%）. At this point， the NIIRS was 4.23. Similarly， the probability of the aircraft in Fig.2（b） was ［1.00 0.92 0.73］， i.e.， the aircraft can be detected， recognized， and identified. The P_C and P_J were ［1.00 0.92 0.73］ and ［1.00 1.00 0.98］， the results showed that the resolution was good enough to visualize the target at this time. It can be observed in Fig.2（b） that the gray level values of the aircraft and the ground were almost the same， and the contrast was low. Thus， the contrast was the main factor responsible for probability changes. Our calculation results were consistent with the subjective observation and NIIRS. The results showed that our method was effective and accurate.

The GF-2 satellite is the first civil optical remote sensing satellite developed by China with a spatial resolution better than 1 m. The resolution of the sub-satellite point can reach a value of 0.8 m. The main technical parameters are shown in Table 3. We used the targets on the land and marine images of GF-2 to carry out the verification experiments.

We selected a car and a truck as the targets in Fig.3. The size of the car is 4.8 m×1.7 m （length × width， the same below）， and the probability P was ［1.00 0.32 0.05］. In addition， the size of the truck is 17.5 m×1.8 m with probabilities P of ［1.00 0.67 0.17］. It is worth noting that the observer had prior knowledge， therefore， sometimes even a small recognition probability can have profound effects. For example， when the recognition probability of a car is 0.32， the car can be hardly recognized. However， when it is on the road， it is easier to be recognized as a car. The NIIRS level of Fig.3 is 4.53， which is basically consistent with our method.

In the marine image， we chose large and small civilian ships as targets， as shown in Fig.4. The probability of the large ship 88.8 m×15.8 m was ［1.00 1.00 1.00］， while the probabilities of the small ships 1 34.4 m×8.0 m and 2 16.8 m×4.8 m were ［0.98 0.63 0.31］ and ［0.62 0.14 0.03］， respectively. The NIIRS level of Fig.4 is 4.53， which is basically consistent with our method.

In general， the detection probability calculated by our method is in good agreement with the subjective judgment of human eye observations.

In this study， a new effectiveness evaluation method was proposed for space-based optical imaging systems to evaluate the effectiveness of optical detection based on the Johnson criteria and MRC. Compared with traditional methods， our method took the characteristics of target and background， atmospheric transmission， detectors， and human eyes into considerations. Additionally， our method can calculate the probability of detection， recognition， and identification of targets in typical backgrounds based on given parameters in the design stage to guide the selection of the remote sensing detectors. Experimental results showed that the detection probability calculated by our method is in good agreement with the subjective judgment of human eye observation. We have prepared 100 subjective experiment sample bank of different scale targets， and will carry out a large number of subjective experiments in the future to verify the correctness of our method.

However， this method does not consider subjective factors， such as the experience and ability of observers， and thus needs to be improved by combining it with deep learning and artificial intelligence technologies. It is also necessary to study the noise types of different detectors and optimize the detection efficiency evaluation model. These constitute work-in-progress and are envisaged to constitute the topics of our future research publication.