Semitransparent photovoltaics (STPV) is a promising solar energy harvesting technology because it can be integrated to harness huge sun-facing areas of modern buildings for electricity generation. Its thermodynamic efficiency limits are of fundamental interest. This work extended the analyses from neutral STPV for building windows to those installed on the surface of colored building envelops according to the principle of detailed balance. Results show that the efficiency limit of STPV for blue building envelops is as high as 28.3%, which represents a 10% absolute enhancement compared to the neutral STPV for building windows (18.1%). These results demonstrate that STPV can be integrated into both windows and envelops of modern buildings, which has the potential to offset the low density of solar energy. This work provides guidelines on the selection and the development of active materials for STPV.