The treatment and criteria for development of unstable Roche lobe overflow (RLOF) that leads to the common envelope (CE) phase have hindered the evolutionary predictions for decades. In particular, the formation of black hole–black hole (BH-BH), black hole–neutron star (BH-NS), and neutron star–neutron star (NS-NS) merging binaries depends sensitively on the CE phase in classical isolated binary evolution model. All these mergers are now reported as LIGO/Virgo sources or source candidates. CE is even considered by some as a mandatory phase in the formation of BH-BH, BH-NS or NS-NS mergers in binary evolution models. At the moment, there is no full first-principles model for development of CE. We employ the Startrack population synthesis code to test the current advancements in studies on stability of RLOF for massive donors to assess their effect on LIGO/Virgo source population. In particular, we allow for more restrictive CE development criteria for massive donors (M>18 Msun). We also test a modified condition for switching between different types of stable mass transfer, thermal or nuclear timescale. Implemented modifications significantly influence basic properties of merging double compact objects, sometimes in non-intuitive way. For one of tested models with restricted CE development criteria, for example, local density merger rates for BH-BH systems increased by a factor of 2-3 due to the emergence of a new dominant formation scenario without any CE phase. We find that the changes in highly uncertain assumptions on RLOF physics may significantly affect (i) local merger rate density, (ii) shape of the mass and mass ratio distributions, and (iii) dominant evolutionary formation (with and without CE) scenarios of LIGO/Virgo sources. Our results demonstrate that without sufficiently strong constraints on RLOF physics, one is not able to draw fully reliable conclusions about the population of double compact object systems based on population synthesis studies.