We study a consistent infrared and ultraviolet regularization scheme for the cosmological perturbations. The infrared divergences are cured by assuming that the Universe undergoes a transition between a non-singular pre-inflationary, radiation-dominated phase and a slow-roll inflationary evolution. The ultraviolet divergences are eliminated via adiabatic subtraction. A consistent regularization of the field fluctuations through this transition is obtained by performing a mode matching for both the gauge invariant Mukhanov variable and its adiabatic expansion. We show that these quantities do not generate ultraviolet divergences other than the standard ones, when evolving through the matching time. We also show how the de Witt-Schwinger expansion, which can be used to construct the counter-terms regularizing the ultraviolet divergences, ceases to be valid well before horizon exit of the scales of interest. Thus, such counter-terms should not be used beyond the time of the horizon exit so it is unlikely that the observed power spectrum is modified by adiabatic subtraction as claimed in the literature. However, the infrared regularization might have an impact on the observed spectrum, and we briefly discuss this possibility.