We discuss the impact of adiabatic renormalization on the power spectrum of scalar and tensor perturbations from inflation. We show that adiabatic regularization is ambiguous as it leads to very different results, for different adiabatic subtraction schemes, both in the range $v\equiv k/(aH) \gsim 0.1$ and in the infrared regime. All these schemes agree in the far ultraviolet, $v\gg 1$. Therefore, we argue that in the far infrared regime, $v\ll 1$, the adiabatic expansion is no longer valid, and the unrenormalized spectra are the physical, measurable quantities. These findings cast some doubt on the validity of the adiabatic subtraction at horizon exit, $v=1$, to determine the perturbation spectra from inflation which has recently advocated in the literature.