Firms mitigate uncertainty in demand and supply by carrying safety stock, planning for excess capacity and diversifying supply sources. In this study, we provide a framework to jointly optimize these three levers in a periodic review infinite horizon setting, and in particular we examine how one can reduce inventory and capacity investments through proper diversification strategies. Observing that a modified base-stock inventory policy is optimal, we find that the capacity-diversification problem is well behaved and characterize the optimal mix of safety stock, excess capacity and extra number of supply sources. We find that higher supply uncertainty results in higher safety stock, more excess capacity, and higher diversification. But safety stock and diversification are non-monotonic in demand uncertainty. Our results can be extended to situations in which suppliers are heterogeneous, and can be used to develop effective heuristics.