In some situations where market information is incomplete or not reliable, building financial term-structures (interest-rate curves, credit curves, volatility surfaces, ...) may be associated with a significant degree of uncertainty. We propose a new arbitrage-free construction method that extends classical spline techniques by additionally allowing for quantification of uncertainty. The proposed method is based on a generalization of kriging regression models to linear equality constraints (market-fit conditions) and shape-preserving constraints (no-arbitrage conditions). Prices of illiquid instruments can also be incorporated when considered as noisy observations. We define the most likely response surface and show how to build confidence bands. The Gaussian process hyper-parameters under the construction constraints can be estimated using cross-validation techniques or maximum likelihood. Based on observed market quotes at different dates, we demonstrate the efficiency of the method by building curves/surfaces and their corresponding confidence bands for OIS discount rates, CDS implied default probabilities and implied volatilities.