A novel Bayesian Augmented-Learning framework, quantifying the uncertainty of spectral representations of stochastic processes in the presence of missing data, is developed. The approach combines additional information (prior domain knowledge) of the physical processes with real, yet incomplete, observations. Bayesian deep learning models are trained to learn the underlying stochastic process, probabilistically capturing temporal dynamics, from the physics-based pre-simulated data. An ensemble of time domain reconstructions are provided through recurrent computations using the learned Bayesian models. Models are characterized by the posterior distribution of model parameters, whereby uncertainties over learned models, reconstructions and spectral representations are all quantified. In particular, three recurrent neural network architectures, (namely long short-term memory, or LSTM, LSTM-Autoencoder, LSTM-Autoencoder with teacher forcing mechanism), which are implemented in a Bayesian framework through stochastic variational inference, are investigated and compared under many missing data scenarios. An example from stochastic dynamics pertaining to the characterization of earthquake-induced stochastic excitations even when the source load data records are incomplete is used to illustrate the framework. Results highlight the superiority of the proposed approach, which adopts additional information, and the versatility of outputting many forms of results in a probabilistic manner.