This paper presents a study into the utilization of Gaussian Processes (GPs) for probabilistic forecasting of residential electricity consumption, photovoltaic (PV) power generation and net demand of a single household. The covariance function that encodes prior belief on the general shape of the time series plays a vital role in the performance of GPs and a common choice is the squared exponential (SE), although it has been argued that the SE is likely suboptimal for physical processes. Therefore, we thoroughly test various (combinations of) covariance functions. Furthermore, in order bypass the substantial learning and inference time accompanied with GPs, we investigate the potential of dynamically updating the hyperparameters using a moving training window and assess the consequences on predictive accuracy. We show that the dynamic GP produces sharper prediction intervals (PIs) than the static GP with significant lower computational burden, but at the cost of the ability to capture sharp peaks. In addition, we examine the difference in accuracy between a direct and indirect forecasting strategy in case of net demand forecasting and show that the latter is prone to producing wider PIs with higher coverage probability.