We evaluate the probability that oil prices affect excess stock returns for U.S. listed firms. The probabilities are obtained from a time-varying multi-factor asset pricing framework estimated using dynamic model averaging techniques, including oil price information among several other possible risk factors. Two widely used oil price measures are considered, one based on raw oil price changes and another based on disentangling the source of oil price changes due to supply-side or demand-side effects. As far as we know our dataset, which comprises 10,118 stock price series with up to 25,372,588 observations between 1995-2018, is the most comprehensive used for this purpose. We develop two "beauty-contests" in which we estimate the multi-factor models separately for individual stocks, for each of the two oil price measures. The results suggests that, when working with daily data (beauty contest 1), oil price changes are a significant (important) determinant for around 1-3% of the sample. When using oil price shocks-as opposed to oil price changes-(beauty contest 2) this percentage increases to 27-45%, suggesting that oil supply and demand shocks (as opposed to oil price changes) can better explain firm-level excess returns, at least for monthly frequency data where such a decomposition is available. We provide evidence that the increase in percentage is only partially attributable to data-frequency, and more likely attributed to the decomposition into supply/demand driven oil price changes. We reconcile differences between our findings and those reported in previous literature on the basis of the fully dynamic nature of our adopted methodology.