We report on the room temperature thermal conductivity of atomic layer deposition-grown amorphous TiO2 and Al2O3 thin films as a function of film thickness and atomic density. For films thinner than 50 nm, we measure an effective thermal conductivity that is reduced with decreasing film thickness. This dependence is attributed to the increased influence of thermal boundary resistances as film thickness is reduced. In addition, we fit for a thickness-independent intrinsic thermal conductivity using a series-resistor model. For films thicker than similar to 50 nm, there is no significant dependence on thickness or substrate. We observe a dependence of the thermal conductivity on density, which agrees well with a differential effective-medium approximation modified minimum limit model.