This article investigates the robustness of strong structural controllability for linear time-invariant and linear time-varying directed networks with respect to structural perturbations, including edge deletions and additions. In this direction, we introduce a new construct referred to as a perfect graph associated with a network with a given set of control nodes. The tight upper bounds on the number of edges that can be added to, or removed from a network, while ensuring strong structural controllability, are then derived. Moreover, we obtain a characterization of critical edge sets, the maximal sets of edges whose any subset can be respectively added to or removed from a network while preserving strong structural controllability. In addition, procedures for combining networks to obtain strongly structurally controllable network of networks are proposed. Finally, controllability conditions are proposed for networks whose edge weights, as well as their structures, can vary over time.