Field-directed colloidal assembly has shown remarkable recent progress in increasing the complexity, degree of control, and multiscale organization of the structures. This has largely been achieved by using particles of complex shapes and polarizabilites (Janus, patchy, shaped, and faceted). We review the fundamentals Of the interactions leading to the directed, assembly of such. structures, the ways to simulate the dynamics of the process, and the effect of particle size, shape,, and properties on the type of structure obtained. We discuss how directional polarization interactions induced by external electric and Magnetic fields can be used to assemble complex particles or particle mixtures into lattices of tailored structure. Examples of such systems include isotropic and anisotropic shaped particles with surface patches, which form networks and crystals of unusual symmetry by dipolar, quadrupolar, and multipolar interactions in external fields. The emerging trends in making reconfigurable and dynamic structures are discussed.