Combining X-ray single crystal and neutron powder diffraction data, the crystal structures of the three polymorphs of Bi4V2O11 were refined. These polymorphs can be described from a common orthorhombic mean cell with approximate parameters: a(m) approximate to 5.5, b(m) approximate to 5.6 and c(m) approximate to 15.3 Angstrom. They are all built upon well-defined Bi2O22+ layers spaced by perovskite-like VO3.5 slabs. A large disorder of the oxygen located in these slabs was observed for the -y phase in which the diffusion is very fast. The decrease in temperature leads to an ordering of the oxygen vacancies in the beta and alpha-polymorphs. The oxygen packing in the beta-polymorph remains close to that of a distorted perovskite with a vacancy located on one site. This perovskite lattice is not maintained in the alpha-polymorph. The beta to a transition is reconstructive with the formation of typical trigonal bipyramids leading to a 6a(m) superstructure. Because of the high parameter to data ratio, the structure of this alpha-polymorph was modelled in the smaller 3a(m) superstructure. This superstructure can be explained by an ordering of some oxygen vacancies along [010]. However, this ordering could be more complex and some disorder may remain in this area, which would explain the low activation energy observed for this alpha-polymorph. The specific structural characteristics of the three polymorphs of Bi4V2O11 are correlated with their corresponding conductivity and activation energy