A new columnar phase Bi11.65Te1.35V5O34-delta (delta similar to 1.3) containing VO4 tetrahedra has been identified for the first time in the Bi2O3-TeO2-V2O5 system. The phase formation and the extent of substitution of Te4+ for Bi3+ ions in order to stabilize V5+ in this composition have been confirmed by the single crystal analysis, combined with the powder X-ray diffraction of the solid state synthesized bulk crystalline samples. The oxide crystallizes in a monoclinic crystal system, space group P2/c, with unit cell parameters a = 11.4616(7) angstrom, b = 5.7131(3) angstrom, c = 23.5090(18) angstrom, beta = 101.071 degrees (6) (Z = 2). The structure retains the basic features of the columnar oxides with the presence of [Bi10.65Te1.35O14](n)(95.35n+) columns along the (010) direction, surrounded by (VO4) tetrahedra placed in the planes parallel to (100) and (001), with an isolated bismuth atom in between the columns. The composition with a limited Te4+ substitution, Bi11.65Te1.35V5O34-delta (delta similar to 1.3), exists with a surprisingly high oxygen deficiency as compared to the stoichiometrically known columnar oxides such as Bi13Mo4VO34, Bi12Te1Mo3V2O34, and Bi11Te2Mo2V3O34. The structure of this interesting member of the columnar family of oxides based on the single crystal X-ray diffraction and the Raman spectroscopic studies indicates the possibility of the distribution of the oxygen vacancies among the VO4 tetrahedral units. Further confirmation for the formation of vanadium stabilized columnar structures has been provided by the successful preparation of Bi11.65Te1.35V4CrO34-delta (delta similar to 0.83) and Bi11.65Te1.35V4WO34-delta (delta similar to 0.83) phases. Preliminary investigation of the photocatalytic efficiencies of the oxides Bi11.65Te1.35V5O34-delta, Bi11.65Te1.35V4CrO34-delta,5, and Bi11.65Te1.35V4WO5 revealed moderate photocatalytic activities for the decomposition of the dyes such as Rhodamine B under UV-vis light irradiation.