Controllability properties for discrete-time, Markovian quantum dynamics are investigated. We find that, while in general the controlled system is not finite-time controllable, feedback control allows for arbitrary asymptotic state-to-state transitions. Under further assumptions on the form of the measurement, we show that finite-time controllability can be achieved in a time that is twice the dimension of the system, and we provide an iterative procedure to design the unitary control actions. (C) 2011 Elsevier Ltd. All rights reserved.