Pattern-driven design method is an important data-driven design method for printed fabric motif design in textiles and clothing industry. We introduce a novel framework for automatic design of color patterns in real-world fabric motif images. The novelty of our work is to formulate the recognition of an underlying color pattern element as a spatial, multi-target tracking, classification, segmentation and similarity association process using a new and efficient color feature encoding method. The proposed design method is based on pattern-driven color pattern recognition and indexing from the element image database. A series of color pattern recognition algorithms are used for color and pattern feature extraction. The local statistical corner features and Markov random field model are used for motif unit tiling detection and conversion. The color feature encoding problem is modeled in a gray-scale color difference optimization problem, which can be solved quickly by existing algorithms. Color pattern feature matching, segmentation and indexing techniques are then used to locate and replace the elements in the motif unit image with similar elements in the database. Experiments show that the approach proposed in this study is effective for color pattern recognition and printed fabric motif design.