This article presents a general framework for modeling memory colors and provides experimental evidence supporting this model for one particular object, i.e., a banana. We propose to characterize memory colors from experimentally determined similarity judgments of apparent object colors with the prototypical color of that object category. The aim of the first experiment was to analyze the memory representation of banana color in the CIELUV color space. To this end, we prepared images imitating different colors of a banana and asked subjects to scale the similarity in color of a banana shown on a CRT display and ct typical ripe banana as it is remembered from their past experience. The relationships between the similarity judgments and chromaticity coordinates representing the manipulated banana samples can be well described by a bivariate normal distribution, Another experiment was carried out to gain more insight into the perception process leading to an appearance of the banana stimuli. Additionally, a sample of banana colors from a fruit market was measured and compared with the similarity judgments,