From Colin Ware, an expert on human perception at the University of New Hampshire: "Research into the capacity of visual working memory is just beginning. Of course if we translate the colors to words, then it is likely that the answer is yes. Color space is 3 dimensional (one dimension is black/white or luminance). Thus there are simply a lot more distinct color names, in the two color dimensions (red-green, yellow-blue) than in the single luminance (black-white) dimension."
If you have access to a big public or university library you can try reading this paper. Ask the librarian to help you find it:
Vogel, E.K., Woodman, G.F. and Luck, S.J. (2001) Storage of features, conjunctions and objects in visual working memory. Journal of experimental Psychology, 27(1) 92-114.