When multiple stimuli appear simultaneously in the visual field, they are not processed independently, but rather interact in a mutually suppressive way, suggesting that they compete for neural representation in visual cortex. The biased competition model of selective attention predicts that the competition can be influenced by both top-down and bottom-up mechanisms. Directed attention has been shown to bias competition in favor of the attended stimulus in extrastriate cortex. Here, we show that suppressive interactions among multiple stimuli are eliminated in extrastriate cortex when they are presented in the context of pop-out displays, in which a single item differs from the others, but not in heterogeneous displays, in which all items differ from each other. The pop-out effects seemed to originate in early visual cortex and were independent of attentional top-down control, suggesting that stimulus context may provide a powerful influence on neural competition in human visual cortex.