In human vision, the perception of localized stimuli is strongly influenced by the presence and nature of surrounding elements. It has been suspected that these contextual effects are linked to the processes of image segmentation and recognition, by enhancing the representation of specific configurations of elementary features which are typical for certain objects or other important aspects of a visual scene. In this contribution, we show that psychophysical detection thresholds for stimulus configurations comprising four Gabor patches of different orientations and spatial frequencies are strongly related to the probabilities that these configurations occur in 'natural' images. This almost perfect match holds for patch distances of 2.8 degrees of visual angle, whereas for 1.4 degrees of visual angle we find strong inhibitory effects, actually leading to increased thresholds for all configurations. Our results suggest that natural image statistics capture specific patterns of local interactions in early, feature-specific layers in visual cortex. This finding indicates that the zoo of contextual interactions actually observed both in psychophysics and electrophysiology may be interpreted in a more systematic way by a careful analysis of the statistical properties of our environment.