In this target article we question the assumption that perception is divided into separate domains of vision, hearing, touch, taste, and smell. We review implications of this assumption for theories of perception and for our understanding of ambient energy arrays (e.g., the optic and acoustic arrays) that are available to perceptual systems. We analyze three hypotheses about relations between ambient arrays and physical reality: (1) that there is an ambiguous relation between ambient energy arrays and physical reality, (2) that there is a unique relation between individual energy arrays and physical reality, and (3) that there is a redundant but unambiguous relation, within or across arrays, between energy arrays and physical reality. This is followed by a review of the physics of motion, focusing on the existence and status of referents for physical motion. Our review indicates that it is not possible, in principle, for there to be a unique relation between physical motion and the structure of individual energy arrays. We argue that physical motion relative to different referents is specified only in the global array, which consists of higher-order relations across different forms of energy. The existence of specificity in the global array is consistent with the idea of direct perception, and so poses a challenge to traditional, inference-based theories of perception and cognition. However, it also presents a challenge to much of the ecological approach to perception and action, which has accepted the assumption of separate senses.