Roland William Fleming

Fleming’s early works focused on the visual perception of the optical properties of surfaces and materials, such as gloss,[19] translucency[20] and transparency.[21] He helped determine the role of visual cues such as motion [22] and binocular stereopsis [23][24][25] in the perception of surface reflectance, especially gloss. A recurring theme within this work was the concept that specular reflections behave unlike surface markings—such as pigmentation patterns or scratches—leading to specific visual cues for identifying specular reflections and therefore glossy surfaces. He also investigated how multi-component patterns of specular reflection lead to hazy glossy appearances.[26] His more recent studies on surface appearance have tested whether artificial neural networks can reproduce the patterns of errors and successes that human observers make when judging material properties.[27][28][29]

In addition to studying how the visual system estimates optical properties of materials, he has also investigated the relationship between other material properties and material categories, and how these are affected by the viewing distance, as in the so-called ‘material-scale ambiguity’.[30]

Fleming also led a number of studies on how the visual system infers the mechanical properties of materials, such as compliance,[31] elasticity,[32] and viscosity [33][34] [35] [36] from optical, shape and motion cues.[37][38] Most of these studies used finite elements computer simulations of liquids or deformable solids interacting with their surroundings. A recurring theme within this body of work is the idea that the visual system represents stimuli in a multi-dimensional space of midlevel visual features, which statistically characterize how shape, motion and appearance evolve over time. He and his colleagues have claimed that such representations facilitate disentangling intrinsic material properties from other factors that also contribute to the proximal stimulus[18][35] such as a flowing liquid’s speed, or the force deforming a compliant solid.

Early in his career, Fleming argued that the visual system infers material properties through heuristics, using simple image statistics that correlate with surface properties under typical viewing conditions.[5][19][20] Later however, he proposed that the visual system uses richer internal models of the appearance of objects and materials under typical viewing conditions—an idea he calls ‘Statistical Appearance Models'.[16][39] Specifically, he has suggested that the visual system acquires the ability to infer material properties (or other distal stimulus properties) by learning generative models of proximal stimuli through unsupervised learning objectives, such as compressing or predictive coding of image content.[40][41] A proof-of-concept of this theory was demonstrated by training an unsupervised artificial neural network model on a dataset of computer rendered images of bumpy, glossy surfaces.[42] Fleming and his colleagues found that the model spontaneously learned to disentangle scene variables—such as lighting and surface reflectance—even though it was given no explicit information about the true values of these variables. Moreover, the model correctly predicted both successes and failures (i.e., illusions) of human gloss perception.

Fleming’s early works focused on the visual estimation of three-dimensional (3D) shape from specular reflections,[43][44] shading [45][46] and texture.[47] He is particularly known as a proponent of the role of ‘orientation fields’ in shape perception.[48][49] Orientation fields refer to spatially varying patterns of the dominant local orientation across the image of a surface, as measured by populations of orientation-selective neurons at each image location.[45][43][44][46][47] Fleming and his colleagues have shown that local image orientation signals tend to vary smoothly across curved surfaces in ways that are systematically related to 3D shape properties.[50] For textured surfaces, local image orientation is related to first-order shape properties, especially surface slant and tilt.[47] For shading patterns and specular reflections, local image orientation structure is related to second-order shape properties, especially the direction of minimum second derivative of surface depths, and the ratio of minimum and maximum second derivative magnitudes.[43][46] He has argued that orientation fields provide a fundamental source of information about shape,[51][52][53] and that their use by the visual system predicts specific illusions of perceived shape, such as when illumination changes.[54]

In addition to the visual estimation of 3D shape, Fleming has also investigated the perceptual organization of shape[55][56] and the use of shape to make additional inferences about objects and their properties,[31][32][33][35][57]—a process he calls ‘Shape Understanding’.[10] Fleming led a number of studies on how the visual system makes inferences about the processes and transformations that have formed objects or altered their shape.[58][59][60][61][62][63][64] A recurring theme within this body of work is that an object’s ‘causal history’ leaves traces in its shape, which can be used to identify which of its features are the result of shape-altering transformations.[59][61] Such transformations include simple spatial distortions[58][59] and more complex biological growth processes.[60] By analogy to the visual system’s ability to separate images depicting transparent surfaces into multiple distinct causes, Fleming and his colleagues refer to the separation of shape into distinct causes as ‘Shape Scission’.[64] An example of this is the ability to distinguish the causes of different shape features that occur when a face or object is fully covered with a cloth veil.[65] Some of the visible features of the surface of the cloth are caused by the textile draping of its own accord, while others are due to the protrusion of the underlying object. Observers can distinguish these causes, even when the hidden object is of unknown shape.[65]

Fleming has also investigated the role of shape in object categorization, especially in one-shot learning of novel object categories from a single (or small number of) exemplars.[66][67] In this context, Fleming and colleagues developed a computational model for predicting the perceived similarity between pairs of two-dimensional (2D) shapes, called ‘ShapeComp’.[68] The model combines a large number of shape features to capture different aspects of shape. He and his colleagues have also studied how shape cues contribute to the visual perception of animacy,[69] and conversely how semantics alter the perceptual organization of shape.[70] Fleming and colleagues have argued that human visual one-shot categorization involves inferring a generative model from the exemplar object.[66][67] They have proposed that this involves segmenting the object into parts, and representing their relations in a way that can be modified to synthesize novel variants belonging to the same category as the exemplar. They claim that this idea is supported by experiments in which participants are presented with a single exemplar and are asked to draw novel variants.[67]

Fleming’s work in computer graphics has mainly focused on perceptually-based approaches to representing and modifying photographic imagery. He contributed to the development of image-based algorithms for altering the material appearance[71] and shape[72] of objects in photographs. His work on orientation fields led to methods for synthesizing images of objects with particular 3D shape and material appearance based on purely 2D image operations.[73] He also contributed to work investigating perceptually-based methods for converting between and presenting conventional (low-dynamic range) and high-dynamic range images.[74][75] He co-authored a text book entitled “Visual Perception from a Computer Graphics Perspective”[76]

Fleming’s work on motor control has focused primarily on the effects of 3D shape[77][78][79] and material properties[78][80][81]—including mass,[80][81] friction[80][82] and rigidity[83]—on grasping. He and his colleagues have investigated various illusions related to grasping,[81][84] including the ‘material-weight illusion,[81] a variant of the size-weight illusion, in which the expected weight of an object is manipulated through its surface material (instead of its volume as in the size-weight illusion). He and his colleagues developed a computational model for predicting human precision grip (thumb and forefinger) grasp locations on objects with varying 3D shape and materials properties.[78] The model combines multiple cost functions related to the properties of the object and the actor’s hand. The model predicted average human grasp locations approximately as well as different individuals’ grasps predict one another.