Details of Grant 

EPSRC Reference:	EP/M021793/1
Title:	Shape and Reflectance Acquisition of Complex Dynamic Scenes
Principal Investigator:	Guillemaut, Dr J
Other Investigators:
Researcher Co-Investigators:
Project Partners:	Double Negative Ltd The Foundry Visionmongers Ltd (UK)
Department:	Vision Speech and Signal Proc CVSSP
Organisation:	University of Surrey
Scheme:	First Grant - Revised 2009
Starts:	01 June 2015	Ends:	31 May 2017	Value (£):	99,140
EPSRC Research Topic Classifications:	Computer Graphics & Visual. Image & Vision Computing
EPSRC Industrial Sector Classifications:	Creative Industries
Related Grants:
Panel History:	Panel Date Panel Name Outcome 14 Apr 2015 EPSRC ICT Prioritisation Panel - Apr 2015 Announced
Summary on Grant Application Form
Scene modelling is central to many applications in our society including quality control in manufacturing, robotics, medical imaging, visual effects production, cultural heritage and computer games. It requires accurate estimation of the scene's shape (its 3D surface geometry) and reflectance (how its surface reflects light). However, there is currently no method capable of capturing the shape and reflectance of dynamic scenes with complex surface reflectance (e.g. glossy surfaces). This lack of generic methods is problematic as it limits the applicability of existing techniques to scene categories which are not representative of the complexity of natural scenes and materials. This project will introduce a general framework to enable the capture of shape and reflectance of complex dynamic scenes thereby addressing an important gap in the field. Current image or video-based shape estimation techniques rely on the assumption that the scene's surface reflectance is diffuse (it reflects light uniformly in all directions) or assume it is known a priori thus limiting the applicatibility to simple scenes. Reflectance estimation requires estimation of a 6-dimensional function (the BRDF) which describes how light is reflected at each surface point as a function of incident light direction and viewpoint direction. Due to high dimensionality, reflectance estimation remains limited to static scenes or requires use of expensive specialist equipment. At present, there is no method capable of accurately capturing both shape and reflectance of general dynamic scenes, yet scenes with complex unknown reflectance properties are omnipresent in our daily lives. The proposed research will address this gap by introducing a novel framework which enables estimation of shape and reflectance for arbitrary dynamic scenes. The approach is based on two key scientific advances which tackle the high dimensionality issue of shape and reflectance estimation. First, a general methodology for decoupling shape estimation from reflectance estimation will be proposed; this will allow decomposition of the original high dimensional problem, which is ill-posed, into smaller sub-problems that are tractable. Second, a space-time formulation of reflectance estimation will be introduced; this will utilise dense surface tracking techniques to extend reflectance estimation to the temporal domain and thereby increase the number of observations available to overcome the inherently low number of observations at a single time instant. This will build on the PI's pioneering research in 3D reconstruction of scenes with arbitrary unknown reflectance properties and his expertise in dynamic scene reconstruction, surface tracking/animation and reflectance estimation. This research represents a radical shift in scene modelling which will result in several major technical contributions: 1) a reflectance independent shape estimation methodology for dynamic scenes, 2) a non-rigid surface tracking method suitable for general scenes with complex and unknown reflectance and 3) a general and scalable reflectance estimation method for dynamic scenes. This will benefit all areas requiring accurate acquisition of shape and reflectance for real-world scenes with complex dynamic shape and reflectance without the requirement for complex and restrictive hardware setups; such scenes are a common occurrence in natural environments, manufacturing (metallic surfaces) and medical imaging (human tissue) but accurate capture of shape is not possible with existing approaches which assume diffuse reflectance and fail dramatically for such cases. This will achieve for the first time accurate modelling of dynamic scenes with arbitrary surface reflectance properties thus opening up novel avenues in scene modelling. The application of this technology will be demonstrated in digital cinema in collaboration with industrial partners to support the development of the next generation of visual effects.
Key Findings
This information can now be found on Gateway to Research (GtR) http://gtr.rcuk.ac.uk
Potential use in non-academic contexts
This information can now be found on Gateway to Research (GtR) http://gtr.rcuk.ac.uk
Impacts
Description This information can now be found on Gateway to Research (GtR) http://gtr.rcuk.ac.uk Summary Date Materialised
Sectors submitted by the Researcher
This information can now be found on Gateway to Research (GtR) http://gtr.rcuk.ac.uk
Project URL:
Further Information:
Organisation Website:	http://www.surrey.ac.uk