============== About ==============

This code package was authored by Renjiao Yi.

This package contains a set of networks and codes for face highlight removal and illumination estimation from a single face image. 

Please read the following papers for details. Please also cite the following work in any resulting publication if you use the codes for academic research purposes.

Renjiao Yi, Chenyang Zhu, Ping Tan and Stephen Lin,
"Faces as Lighting Probes via Unsupervised Deep Highlight Extraction" (to appear in ECCV2018)
https://arxiv.org/abs/1803.06340v2 (with supplementary materials)

If you have any questions about the codes, please send emails to renjiaoy@sfu.ca

All rights reserved.

========== Testing routine ===========
1. Download codes for "Direct Intrinsics: Learning Albedo-Shading Decomposition by Convolutional Regression" (https://github.com/tnarihi/direct-intrinsics) and install their caffe branch

2. Replace data_layers.py, loss_layers.py and vision_layers.py in "$WhereYouPutCaffeDir$/modified_caffe/caffe_helper/python/caffe_helper/layers" with our provided ones

2. make sure you have these depencies:
BOOST/1.61.0
OPENCV/3.1.0-CUDA80
PYTHON/2.7.11-SYSTEM

and make sure CAFFE_ROOT and PYTHONPATH are updated:
export CAFFE_ROOT=$WhereYouPutCaffeDir$/modified_caffe/caffe-future:$WhereYouPutCaffeDir$/modified_caffe/caffe_helper
export PYTHONPATH=$PYTHONPATH\:$WhereYouPutCaffeDir$/modified_caffe/caffe-future/python:$WhereYouPutCaffeDir$/modified_caffe/caffe_helper/python 

3. Before running Highlight-net, run "preprocessphoto.m" first to preprocess the face photo, roughly crop the face region before preprocessing would provide better results for landmark detection 
(this step require the landmark detection codes by "From Pixels to Response Maps: Discriminative Image Filtering for Face Alignment in the Wild", download their codes and put Demo0.m into the folder where Demo.m is, add the folder to matlab paths)

4. run testing script of highlight (put ***-meannorm.bmp to Renjiao_data/2/, write the file name to testing-hnet.txt, then run testing script/rjpredict-highlight.py)

5. use preASnet.m to prepare input files for albedo-shading net, run postprocess.m to get highlight removal results on original images (the original face images before preprocessing)

6. run testing script of AS-net (put ***-hf.bmp to Renjiao_data/, write the file name to testing-asnet.txt, then run testing script/rjpredict-albedo.py and testing script/rjpredict-shading.py)

7. run highlight2illumination.m to get estimated illumination, generate HDR file named "result.hdr"

========== Files provided ===========
snapshot__iter_2035.caffemodel: finetuned Albedo-shading net
snapshot__iter_28154.caffemodel: finetuned Highlight-Net
snapshot__iter_40001.caffemodel: pretrained Highlight-Net

matlab: matlab files
pred: testing output dir
Renjiao_data: testing input dir
testing scripts: scripts for testing
training scripts: Highlight-Net finetuning scripts