In the course, I was introduced to many different computer vision techniques and methods. Learning about optics, light field photography, edge detection techniques, and new ways of capturing and manipulating images were valuable as is was the main way people communicate with one another and as photography is the major communication tool available to us. In particular, I found detection and recognition technology (face, smile, gesture recognition) particularly fascinating because it allowed people to interact with digital devices in a more perceivable manner.
Because I believe that 3D environment provides a more realistic visual and spacial experience, I decided to design a 3D scene and relight it from detected face positions, hoping to use the technique in my future applications for entertainment. Below are the results. When I moved my fact to the right, the scene was relighted from right top and when I moved to left, the scene was relighted from the left top. (Because I was facing the webcam, the direction of my head movement is reversed)
* Used Microsoft Kinect on Mac
Here is the result I got by multiplying a number of different images is Photoshop after adjusting their color channels individually. The model that I worked from is shown below:
Assignment 2Light Fields
Light Fields certainly blew my mind away. By taking multiple pictures along the horizontal (small distances apart) and shift-adding them allowed refocusing effect at a certain depth of field. Shree Nayar's idea of generalized optics and processing were centered around the fact that lights, in fact, are and should be treated as rays. Capturing all the rays of light allows us to refocus at a certain depth and this is the basic principle that is applied to the Lytro cameras, where an array of micro-lenses capture the light information. In this experiment, we were asked to manually and computationally creating this light field effect by shifting the images by a certain amount and adding them together.
In Maya, I created a color blind test chart consisting of many spheres, two eye balls behind the test chart and a vision test chart in the back. (More information on my Assignment Page)
Images below are the results. From the images above, the vision chart in the back was not visible through the colorful spheres (occluders), but in my result image4 below, the alphabets can be read clearly.
Assignment 3Edge Detection
Using multi-flash camera that shoots light at objects from four different directions (from top, bottom, right and left) makes shadows in four different directions. These shadows can be used to detect the edge of the 3d objects and produce a contour drawing effect. Below are the results I got from taking multiple pictures with different light sources to capture the different orientation of shadows. (More information on my Assignment Page)
Webpage designed by Hye Soo Yang
Past Class Work
Here are the results of my experiments this fall.
Assignment 1 Mixing the color channel
We had to play around with color channels of multiple images. I liked the fact that shadows also changed in color when the color channel was adjusted. Therefore, I decided to make a flower out of shadows (More information on my Assignment Page)
Integration of Hand gesture recognition system
As I was working on hand gesture recognition system for my UROP, I decided to integrate the system in my scene. I was inspired by teletubbies in which the sun was a smiling face of a baby and decided to make my objects into teletubbies. The number of fingers open designates which of the four teletubbies should appear. This system combined with the face recognition system made the interface more responsive to my motions and actions, which I enjoyed myself very much. I could envision children and young students having fun with a game interface that uses such systems.