It's a dialogue between shader and reality. The metamorphoses of touching can raise people’s brittleness about boundary of dematerialized shader and actual screen. The screen can be regarded as the spectrum of experiencer’s volatile emotions. Distorting the limits and immersing the spectator into a short detachment of reality. Nothing is uncounted experience, creating an immaterial boundary is the process of people wander their heart. It’s all start from the screen seduced you to interact with it.
Basically, I used Kinect to detect depth image and I read depth gray image into brightness color pixels. Then I used certain range of pixels to trigger each single scenes.
The fabric I chosen is spandex. Through measuring, the elasticity of spandex can be pulled off more than 20cm, which is quite good for depth detecting. The problem is the ratio of projector I used is 16 : 9, but the proportion of Kinect 2 depth image is 6 : 9(approximate to 640 * 480P). Thus, when mapping to the real screen, I could not get the accurate position output. The tricky way to solve is that mapping to the same ratio. In this project, I need to figure out noises problem too as the leg of timber structure can influence detection of Kinect. So the solution ended up with mapping specific area in gray depth image to frame which has already been changed mapping proportion. Consequently, I wrote this in the nested for loop when I go through each pixels of depth image.
In total, there are 8 different scenes, which all expressing lexical distortions of shader and real life. Blurring the boundary and submerging the audience into a ‘transition’ environment.
The first scene, when you put your hand in the left top corner, the iridescent metamorphosis will become blur, there will be less and less drape. By contrast, if you slightly press your hand towards right direction, it will come out peristaltic and dynamic drape.
In this stage, the ripple will follow your hand position as long as you press the screen. It will go back to original place when you release your hand. In here is your hand instead of not hands, otherwise the flickering ripple will jump between your hands where no matter which one is touching.
In this scenery, what the interaction behavior is simply sliding the hands between left and right corners. Then you will see the cloth wireframe flutter lightly, along with your hands orientation.
scene 4, Black and white moving strips will make people vertigo. But you can control it, if you insert your hand or whichever part of your body into middle screen position(I’m not really mean insert). Then you can be able to slow it down. Also, the middle part would become the largest concentration. If moving towards left side, you will make a cluster of intensive lighting strips toward right side, vice versa.