Paper Reading #10: Enabling Beyond-Surface Interactions for Interactive Surface with An Invisible Projection

Comment 1: http://csce436-nabors.blogspot.com/2011/02/reading-10-soylent-word-processor-with.html
Comment 2: http://chiblog.sjmorrow.com/2011/02/paper-reading-10-soylent-word-processor.html

Enabling Beyond-Surface Interactions for Interactive Surface with An Invisible Projection
Li-Wei Chan, Hsiang-Tao Wu, Hui-Shan Kao, Ju-Chun Ko, Home-Ru Lin, Mike Y. Chen, Jane Hsu, Yi-Ping Hung
23nd annual ACM symposium on User interface software and technology

In this paper the authors explore the idea of having a multi-touch, multi-display 3D viewing surface computer for general use. The system is intended to be used for viewing of different materials on a large surface that allows the user to use not only multi-touch but also other forms of viewing and display to enhance the users experience. The state of art work has shown similar concepts to be working through the use of tools or mouse but no one has yet to design a system that uses touch as well as other tools to enhance the user experience. The system is designed by having a large tabletop surface with a glass pane as a display surface. The table has two different camera projections from the bottom one displaying the color image and the other an infrared projection for use by tools and the multi-touch. It also includes mirrors underneath to help the image from refracting through the bottom too much as well as additional IR cameras on the top and bottom to help facilitate the use of additional tools. The system essentially works by projecting an image and an infrared grid then translating the touch view or camera view through a series of filters completing with a Kalman filter and then translating this into (essentially) a mouse click that then interacts with the application. The system has can sense objects that are placed on the table and can view these with a "region of interest" projection enabling the use of various pointer devices. The system is primary used through the use of a i-m view that allows the user to see an area of interest in a 3D display. Essentially the device is the side of an IPad and allows the user to view the 2D surface and then the image is calibrated to display in a 3D fashion. The newest idea they are also showing is the use of an i-m camera as well as an i-m lamp. These devices are similar to what they sound like allowing the user to have a fixed (lamp) or mobile (camera) view of the surface that displays a more detailed 3D Image of the area they are viewing. These devices allow the user to get a smaller more detailed view of a certain area and allows for exploring of a smaller 3D area. When developing a prototype for this they found that users were easily able to use the i-m view and i-m lamp with their intended uses but found the i-m camera rather difficult to use as due to the frequency and velocity of movement the camera was never able to "focus" and was in some cases unable to provide detailed images. They plan to add a "jitter reduction" so smaller movements by the user are largely ignored and the camera adjusts its focus to adapt. They also found the users looking at the table in a variety of orientations and found it more difficult to view at certain angles. Censors that will orient views from each side will be added to aid in this process. They considered the first view a success and were allowed to refine their technology with the resulting user studies.

I think this is one of the most interesting articles I have read from a computer science standpoint. I am trying to visualize the amount of code that needs to go into something like this to enable the user to use a screen on a 2D surface and have a 3D image of the area projected onto their screen. I think the use of IR technology greatly assists in this and while the report did go into very fine detail about the various devices and technology used it was interesting to see the various modifications and updates that the group implemented. I think the most intriguing idea that was proposed are the lamp and camera ideas that allow the users to focus in on a particular area and get a close up view with a more detailed image. I would think for viewing maps, which seemed to be their primary use for this system, this would be a large point of interest and would need to be a feature that is available in any similar system. I would think that they would have guessed that the users might be moving the camera quite fast and been able to adapt this. From reading the article they made it sound like the users moved the device much faster than intended and that was the source of confusion but I cannot be sure because the idea is still listed as a source of contention in the user feedback. I don't think that the group does a good enough job explaining what the users meant by having strange orientation problems and if they want to improve on this need to expand the idea more or give a more concise example of what the problem is. The paper is very well fleshed out and the ideas and examples given are very well done. Despite this essentially being a professional paper on their work this really makes me want to go and try this system and see how well it does work. I wonder if the system could be expanded to 3D goggles so that the user would not have to hold the i-m view display but rather just look with glasses and then use the lamp and camera for their intended uses. I also wonder if more research is going to be done on this kind of computing with the rise of 3D technologies in the market.