Sixth Sense Technology and Its New Applications

Sixth Sense Technology and Its New Applications

Deepesh Sharma

Department of Electronics & Communication Vivekananda Institute of Technology, Jaipur Jaipur, India

Abstract During the attention at a new object our senses try to sense and analyze that particular object and make us familiar with them. But to know the complete information about a particular object we should go through it in detail either by enquiring to someone who is already familiar or by surfing on internet. In this paper, we tried to design an application for sixth sense enabled college. This application can be used by the institutions to know about the academic qualifications of a particular student, student can also know about the upcoming events and other information from notice boards without actually physically being present near the notice board. Anyone can get the review of a book and its availability in the library. Technology related to sixth sense, its advantages and applications are also presented in this paper.

Keywords Sixth Sense Technology, Gesture recognition, Evolution, Cloud computing, Applications.

INTRODUCTION

The modern era is ruled by computers. Computers have become indispensable part in almost every field like offices, hospitals, educational institutions, super markets, households etc. Many types of software have been enabling us to do our task in a much easier way. For each and every word we tend rely on computers for example booking an online ticket, paying telephone bills, online shopping etc. With the passage of time many things have changed and many new things have come into limelight and computers are no such exceptions. Scientists are trying to develop a technology to connect the physical world with the digital world. Humans are blessed with five senses viz. eyes, ears, nose, tongue and skin which perform their specific functions i.e. eyes for the purpose seeing, ears for the purpose of listening, tongue for the purpose of tasting, skin for the purpose of touching and nose for purpose of smelling as shown in Fig 1. Like senses enables us to perceive information about the environment in different ways there could be another sense which would enable us to sense the things which are beyond biological arena.

Fig 1: Five Senses of Humans

Sixth sense technology can replace every task through hand gestures. It can be used in colleges, supermarkets, schools, map navigator, jewelry shops etc. Using hand gestures we can zoom in or zoom out or pan like any other multi touch system does. Sixth sense technology can also be used to check the quality of a particular object, directions to particular location, etc. Augmented Reality [4] is the technology which can act as a bridge physical world and digital world.

Augmented reality bridges virtual world with the real world. The concept of augmented reality can be combined with sixth sense in order to have multitasking gadgets that can reduce the burden on machines. As more and more sixth sense technology will penetrate, the task for humans will become simpler and efficient.

1. SIXTH SENSE TECHNOLOGY

   We know, previously many technologies have evolved such as augmented reality where the superimposition of graphics, audio, and other enhancements were used. Unlike virtual reality, augmented reality does not forge a simulation of reality, instead it blurs the line to show the difference between what is real and what is computer generated by enhancing five senses. In simple words, augmented reality is changing the way our world is perceived and sixth sense technology is making it appear in real.

   Initially Steve Mann proposed the idea of a neck worn projector with a camera for implementing sixth sense technology. Later, Pattie Maes and Pranav Mistry carried his work and developed augmented system which they call Sixth Sense, presented in TED conference [4]. The Sixth Sense device assumes any plane surface as an interface with which we can interact physically.

   The prototype for the SIXTH SENSE is an on mode wearable gestural interface that augments the physical world around us with the digital information. The sixth sense prototype is made using very common and readily available equipment like pocket projector, a mirror, mobile components, colored caps and a camera as shown in Fig 2. The Bluetooth gives the connectivity between the projector and the camera. Natural hand gestures are used to interact with the digital data.

2. GESTURE RECOGNITION

   It is a technology which is aimed to interpret human gestures with the help of mathematical algorithms. Gesture recognition technique basically based on the emotion recognition from the face and hand gesture recognition. Gender recognition technique allows humans to interact

   with computers in a more direct way without using any external interfacing devices. It can provide a much better alternative to text user interfaces and graphical user interfaces which requires the need of a keyboard or mouse to interact with the computer.

   An interface which solely relies on the gestures requires precise hand pose tracking. Earlier the versions of the gesture recognition processes specializes in the type of hand gloves which provides the information according to the hand positions and orientation and along with the fluxes of the fingers. In all of the Sixth Sense devices, the colored bands are used for same processing. Once hand pose has been captured the gestures can be recognized using different techniques.

   The statistical templates or the neural network approaches are the commonly used techniques used for the recognition purposes. This technique has a high accuracy usually showing accuracy of more than 95%. Time dependent neural network can also be used for real time recognition of the gestures

3. SIXTH SENSE PROTOTYPE

   In the sixth sense technology, the uses of various equipment are Camera to capture the image of the object which is found in its view and it can be considered as digital eye.

   Fig 2: Motion Capture [4]

   Projector is used project the visual information on to the surfaces and objects making them as physical interfaces. Mirror is used for the purpose of adjusting the angle between the projector and the screen. It reflects the rays coming from the projector on to the screen. A smart phone with an access to Internet is used in order to process the video data. Colored Caps are placed at the tip of users fingers.

   They are marked with red, yellow, green and also with blue tapes. These help the camera to recognize the gestures. All these hardware components are further coupled in a pendent. Gesture recognition is a term used to interpret the gestures through mathematical algorithms and mathematical gestures.

   Gesture recognition can be seen as a way for computers to perceive human body language, thus building a bridge between machines and humans than primitive text user interfaces. Gesture recognition can be conducted with techniques from computer vision as well as with the image processing.

   The digital vision in the field of science and technology of machines that is concerned along with the theories behind artificially designed systems that extracts the information from processed images. In the Sixth Sense technology, the gesture recognition is been considered in order for the better understanding of the motion from the fingers.

4. ADVANTAGES OF SIXTH SENSE

   The Sixth sense technology featuring along with a user guide that makes it easier to use. There are various important gestures which an individual can easily learn without going through tutorials. The various advatages of the Sixth Sense Technology are as follows.

   Capturing Images

   In order to take a snap of a picture, the user has to make a rectangular division in air as shown in Fig 3. The picture would be taken in that division and will be transferred in memory. The taken image can easily be resized and edited whenever needed for the projection on any hard surface like wall or a screen.

   Fig 3: Capturing image method for rectangular division

   Cell Phone Dialer

   Cell phones can also get equipped with sixth sense technology. Instead of carrying a cell phone in hand, a person can project the keypad on his palms and make calls just by touching the digital keys(acting as virtual keys) and press the keys. There will not be any limitation for screen and any medium can become act as output medium [3]. Sixth sense in other words will merge the digital and real world together.

   Reading Books

   In library, a person can easily identify whether a particular book is interesting or not based on the rating and reviews of the book which is viewed. This makes things easier and time efficient. People can easily decide whether to go for that book or look for another book and thus saving precious time.

   Video Newspaper

   Instead of going through a newspaper which has static images a person can view a video regarding particular news as shown in Fig 4. In this case, Sixth Sense pulled up a video online pertaining to the same news that is published in the newspaper. This device can be used by all the people easily because it uses hand gestures which everyone uses in daily life.

   Fig 4: Video Newspaper [2]

   Quality Check

   In shopping malls, one can easily check the quality of food with this technology, quality and reviews of other products they wish to buy. This process reduces the customer- complaints in the malls hence increasing the profits and services to the customers.

5. VARIOUS APPLICATIONS

   There are various applications of Sixth Sense which exhibits the efficiency, viability and flaccidity of the system.

   Make a Call: This technology enables the user to call without using the dialer. The information of the dialer would be projected on the device as soon as the user brings the palm in front of the device. The user do not need any cell phone to make the calls, rather than a virtual keypad would be used for the projection on the palm.

   The Map: The map application allows the user to navigate a map that displayed on a nearby surface by using our hand gestures, which is similar to the gestures supported by Multi-Touch based systems permitting the user to zoom in, zoom out as well as pan using intuitive hand movements.

   3D Drawing Application: The drawing application of this device allows the user draw on any surface by tracking the tips movements of the users index finger.

   The Clock: The user simply needs to make a gesture of a circle on the wrist and the clock with current time will be projected on the users hand. 4.6 Motion Capture: Using fingers, the user can capture photos and there no need to carry an extra gadgets. The box created by fingers act as a frame. These photos could be edited also or shared with people. 4.7 Video Newspapers: This device also recognizes articles in the newspaper. Also it can retrieve the latest related stories or video stream from the internet and then display them on pages for user. It had said goodbye to dictionary and encyclopedia.

   Book Information: For book lovers it has proved to be a boon as you just open any book and find the ratings of the book and can also move to any page and get additional information. Similarly, we can find information regarding any product.

   Get Flight Updates: The device can also tell the user whether the flight is delayed or is on time by looking at the ticket giving complete information by what time flight is delayed.

   Fig 6: Flight Enquiry

6. VARIOUS SECURITY THREATS

   Every technology available in the market faces some severe security threats. With sixth sense devices also there are some security concerns due to which it is not rampantly used market. There are several Cyber warfare with Sixth Sense. Cyber warfare also known as Cyber War are some virtual conflict initiated to attack on enemys computer and information system. Using Internet these attacks can disable financial and organizational systems by stealing or altering classified data to undetermined network websites and services. It can lead to military and financial computer systems at risk and Security breaches.

   A Study on – The Sixth Sense Technology reveals that its Various Security Threats 667 are some weaknesses in the device that can reduce the accuracy of the data.

   Let us study all the security issues:

   • No Privacy: When a person is taking a picture using Sixth Sense technology, he just snaps with his fingers without even intimating the person or thing being clicked. Person without giving any indication is taking a picture which can just be captured by hand gestures and the device is also not that big. Also people dont want to be captured on a camera of any unknown person. This is invading others privacy which can be a huge problem.

   • Intelligence opening doors for criminals: This device can help in collecting good amount of information without getting caught or being tagged as a suspicious person. When the person having this device enter any office or building can use PICTURE application to collect all the relative and secret information about that office or building. Secondly, as far as OSINT is concerned it is endless. OSINT stands for Open Source Intelligence; that is collected from publically available sources. By using the Facebook database one can recognize people on the streets; knowing what they do, where they live, where the work, how much they earn, etc. All the personal information can be extracted. This will definitely opening doors for criminals also.

   • Tool for Hackers: A hacker can also use this device for attacking an online environment using functions on the device to break in building. It can lead to consequences of Hacktivism which is closer to cyber terrorism. Hacktivism can be understood as an activity that can cause malicious, destructive, and undermining the security of the Internet as a technical, economic and political platform. Hackers can use this device as a weapon to feed or can provide information to Hacktivist army.

   • Cyber Security: Sixth Sense Device could actually be a real trouble if one can communicate and share information with other Sixth Sense devices means one can manipulate them and infect them with malware. Malwares are software used to disrupt computer operations gather sensitive information or gain access to private computer system. It could change credentials, steal credentials, provide wrong information or wrong agenda.

   • Health Issues: When the device is projecting on hard surface, it is not only private for the user using it, rather people around him can see the projection very easily since it is very definite and exact. As we all know that projection is much better at night time and dark areas than morning and bright areas. This is a bigger issue from health perspective that vision of the user can be damaged. The device should be able to shift its projection techniques during different times of the day. Sixth Sense could be used as a weapon for mass destruction also. We cannot use it in Medical Environment. Though this device will allow us to recognize people but it is an infringement for our privacy. Pranav Mistry is trying to overcome these security threats. Pranav Mistry said This prototype needs some serious engineering & programming.

7. FUTURE ENHANCEMENTS

   As this technology will emanate may be new devices and new technologies based on it will also come up. Sixth Sense Device is totally different from the computers as it enables one to compute and browse on any surface that we can find around. The foremos thing is to overcome all the security threats possibilities discussed in previous section; as many new technologies came and died due to security issues and threats. Secondly, to get rid of the color marker, integrate camera and projector inside the mobile computing device itself. Thirdly, to implement this technology in various areas like gamming, education system etc. Fourthly, there could be a 3D gesture tracking as in MS Kinect device. Fifthly, in this world of smart phones where smart phone as available at cheaper prices with more features .Pranav should try to reduce its actual cost from $350. Last but not least which can be benefit for the society also to make sixth sense work as fifth sense for disabled people.

8. CONCLUSIONS

Sixth Sense Technology recognizes the objects around us and makes us familiar with the information in any way we need without the use of microelectronic gadgets using simple hand gestures. It can be used as 5th sense for handicapped people.

Individuals can develop their own applications. Sixth sense technology clearly became a transparent user interface thus making life easier, faster and comfortable. As technology upgrades, many devices hit the market and leave soon in case they are not efficient. The security reasons are to be considered while using sixth sense technology.

Authorized access to any application plays a major role. For instance, while using sixth sense technology in libraries, there should be a separate method for faculty and students

to access them as it may lead to chaos. Sixth sense technology is very different from other technologies and is a new topic for hackers and other people.

We got rid of coloured markers in recent years. However, we should try to make it more secure and avoid radio waves which are harmful and could cause brain cancer to children. Camera and projector could be incorporated into a mobile device than using them separately.

REFERENCES

[1]. https://en.wikipedia.org/wiki/SixthSense

[2]. https://www.engineersgarage.com/articles/sixth-sense- technology

[3]. ids.nic.in/technical_letter/Tnl_Jan2014/Sixth%20Sense Technology.pdf

[4]. dspace.cusat.ac.in/jspui/bitstream/…/1/SIXTH%20SEN SE%20TECHNOLOGY.pdf

[5]. https://www.quora.com/What-is-sixth-sense- technology

[6]. https://www.scribd.com/doc/46500010/Sixth-sense- technology-seminar-report

[7]. https://www.crazyengineers.com … Project Ideas & Seminar Topics

[8]. https://www.pcworld.com/article/163072/wearable_gad get.html