e-skin camera-free gesture tracking provides you immersive AR/VR/MR experience and significant motion analysis using machine learning
Localize: Japanese (日本語) | Chinese (中文)
With e-skin, you can use, recognize and analyze human motion data anytime, anywhere.
We created e-skin smart apparel by first focusing on the feel of the shirt so that it maintains the comfort, durability and machine washability of a regular shirt. We believe this is imperative for it to be fully embraced by all.
In the near future, connected clothing will be the most natural way for people to interact with internet, and other people. We do not refer to e-skin a ‘wearable device’ because it is just a shirt, and a shirt is always to be worn.
Our e-skin apparel unobtrusively moves and stretches with the body’s movements. The proprietary stretchable electronics were developed in Professor Takao Someya’s Lab at the University of Tokyo. E-skin provides a new intuitive and immersive input for interacting with virtual reality content and games as well as means for analyzing exercises and athletic sports. Since it is wireless and camera-free, e-skin can be used for many hours both inside and outside.
We are here on Kickstarter to provide a new, open source medium to the community of passionate developers, designers, artists, and musicians to create user experiences to share and learn from. We believe there are a variety of needs where e-skin can provide a solution to satisfy and help people and we want to provide the tools to empower this. Let’s change the world together with e-skin.
We’ve integrated 14 strain sensors and a 6-axis accelerometer and gyroscope to capture and track upper body motion as well as respiration to be used as an input controller in game, or analyzing form in sports / fitness activity or more broadly as an upper body activity tracker.
Data from the shirt is sent through the attached e-skin Hub over Bluetooth.
An environment to support machine learning will also be provided. Original gesture could be recognized through machine learning, and new applications can be developed using it.
By machine learning, not only specific form such as punching, jumping, but the daily movement and posture can also be recognized.
e-skin can work with the Microsoft HoloLens. You may expand the scope of gesture input and create more AR content development possibilities using e-skin.
Development Environment: Unity, UNREAL ENGINE*
*UNREAL ENGINE coming soon
A configuration screen is designed in which you can intuitively understand the e-skin status. You can see the state of communication and individual sensor response in it.
e-skin Store will be published to support every e-skin application.
Share your application and broaden the e-skin world!
Here is the way how our e-skin tracks user’s motion.
The resistance of the strain sensors on e-skin shirt changes when they stretch.
We have developed the algorithm to recognize human movement (arm bending etc.) by measuring the voltage drop in each channel in the e-skin Hub. The Hub can also detect movement like jumping or stepping through the 6-axis accelerometer and gyroscope.
Let us introduce some application examples using e-skin.
e-skin Game Demo
With e-skin, the user’s gestures (running, jumping, punching, crouching) were integrated into a game demo that was shown at CES2017. In the demo, the strain sensors and accelerometer/gyroscope module recognize the physical actions of the person wearing the shirt. The information gathered from the sensors on the shirt is sent over Bluetooth to a host laptop running the game where the physical movements are rendered as the avatar’s movements.
e-skin Dance Demo
This dance demo was also shown at CES. As the woman moves, the sensors stretch with her movement and the electronic signature from the sensor response creates triggers for various sound effects. Dancers usually dance with music, but in this case dancers can create the music.
e-skin Golf Demo
This is another demo that was shown at CES2017. In this video, we developed a series of golf swing algorithms that were able to correctly identify correct and incorrect swing form by analyzing the realtime strain sensor and accelerometer/gyroscope. This was accomplished using only the data from the e-skin shirt and sending it over Bluetooth to a smart phone to process.
e-skin + HoloLens Demo
Here is some footage that shows a real demo of our e-skin shirt used with a Microsoft HoloLens. We connected our e-skin directly to a Microsoft HoloLens and control the avatar with the movements of shoulder and elbow. The e-skin enables motion input outside the eyesight of the goggle.
e-skin App:Windows、Android、Mac、iOS
*Mac, iOS to be prepared.
3 applications for sample demo.
Running / Fitness application
Control characters with your movement in e-skin. Check calories burned by playing it.
*This application works with Windows PC only.
Yoga application
Control sounds, lights with your movement in e-skin. Make your exercise log.
Data Tracker application (for developer)
Save your log data in e-skin and visualize with a graphical interface.
Durable : more than 10,000 stretches at a 50% strain rate
Washable : more than 100 wash cycles based on ISO 6330 4N-A standards
ONLY HERE! Kickstarter special edition
*Material and performance are the same as the standard black e-skin.
* Note: E-skin offered as Reward is “e-skin Personal”. Please refer to: About e-skin Personal license
We have assembled a team experienced in designing, prototyping, manufacturing and testing electronics, textiles, and software to minimize the risks for delivering a new platform to our early backers. We have already started producing and delivering e-skin products to our corporate and institution customers around the world. As with all new product offerings there will always be glitches and bugs that need to be worked out. We understand this and have manufacturing partners and design teams that will be able to further refine our products and processes to give you the highest quality product.
We are extremely proud of the product we have assembled, but want to clarify that our e-skin might not always be the most appropriate solution for you. For instance, e-skin is not a replacement for motion capture systems used for 3D animation creation. The reason for this is attributed to the inherent positional inaccuracy of stain sensors since the strain sensors on e-skin apparel detect the stretches from the shirt. For such applications, we recommend you stick with a traditional motion capture system.
However, e-skin apparel can be more useful and insightful for recognizing what kinds of activities the wearer is doing without being constrained by a camera. In some cases, it will recognize small differences indiscernible by eye. If you are unsure whether or not our product is an appropriate solution for your needs and applications, please feel free to ask us.