Doctoral Work
Thesis: Contribution to the Study of Usability and Haptic Feedback of Encountered-Type Haptic Displays
Use case: Prototyping vehicles using virtual reality and haptic interfaces
My doctoral work was part of the LobbyBot project. The goal was to design and test new interaction techniques for integrating a robotic arm that provided haptic feedback in virtual reality. The integration of my research work in this use case was evaluated by experts from Clarté and the Renault Group.
Activities
Designed a way to interact with a novel haptic interface so it could be used for automobile prototyping
Augmented the haptic rendering capabilities of a haptic interface
Designed visual feedback for avoiding collisions between the user and the haptic interface without degrading immersion
Role
PhD student
VR developer
UX researcher
UI/UX designer
Tools
Unity
C#
SteamVR
Adobe Illustrator
MATLAB
Thesis research
Touching elements in virtual reality is a challenge that can be addressed by different approaches in haptics. One of these approaches consists of using a robotic display to bring a surface to be touched by the user at the right time and at the right moment.
This type of device is called an encountered-type haptic display (ETHD).
This ambitious type of device has many challenges to address since the system setup is quite complex. The major challenges identified by academia and the industry are increasing usability and haptic rendering capabilities.
These two factors were the axes of my doctoral research. The contributions of my research works in these research axes were published in scientific journals and conferences.
Contribution 1
User safety
A major challenge of ETHDs is that the robotic display has to be hidden from the users’ view. User safety needed to be considered when having an unseen moving robot in front of them. However, showing the robot in the virtual environment would break user immersion.
I conducted a focus group session with academia and industry experts to determine the most important factors that visual feedback needs to have for conveying a warning to the user without degrading immersion.
Once I gathered these factors, I designed a set of visual warning cues that alerted users when they approached the robot.
In the end, the LobbyBot consortium integrated some of these techniques into their demo.
Contribution 2
Interaction techniques for surface exploration
ETHDs have several limitations in matters of speed and shape displaying to minimize the risks of accidental collisions with the users. However, this also constrains the size and the speed used for haptic surface rendering.
I designed a set of interaction techniques that informed users about the system’s state. These techniques allowed users to select an area they desired to touch in a virtual environment.
Some of these techniques allowed users to be aware of the time required for the system to render haptic feedback from one location to another.
These techniques were compared in user studies to evaluate their performance in a surface exploration task.
Results suggest that users prefer an interaction technique that uses a detailed gesture (such as pressing the trigger of the VR controller) for setting and confirming the area they want to touch in VR.
Contribution 3
Large, multi-textured surface rendering
The surface placed by the robot was limited to a small and static prop with a limited set of textures. An automobile prototyping scenario not only demands a large number of textures but, more importantly, it demands a larger area to touch.
How to give the sensation of touching a large surface while having a reduced prop size?
I designed a novel approach for rendering large surfaces using a multi-textured rotating prop. Users were able to touch larger surfaces by sliding their fingers across virtual surfaces while the robot rotated a multi-textured prop in synchrony with the users’ finger positions.
In order to properly couple the users’ finger position with the prop, I designed an interaction technique that informed the users about the system’s state and position. The technique indicated the areas users could touch and guided them through the surface to be explored.
I ran user studies to compare the sensations between touching an actual large surface and touching a surface that rotates and displaces under the user’s finger.
Results suggested that the rotating prop approach serves as a means to convey the sensation of touching a large surface.
Contribution 4
Object manipulation
An important challenge for ETHDs is to allow object manipulation. This is hard due to the fact that most robotic systems have the surface to be touched directly attached to their end effector.
So, what could happen if users could detach the tangible surface from the end effector?
Would this mean that the robotic system would act as some sort of butler?
Working in collaboration with Dr. Thomas Howard, we created Alfred: the haptic butler. By having a detached tangible object, users could manipulate and relocate objects in a virtual environment.
We designed several positioning algorithms for optimizing the robot’s speed, and we ran a set of simulations to determine their efficiency.