The HPI bachelor student Julia Joch has won the second prize at this year's DMEA young talent award with her bachelor thesis titled "Gesture recognition in daily life as a means of noninvasive labeling in medical studies"
In April this year, our bachelor Julia Joch presented her bachelor thesis at the DMEA Young Talent Award. We are very happy to announce that with her excellent work, she took second place in the category best bachelor thesis. The title of her thesis was "Gesture recognition in daily life as a means of noninvasive labeling in medical studies" and she was supervised by PhD student Kristina Kirsten. The aim of her work was to find a simpler approach to annotate data. Therefore, she used predefined hand gestures that could be recognized by the phone to enrich collected data with additional information without using the phone. This could mean for participants in medical studies that instead of filling out a questionnaire on their cell phone every two hours, they simply perform a predefined gesture with their hand in certain situations.
Thesis abstract
Supervised machine learning is already an established approach to many scientific problems. It has been applied in medical contexts as well, such as stress recognition in daily life. However, when applying this technique to real-life contexts, scientists are confronted with the challenge of gathering labeled data over an extended period. Labeling this data can become problematic, as constant supervision, similar to a laboratory setting, is neither feasible nor desired. Therefore, participants of such studies have to label their data themselves. Currently, such self-indicating processes are implemented via apps, such as the real-world study framework SensorHub. Nevertheless, this process can become very obtrusive in daily life and might even influence the results. For example, in studies where participants need to indicate their stress levels frequently, labels get missed in situations where it would be inappropriate to take the phone. Unfortunately, these situations are often important meetings, car rides, or personal encounters, which are especially stressful and noteworthy for stress studies. Consequently, missing these labels presents a significant problem. This thesis aims to provide an unobtrusive solution to labeling data in real-world studies. It aims to provide a set of predefined gestures that can be detected in everyday life by accelerometer and gyroscope sensors of wearable devices on the wrist. , the labeling process on the phone can be replaced by performing one of these gestures as a label. Two approaches were compared and achieved promising results with Matthews Correlation Coefficients of up to 0.789 for a Random Forest and up to 0.835 for a Convolutional Neural Network.
In the following video Julia presents her thesis in German.
