Video based detection of driver fatigue

Abstract

Bu doktora tezinde yuze uygulanan bilgisayar goru teknikleri kullanilarak surucude uykululugun sezimi problemi ele alinmistir. Ozellikle uykulu goruntu kesitlerinin uykusuz goruntu kesitlerinden yuz ifadeleri araciligiyla ayrilabilirligi kesfedilmeye calisilmaktadir. Gecmiste uykululugun sezimi ve tahmini icin cesitli yaklasimlar onerilmistir. Uykuda suruculugunun seziminde bilgisayarla goru yaklasimlarinin carpici ve mudehaleci olmayan ozellikleri son yillarda bu yaklasimlara ilgiyi arttirmaktadir. Bu yaklasimla calisan onceki calismalar uykulu surucu seziminde baslica varsayimlar olan goz kirpma hizi, goz kapama, ve esneme gibi uygun davranislara odaklanmaktadir. Burada makine ogrenme tekniklerini kullanarak uykululuk kesitlerinde gercek insan davranisini bulmayi hedeflemekteyiz. Yuz hareket kodlama sistemi otomatik siniflandiricilarinin ciktilari uykulu surucu seziminde kullanilmaktadir. Bu hareket birimleri goz kapama esneme ve de birkac ek yuz hareketini barindirmaktadir. Ayni zamanda bas hareketi verileri tezin bir bolumunde otomatik goruntu tabanli bas pozisyonu sezici yardimiyla ve de diger bolumlerde ivmeolcer araciligiyla toplanmistir. Bu olculer ogrenme tabanli siniflandiricilar olan Lojistik Baglanim Siniflandiricilarina (MLR) ve Adaboost'a gecirilmistir. Sistem bir bilgisayar surus simulasyonu kullanan deneklerin uykulu ve uykusuz kesitlerini kisi bagimsiz testlerde % 98 dogrulukla tahmin edebilmektedir. Bu gercek uykululugun seziminde en yuksek tahmin oranidir. Ayrica analiz uykululukta insan yuz davranisi icin yeni bilgiler ortaya koymaktadir. Uykulu hallerin ince ayrimi baska bir veri tabaninda arastirilmistir. Bireysel yuz hareket birimlerinin ne derecede orta ve ileri dereceli uykuluk farkini tespit edebilecegi calisilmistir. Sinyal isleme teknikleri ve makina ogrenme yontemleri kullanilarak kisi bagimsiz ileri derecede uykululukk sezim sistemi kurulmustur. Zamandaki dinamik bilgi zamansal filtre bankasi kullanilarak cikarilmistir. Bireysel hareket unitelerinin tahmin gucu MLR tabanli siniflandiricilar kullanilarak arastirilmistir. En iyi performansi veren bes hareket birimi insan bagimsiz bir sistem icin belirlenmistir. Sistem 5 hareket unitesinin ozniteliklerini birlestiren bir siniflandirici icin daha zorlu bir veri kumesinde % 96 dogruluk gostermektedir. Ayrica analiz degisik seviyelerdeki uykululuk icin yeni belirtecler ortaya koymaktadir.

The thesis addresses the problem of drowsy driver detection using computer vision techniques applied to the human face. Specifically we explore the possibility of discriminating drowsy from alert video segments using facial expressions automatically extracted from video. Several approaches were proposed for the detection and prediction of drowsiness. There has recently been increasing interest in computer vision approaches as it is a prominent and a non-invasive approach for detecting drowsiness. Previous studies with this approach detect driver drowsiness primarily by making pre-assumptions about the relevant behavior, focusing on blink rate, eye closure, and yawning. Here we employ machine learning to datamine actual human behavior during drowsiness episodes. Outputs of automatic classifiers of the Facial Action Coding system were used for detecting drowsiness. These facial actions include blinking and yawn motions, as well as a number of other facial movements. In addition, head motion was collected through and automatic video based head pose detector and via an accelerometer in other parts of the thesis. These measures were passed to learning-based classifiers such as Adaboost and multinomial logistic regression. The system was able to predict sleep and crash episodes during a driving computer game with 98% accuracy across subjects. This is the highest prediction rate reported to date for detecting real drowsiness. Moreover, the analysis revealed new information about human facial behavior during drowsy driving. Fine discrimination of drowsy states were also explored on a separate dataset. The degree to which individual facial action units can predict the difference between moderately drowsy to acutely drowsy is studied. Signal processing techniques and machine learning methods are employed to build a person independent acute drowsiness detection system. Temporal dynamics are captured using a bank of temporal filters. Individual action unit predictive power was explored with an MLR based classifier. Best performing five action units were determined for a person independent system. The system was able to obtain 0.96 ROC for a more challenging dataset with the combined features of the best performing 5 action units. Moreover the analysis revealed new markers for different levels of drowsiness.