Derin öğrenme yöntemi ile optik uydu görüntülerinden gemi tespiti

Abstract

Deniz yollarının ticari/askeri alanda etkin şekilde kullanımı ve kontrolü; devletler için gün geçtikçe daha da artan ve önem arz eden bir husus haline gelmektedir. Bu önemli alanların gözetimi ve denetimi; gözlemci unsurun (gemi, insanlı/insansız hava aracı, vb.) olay yerine direkt olarak gitmesi ve alanı incelemesi suretiyle icra edilebilir, ancak bu şekilde bir yaklaşım ile bölgeye intikal ve kısıtlı görüş açısı/kapsama alanı ile gözlem işlemleri sonucunda zaman sarfiyatına sebebiyet verebileceği gibi savaş, terör veya kriz ortamlarında sıcak bölgeye yakınlığından dolayı gözlemci unsur büyük bir risk altında bulunacaktır. Tam bu noktada uzaktan algılama yöntemleri faydalı çözümler sunmakta, özellikle uydu görüntüleri ile yapılan uzaktan algılama ile operator güvenli bir şekilde, kendini ifşaa etmeden, kısa zamanda büyük alanları inceleyebilmekte ve aradığı unsuru tespit edebilmektedir.Bu çalışma kapsamında, Tensorflow Nesne Tespiti API'sinin nesne tanıma üzerine halihazırda eğitilmiş modellerinin içerisinde gemi resmi bulunduran optik uydu görüntüleri ile eğitilmesi yolu ile optik uydu görüntülerinden gemi tespit eden algoritmalara ilave bir seçenek eklemektir. Gemi tespit metodu tasarlanırken, lisans sorunu yaşanmaksızın her ortamda kullanılabilmesi için sistemin açık kaynak koduna sahip olması, hızlı çalışması ve kullanımının kolay olması hedeflenmiştir.Birinci bölümde, deniz yollarının önemi ve izlenme usulleri hakkında temel bilgilerden, bahse konu alan ile ilgili olarak uzaktan algılama yöntemlerinin kullanımından, yararından ve birbirleri ile karşılaştırılmasından, literatür araştırmasından, tezin konusundan, amacından ve kapsamından bahsedilmiştir.İkinci bölümde, derin öğrenme ve kullanım alanları, yapay sinir ağları ve konvolüsyonel sinir ağları hakkında özet bilgiler verilmiştir.Üçüncü bölümde, tez çalışması kapsamında kullanılan kütüphaneler, modelin eğitilmesi ve deneysel sonuçlardan bahsedilmiştir. Tezin amacı doğrultusunda gemi tespit yazılımının kaynak kodu Python 3.7 yazılım dili ile hazırlanmıştır. Python kütüphaneleri için Anaconda programı kurulmuştur. Derin öğrenme kütüphanesi olarak Google tarafından oluşturulmuş Tensorflow kütüphanesi, bilgisayarlı görü kütüphanesi olarak OpenCV kütüphanesi kullanılmıştır. Eğitilmiş model olarak; Tensorflow Object Detection Model Zoo'dan `faster rcnn inception v2_coco, faster rcnn resnet101_coco, ssd_inception_v2_coco` modelleri kullanılmıştır. Modellerin eğitiminde kullanılan gemi görüntü seti, çözünürlüğü 80 * 80 * 3, RGB bandı, 96 dpi olan 2085 adet görüntüden oluşmaktadır. Modellerin eğitimi Tensorboard arayüzü üzerinden kayıp (loss) grafikleri vasıtasıyla takip edilmiştir. Modeller arasındaki gemi tespit performansı sergileyen modelin gemi tespit performansının artırılması için değişken çözünürlüğe sahip (987 * 804 * 3, 1155*820*3, vb.), ilk gemi görüntü setinin çözünürlüğünden yaklaşık 10 kat büyük, RGB bandı, 96 dpi olan 1056 adet görüntüden oluşan görüntü seti ile model tekrar eğitilmiştir. Bahse konu modelin gemi tespit performansında iyileşme olduğu gözlenmiştir.Dördüncü bölümde sonuçlar ve öneriler kısmı bulunmaktadır.

The effective use and control of maritime routes in the commercial / military area is an increasing and important need for states. As a result of the rapid development in the shipbuilding industry, the number of ships and their size increase day by day. This poses a major threat to safe passage, especially in narrow traffic channels. In order to ensure the safety of the cruises within this scope; some ports allocate special anchorages and evacuation channels, but these canals and anchorages can be used illegally by ordinary ships. In addition, as another type of illegal use; sometimes fishing vessels can occupy the main channels irregularly as fishing areas. Due to the recent decline in the amount of fish reserves in the world, ship detection has become a more effective and efficient method of monitoring fishermen to identify illegal fishing activities. Remote monitoring of the regions of interest in terms of detection and tracking of foreign/enemy elements threatening the security of the area in the seas makes a major contribution to the early warning of military intelligence, crime and anti-terrorism units. It is also useful for detecting the lost ships, boats, aeroplanes, debris, containers, etc. For these issues; the observer can check these areas directly by going there but it will cost time and in case of terror or war the observer will be in a huge risk. At this point `remote sensing` offers a great solution. Surface of the sea can be observed by radar, video cameras, optical satellite imagery or synthetic aperture radar (SAR) imagery. In case of war, active transmission devices can be dangerous for observer because enemy electronic warfare devices can detect your coordinates according to your transmission. In the war environment, this is never desired. So that devices like video cameras, satellite imageries are much more useful. Compared to satellite imaging systems, the viewing distance and field of view of the video cameras are more limited, more susceptible to damage or hitting by enemy. Compared to optical satellite imagery, SAR imagery is much more useful at night because it doesn't need a lightning source but its images are usually with high-level speckles and, insensitive to wood materials. Compared to SAR imagery, optical satellite imagery has higher resolution and more detailed information but it needs lightning source so it can't be used at night. Deep learning is a kind of machine learning method that enables computers to learn from acquired experience and information and to extract useful patterns from raw data. It can learn information from pictures, texts or sounds. Based on the knowledge it learned, it can develop more advanced works, or even can work ahead of human abilities. It has a wide range of applications from driverless vehicles to satellite and defense systems, medical research to industrial automation systems, electronic devices and advertising. Nowadays, thanks to deep learning, autonomous vehicles are developed which can stop when they see the red light, park itself when it finds empty parking lot, follow the vehicles in front and rear, decide how to behave against objects that suddenly come to the road. We know that objects in the world can be seen very clearly from satellites. What would happen if we could transfer this ability to the automated system? Deep Learning seems to have done this a lot. In medical research (especially in cancer screening), it is targeted to automatically scan items that can escape the eyes of technicians. It is aimed to detect abnormal situations with deep learning. With the help of deep learning in the field of Industrial Automation, it is planned to reduce the damage caused by occupational safety and work accidents. It is aimed at working by hearing and interpreting speech rather than manual intervention in the management of electronic devices. Many deep learning methods use architectures called Artificial Neural Networks (ANN). It is the system developed for the purpose of technically realizing the ability to discover, create and derive new information by means of learning which is the most basic feature of human brain. This system consists of artificial cells that run in parallel and consist of hierarchically connected network nodes. ANN consists of input, hidden and output layers. Although the number of neurons in each layer can be more than one, the layer that has the most impact on the learning ability of the network is the hidden layer.Convolutional neural network (CNN) in machine learning is a deep feed artificial neural network which has been successfully applied to the analysis of images. It is a multi-layer network architecture developed from the classic neural network process. CNN usualy consists of input layer, convolution layer, pooling layer, full connection layer and output layer. It can execute feature extraction and mapping through fast training, and has high prediction accuracy, so it is often applied to classification and estimation.The purpose of this work is to add an additional option to the ship detection algorithms from optical satellite imagery by training the Tensorflow Object Detection Application Programming Interface (API). When the ship detection method is designed, it is aimed that the system should have open source code, work quickly and be easy to use.In the first chapter, the basic information about the importance and monitoring methods of maritime routes, the usage of remote sensing methods, the benefits and comparison with each other, the literature research, the subject, the purpose and the scope of the thesis are mentioned.In the second chapter, deep learning, ANN and CNN are metioned. In the third chapter, the libraries used in the thesis work, the training of the models and the experimental results are mentioned. When the ship detection method is designed, it is aimed that the system should have open source code, work quickly, can be trained, can be operated via a avarage laptop and be easy to use. For this purpose, source code is written via Python 3.7.. Anaconda is installed for Python libraries. Tensorflow (made by Google) is installed which is an open source machine learning library for research and production. OpenCV is installed for computer vision library. Training a single `object detection model` can take weeks to months, even with GPU. For CPU it will be done much more long time. Therefore, usage of an already trained model and training it for your object (for this work it is ship) is much more useful and time saver. For this purpose, Tensorflow Object Detection API's models (trained for object detection by Google) are used for already trained object detection model. They are trained with optical satellite images to detect ships. `faster rcnn inception v2_coco`, `faster rcnn resnet101_coco` and `ssd_inception_v2_coco` have been used as trained models from Tensorflow Object Detection Model Zoo. This models are trained by Google via Coco Image Dataset which is a large-scale object detection, segmentation, and captioning dataset. Next; a number of additional libraries, such as `matplotlib, pandas` are installed. Then ship image sets are downloaded and images which has ship labelled as `ship (in Turkish: Gemi)` with LabelImg. Non-ship images are erased. Dataset is consist of 2085 images whose resolution is 80*80*3, RGB band., 96 dpi. Training of the models is observed through loss graphs via Tensorboard interface. In order to increase the ship detection performance of the model which has the best ship detection performance between the models, it is retrained with additional image set. The additional image set consists of 1056 images with 96 dpi, in the RGB band with variable resolution (987 * 804 * 3, 1155 * 820 * 3, etc.) which are approximately 10 times larger than the resolution of the first ship image set. It is observed that the ship detection performance of this model is improved.The fourth section contains the results and recommendations.