深度学习车道线检测
以前简单看过车道线检测思路，现在看到⽐较详细⽂章，以后系统学习。
第九章：车道线检测 1. 车道线检测相关理论知识点 [ 27:19 ]
最近在⽤深度学习的⽅法进⾏车道线检测，现总结如下：
⽬前，对于车道线检测的⽅法主要分为两⼤类，⼀是基于传统机器视觉的⽅法，⼆是基于深度学习⼤⽅法。
⼀、基于传统机器视觉的⽅法
1. 边缘检测+霍夫变换
⽅法流程：彩⾊图像转灰度，模糊处理，边缘检测，霍夫变换
这种⽅法⼀般能够检测出简单场景下的车辆⽬前⾏驶的两条车道线，以及偶尔的相邻车道（依赖前视相机的⾓度）。该⽅法可以利⽤霍夫变换的结果（线的斜率），进⼀步过滤出左右车道线。不过同时，该⽅法也依赖于边缘检测的结果，所以调参（边缘检测、霍夫变换）以及其他的trick（roi选取等等）是很重要的。
2. 颜⾊阈值
⽅法流程：将图像转颜⾊空间（⼀般HSV），对新的color space中的各个通道设置阈值（⼤于阈值取值为1，⼩于取值为0），得到结果。
该⽅法依赖于各通道的阈值的选取，只需要调整⼏个阈值参数，但个⼈认为该⽅法鲁棒性会较差，例如当前车辆前⽅的车辆可能会被全部置1。
3. 透视变换
⽅法流程：获取透视变换矩阵，透视变换，车道线检测（1或者2）
该⽅法的优点是将前视摄像头抓拍的图像转为鸟瞰图，能够检测到多条线。其关键在于透视变换矩阵的准确性（不考虑转换后的车道线检测），对于转换后的鸟瞰图，可以通过上述两种⽅式检测车道线。
在实际场景中，传统⽅法的鲁棒性确实不⾏，除去光照和邻近车辆的影响外，车道中间的指⽰箭头和⼈⾏道也是此类算法很难处理的挑战。⼆、基于深度学习的⽅法
⼩编主要讲的是如何通过训练⼀个深度神经⽹络对车道线进⾏语义分割，从⽽实现车道线检测。SegN
et⽹络是⼀种很有趣的图像分割技术，是⼀种encoding-decoding的结构，在使⽤时可直接调⽤标准的模型结构。⼩编训练的⽹络便是基于SegNet⽹络构建的。
初始的⽹络结构如下图：
1.数据集
原图：
分割图（label）：
原图点此处下载()
数据集labels点此处下载()
初版的⽹络model如下：
""" This file contains code for a fully convolutional
(i.e. contains zero fully connected layers) neural network
for detecting lanes. This version assumes the inputs
to be road images in the shape of 80 x 160 x 3 (RGB) with
the labels as 80 x 160 x 1 (just the G channel with a
re-drawn lane). Note that in order to view a returned image,
the predictions is later stacked with zero'ed R and B layers
and added back to the initial road image.
"""
import numpy as np
import pickle
from sklearn.utils import shuffle
del_selection import train_test_split
# Import necessary items from Keras
dels import Sequential
from keras.layers import Activation, Dropout, UpSampling2D, concatenate
from keras.layers import Conv2DTranspose, Conv2D, MaxPooling2D
from alization import BatchNormalization
from keras.preprocessing.image import ImageDataGenerator
from keras import regularizers
# Load training images
train_images = pickle.load(open("full_CNN_train.p", "rb" ))
# Load image labels
labels = pickle.load(open("full_CNN_labels.p", "rb" ))
# Make into arrays as the neural network wants these
train_images = np.array(train_images)
labels = np.array(labels)
# Normalize labels - training images get normalized to start in the network
labels = labels / 255
# Shuffle images along with their labels, then split into training/validation sets
train_images, labels = shuffle(train_images, labels)
# Test size may be 10% or 20%
X_train, X_val, y_train, y_val = train_test_split(train_images, labels, test_size=0.1)
# Batch size, epochs and pool size below are all paramaters to fiddle with for optimization batch_size = 16
epochs = 10
pool_size = (2, 2)
input_shape = X_train.shape[1:]
### Here is the actual neural network ###
model = Sequential()
# Normalizes incoming inputs. First layer needs the input shape to work
model.add(BatchNormalization(input_shape=input_shape))
# Below layers were re-named for easier reading of model summary; this not necessary
# Conv Layer 1
model.add(Conv2D(8, (3, 3), padding='valid', strides=(1,1), activation = 'relu', name = 'Conv1'))
# Conv Layer 2
model.add(Conv2D(16, (3, 3), padding='valid', strides=(1,1), activation = 'relu', name = 'Conv2'))
# Pooling 1
model.add(MaxPooling2D(pool_size=pool_size))
# Conv Layer 3
model.add(Conv2D(16, (3, 3), padding='valid', strides=(1,1), activation = 'relu', name = 'Conv3')) model.add(Dropout(0.2))
model.add(Conv2D(32, (3, 3), padding='valid', strides=(1,1), activation = 'relu', name = 'Conv4'))
model.add(Dropout(0.2))
# Conv Layer 5
model.add(Conv2D(32, (3, 3), padding='valid', strides=(1,1), activation = 'relu', name = 'Conv5'))
model.add(Dropout(0.2))
# Pooling 2
model.add(MaxPooling2D(pool_size=pool_size))
# Conv Layer 6
model.add(Conv2D(64, (3, 3), padding='valid', strides=(1,1), activation = 'relu', name = 'Conv6'))
model.add(Dropout(0.2))
# Conv Layer 7
model.add(Conv2D(64, (3, 3), padding='valid', strides=(1,1), activation = 'relu', name = 'Conv7'))
model.add(Dropout(0.2))
# Pooling 3
model.add(MaxPooling2D(pool_size=pool_size))
# Upsample 1
model.add(UpSampling2D(size=pool_size))
# Deconv 1
model.add(Conv2DTranspose(64, (3, 3), padding='valid', strides=(1,1), activation = 'relu', name = 'Deconv1')) model.add(Dropout(0.2))
# Deconv 2
model.add(Conv2DTranspose(64, (3, 3), padding='valid', strides=(1,1), activation = 'relu', name = 'Deconv2')) model.add(Dropout(0.2))
# Upsample 2
model.add(UpSampling2D(size=pool_size))
# Deconv 3
model.add(Conv2DTranspose(32, (3, 3), padding='valid', strides=(1,1), activation = 'relu', name = 'De
conv3')) model.add(Dropout(0.2))
# Deconv 4
model.add(Conv2DTranspose(32, (3, 3), padding='valid', strides=(1,1), activation = 'relu', name = 'Deconv4')) model.add(Dropout(0.2))
# Deconv 5
model.add(Conv2DTranspose(16, (3, 3), padding='valid', strides=(1,1), activation = 'relu', name = 'Deconv5')) model.add(Dropout(0.2))
<_layer('Conv4')
# Upsample 3
model.add(UpSampling2D(size=pool_size))
# Deconv 6
model.add(Conv2DTranspose(16, (3, 3), padding='valid', strides=(1,1), activation = 'relu', name = 'De
conv6'))
# Final layer - only including one channel so 1 filter
model.add(Conv2DTranspose(1, (3, 3), padding='valid', strides=(1,1), activation = 'relu', name = 'Final'))
### End of network ###
# Using a generator to help the model use less data
# Channel shifts help with shadows slightly
datagen = ImageDataGenerator(channel_shift_range=0.2)
# Compiling and training the model
modelpile(optimizer='Adam', loss='mean_squared_error')
model.fit_generator(datagen.flow(X_train, y_train, batch_size=batch_size), steps_per_epoch=len(X_train)/batch_size,
epochs=epochs, verbose=1, validation_data=(X_val, y_val))
# Freeze layers since training is done
modelpile(optimizer='Adam', loss='mean_squared_error')
# Save model architecture and weights
model.save('full_CNN_model.h5')
# Show summary of model
model.summary()
上述model的建⽴参考链接()
三、改进⽹络的model
上述的⽹络model在车道线检测的时候效果并不稳定，所以对其进⾏了改进。
1.增加了⽹络的层数，修改了每层卷积核的个数。
2.受Unet⽹络的启发，采⽤并联跳跃结构将encoding的feature map 与 decoding的feature map进⾏连接，这样可以在进⾏分类预测时利⽤多层信息。
3.将UpSampling2D改为Conv2DTranspose实现上采样的过程，UpSampling2D直接采⽤原像素值进⾏填补不存在学习的过程，⽽Conv2DTranspose存在学习的过程，效果更好。
训练
改进后的⽹络model如下：
import numpy as np
import pickle
import pickle#import cv2
from sklearn.utils import shuffle
del_selection import train_test_split
# Import necessary items from Keras
dels import Model
from keras.layers import Activation, Dropout, UpSampling2D, concatenate, Input
from keras.layers import Conv2DTranspose, Conv2D, MaxPooling2D
from alization import BatchNormalization
from keras.preprocessing.image import ImageDataGenerator
from keras.utils import plot_model
from keras import regularizers
# Load training images
train_images = pickle.load(open("full_CNN_train.p", "rb" ))
# Load image labels
labels = pickle.load(open("full_CNN_labels.p", "rb" ))
# Make into arrays as the neural network wants these
train_images = np.array(train_images)
labels = np.array(labels)
# Normalize labels - training images get normalized to start in the network
labels = labels / 255
# Shuffle images along with their labels, then split into training/validation sets