如何提高深度学习的准确性和验证准确性

Question

我正在用自己的数据训练 CNN，我在相同的数据上尝试了 resnet50 和 resnet101 以及我自己的模型，准确度为 63，验证准确度为 0.08。我知道问题出在我的数据上，我想在拆分数据之前尝试对数据进行洗牌，但我的数据分为 26 个不同的类，如何在将数据拆分为训练集和验证集之前对数据进行洗牌。我的数据集超过 36K 图像。

(trainX, testX, trainY, testY) = train_test_split(data, labels,

    test_size=0.25, stratify=labels, random_state=42)

# initialize the training data augmentation object

trainAug = ImageDataGenerator(

    rotation_range=30,

    zoom_range=0.15,

    width_shift_range=0.2,

    height_shift_range=0.2,

    shear_range=0.15,

    horizontal_flip=True,

    fill_mode="nearest")

# initialize the validation/testing data augmentation object (which

# we'll be adding mean subtraction to)

valAug = ImageDataGenerator()

# define the ImageNet mean subtraction (in RGB order) and set the

# the mean subtraction value for each of the data augmentation

# objects

mean = np.array([123.68, 116.779, 103.939], dtype='float32')

trainAug.mean = mean

valAug.mean = mean

model = Sequential()

# The first two layers with 32 filters of window size 3x3

model.add(Conv2D(32, (5, 5), padding='same', activation='relu', input_shape=(224, 224, 3)))

model.add(Conv2D(32, (5, 5), activation='relu'))

model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Dropout(0.25))

model.add(Conv2D(64, (5, 5), padding='same', activation='relu'))

model.add(Conv2D(64, (5, 5), activation='relu'))

model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Dropout(0.25))

model.add(Conv2D(128, (3, 3), padding='same', activation='relu'))

model.add(Conv2D(128, (3, 3), activation='relu'))

model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Dropout(0.25))

model.add(Flatten())

model.add(Dense(512, activation='relu'))

model.add(Dropout(0.5))

model.add(Dense(labels, activation='softmax'))

Answer 1

您可以使用 ImageDataGenerator 的验证 split 关键字自动拆分训练数据和测试数据。train_datagen = ImageDataGenerator(rescale=1./255,    shear_range=0.2,    zoom_range=0.2,    horizontal_flip=True,    validation_split=0.2) # set validation splittrain_generator = train_datagen.flow_from_directory(    train_data_dir,    target_size=(img_height, img_width),    batch_size=batch_size,    class_mode='binary',    subset='training') # set as training datavalidation_generator = train_datagen.flow_from_directory(    train_data_dir, # same directory as training data    target_size=(img_height, img_width),    batch_size=batch_size,    class_mode='binary',    subset='validation') # set as validation datamodel.fit_generator(    train_generator,    steps_per_epoch = train_generator.samples // batch_size,    validation_data = validation_generator,     validation_steps = validation_generator.samples // batch_size,    epochs = nb_epochs)当ImageDataGenerator自动打乱您的输入数据时，您使用ImageDataGenerator的数据会被打乱和分割。在你的情况下，你需要flow而不是flow_from_directory