-
[CNN] AlexNet머신러닝 & 딥러닝 2022. 1. 22. 21:49
AlexNet 특징
- 2012년에 성능 증명.
- Activation 함수로 ReLU 함수를 첫 사용하였다.
- MaxPooling, Overlapping Pooling을 적용하였다.
- Local Response Normalization (LRN) 사용
- Dropout Layer, weight에 decay 적용 사용. (이 때 처음 사용)
- Data Augmentation 적용.
- 11x11, 5x5 사이즈의 큰 사이즈의 Kernel 적용 (지금은 상상할 수 x).
- 많은 weight parameter 갯수로 인하여 컴퓨팅 연산량이 크게 증가 함 (그 당시 하드웨어 성능은 좋지 않았음). 이를 극복하기 위하여 병렬 GPU를 활용할 수 있도록 CNN 모델을 병렬화하였다.
전체 프로세스
- create_alexnet -> model 생성
- 데이터셋 나누기 (Train, Test, Validation) & 전처리 (255 나누기, 원-핫 인코딩)
- 원본 이미지 크기 조정 (32x32) -> (128x128)
- 모델 학습
- 테스트 데이터 평가
import numpy as np import pandas as pd import os1. create_alexnet -> model 생성
from tensorflow.keras.models import Model from tensorflow.keras.layers import Input, Dense , Conv2D , Dropout , Flatten , Activation, MaxPooling2D , GlobalAveragePooling2D from tensorflow.keras.optimizers import Adam , RMSprop from tensorflow.keras.layers import BatchNormalization from tensorflow.keras.callbacks import ReduceLROnPlateau , EarlyStopping , ModelCheckpoint , LearningRateScheduler from tensorflow.keras import regularizers # input shape, classes 개수, kernel_regularizer등을 인자로 가져감. def create_alexnet(in_shape=(227, 227, 3), n_classes=10, kernel_regular=None): # 첫번째 CNN->ReLU->MaxPool, kernel_size를 매우 크게 가져감(11, 11). 지금은 (11, 11) 사용하지 않음. input_tensor = Input(shape=in_shape) x = Conv2D(filters= 96, kernel_size=(11,11), strides=(4,4), padding='valid')(input_tensor) x = Activation('relu')(x) # LRN을 대신하여 Batch Normalization 적용. x = BatchNormalization()(x) x = MaxPooling2D(pool_size=(3,3), strides=(2,2))(x) # 두번째 CNN->ReLU->MaxPool. kernel_size=(5, 5) x = Conv2D(filters=256, kernel_size=(5,5), strides=(1,1), padding='same',kernel_regularizer=kernel_regular)(x) x = Activation('relu')(x) x = BatchNormalization()(x) x = MaxPooling2D(pool_size=(3,3), strides=(2,2))(x) # 3x3 Conv 2번 연속 적용. filters는 384개 x = Conv2D(filters=384, kernel_size=(3,3), strides=(1,1), padding='same', kernel_regularizer=kernel_regular)(x) x = Activation('relu')(x) x = BatchNormalization()(x) x = Conv2D(filters=384, kernel_size=(3,3), strides=(1,1), padding='same', kernel_regularizer=kernel_regular)(x) x = Activation('relu')(x) x = BatchNormalization()(x) # 3x3 Conv를 적용하되 filters 수를 줄이고 maxpooling을 적용 x = Conv2D(filters=256, kernel_size=(3,3), strides=(1,1), padding='same', kernel_regularizer=kernel_regular)(x) x = Activation('relu')(x) x = BatchNormalization()(x) x = MaxPooling2D(pool_size=(3,3), strides=(2,2))(x) # Dense 연결을 위한 Flatten x = Flatten()(x) # Dense + Dropout을 연속 적용. x = Dense(units = 4096, activation = 'relu')(x) x = Dropout(0.5)(x) x = Dense(units = 4096, activation = 'relu')(x) x = Dropout(0.5)(x) # 마지막 softmax 층 적용. output = Dense(units = n_classes, activation = 'softmax')(x) model = Model(inputs=input_tensor, outputs=output) model.summary() return modelmodel = create_alexnet(in_shape=(227, 227, 3), n_classes=10, kernel_regular=regularizers.l2(l2=1e-4))Model: "model" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_1 (InputLayer) [(None, 227, 227, 3)] 0 conv2d (Conv2D) (None, 55, 55, 96) 34944 activation (Activation) (None, 55, 55, 96) 0 batch_normalization (BatchN (None, 55, 55, 96) 384 ormalization) max_pooling2d (MaxPooling2D (None, 27, 27, 96) 0 ) conv2d_1 (Conv2D) (None, 27, 27, 256) 614656 activation_1 (Activation) (None, 27, 27, 256) 0 batch_normalization_1 (Batc (None, 27, 27, 256) 1024 hNormalization) max_pooling2d_1 (MaxPooling (None, 13, 13, 256) 0 2D) conv2d_2 (Conv2D) (None, 13, 13, 384) 885120 activation_2 (Activation) (None, 13, 13, 384) 0 batch_normalization_2 (Batc (None, 13, 13, 384) 1536 hNormalization) conv2d_3 (Conv2D) (None, 13, 13, 384) 1327488 activation_3 (Activation) (None, 13, 13, 384) 0 batch_normalization_3 (Batc (None, 13, 13, 384) 1536 hNormalization) conv2d_4 (Conv2D) (None, 13, 13, 256) 884992 activation_4 (Activation) (None, 13, 13, 256) 0 batch_normalization_4 (Batc (None, 13, 13, 256) 1024 hNormalization) max_pooling2d_2 (MaxPooling (None, 6, 6, 256) 0 2D) flatten (Flatten) (None, 9216) 0 dense (Dense) (None, 4096) 37752832 dropout (Dropout) (None, 4096) 0 dense_1 (Dense) (None, 4096) 16781312 dropout_1 (Dropout) (None, 4096) 0 dense_2 (Dense) (None, 10) 40970 ================================================================= Total params: 58,327,818 Trainable params: 58,325,066 Non-trainable params: 2,752 _________________________________________________________________2. 데이터셋 나누기 (Train, Test, Validation) & 전처리 (255 나누기, 원-핫 인코딩)
CIFAR10 데이터세트를 이용하여 AlextNet 학습 및 성능 테스트
from tensorflow.keras.datasets import cifar10 # 전체 6만개 데이터 중, 5만개는 학습 데이터용, 1만개는 테스트 데이터용으로 분리 (train_images, train_labels), (test_images, test_labels) = cifar10.load_data() print("train dataset shape:", train_images.shape, train_labels.shape) print("test dataset shape:", test_images.shape, test_labels.shape)Downloading data from https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz 170500096/170498071 [==============================] - 3s 0us/step 170508288/170498071 [==============================] - 3s 0us/step train dataset shape: (50000, 32, 32, 3) (50000, 1) test dataset shape: (10000, 32, 32, 3) (10000, 1)학습/검증/테스트 데이터 세트로 나누고 데이터 전처리 수행
- 학습/검증/테스트 데이터로 분할. 검증 데이터는 학습 데이터의 20% 할당.
- 레이블의 원-핫 인코딩과 이미지 픽셀값의 스케일링 적용
import tensorflow as tf import numpy as np import pandas as pd import random as python_random from tensorflow.keras.utils import to_categorical from sklearn.model_selection import train_test_split from tensorflow.keras.datasets import cifar10 def zero_one_scaler(image): return image/255.0 def get_preprocessed_ohe(images, labels, pre_func=None): # preprocessing 함수가 입력되면 이를 이용하여 image array를 scaling 적용. if pre_func is not None: images = pre_func(images) # OHE 적용 oh_labels = to_categorical(labels) # 데이터프레임에서 원-핫 인코딩 하면 get_dummies 씀. 넘파이에 적용할 때는 to_categorical. return images, oh_labels # 학습/검증/테스트 데이터 세트에 전처리 및 OHE 적용한 뒤 반환 def get_train_valid_test_set(train_images, train_labels, test_images, test_labels, valid_size=0.15, random_state=2021): # 학습 및 테스트 데이터 세트를 0 ~ 1사이값 float32로 변경 및 OHE 적용. train_images, train_oh_labels = get_preprocessed_ohe(train_images, train_labels) test_images, test_oh_labels = get_preprocessed_ohe(test_images, test_labels) # 학습 데이터를 검증 데이터 세트로 다시 분리 tr_images, val_images, tr_oh_labels, val_oh_labels = train_test_split(train_images, train_oh_labels, test_size=valid_size, random_state=random_state) return (tr_images, tr_oh_labels), (val_images, val_oh_labels), (test_images, test_oh_labels ) # CIFAR10 데이터 재 로딩 및 Scaling/OHE 전처리 적용하여 학습/검증/데이터 세트 생성. (train_images, train_labels), (test_images, test_labels) = cifar10.load_data() print(train_images.shape, train_labels.shape, test_images.shape, test_labels.shape) (tr_images, tr_oh_labels), (val_images, val_oh_labels), (test_images, test_oh_labels) = \ get_train_valid_test_set(train_images, train_labels, test_images, test_labels, valid_size=0.2, random_state=2021) print(tr_images.shape, tr_oh_labels.shape, val_images.shape, val_oh_labels.shape, test_images.shape, test_oh_labels.shape)(50000, 32, 32, 3) (50000, 1) (10000, 32, 32, 3) (10000, 1) (40000, 32, 32, 3) (40000, 10) (10000, 32, 32, 3) (10000, 10) (10000, 32, 32, 3) (10000, 10)# 이미지 사이즈가 너무 작으면 모델의 MaxPooling에서 오류 발생. 이미지가 (32, 32) 이러면 나중에 maxpooling 할 때 이미 (2,2) 상태여서 에러난다. model = create_alexnet(in_shape=(128, 128, 3), n_classes=10, kernel_regular=regularizers.l2(l2=1e-4))Model: "model_1" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_2 (InputLayer) [(None, 128, 128, 3)] 0 conv2d_5 (Conv2D) (None, 30, 30, 96) 34944 activation_5 (Activation) (None, 30, 30, 96) 0 batch_normalization_5 (Batc (None, 30, 30, 96) 384 hNormalization) max_pooling2d_3 (MaxPooling (None, 14, 14, 96) 0 2D) conv2d_6 (Conv2D) (None, 14, 14, 256) 614656 activation_6 (Activation) (None, 14, 14, 256) 0 batch_normalization_6 (Batc (None, 14, 14, 256) 1024 hNormalization) max_pooling2d_4 (MaxPooling (None, 6, 6, 256) 0 2D) conv2d_7 (Conv2D) (None, 6, 6, 384) 885120 activation_7 (Activation) (None, 6, 6, 384) 0 batch_normalization_7 (Batc (None, 6, 6, 384) 1536 hNormalization) conv2d_8 (Conv2D) (None, 6, 6, 384) 1327488 activation_8 (Activation) (None, 6, 6, 384) 0 batch_normalization_8 (Batc (None, 6, 6, 384) 1536 hNormalization) conv2d_9 (Conv2D) (None, 6, 6, 256) 884992 activation_9 (Activation) (None, 6, 6, 256) 0 batch_normalization_9 (Batc (None, 6, 6, 256) 1024 hNormalization) max_pooling2d_5 (MaxPooling (None, 2, 2, 256) 0 2D) flatten_1 (Flatten) (None, 1024) 0 dense_3 (Dense) (None, 4096) 4198400 dropout_2 (Dropout) (None, 4096) 0 dense_4 (Dense) (None, 4096) 16781312 dropout_3 (Dropout) (None, 4096) 0 dense_5 (Dense) (None, 10) 40970 ================================================================= Total params: 24,773,386 Trainable params: 24,770,634 Non-trainable params: 2,752 _________________________________________________________________3. 원본 이미지 크기 조정 (32x32) -> (128x128)
CIFAR10 원본 이미지 크기 32x32 를 128x128로 증가 시키는 Sequence Dataset 생성
- 128x128로 CIFAR10 모든 이미지 배열값을 증가시키면 RAM 부족 발생.
- 배치 크기 만큼의 개수만 원본 이미지를 128x128로 증가 시킨 뒤(opencv의 resize()), 이를 모델에 입력하는 로직으로 Sequence Dataset 구성.
(1) init : 내부 변수로 할당.
(2) len : 데이터를 몇 회 가져와야 하는지 계산.
(3) getitem : 이미지와 labe을 batch size 만큼 가져온 뒤 사이즈 조정.
IMAGE_SIZE = 128 BATCH_SIZE = 64from tensorflow.keras.utils import Sequence import cv2 import sklearn # 입력 인자 images_array labels는 모두 numpy array로 들어옴. # 인자로 입력되는 images_array는 전체 32x32 image array임. class CIFAR_Dataset(Sequence): def __init__(self, images_array, labels, batch_size=BATCH_SIZE, augmentor=None, shuffle=False, pre_func=None): ''' 파라미터 설명 images_array: 원본 32x32 만큼의 image 배열값. labels: 해당 image의 label들 batch_size: __getitem__(self, index) 호출 시 마다 가져올 데이터 batch 건수 augmentor: albumentations 객체 shuffle: 학습 데이터의 경우 epoch 종료시마다 데이터를 섞을지 여부 ''' # 객체 생성 인자로 들어온 값을 객체 내부 변수로 할당. # 인자로 입력되는 images_array는 전체 32x32 image array임. self.images_array = images_array self.labels = labels self.batch_size = batch_size self.augmentor = augmentor self.pre_func = pre_func # train data의 경우 self.shuffle = shuffle if self.shuffle: # 객체 생성시에 한번 데이터를 섞음. #self.on_epoch_end() pass # Sequence를 상속받은 Dataset은 batch_size 단위로 입력된 데이터를 처리함. # __len__()은 전체 데이터 건수가 주어졌을 때 batch_size단위로 몇번 데이터를 반환하는지 나타남 def __len__(self): # batch_size단위로 데이터를 몇번 가져와야하는지 계산하기 위해 전체 데이터 건수를 batch_size로 나누되, 정수로 정확히 나눠지지 않을 경우 1회를 더한다. return int(np.ceil(len(self.labels) / self.batch_size)) # batch_size 단위로 image_array, label_array 데이터를 가져와서 변환한 뒤 다시 반환함 # 인자로 몇번째 batch 인지를 나타내는 index를 입력하면 해당 순서에 해당하는 batch_size 만큼의 데이타를 가공하여 반환 # batch_size 갯수만큼 변환된 image_array와 label_array 반환. def __getitem__(self, index): # index는 몇번째 batch인지를 나타냄. # batch_size만큼 순차적으로 데이터를 가져오려면 array에서 index*self.batch_size:(index+1)*self.batch_size 만큼의 연속 데이터를 가져오면 됨 # 32x32 image array를 self.batch_size만큼 가져옴. images_fetch = self.images_array[index*self.batch_size:(index+1)*self.batch_size] if self.labels is not None: label_batch = self.labels[index*self.batch_size:(index+1)*self.batch_size] # 만일 객체 생성 인자로 albumentation으로 만든 augmentor가 주어진다면 아래와 같이 augmentor를 이용하여 image 변환 # albumentations은 개별 image만 변환할 수 있으므로 batch_size만큼 할당된 image_name_batch를 한 건씩 iteration하면서 변환 수행. # 변환된 image 배열값을 담을 image_batch 선언. image_batch 배열은 float32 로 설정. image_batch = np.zeros((images_fetch.shape[0], IMAGE_SIZE, IMAGE_SIZE, 3), dtype='float32') # batch_size에 담긴 건수만큼 iteration 하면서 opencv image load -> image augmentation 변환(augmentor가 not None일 경우)-> image_batch에 담음. for image_index in range(images_fetch.shape[0]): #image = cv2.cvtColor(cv2.imread(image_name_batch[image_index]), cv2.COLOR_BGR2RGB) # 원본 image를 IMAGE_SIZE x IMAGE_SIZE 크기로 변환 image = cv2.resize(images_fetch[image_index], (IMAGE_SIZE, IMAGE_SIZE)) # 만약 augmentor가 주어졌다면 이를 적용. if self.augmentor is not None: image = self.augmentor(image=image)['image'] # 만약 scaling 함수가 입력되었다면 이를 적용하여 scaling 수행. if self.pre_func is not None: image = self.pre_func(image) # image_batch에 순차적으로 변환된 image를 담음. image_batch[image_index] = image return image_batch, label_batch # epoch가 한번 수행이 완료 될 때마다 모델의 fit()에서 호출됨. def on_epoch_end(self): if(self.shuffle): #print('epoch end') # 원본 image배열과 label를 쌍을 맞춰서 섞어준다. scikt learn의 utils.shuffle에서 해당 기능 제공 self.images_array, self.labels = sklearn.utils.shuffle(self.images_array, self.labels) else: passdef zero_one_scaler(image): return image/255.0 tr_ds = CIFAR_Dataset(tr_images, tr_oh_labels, batch_size=BATCH_SIZE, augmentor=None, shuffle=True, pre_func=zero_one_scaler) val_ds = CIFAR_Dataset(val_images, val_oh_labels, batch_size=BATCH_SIZE, augmentor=None, shuffle=False, pre_func=zero_one_scaler) print(next(iter(tr_ds))[0].shape, next(iter(val_ds))[0].shape) print(next(iter(tr_ds))[1].shape, next(iter(val_ds))[1].shape) # 원핫인코딩 했으므로 (64, 10) print(next(iter(tr_ds))[0][0])(64, 128, 128, 3) (64, 128, 128, 3) (64, 10) (64, 10) [[[0.6431373 0.68235296 0.69411767] [0.6431373 0.68235296 0.69411767] [0.63529414 0.67058825 0.68235296] ... [0.28627452 0.2901961 0.27058825] [0.27450982 0.28235295 0.26666668] [0.27450982 0.28235295 0.26666668]] [[0.6431373 0.68235296 0.69411767] [0.6431373 0.68235296 0.69411767] [0.63529414 0.67058825 0.68235296] ... [0.28627452 0.2901961 0.27058825] [0.27450982 0.28235295 0.26666668] [0.27450982 0.28235295 0.26666668]] [[0.6509804 0.6901961 0.7019608 ] [0.6509804 0.6901961 0.7019608 ] [0.6431373 0.6784314 0.6901961 ] ... [0.28627452 0.2901961 0.27058825] [0.2784314 0.28235295 0.26666668] [0.2784314 0.28235295 0.26666668]] ... [[0.9137255 0.85882354 0.8784314 ] [0.9137255 0.85882354 0.8784314 ] [0.90588236 0.84705883 0.8627451 ] ... [0.26666668 0.28235295 0.26666668] [0.27450982 0.28627452 0.27058825] [0.27450982 0.28627452 0.27058825]] [[0.91764706 0.8627451 0.88235295] [0.91764706 0.8627451 0.88235295] [0.90588236 0.8509804 0.8666667 ] ... [0.2627451 0.2784314 0.2627451 ] [0.27058825 0.28235295 0.26666668] [0.27058825 0.28235295 0.26666668]] [[0.91764706 0.8627451 0.88235295] [0.91764706 0.8627451 0.88235295] [0.90588236 0.8509804 0.8666667 ] ... [0.26666668 0.2784314 0.2627451 ] [0.27058825 0.28235295 0.26666668] [0.27058825 0.28235295 0.26666668]]]4. 모델 학습
Input 크기가 128x128x3 인 AlexNet 모델을 생성하고 epochs는 30으로 설정하고 학습
model = create_alexnet(in_shape=(128, 128, 3), n_classes=10, kernel_regular=regularizers.l2(l2=1e-4)) model.compile(optimizer=Adam(lr=0.001), loss='categorical_crossentropy', metrics=['accuracy']) # 5번 iteration내에 validation loss가 향상되지 않으면 learning rate을 기존 learning rate * 0.2로 줄임. rlr_cb = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=5, mode='min', verbose=1) ely_cb = EarlyStopping(monitor='val_loss', patience=10, mode='min', verbose=1) history = model.fit(tr_ds, epochs=30, #steps_per_epoch=int(np.ceil(tr_images.shape[0]/BATCH_SIZE)), validation_data=val_ds, #validation_steps=int(np.ceil(val_images.shape[0]/BATCH_SIZE)), callbacks=[rlr_cb, ely_cb] )Model: "model_2" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_3 (InputLayer) [(None, 128, 128, 3)] 0 conv2d_10 (Conv2D) (None, 30, 30, 96) 34944 activation_10 (Activation) (None, 30, 30, 96) 0 batch_normalization_10 (Bat (None, 30, 30, 96) 384 chNormalization) max_pooling2d_6 (MaxPooling (None, 14, 14, 96) 0 2D) conv2d_11 (Conv2D) (None, 14, 14, 256) 614656 activation_11 (Activation) (None, 14, 14, 256) 0 batch_normalization_11 (Bat (None, 14, 14, 256) 1024 chNormalization) max_pooling2d_7 (MaxPooling (None, 6, 6, 256) 0 2D) conv2d_12 (Conv2D) (None, 6, 6, 384) 885120 activation_12 (Activation) (None, 6, 6, 384) 0 batch_normalization_12 (Bat (None, 6, 6, 384) 1536 chNormalization) conv2d_13 (Conv2D) (None, 6, 6, 384) 1327488 activation_13 (Activation) (None, 6, 6, 384) 0 batch_normalization_13 (Bat (None, 6, 6, 384) 1536 chNormalization) conv2d_14 (Conv2D) (None, 6, 6, 256) 884992 activation_14 (Activation) (None, 6, 6, 256) 0 batch_normalization_14 (Bat (None, 6, 6, 256) 1024 chNormalization) max_pooling2d_8 (MaxPooling (None, 2, 2, 256) 0 2D) flatten_2 (Flatten) (None, 1024) 0 dense_6 (Dense) (None, 4096) 4198400 dropout_4 (Dropout) (None, 4096) 0 dense_7 (Dense) (None, 4096) 16781312 dropout_5 (Dropout) (None, 4096) 0 dense_8 (Dense) (None, 10) 40970 ================================================================= Total params: 24,773,386 Trainable params: 24,770,634 Non-trainable params: 2,752 _________________________________________________________________ Epoch 1/30/usr/local/lib/python3.7/dist-packages/keras/optimizer_v2/adam.py:105: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead. super(Adam, self).__init__(name, **kwargs)625/625 [==============================] - 29s 32ms/step - loss: 1.9866 - accuracy: 0.3658 - val_loss: 1.6507 - val_accuracy: 0.4430 - lr: 0.0010 Epoch 2/30 625/625 [==============================] - 20s 31ms/step - loss: 1.4828 - accuracy: 0.5307 - val_loss: 1.8877 - val_accuracy: 0.4387 - lr: 0.0010 Epoch 3/30 625/625 [==============================] - 20s 32ms/step - loss: 1.2710 - accuracy: 0.6209 - val_loss: 1.5578 - val_accuracy: 0.5333 - lr: 0.0010 Epoch 4/30 625/625 [==============================] - 21s 34ms/step - loss: 1.1307 - accuracy: 0.6819 - val_loss: 1.4284 - val_accuracy: 0.5889 - lr: 0.0010 Epoch 5/30 625/625 [==============================] - 19s 31ms/step - loss: 1.0626 - accuracy: 0.7215 - val_loss: 1.3497 - val_accuracy: 0.6361 - lr: 0.0010 Epoch 6/30 625/625 [==============================] - 19s 31ms/step - loss: 1.0067 - accuracy: 0.7527 - val_loss: 1.3321 - val_accuracy: 0.6553 - lr: 0.0010 Epoch 7/30 625/625 [==============================] - 19s 30ms/step - loss: 0.9638 - accuracy: 0.7816 - val_loss: 1.2152 - val_accuracy: 0.7098 - lr: 0.0010 Epoch 8/30 625/625 [==============================] - 20s 32ms/step - loss: 0.9433 - accuracy: 0.7988 - val_loss: 1.2119 - val_accuracy: 0.7142 - lr: 0.0010 Epoch 9/30 625/625 [==============================] - 19s 31ms/step - loss: 0.9117 - accuracy: 0.8181 - val_loss: 1.2229 - val_accuracy: 0.7283 - lr: 0.0010 Epoch 10/30 625/625 [==============================] - 19s 31ms/step - loss: 0.9235 - accuracy: 0.8294 - val_loss: 1.3754 - val_accuracy: 0.6726 - lr: 0.0010 Epoch 11/30 625/625 [==============================] - 20s 31ms/step - loss: 0.9047 - accuracy: 0.8453 - val_loss: 1.3700 - val_accuracy: 0.7090 - lr: 0.0010 Epoch 12/30 625/625 [==============================] - 19s 30ms/step - loss: 0.8706 - accuracy: 0.8601 - val_loss: 1.2489 - val_accuracy: 0.7542 - lr: 0.0010 Epoch 13/30 624/625 [============================>.] - ETA: 0s - loss: 0.8357 - accuracy: 0.8755 Epoch 00013: ReduceLROnPlateau reducing learning rate to 0.00020000000949949026. 625/625 [==============================] - 20s 31ms/step - loss: 0.8360 - accuracy: 0.8754 - val_loss: 1.3498 - val_accuracy: 0.7315 - lr: 0.0010 Epoch 14/30 625/625 [==============================] - 19s 30ms/step - loss: 0.6217 - accuracy: 0.9428 - val_loss: 1.1216 - val_accuracy: 0.8155 - lr: 2.0000e-04 Epoch 15/30 625/625 [==============================] - 19s 30ms/step - loss: 0.5046 - accuracy: 0.9729 - val_loss: 1.2450 - val_accuracy: 0.8153 - lr: 2.0000e-04 Epoch 16/30 625/625 [==============================] - 19s 31ms/step - loss: 0.4526 - accuracy: 0.9806 - val_loss: 1.3768 - val_accuracy: 0.8160 - lr: 2.0000e-04 Epoch 17/30 625/625 [==============================] - 19s 30ms/step - loss: 0.4202 - accuracy: 0.9861 - val_loss: 1.4376 - val_accuracy: 0.8020 - lr: 2.0000e-04 Epoch 18/30 625/625 [==============================] - 19s 30ms/step - loss: 0.4059 - accuracy: 0.9850 - val_loss: 1.4634 - val_accuracy: 0.8035 - lr: 2.0000e-04 Epoch 19/30 624/625 [============================>.] - ETA: 0s - loss: 0.3853 - accuracy: 0.9880 Epoch 00019: ReduceLROnPlateau reducing learning rate to 4.0000001899898055e-05. 625/625 [==============================] - 19s 30ms/step - loss: 0.3852 - accuracy: 0.9880 - val_loss: 1.5160 - val_accuracy: 0.8013 - lr: 2.0000e-04 Epoch 20/30 625/625 [==============================] - 19s 30ms/step - loss: 0.3587 - accuracy: 0.9940 - val_loss: 1.4752 - val_accuracy: 0.8195 - lr: 4.0000e-05 Epoch 21/30 625/625 [==============================] - 19s 30ms/step - loss: 0.3453 - accuracy: 0.9972 - val_loss: 1.5581 - val_accuracy: 0.8203 - lr: 4.0000e-05 Epoch 22/30 625/625 [==============================] - 19s 31ms/step - loss: 0.3381 - accuracy: 0.9979 - val_loss: 1.6077 - val_accuracy: 0.8211 - lr: 4.0000e-05 Epoch 23/30 625/625 [==============================] - 19s 30ms/step - loss: 0.3300 - accuracy: 0.9987 - val_loss: 1.7367 - val_accuracy: 0.8199 - lr: 4.0000e-05 Epoch 24/30 624/625 [============================>.] - ETA: 0s - loss: 0.3236 - accuracy: 0.9988 Epoch 00024: ReduceLROnPlateau reducing learning rate to 8.000000525498762e-06. 625/625 [==============================] - 19s 30ms/step - loss: 0.3236 - accuracy: 0.9988 - val_loss: 1.7716 - val_accuracy: 0.8217 - lr: 4.0000e-05 Epoch 00024: early stopping5. 테스트 데이터 평가
test_ds = CIFAR_Dataset(test_images, test_oh_labels, batch_size=BATCH_SIZE, augmentor=None, shuffle=False, pre_func=zero_one_scaler) model.evaluate(test_ds)157/157 [==============================] - 3s 21ms/step - loss: 1.9035 - accuracy: 0.8051[1.9034550189971924, 0.8051000237464905]'머신러닝 & 딥러닝' 카테고리의 다른 글
[CNN] GoogLeNet (0) 2022.01.23 [CNN] VGG (0) 2022.01.23 [Image] Albumentations (0) 2022.01.19 [Image] 이미지 증강 (Image Augmentation) (0) 2022.01.13 [신경망] 뉴스 기사 분류 (Multiclass classification) (0) 2021.12.23