tensorflow2에서 손실함수 동작 원리 실습해보기

tensorflow2에서 손실함수 동작 원리 실습¶

모듈 불러오기

In [1]:

import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt

가상의 훈련 x, y값 설정

In [2]:

x_train = [1,2,3,4]
y_train = [0,-1,-2,-3]

모델 keras의 Neural Net, Sequential model load

In [3]:

model = tf.keras.Sequential()

예시로 1개 층, 1개 노드를 갖춘 단순 모델 설계

In [4]:

model.add(tf.keras.layers.Dense(units = 1, input_dim = 1))

In [6]:

opt = tf.keras.optimizers.SGD(lr = 0.1)
model.compile(loss = 'mse', optimizer = opt)

In [7]:

hist = model.fit(x_train, y_train , epochs = 100)

Train on 4 samples
Epoch 1/100
4/4 [==============================] - 0s 123ms/sample - loss: 13.7163
Epoch 2/100
4/4 [==============================] - 0s 2ms/sample - loss: 6.3212
Epoch 3/100
4/4 [==============================] - 0s 3ms/sample - loss: 2.9913
Epoch 4/100
4/4 [==============================] - 0s 3ms/sample - loss: 1.4873
Epoch 5/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.8034
Epoch 6/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.4884
Epoch 7/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.3394
Epoch 8/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.2655
Epoch 9/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.2256
Epoch 10/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.2014
Epoch 11/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.1847
Epoch 12/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.1716
Epoch 13/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.1605
Epoch 14/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.1506
Epoch 15/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.1415
Epoch 16/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.1330
Epoch 17/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.1252
Epoch 18/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.1178
Epoch 19/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.1108
Epoch 20/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.1043
Epoch 21/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0981
Epoch 22/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0923
Epoch 23/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0869
Epoch 24/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0818
Epoch 25/100
4/4 [==============================] - 0s 4ms/sample - loss: 0.0769
Epoch 26/100
4/4 [==============================] - 0s 4ms/sample - loss: 0.0724
Epoch 27/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0681
Epoch 28/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0641
Epoch 29/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0603
Epoch 30/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0568
Epoch 31/100
4/4 [==============================] - 0s 4ms/sample - loss: 0.0534
Epoch 32/100
4/4 [==============================] - 0s 4ms/sample - loss: 0.0503
Epoch 33/100
4/4 [==============================] - 0s 4ms/sample - loss: 0.0473
Epoch 34/100
4/4 [==============================] - 0s 4ms/sample - loss: 0.0445
Epoch 35/100
4/4 [==============================] - 0s 5ms/sample - loss: 0.0419
Epoch 36/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0394
Epoch 37/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0371
Epoch 38/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0349
Epoch 39/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0328
Epoch 40/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0309
Epoch 41/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0291
Epoch 42/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0274
Epoch 43/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0258
Epoch 44/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0242
Epoch 45/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0228
Epoch 46/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0215
Epoch 47/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0202
Epoch 48/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0190
Epoch 49/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0179
Epoch 50/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0168
Epoch 51/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0158
Epoch 52/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0149
Epoch 53/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0140
Epoch 54/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0132
Epoch 55/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0124
Epoch 56/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0117
Epoch 57/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0110
Epoch 58/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0103
Epoch 59/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0097
Epoch 60/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0092
Epoch 61/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0086
Epoch 62/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0081
Epoch 63/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0076
Epoch 64/100
4/4 [==============================] - 0s 4ms/sample - loss: 0.0072
Epoch 65/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0068
Epoch 66/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0064
Epoch 67/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0060
Epoch 68/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0056
Epoch 69/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0053
Epoch 70/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0050
Epoch 71/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0047
Epoch 72/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0044
Epoch 73/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0042
Epoch 74/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0039
Epoch 75/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0037
Epoch 76/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0035
Epoch 77/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0033
Epoch 78/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0031
Epoch 79/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0029
Epoch 80/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0027
Epoch 81/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0026
Epoch 82/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0024
Epoch 83/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0023
Epoch 84/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0021
Epoch 85/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0020
Epoch 86/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0019
Epoch 87/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0018
Epoch 88/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0017
Epoch 89/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0016
Epoch 90/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0015
Epoch 91/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0014
Epoch 92/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0013
Epoch 93/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0012
Epoch 94/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0012
Epoch 95/100
4/4 [==============================] - 0s 3ms/sample - loss: 0.0011
Epoch 96/100
4/4 [==============================] - 0s 2ms/sample - loss: 0.0010
Epoch 97/100
4/4 [==============================] - 0s 2ms/sample - loss: 9.6644e-04
Epoch 98/100
4/4 [==============================] - 0s 3ms/sample - loss: 9.0944e-04
Epoch 99/100
4/4 [==============================] - 0s 3ms/sample - loss: 8.5580e-04
Epoch 100/100
4/4 [==============================] - 0s 2ms/sample - loss: 8.0533e-04

In [8]:

model.summary()

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 1)                 2         
=================================================================
Total params: 2
Trainable params: 2
Non-trainable params: 0
_________________________________________________________________

In [12]:

y_pred = model.predict(np.array([5,4]))
print(y_pred)

[[-3.9528065]
 [-2.9757173]]

In [13]:

# plot training & val loss values
plt.plot(hist.history['loss'])
plt.title('Model loss graph')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'])
plt.show()

손실 함수의 그래프 단순한 모델이지만 loss가 낮아지는 방향으로 계속해서 움직인다.