前向传播与反向传播：用计算图理解参数更新

weight, x --乘法--> weighted
weighted, bias --加法--> prediction
prediction, y --减法--> error
error --平方与平均--> loss

d(loss) / d(weight)
= d(loss) / d(prediction) * d(prediction) / d(weight)

parameter = parameter - learning_rate * gradient

import numpy as np

x = np.array([1.0, 2.0, 3.0, 4.0])
y = np.array([3.0, 5.0, 7.0, 9.0])
weight = 0.5
bias = 0.0
learning_rate = 0.05


def mse(current_weight: float, current_bias: float) -> float:
    prediction = current_weight * x + current_bias
    return float(np.mean((prediction - y) ** 2))


before = mse(weight, bias)
prediction = weight * x + bias
error = prediction - y
weight_gradient = 2 * np.mean(error * x)
bias_gradient = 2 * np.mean(error)

epsilon = 1e-5
numeric_weight_gradient = (
    mse(weight + epsilon, bias) - mse(weight - epsilon, bias)
) / (2 * epsilon)
numeric_bias_gradient = (
    mse(weight, bias + epsilon) - mse(weight, bias - epsilon)
) / (2 * epsilon)

print("analytic:", round(float(weight_gradient), 6), round(float(bias_gradient), 6))
print("numeric:", round(numeric_weight_gradient, 6), round(numeric_bias_gradient, 6))

weight -= learning_rate * weight_gradient
bias -= learning_rate * bias_gradient
after = mse(weight, bias)
print("loss:", round(before, 6), "->", round(after, 6))
print("parameters:", round(float(weight), 3), round(float(bias), 3))

前向传播 -> 计算损失 -> 清空旧梯度
-> 反向传播 -> 更新参数 -> 进入下一批