https://arxiv.org/abs/1502.03167
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful param
arxiv.org
▲ batch normalization 논문
논문리뷰 ㄴㄴ
간단 개념정리
문제 정의
→ internal covariate shift
→ 모든 레이어에 대한 whitening 대신에 배치놈 도입
→ why? cost & 모든곳에서 미분이 가능하지 x
참고자료
https://youtu.be/m61OSJfxL0U?feature=shared
'Study > 딥러닝' 카테고리의 다른 글
| TTS 모델 조사 (0) | 2024.01.28 |
|---|---|
| Regularization (0) | 2023.12.17 |
| Pytorch Tensorboard (1) | 2023.12.12 |
| 파이토치 (0) | 2023.12.11 |
| Weight Initialization (1) | 2023.10.22 |