How to Break GPU Memory Boundaries Even with Large Batch Sizes
· It is now clearly noticeable that increasing the batch size will directly result in increasing the required GPU memory. In many cases, not having enough GPU memory prevents us from increasing the batch size. Let's now see how we could break the GPU memory boundaries and still use larger batch sizes.