NVLink, InfiniBand and SpectrumX

1. Concepts

• thread
• thread block
• Warp, a warp is 32 threads inside a thread block

GPU latency hiding

• A fundamental design differences for GPU to differenciate from CPU
• Implemented inside the SM