NVLink, InfiniBand and SpectrumX

less than 1 minute read

1. Concepts

  • thread
  • thread block
  • Warp, a warp is 32 threads inside a thread block

GPU latency hiding

  • A fundamental design differences for GPU to differenciate from CPU
  • Implemented inside the SM

Tags:

Categories:

Updated:

You May Also Enjoy

Something about IRA

September 01 2025

I’m starting a new series, of Finance. Looked into IRA, backdoor, and mega backdoor. These CPA domain knowledge shouldn’t be too hard for STEM background ppl...

Ray continue on Two H200x8 nodes

August 28 2025

Issues and fixes GPU limitations Setting export CUDA_VISIBLE_DEVICES=0,1,... at the top of _start_ray.sh does NOT work Instead, add -e CU...