Free Access NVIDIA NCP-AII New Release

The correct option is export HPL_OOC_SAFE_SIZE=4.0. NVIDIA HPL out-of-core mode allows matrix data that exceeds GPU memory capacity to be placed in host memory, but the GPU still needs reserved memory for drivers and runtime overhead. NVIDIA documents HPL_OOC_SAFE_SIZE as the amount of GPU memory, in GiB, reserved for the driver and not used by HPL out-of-core mode; increasing it is recommended when GPU out-of-memory errors occur. HPL_OOC_MODE=0 disables out-of-core mode, which would not help run a larger 40B matrix. HPL_OOC_NUM_STREAMS=8 changes the number of CUDA streams used for out-of-core operations, but it does not reserve driver memory. HPL_OOC_MAX_GPU_MEM=90 limits total GPU memory use, but the specific variable intended to leave safe driver space is HPL_OOC_SAFE_SIZE. During cluster burn-in, this setting helps preserve test validity while avoiding false failures caused by memory reservation issues rather than actual hardware instability.

The correct answer is Fat-Tree topology with rail-optimized design. NVIDIA DGX BasePOD and similar AI Factory architectures rely on high-performance InfiniBand fabrics to support distributed GPU training, low-latency communication, and scalable bandwidth between compute nodes. A Fat-Tree topology provides multiple high-bandwidth paths between servers and switches, helping reduce oversubscription and improve bisection bandwidth. Rail-optimized design is especially important in DGX environments because each GPU or network adapter path can be aligned to a specific network rail, improving predictable communication behavior for NCCL collectives such as all-reduce and reduce-scatter. A simple ring topology is not suitable because it introduces unnecessary hop latency and limited failure tolerance. A single flat Ethernet network for all traffic does not meet the dedicated high-performance InfiniBand fabric requirement. A star topology with one central switch can become a bottleneck and single point of failure. For BasePOD-scale AI workloads, the fabric must support high availability, bandwidth consistency, and efficient East-West GPU communication, making a rail-optimized Fat-Tree the correct design.

NVIDIA NeMo and large language model (LLM) training workloads are extremely demanding on HBM (High Bandwidth Memory). Out-of-Memory (OOM) errors often occur not because the total dataset is too large, but because memory fragmentation or sudden spikes in allocation (spikes during data shuffling or batch loading) exceed the available GPU memory. To mitigate this during intensive burn-in tests, engineers utilize the RMM (RAPIDS Memory Manager) library, which provides an asynchronous allocator. Enabling RMM asynchronous allocation allows the system to pre-allocate a pool of memory and manage it more efficiently than the standard CUDA allocator, reducing the overhead of constant allocations and deallocations. Furthermore, setting a specific blocksize (e.g., 1GB) for data loading ensures that the data ingestion pipeline reads data in manageable, deterministic chunks. This prevents the system from attempting to load massive files entirely into memory at once, which is the primary cause of OOMs in question-answering tasks involving large Parquet or JSON datasets. Switching to FP32 (Option B) would actually double the memory footprint and increase the likelihood of an OOM error.

NVIDIA DGX BasePOD and SuperPOD reference architectures mandate strict network segmentation to ensure performance, security, and manageability.

Compute Network: Typically InfiniBand (or high-speed Spectrum-X Ethernet), dedicated solely to GPU-to-GPU collective communications (NCCL).

Storage Network: A high-bandwidth Ethernet or InfiniBand fabric specifically for data ingestion and model checkpointing, often utilizing GPUDirect Storage (GDS).

Management Network: Used for standard cluster administration, SSH, and software orchestration (e.g., Bright Cluster Manager or Kubernetes control plane traffic).

Out-of-Band (OOB) Network: A physically isolated network connected to the BMC ports for low-level system monitoring, power control, and remote console access, even when the OS is down.

A single VLAN (Option A) would cause massive congestion during training, as storage and management traffic would compete with high-frequency compute packets. The four-network model ensures that a " storm " in the storage fabric does not prevent an administrator from accessing the system via the management or OOB networks, which is essential for maintaining an AI Factory at scale.