Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,10 +185,10 @@ The HIP path builds the same sources for AMD GPUs in place of CUDA. On Windows:

On Linux:
```
cmake --preset ninja-hip -DCMAKE_HIP_ARCHITECTURES=gfx1100 -DCMAKE_PREFIX_PATH=/opt/rocm
cmake --preset ninja-hip -DCMAKE_HIP_ARCHITECTURES="gfx10-3-generic;gfx11-generic;gfx12-generic" -DCMAKE_PREFIX_PATH=/opt/rocm
cmake --build --preset ninja-hip-release
```
`CMAKE_HIP_ARCHITECTURES` selects the target architecture (`gfx1100` for RDNA3, `gfx90a` for CDNA2 / MI200). If omitted, it is auto-detected from the GPUs of the host. `CMAKE_PREFIX_PATH` lets `find_package(hip)` locate the ROCm installation when CMake uses the vcpkg toolchain. Adjust it if ROCm is installed elsewhere.
`CMAKE_HIP_ARCHITECTURES` lists the target architectures and has to cover every AMD GPU that `rocminfo` shows, including integrated GPUs that ALIEN does not use. The list above covers RDNA2, RDNA3 and RDNA4 like the Windows build. Other GPUs need their own entry, for example `gfx90a` for CDNA2 / MI200 or `gfx90c` for the integrated Vega GPU of many Ryzen 4000 and 5000 processors. If the option is omitted, the architectures of the GPUs in the computer are detected, which shortens the build. `CMAKE_PREFIX_PATH` lets `find_package(hip)` locate the ROCm installation when CMake uses the vcpkg toolchain. Adjust it if ROCm is installed elsewhere.

**AMD GPUs (SCALE)**

Expand All @@ -197,6 +197,7 @@ Alternatively, the unchanged CUDA sources can be compiled for AMD GPUs with [SCA
cmake --preset ninja -DCMAKE_CUDA_COMPILER=/opt/scale/bin/nvcc -DCMAKE_CUDA_ARCHITECTURES=gfx1100
cmake --build --preset ninja-release
```
Replace `gfx1100` with the architecture of your GPU, which `rocminfo` shows.

### Command-line interface

Expand Down
5 changes: 5 additions & 0 deletions external/hip/cuda_to_hip.h
Original file line number Diff line number Diff line change
Expand Up @@ -42,12 +42,17 @@ using cudaError_t = hipError_t;
#define cudaErrorMemoryAllocation hipErrorOutOfMemory
#define cudaErrorIllegalAddress hipErrorIllegalAddress
#define cudaErrorLaunchFailure hipErrorLaunchFailure
#define cudaErrorInvalidDeviceFunction hipErrorInvalidDeviceFunction
#define cudaErrorNoKernelImageForDevice hipErrorNoBinaryForGpu
#define cudaErrorInvalidPtx hipErrorInvalidKernelFile

// --- device management ---
#define cudaDeviceProp hipDeviceProp_t
#define cudaGetDeviceCount hipGetDeviceCount
#define cudaGetDeviceProperties hipGetDeviceProperties
#define cudaOccupancyMaxActiveBlocksPerMultiprocessor hipOccupancyMaxActiveBlocksPerMultiprocessor
#define cudaFuncAttributes hipFuncAttributes
#define cudaFuncGetAttributes hipFuncGetAttributes
#define cudaMemGetInfo hipMemGetInfo
#define cudaSetDevice hipSetDevice
#define cudaDeviceSynchronize hipDeviceSynchronize
Expand Down
38 changes: 35 additions & 3 deletions source/Engine/Impl/SimulationCudaFacade.cu
Original file line number Diff line number Diff line change
Expand Up @@ -730,6 +730,37 @@ void _SimulationCudaFacade::initCuda()
log(Priority::Important, "device " + std::to_string(_gpuInfo.deviceNumber) + " selected");
}

namespace
{
std::string getArchitectureDescription(cudaDeviceProp const& prop)
{
#if defined(USE_HIP)
return std::string("architecture ") + prop.gcnArchName;
#else
return "compute capability " + std::to_string(prop.major) + "." + std::to_string(prop.minor);
#endif
}

#if defined(USE_HIP)
void checkGpuCodeForDevice(int deviceNumber)
{
cudaDeviceProp prop;
CHECK_FOR_DEVICE_ERRORS(cudaGetDeviceProperties(&prop, deviceNumber));
CHECK_FOR_DEVICE_ERRORS(cudaSetDevice(deviceNumber));

cudaFuncAttributes attributes;
auto result = cudaFuncGetAttributes(&attributes, reinterpret_cast<void const*>(cudaNextTimestep_physics_calcFluidForces));
if (result == cudaErrorNoKernelImageForDevice || result == cudaErrorInvalidDeviceFunction || result == cudaErrorInvalidPtx) {
throw std::runtime_error(
"The GPU code of this build could not be loaded for " + std::string(prop.name) + " (" + getArchitectureDescription(prop) + "). "
+ "A build from the sources needs the architectures of all AMD GPUs of this computer in CMAKE_HIP_ARCHITECTURES, integrated ones included. "
+ "Error code: " + std::to_string(result));
}
CHECK_FOR_DEVICE_ERRORS(result);
}
#endif
}

auto _SimulationCudaFacade::checkAndReturnGpuInfo() -> GpuInfo
{
static std::mutex mutex;
Expand Down Expand Up @@ -764,7 +795,7 @@ auto _SimulationCudaFacade::checkAndReturnGpuInfo() -> GpuInfo

auto discrete = prop.integrated == 0;
std::stringstream stream;
stream << "device " << deviceNumber << ": " << prop.name << " with compute capability " << prop.major << "." << prop.minor << ", "
stream << "device " << deviceNumber << ": " << prop.name << " with " << getArchitectureDescription(prop) << ", "
<< (discrete ? "discrete" : "integrated") << ", " << prop.totalGlobalMem / (1024 * 1024) << " MB";
log(Priority::Important, stream.str());

Expand All @@ -778,10 +809,11 @@ auto _SimulationCudaFacade::checkAndReturnGpuInfo() -> GpuInfo
std::memcpy(result.gpuUuid.data(), &prop.uuid, result.gpuUuid.size());
}
}
#if !defined(USE_HIP)
#if defined(USE_HIP)
checkGpuCodeForDevice(result.deviceNumber);
#else
// The CUDA compute-capability heuristic (major*100+minor, gated at >= 705)
// is meaningless for AMD GPUs, where prop.major/minor encode the gfx arch.
// On HIP accept the selected device unconditionally.
if (selectedComputeCapability < 705) {
throw std::runtime_error("No CUDA device with compute capability of 7.5 or higher found.");
}
Expand Down