Chapter 12: Dynamic CPU Probing & Zero-Branching Dispatch Tables
"Compiling with
-march=nativeis a luxury for personal benchmarks. Distributing production binaries requires running everywhere—from an old dual-core cloud VM to a 128-core workstation—without crashing onSIGILLand without sacrificing a single FLOP."
When you distribute a high-performance computer vision or machine learning library, you encounter the Heterogeneous Silicon Dilemma:
1. If you compile your entire binary with -mavx2 or -march=armv8.4-a, executing that binary on an older CPU instantly triggers an illegal instruction exception (SIGILL) and crashes.
2. If you compile with conservative generic flags (e.g. -march=x86-64), your binary runs on every machine, but wastes \(80\%\text{--}90\%\) of modern silicon performance.
3. If you insert if (cpu_has_avx2()) checks inside your inner loops, you pollute the branch predictor, bloat code size, and introduce branch evaluation latency in the hottest instruction streams.
In this chapter, we design and implement a Zero-Branching Hardware Abstraction Layer (HAL). We inspect hardware capabilities at startup using cpuid and system kernel queries, binding optimal SIMD micro-kernels to an immutable function dispatch table.
12.1 Hardware Feature Detection Across Architectures
x86-64 Architecture: The cpuid Instruction
On x86 processors, the cpuid instruction queries processor identification and feature leaves:
#if defined(__x86_64__) || defined(_M_X64)
#include <cpuid.h>
typedef struct {
uint32_t eax, ebx, ecx, edx;
} rcre_cpuid_regs;
static inline rcre_cpuid_regs rcre_cpuid(uint32_t leaf, uint32_t subleaf) {
rcre_cpuid_regs regs;
__cpuid_count(leaf, subleaf, regs.eax, regs.ebx, regs.ecx, regs.edx);
return regs;
}
#endif
1, Register ECX, Bit 28.
- FMA3: Leaf 1, Register ECX, Bit 12.
- AVX2: Leaf 7, Subleaf 0, Register EBX, Bit 5.
- AVX-512 Foundation: Leaf 7, Subleaf 0, Register EBX, Bit 16.
ARM64 Architecture: OS Kernel Feature Queries
Unlike x86, ARM64 restricts direct user-space instruction querying. User software must query operating system kernel capabilities:
- Linux: getauxval(AT_HWCAP) or getauxval(AT_HWCAP2).
#include <sys/auxv.h>
#include <asm/hwcap.h>
bool has_neon = (getauxval(AT_HWCAP) & HWCAP_ASIMD) != 0;
sysctlbyname().
#include <sys/sysctl.h>
int val = 0; size_t len = sizeof(val);
sysctlbyname("hw.optional.neon", &val, &len, NULL, 0);
12.2 The Branching Hazard in Hot Loops
Consider the naive implementation of a hardware-adaptive vector operation:
// ANTI-PATTERN: Branching inside hot inner loops
void vector_add(float* dst, const float* a, const float* b, size_t n) {
for (size_t i = 0; i < n; ++i) {
if (g_cpu_features.has_avx2) {
// AVX2 branch
} else if (g_cpu_features.has_neon) {
// NEON branch
} else {
// Scalar fallback
}
}
}
The Cost of In-Loop Dynamic Dispatch
- Branch Predictor Pressure: Every iteration evaluates branch conditions. While the branch outcome is invariant, the Branch Target Buffer (BTB) must retain branch entries, displacing application branching history.
- Instruction Pipeline Inefficiency: The branch instruction prevents the compiler from emitting uninterrupted vector streaming instructions.
- Register Allocation Overhead: The compiler must preserve registers across conditionally executed code paths.
12.3 Zero-Branching Function Dispatch Tables
The textbook architectural solution is Initialization-Time Function Table Binding. We encapsulate hardware-accelerated kernels behind an immutable table of function pointers:
typedef float (*rcre_dot_fn)(const float* a, const float* b, size_t n);
typedef void (*rcre_gemm_fn)(const float* A, const float* B, float* C, size_t M, size_t N, size_t K);
typedef struct {
rcre_dot_fn dot;
rcre_gemm_fn gemm;
} RCRE_HAL_Table;
extern RCRE_HAL_Table rcre_hal;
Single Initialization Step
At library startup (or upon first invocation via a one-time guard), the CPU capabilities are probed, and rcre_hal is populated with pointers directly targeting the highest-tier instruction set supported by the host machine:
+-----------------------------------+
| rcre_hal_init() |
+-----------------+-----------------+
|
[Detect CPU Features]
|
+-------------------------+-------------------------+
| (Host: Apple M4) | (Host: Core i9)
v v
+-----------------------+ +-----------------------+
| rcre_hal.dot = neon | | rcre_hal.dot = avx2 |
| rcre_hal.gemm = neon | | rcre_hal.gemm = avx2 |
+-----------------------+ +-----------------------+
Inside your hot computing kernels, call the dispatch table with zero branching:
The indirect call overhead is amortized across thousands of vector operations, achieving 100% full SIMD execution speed with zero runtime branch penalties.12.4 GNU Indirect Functions (ifunc)
On modern ELF platforms (Linux glibc), the dynamic linker supports STT_GNU_IFUNC. An indirect function resolver executes during dynamic linking before main() begins:
#if defined(__GNUC__) && !defined(__APPLE__)
float rcre_dot_dispatch(const float* a, const float* b, size_t n)
__attribute__((ifunc("rcre_dot_resolve")));
static rcre_dot_fn rcre_dot_resolve(void) {
if (rcre_probe_cpu().has_avx2) return rcre_dot_avx2;
return rcre_dot_scalar;
}
#endif
ifunc, the dynamic linker patches the Global Offset Table (GOT) entry directly with the chosen function pointer. Subsequent calls incur zero indirect branch overhead, performing an identical direct jump as a static function call!
12.5 Chapter Project: The Universal HAL Dispatch Table
Project Objective
Implement rcre_hal.h and main.c supporting:
1. Dynamic CPU feature probe identifying SIMD extensions (ARM NEON, SSE, AVX, AVX2).
2. Function pointer dispatch table binding optimal kernels at runtime.
3. Comparative benchmark verifying zero-overhead dispatch versus in-loop branching.
Applied Tasks
- Execute feature detection and print detected capabilities.
- Validate numerical equivalence across all available execution paths.
- Run under AddressSanitizer and verify zero memory leaks and zero segmentation faults.