Approach: implement an SPSC ring buffer using a power-of-two sized array, separate atomic head (write index) and tail (read index) with relaxed/acquire-release ordering to minimize fences. Use cache-line padding to avoid false sharing. Provide push/pop for small messages (copy into slots).cpp
#include <atomic>
#include <cstdint>
#include <vector>
#include <cstring>
#include <cassert>
template<size_t N>
class SpscRing {
static_assert((N & (N-1))==0, "N must be power of two");
struct alignas(64) AlignedIndex { std::atomic<uint32_t> v; char pad[64 - sizeof(std::atomic<uint32_t>)]; };
AlignedIndex head{}; // producer index (next write)
AlignedIndex tail{}; // consumer index (next read)
struct Slot { alignas(64) char data[64]; }; // message max 64 bytes; align to cache line
std::vector<Slot> buffer;
public:
SpscRing(): buffer(N) { head.v.store(0, std::memory_order_relaxed); tail.v.store(0, std::memory_order_relaxed); }
bool push(const void* src, size_t len) {
assert(len <= sizeof(buffer[0].data));
uint32_t h = head.v.load(std::memory_order_relaxed);
uint32_t t = tail.v.load(std::memory_order_acquire); // see consumer progress
if (((h + 1) & (N-1)) == (t & (N-1))) return false; // full
std::memcpy(buffer[h & (N-1)].data, src, len);
std::atomic_thread_fence(std::memory_order_release); // ensure data visible before index update
head.v.store(h + 1, std::memory_order_relaxed);
return true;
}
bool pop(void* dst, size_t* out_len) {
uint32_t t = tail.v.load(std::memory_order_relaxed);
uint32_t h = head.v.load(std::memory_order_acquire); // see producer progress
if ((t & (N-1)) == (h & (N-1))) return false; // empty
std::memcpy(dst, buffer[t & (N-1)].data, sizeof(buffer[0].data));
std::atomic_thread_fence(std::memory_order_release);
tail.v.store(t + 1, std::memory_order_relaxed);
if(out_len) *out_len = sizeof(buffer[0].data);
return true;
}
};
Memory ordering rationale:- Producer writes data to slot, uses release fence before advancing head so consumer's acquire load of head sees data.- Consumer uses acquire when reading head to ensure it sees written payload.- Relaxed for index increments because SPSC guarantees there's only one writer/one reader.- Fences and acquire/release minimize costly full barriers.Avoiding false sharing:- Align and pad indices and slots to 64-byte cache lines so head/tail and slot data sit on separate lines.Testing under contention:- Unit tests: single-producer single-consumer correctness with sequences, boundary wrap.- Stress test: run producer at high rate with random-sized messages, validate sequence numbers embedded in messages, run for long durations and detect lost/duplicated messages.- Race fuzz: vary thread priorities, CPU affinity (pin threads to different cores), run under sanitizers (TSAN) to detect ordering issues.- Throughput/latency measurement: microbench with timestamp in payload to measure round-trip latency.Alternatives:- Use sequence numbers per-slot to avoid fences entirely (Lamport-style) or use C++20 atomic_ref for fine-grained control.