rt/snapraid
1
0
Fork 0
mirror of https://github.com/amadvance/snapraid.git synced 2026-10-26 19:01:57 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

Add SMHasher3 adapters and validation results for SnapRAID hash functions

Browse source
This commit is contained in:
Andrea Mazzoleni 2026-08-17 15:40:01 +02:00
parent b8ffdbd1f7
commit e241b9b43d
7 changed files with 12170 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

3777
smhasher3/MurmurHash3-128-snapraid.txt Normal file
View file

File diff suppressed because it is too large Load diff

3744
smhasher3/MuseAir-128.bfast-snapraid.txt Normal file
View file

File diff suppressed because it is too large Load diff

136
smhasher3/README Normal file
View file

@ -0,0 +1,136 @@
SMHasher3 validation files for SnapRAID
=======================================
This directory contains SMHasher3 adapters and test results for the 128-bit
non-cryptographic hash functions used by SnapRAID.
The adapters reproduce the SnapRAID implementations while adapting their
128-bit seed interfaces to the 64-bit seed_t exposed by SMHasher3.
SMHasher3 version
=================
commit 51d3cd1ac0aa4934f6aacb44d9d234f50300b6e3
Author: Frank J. T. Wojcik <fwojcik@uw.edu>
Date: Tue Aug 4 11:58:32 2026 -0700
Fix whitespace (NFC)
Files and results
=================
museair_snapraid.cpp
MuseAir BFast-128 as used by SnapRAID. Inputs shorter than 32 bytes
are zero-padded to 32 bytes and processed with the Long path while
retaining the original input length.
MuseAir-128.bfast-snapraid.txt
SMHasher3 output for museair_snapraid.cpp.
Result comparison:
The SnapRAID variant passes all 252 tests. The official MuseAir
BFast-128 also passes its complete test set. The different short-input
path used by SnapRAID shows no statistical regression. Small differences
in the p-value distribution are not significant enough to rank either
version as better.
spooky2_snapraid.cpp
SpookyHash V2 128-bit as used by SnapRAID. SnapRAID always uses the
Long path instead of the standard Short path below 192 bytes.
SpookyHash2-128-snapraid.txt
SMHasher3 output for spooky2_snapraid.cpp.
Result comparison:
The SnapRAID variant passes all 252 tests. The official SpookyHash V2
128-bit implementation has strong BIC failures on several short input
lengths. Those failures come from the Short path. Always using the Long
path, as SnapRAID does, removes the observed BIC weakness.
murmur3_snapraid.cpp
MurmurHash3 x86_128 as used by SnapRAID, with the SnapRAID 128-bit
seed interface.
MurmurHash3-128-snapraid.txt
SMHasher3 output for murmur3_snapraid.cpp.
Result comparison:
The SnapRAID variant passes 179 of 252 tests, compared with 154 of 250
for the official MurmurHash3 x86_128 int32 variant. Most data-path
failures are the same, including BIC, Text, TwoBytes, PerlinNoise and
Bitflip. The main improvement is in seed tests because the SnapRAID
adapter uses all 64 bits of seed_t, while the official SMHasher3 wrapper
uses only the low 32 bits. MurmurHash3 remains substantially weaker than
the newer hashes and is retained by SnapRAID only for legacy use.
Seed mapping
============
SMHasher3 exposes one 64-bit seed_t, while SnapRAID uses 128-bit seeds.
For MuseAir and SpookyHash V2:
seed[0..7] = seed_t
seed[8..15] = seed_t
For MurmurHash3 x86_128 this gives:
h1 = low32(seed_t)
h2 = high32(seed_t)
h3 = low32(seed_t)
h4 = high32(seed_t)
This intentionally tests the same diagonal seed mapping used by the other
SnapRAID adapters.
Building
========
Copy the adapters into the SMHasher3 hashes directory:
hashes/museair_snapraid.cpp
hashes/spooky2_snapraid.cpp
hashes/murmur3_snapraid.cpp
Add them to hashes/Hashsrc.cmake and rebuild SMHasher3.
Check the registrations with:
./SMHasher3 --list | grep -i snapraid
Generating the result files
===========================
./SMHasher3 MuseAir-128.bfast-snapraid \
--extra --notest=Speed,Hashmap --ncpu=16 \
> MuseAir-128.bfast-snapraid.txt 2>&1
./SMHasher3 SpookyHash2-128-snapraid \
--extra --notest=Speed,Hashmap --ncpu=16 \
> SpookyHash2-128-snapraid.txt 2>&1
./SMHasher3 MurmurHash3-128.snapraid \
--extra --notest=Speed,Hashmap --ncpu=16 \
> MurmurHash3-128-snapraid.txt 2>&1
Source relationship
===================
The authoritative implementations remain the SnapRAID source files:
cmdline/museair.c
cmdline/spooky2.c
cmdline/murmur3.c
SnapRAID repository:
https://github.com/amadvance/snapraid
Whenever one of these implementations changes, compare the corresponding
adapter again with the SnapRAID source and rerun the SMHasher3 test.

3744
smhasher3/SpookyHash2-128-snapraid.txt Normal file
View file

File diff suppressed because it is too large Load diff

257
smhasher3/murmur3_snapraid.cpp Normal file
View file

@ -0,0 +1,257 @@
/*
* MurmurHash3 x86_128 SnapRAID adapter for SMHasher3.
*
* Based on SnapRAID cmdline/murmur3.c, itself derived from
* MurmurHash3.cpp revision r136.
*
* SnapRAID accepts a full 128-bit seed and initializes the four 32-bit
* state words independently.
*
* SMHasher3 exposes one 64-bit seed_t. As with the other SnapRAID
* adapters, duplicate that complete 64-bit seed into both halves of the
* 128-bit SnapRAID seed:
*
* seed[0..7] = seed
* seed[8..15] = seed
*
* Therefore, for
*
* seed = (seed_hi << 32) | seed_lo
*
* the MurmurHash3 state is initialized as:
*
* h1 = seed_lo
* h2 = seed_hi
* h3 = seed_lo
* h4 = seed_hi
*
* This intentionally tests only the diagonal 64-bit subset of the full
* 128-bit SnapRAID seed space.
*
* The SnapRAID source is GPL-3.0-or-later.
*/
#include "Platform.h"
#include "Hashlib.h"
//------------------------------------------------------------------------------
static FORCE_INLINE uint32_t snapraid_murmur3_fmix32(uint32_t h) {
h ^= h >> 16;
h *= UINT32_C(0x85ebca6b);
h ^= h >> 13;
h *= UINT32_C(0xc2b2ae35);
h ^= h >> 16;
return h;
}
//------------------------------------------------------------------------------
template <bool bswap>
static void MurmurHash3_x86_128_SnapRAID(
const void * in,
const size_t len,
const seed_t seed,
void * out) {
const uint8_t * data = (const uint8_t *)in;
const size_t nblocks = len / 16;
/*
* Equivalent to a SnapRAID 128-bit seed formed by concatenating two
* identical copies of the 64-bit SMHasher3 seed.
*/
const uint32_t seed_lo = (uint32_t)(uint64_t)seed;
const uint32_t seed_hi = (uint32_t)((uint64_t)seed >> 32);
uint32_t h1 = seed_lo;
uint32_t h2 = seed_hi;
uint32_t h3 = seed_lo;
uint32_t h4 = seed_hi;
const uint32_t c1 = UINT32_C(0x239b961b);
const uint32_t c2 = UINT32_C(0xab0e9789);
const uint32_t c3 = UINT32_C(0x38b34ae5);
const uint32_t c4 = UINT32_C(0xa1e38b93);
// body
const uint8_t * blocks = data;
for (size_t i = 0; i < nblocks; i++) {
uint32_t k1 = GET_U32<bswap>(blocks, i * 16 + 0);
uint32_t k2 = GET_U32<bswap>(blocks, i * 16 + 4);
uint32_t k3 = GET_U32<bswap>(blocks, i * 16 + 8);
uint32_t k4 = GET_U32<bswap>(blocks, i * 16 + 12);
k1 *= c1;
k1 = ROTL32(k1, 15);
k1 *= c2;
h1 ^= k1;
h1 = ROTL32(h1, 19);
h1 += h2;
h1 = h1 * 5 + UINT32_C(0x561ccd1b);
k2 *= c2;
k2 = ROTL32(k2, 16);
k2 *= c3;
h2 ^= k2;
h2 = ROTL32(h2, 17);
h2 += h3;
h2 = h2 * 5 + UINT32_C(0x0bcaa747);
k3 *= c3;
k3 = ROTL32(k3, 17);
k3 *= c4;
h3 ^= k3;
h3 = ROTL32(h3, 15);
h3 += h4;
h3 = h3 * 5 + UINT32_C(0x96cd1c35);
k4 *= c4;
k4 = ROTL32(k4, 18);
k4 *= c1;
h4 ^= k4;
h4 = ROTL32(h4, 13);
h4 += h1;
h4 = h4 * 5 + UINT32_C(0x32ac3b17);
}
// tail
const uint8_t * tail = data + nblocks * 16;
uint32_t k1 = 0;
uint32_t k2 = 0;
uint32_t k3 = 0;
uint32_t k4 = 0;
switch (len & 15) {
case 15:
k4 ^= (uint32_t)tail[14] << 16;
/* FALLTHROUGH */
case 14:
k4 ^= (uint32_t)tail[13] << 8;
/* FALLTHROUGH */
case 13:
k4 ^= (uint32_t)tail[12];
k4 *= c4;
k4 = ROTL32(k4, 18);
k4 *= c1;
h4 ^= k4;
/* FALLTHROUGH */
case 12:
k3 ^= (uint32_t)tail[11] << 24;
/* FALLTHROUGH */
case 11:
k3 ^= (uint32_t)tail[10] << 16;
/* FALLTHROUGH */
case 10:
k3 ^= (uint32_t)tail[9] << 8;
/* FALLTHROUGH */
case 9:
k3 ^= (uint32_t)tail[8];
k3 *= c3;
k3 = ROTL32(k3, 17);
k3 *= c4;
h3 ^= k3;
/* FALLTHROUGH */
case 8:
k2 ^= (uint32_t)tail[7] << 24;
/* FALLTHROUGH */
case 7:
k2 ^= (uint32_t)tail[6] << 16;
/* FALLTHROUGH */
case 6:
k2 ^= (uint32_t)tail[5] << 8;
/* FALLTHROUGH */
case 5:
k2 ^= (uint32_t)tail[4];
k2 *= c2;
k2 = ROTL32(k2, 16);
k2 *= c3;
h2 ^= k2;
/* FALLTHROUGH */
case 4:
k1 ^= (uint32_t)tail[3] << 24;
/* FALLTHROUGH */
case 3:
k1 ^= (uint32_t)tail[2] << 16;
/* FALLTHROUGH */
case 2:
k1 ^= (uint32_t)tail[1] << 8;
/* FALLTHROUGH */
case 1:
k1 ^= (uint32_t)tail[0];
k1 *= c1;
k1 = ROTL32(k1, 15);
k1 *= c2;
h1 ^= k1;
}
// finalization
h1 ^= (uint32_t)len;
h2 ^= (uint32_t)len;
h3 ^= (uint32_t)len;
h4 ^= (uint32_t)len;
h1 += h2;
h1 += h3;
h1 += h4;
h2 += h1;
h3 += h1;
h4 += h1;
h1 = snapraid_murmur3_fmix32(h1);
h2 = snapraid_murmur3_fmix32(h2);
h3 = snapraid_murmur3_fmix32(h3);
h4 = snapraid_murmur3_fmix32(h4);
h1 += h2;
h1 += h3;
h1 += h4;
h2 += h1;
h3 += h1;
h4 += h1;
PUT_U32<bswap>(h1, (uint8_t *)out, 0);
PUT_U32<bswap>(h2, (uint8_t *)out, 4);
PUT_U32<bswap>(h3, (uint8_t *)out, 8);
PUT_U32<bswap>(h4, (uint8_t *)out, 12);
}
//------------------------------------------------------------------------------
REGISTER_FAMILY(murmur3_snapraid,
$.src_url = "https://github.com/amadvance/snapraid/",
$.src_status = HashFamilyInfo::SRC_ACTIVE
);
REGISTER_HASH(MurmurHash3_128__snapraid,
$.desc =
"MurmurHash3 x86_128 with SnapRAID 128-bit seed interface",
$.hash_flags =
0,
$.impl_flags =
FLAG_IMPL_MULTIPLY |
FLAG_IMPL_ROTATE |
FLAG_IMPL_LICENSE_GPL3,
$.bits = 128,
$.verification_LE = 0x260E7FCF,
$.verification_BE = 0x37086199,
$.hashfn_native = MurmurHash3_x86_128_SnapRAID<false>,
$.hashfn_bswap = MurmurHash3_x86_128_SnapRAID<true>
);

301
smhasher3/museair_snapraid.cpp Normal file
View file

@ -0,0 +1,301 @@
/*
* MuseAir v2
* By K--Aethiax
*
* SnapRAID BFast-128 long-path variant.
*
* For len < 32, the input is zero-padded to 32 bytes and processed
* through the long path. The original length is preserved for
* finalization.
*
* Released into the public domain under the CC0 1.0 license.
*/
#include "Platform.h"
#include "Hashlib.h"
#include "Mathmult.h"
//------------------------------------------------------------
#define U64x(N) ((N) * 8)
/* AiryAi(0) fractional part calculated by Y-Cruncher */
static const uint64_t MUSEAIR_CONSTANT[13] = {
UINT64_C(0x5ae31e589c56e17a),
UINT64_C(0x96d7bb04e64f6da9),
UINT64_C(0x7ab1006b26f9eb64),
UINT64_C(0x21233394220b8457),
UINT64_C(0x047cb9557c9f3b43),
UINT64_C(0xd24f2590c0bcee28),
UINT64_C(0x33ea8f71bb6016d8),
UINT64_C(0xb5d2697595d0a01f),
UINT64_C(0x9bb30a32f00e2b4f),
UINT64_C(0x4acea09317a429d1),
UINT64_C(0xc2b2435dfdd545c6),
UINT64_C(0xfda811a785572a42),
UINT64_C(0xe5f50676bf67137b)
};
static const uint64_t MASK_I = UINT64_C(01555555555555555555555);
static const uint64_t MASK_J = UINT64_C(01333333333333333333333);
static const uint64_t MASK_K = UINT64_C(00666666666666666666666);
//------------------------------------------------------------
template <bool bswap>
static NEVER_INLINE void museair_hash_snapraid(
const uint8_t * bytes,
const size_t len,
const seed_t seed_a,
const seed_t seed_b,
uint64_t * out_lo,
uint64_t * out_hi) {
const uint8_t * p = bytes;
size_t q = len;
uint8_t buf[32];
/*
* SnapRAID extension:
* zero-pad short inputs and use the MuseAir long path.
*
* Keep 'len' unchanged because the original length is mixed
* into finalization.
*/
if (unlikely(q < U64x(4))) {
memset(buf, 0, sizeof(buf));
if (q > 0) {
memcpy(buf, p, q);
}
p = buf;
q = U64x(4);
}
uint64_t i, j, k;
uint64_t lo0, lo1, lo2, lo3, lo4;
uint64_t lo5 = MUSEAIR_CONSTANT[6];
uint64_t hi0, hi1, hi2, hi3, hi4;
uint64_t hi5 = MUSEAIR_CONSTANT[6];
uint64_t state[6] = {
MUSEAIR_CONSTANT[0],
MUSEAIR_CONSTANT[1],
MUSEAIR_CONSTANT[2],
MUSEAIR_CONSTANT[3],
MUSEAIR_CONSTANT[4],
MUSEAIR_CONSTANT[5]
};
/*
* MuseAir v2 BFast-128 seed initialization.
*/
state[0] ^= seed_a & MASK_I;
state[1] ^= seed_b & MASK_J;
state[2] ^= seed_a & MASK_K;
state[3] ^= seed_b & MASK_I;
state[4] ^= seed_a & MASK_J;
state[5] ^= seed_b & MASK_K;
/*
* Main 96-byte loop.
*
* Note the strict > rather than >=. This is intentional and
* matches MuseAir v2 / SnapRAID.
*/
if (unlikely(q > U64x(12))) {
do {
state[0] ^= GET_U64<bswap>(p, U64x(0));
state[1] ^= GET_U64<bswap>(p, U64x(1));
MathMult::mult64_128(lo0, hi0, state[0], state[1]);
state[0] = lo5 ^ hi0;
state[1] ^= GET_U64<bswap>(p, U64x(2));
state[2] ^= GET_U64<bswap>(p, U64x(3));
MathMult::mult64_128(lo1, hi1, state[1], state[2]);
state[1] = lo0 ^ hi1;
state[2] ^= GET_U64<bswap>(p, U64x(4));
state[3] ^= GET_U64<bswap>(p, U64x(5));
MathMult::mult64_128(lo2, hi2, state[2], state[3]);
state[2] = lo1 ^ hi2;
state[3] ^= GET_U64<bswap>(p, U64x(6));
state[4] ^= GET_U64<bswap>(p, U64x(7));
MathMult::mult64_128(lo3, hi3, state[3], state[4]);
state[3] = lo2 ^ hi3;
state[4] ^= GET_U64<bswap>(p, U64x(8));
state[5] ^= GET_U64<bswap>(p, U64x(9));
MathMult::mult64_128(lo4, hi4, state[4], state[5]);
state[4] = lo3 ^ hi4;
state[5] ^= GET_U64<bswap>(p, U64x(10));
state[0] ^= GET_U64<bswap>(p, U64x(11));
MathMult::mult64_128(lo5, hi5, state[5], state[0]);
state[5] = lo4 ^ hi5;
p += U64x(12);
q -= U64x(12);
} while (likely(q > U64x(12)));
state[0] ^= lo5;
}
/*
* Tail.
*/
lo0 = 0;
lo1 = 0;
lo2 = 0;
lo3 = 0;
hi0 = state[1];
hi1 = state[2];
hi2 = state[3];
hi3 = state[4];
if (q > U64x(4)) {
state[0] ^= GET_U64<bswap>(p, U64x(0));
state[1] ^= GET_U64<bswap>(p, U64x(1));
MathMult::mult64_128(lo0, hi0, state[0], state[1]);
if (q > U64x(6)) {
state[1] ^= GET_U64<bswap>(p, U64x(2));
state[2] ^= GET_U64<bswap>(p, U64x(3));
MathMult::mult64_128(lo1, hi1, state[1], state[2]);
if (q > U64x(8)) {
state[2] ^= GET_U64<bswap>(p, U64x(4));
state[3] ^= GET_U64<bswap>(p, U64x(5));
MathMult::mult64_128(lo2, hi2, state[2], state[3]);
if (q > U64x(10)) {
state[3] ^= GET_U64<bswap>(p, U64x(6));
state[4] ^= GET_U64<bswap>(p, U64x(7));
MathMult::mult64_128(lo3, hi3,
state[3], state[4]);
}
}
}
}
/*
* Always mix the final 32 bytes.
*/
state[4] ^= GET_U64<bswap>(p + q - U64x(4), 0);
state[5] ^= GET_U64<bswap>(p + q - U64x(3), 0);
MathMult::mult64_128(lo4, hi4, state[4], state[5]);
state[5] ^= GET_U64<bswap>(p + q - U64x(2), 0);
state[0] ^= GET_U64<bswap>(p + q - U64x(1), 0);
MathMult::mult64_128(lo5, hi5, state[5], state[0]);
/*
* Epilogue.
*/
i = (state[0] - state[1]) ^ MUSEAIR_CONSTANT[7];
j = (state[2] - state[3]) ^ MUSEAIR_CONSTANT[8];
k = (state[4] - state[5]) ^ MUSEAIR_CONSTANT[9];
int rot = len & 63;
i = ROTL64(i, rot);
j = ROTR64(j, rot);
k = k - len;
i = i - (lo3 ^ hi3) - (lo4 ^ hi4);
j = j - (lo5 ^ hi5) - (lo0 ^ hi0);
k = k - (lo1 ^ hi1) - (lo2 ^ hi2);
MathMult::mult64_128(lo0, hi0, i, j);
MathMult::mult64_128(lo1, hi1, j, k);
MathMult::mult64_128(lo2, hi2, k, i);
/*
* BFast state update.
*/
i = lo2 ^ hi0;
j = lo0 ^ hi1;
k = lo1 ^ hi2;
/*
* 128-bit finalization.
*/
MathMult::mult64_128(
lo3, hi3, i, MUSEAIR_CONSTANT[10]);
MathMult::mult64_128(
lo4, hi4, j, MUSEAIR_CONSTANT[11]);
MathMult::mult64_128(
lo5, hi5, k, MUSEAIR_CONSTANT[12]);
*out_lo = lo3 ^ hi4 ^ lo5;
*out_hi = hi3 ^ lo4 ^ hi5;
}
//------------------------------------------------------------
template <bool bswap>
static void MuseAirSnapRAID(
const void * in,
const size_t len,
const seed_t seed,
void * out) {
uint64_t out_lo;
uint64_t out_hi;
/*
* SMHasher3 only supplies one 64-bit seed.
*
* MuseAir v2's SMHasher3 adapter uses the same seed for both
* halves of the B128 seed.
*/
museair_hash_snapraid<bswap>(
(const uint8_t *)in,
len,
seed,
seed,
&out_lo,
&out_hi);
out_lo = COND_BSWAP(out_lo, isBE());
PUT_U64<false>(out_lo, (uint8_t *)out, 0);
out_hi = COND_BSWAP(out_hi, isBE());
PUT_U64<false>(out_hi, (uint8_t *)out, 8);
}
//------------------------------------------------------------
REGISTER_FAMILY(museair_snapraid,
$.src_url = "https://github.com/amadvance/snapraid/",
$.src_status = HashFamilyInfo::SRC_ACTIVE
);
REGISTER_HASH(MuseAir_128__bfast_snapraid,
$.desc =
"MuseAir v2, BFast-128, SnapRAID long-path short-input extension",
$.hash_flags =
FLAG_HASH_ENDIAN_INDEPENDENT,
$.impl_flags =
FLAG_IMPL_MULTIPLY_64_128 |
FLAG_IMPL_ROTATE_VARIABLE |
FLAG_IMPL_CANONICAL_LE |
FLAG_IMPL_LICENSE_PUBLIC_DOMAIN,
$.bits = 128,
$.verification_LE = 0x427E5CD7,
$.verification_BE = 0xED88557C,
$.hashfn_native = MuseAirSnapRAID<false>,
$.hashfn_bswap = MuseAirSnapRAID<true>
);

211
smhasher3/spooky2_snapraid.cpp Normal file
View file

@ -0,0 +1,211 @@
/*
* SpookyHash V2
* By Bob Jenkins, public domain.
*
* SnapRAID long-path variant.
*
* SnapRAID deliberately does not use SpookyHash's Short() optimization for
* messages shorter than 192 bytes. All input lengths are processed through
* the 12-word (96-byte) long path, matching cmdline/spooky2.c.
*
* SnapRAID uses a 128-bit seed. SMHasher3 supplies one 64-bit seed, so this
* adapter uses the same value for both 64-bit seed halves. This is the same
* convention used by SMHasher3's regular SpookyHash adapter.
*/
#include "Platform.h"
#include "Hashlib.h"
//------------------------------------------------------------------------------
static const size_t SPOOKY_NUM_VARS = 12;
static const size_t SPOOKY_BLOCK_SIZE = SPOOKY_NUM_VARS * 8;
static const uint64_t SPOOKY_CONST = UINT64_C(0xdeadbeefdeadbeef);
template <bool bswap>
static FORCE_INLINE void SpookyMix(
const uint8_t * data,
uint64_t & s0, uint64_t & s1, uint64_t & s2, uint64_t & s3,
uint64_t & s4, uint64_t & s5, uint64_t & s6, uint64_t & s7,
uint64_t & s8, uint64_t & s9, uint64_t & s10, uint64_t & s11) {
s0 += GET_U64<bswap>(data, 8 * 0); s2 ^= s10; s11 ^= s0; s0 = ROTL64(s0, 11); s11 += s1;
s1 += GET_U64<bswap>(data, 8 * 1); s3 ^= s11; s0 ^= s1; s1 = ROTL64(s1, 32); s0 += s2;
s2 += GET_U64<bswap>(data, 8 * 2); s4 ^= s0; s1 ^= s2; s2 = ROTL64(s2, 43); s1 += s3;
s3 += GET_U64<bswap>(data, 8 * 3); s5 ^= s1; s2 ^= s3; s3 = ROTL64(s3, 31); s2 += s4;
s4 += GET_U64<bswap>(data, 8 * 4); s6 ^= s2; s3 ^= s4; s4 = ROTL64(s4, 17); s3 += s5;
s5 += GET_U64<bswap>(data, 8 * 5); s7 ^= s3; s4 ^= s5; s5 = ROTL64(s5, 28); s4 += s6;
s6 += GET_U64<bswap>(data, 8 * 6); s8 ^= s4; s5 ^= s6; s6 = ROTL64(s6, 39); s5 += s7;
s7 += GET_U64<bswap>(data, 8 * 7); s9 ^= s5; s6 ^= s7; s7 = ROTL64(s7, 57); s6 += s8;
s8 += GET_U64<bswap>(data, 8 * 8); s10 ^= s6; s7 ^= s8; s8 = ROTL64(s8, 55); s7 += s9;
s9 += GET_U64<bswap>(data, 8 * 9); s11 ^= s7; s8 ^= s9; s9 = ROTL64(s9, 54); s8 += s10;
s10 += GET_U64<bswap>(data, 8 * 10); s0 ^= s8; s9 ^= s10; s10 = ROTL64(s10, 22); s9 += s11;
s11 += GET_U64<bswap>(data, 8 * 11); s1 ^= s9; s10 ^= s11; s11 = ROTL64(s11, 46); s10 += s0;
}
static FORCE_INLINE void SpookyEndPartial(
uint64_t & h0, uint64_t & h1, uint64_t & h2, uint64_t & h3,
uint64_t & h4, uint64_t & h5, uint64_t & h6, uint64_t & h7,
uint64_t & h8, uint64_t & h9, uint64_t & h10, uint64_t & h11) {
h11 += h1; h2 ^= h11; h1 = ROTL64(h1, 44);
h0 += h2; h3 ^= h0; h2 = ROTL64(h2, 15);
h1 += h3; h4 ^= h1; h3 = ROTL64(h3, 34);
h2 += h4; h5 ^= h2; h4 = ROTL64(h4, 21);
h3 += h5; h6 ^= h3; h5 = ROTL64(h5, 38);
h4 += h6; h7 ^= h4; h6 = ROTL64(h6, 33);
h5 += h7; h8 ^= h5; h7 = ROTL64(h7, 10);
h6 += h8; h9 ^= h6; h8 = ROTL64(h8, 13);
h7 += h9; h10 ^= h7; h9 = ROTL64(h9, 38);
h8 += h10; h11 ^= h8; h10 = ROTL64(h10, 53);
h9 += h11; h0 ^= h9; h11 = ROTL64(h11, 42);
h10 += h0; h1 ^= h10; h0 = ROTL64(h0, 54);
}
template <bool bswap>
static FORCE_INLINE void SpookyEnd(
const uint8_t * data,
uint64_t & h0, uint64_t & h1, uint64_t & h2, uint64_t & h3,
uint64_t & h4, uint64_t & h5, uint64_t & h6, uint64_t & h7,
uint64_t & h8, uint64_t & h9, uint64_t & h10, uint64_t & h11) {
h0 += GET_U64<bswap>(data, 8 * 0);
h1 += GET_U64<bswap>(data, 8 * 1);
h2 += GET_U64<bswap>(data, 8 * 2);
h3 += GET_U64<bswap>(data, 8 * 3);
h4 += GET_U64<bswap>(data, 8 * 4);
h5 += GET_U64<bswap>(data, 8 * 5);
h6 += GET_U64<bswap>(data, 8 * 6);
h7 += GET_U64<bswap>(data, 8 * 7);
h8 += GET_U64<bswap>(data, 8 * 8);
h9 += GET_U64<bswap>(data, 8 * 9);
h10 += GET_U64<bswap>(data, 8 * 10);
h11 += GET_U64<bswap>(data, 8 * 11);
SpookyEndPartial(h0, h1, h2, h3, h4, h5,
h6, h7, h8, h9, h10, h11);
SpookyEndPartial(h0, h1, h2, h3, h4, h5,
h6, h7, h8, h9, h10, h11);
SpookyEndPartial(h0, h1, h2, h3, h4, h5,
h6, h7, h8, h9, h10, h11);
}
//------------------------------------------------------------------------------
// Exact SnapRAID SpookyHash2 long-path algorithm.
//
// Unlike the upstream SpookyHash2 one-shot Hash128(), this function NEVER
// switches to Short() for len < 192. This is intentional and is the defining
// behavior of the SnapRAID variant.
template <bool bswap>
static NEVER_INLINE void SpookyHash2SnapRAIDCore(
const void * data,
const size_t size,
const uint64_t seed_a,
const uint64_t seed_b,
uint64_t * out_lo,
uint64_t * out_hi) {
uint64_t h0, h1, h2, h3, h4, h5;
uint64_t h6, h7, h8, h9, h10, h11;
h9 = seed_a;
h10 = seed_b;
h0 = h3 = h6 = h9;
h1 = h4 = h7 = h10;
h2 = h5 = h8 = h11 = SPOOKY_CONST;
const uint8_t * p = (const uint8_t *)data;
size_t remaining = size;
while (remaining >= SPOOKY_BLOCK_SIZE) {
SpookyMix<bswap>(p,
h0, h1, h2, h3, h4, h5,
h6, h7, h8, h9, h10, h11);
p += SPOOKY_BLOCK_SIZE;
remaining -= SPOOKY_BLOCK_SIZE;
}
/*
* SnapRAID tail:
* - copy 0..95 remainder bytes
* - zero-fill the rest
* - store the remainder length in byte 95
* - run SpookyHash2 End() directly
*/
alignas(16) uint8_t buf[SPOOKY_BLOCK_SIZE];
memset(buf, 0, sizeof(buf));
if (remaining > 0) {
memcpy(buf, p, remaining);
}
buf[SPOOKY_BLOCK_SIZE - 1] = (uint8_t)remaining;
SpookyEnd<bswap>(buf,
h0, h1, h2, h3, h4, h5,
h6, h7, h8, h9, h10, h11);
*out_lo = h0;
*out_hi = h1;
}
//------------------------------------------------------------------------------
/*
* SMHasher3 supplies one 64-bit seed, while SnapRAID supplies two independent
* 64-bit halves. Duplicate the SMHasher3 seed into both halves, matching the
* convention used by SMHasher3's regular SpookyHash adapter.
*/
template <bool bswap>
static void SpookyHash2SnapRAID(
const void * in,
const size_t len,
const seed_t seed,
void * out) {
uint64_t out_lo;
uint64_t out_hi;
SpookyHash2SnapRAIDCore<bswap>(
in,
len,
(uint64_t)seed,
(uint64_t)seed,
&out_lo,
&out_hi);
/* SnapRAID stores the 128-bit digest in canonical little-endian order. */
out_lo = COND_BSWAP(out_lo, isBE());
out_hi = COND_BSWAP(out_hi, isBE());
PUT_U64<false>(out_lo, (uint8_t *)out, 0);
PUT_U64<false>(out_hi, (uint8_t *)out, 8);
}
//------------------------------------------------------------------------------
REGISTER_FAMILY(spooky2_snapraid,
$.src_url = "https://github.com/amadvance/snapraid/",
$.src_status = HashFamilyInfo::SRC_ACTIVE
);
REGISTER_HASH(SpookyHash2_128_snapraid,
$.desc =
"SpookyHash v2 128-bit, SnapRAID long-path-only variant",
$.hash_flags =
FLAG_HASH_ENDIAN_INDEPENDENT,
$.impl_flags =
FLAG_IMPL_ROTATE |
FLAG_IMPL_CANONICAL_LE |
FLAG_IMPL_LICENSE_GPL3,
$.bits = 128,
$.verification_LE = 0x965BDDF8,
$.verification_BE = 0x2EF470E2,
$.hashfn_native = SpookyHash2SnapRAID<false>,
$.hashfn_bswap = SpookyHash2SnapRAID<true>
);