stringzilla
Fast string search, hashing and sorting via
StringZilla's SIMD kernels
(Apache-2.0), as a native dependency (ADR 0005). The library picks its
kernels at runtime from CPUID — one build runs everywhere, and
capabilities() says what it picked.
import stringzilla
stringzilla.find("hello world", "world") // Some(6)
stringzilla.find_last("banana", "an") // Some(3)
stringzilla.count("banana", "an") // 2
stringzilla.find_byte("banana", 97) // Some(1) — the byte 'a'
stringzilla.compare("abc", "abd") // Lt
stringzilla.hash("hello", 0) // 2641555913580541882
stringzilla.sort(["b", "a", "c"]) // Ok(["a", "b", "c"])
Everything performs <ffi>. Offsets and counts are in bytes (UTF-8),
not characters. Empty needles follow std/string's Rust semantics:
find matches at 0, find_last at the end, count at every gap.
API
| Function | Result |
|---|---|
find(haystack, needle) | Option<Int> — first byte offset |
find_last(haystack, needle) | Option<Int> — last byte offset |
count(haystack, needle) | Int — non-overlapping occurrences |
find_byte(s, byte), find_byte_last(s, byte) | Option<Int>; byte is 0–255 |
compare(a, b) | Order, byte-wise — the order Ord<String> uses |
hash(s, seed) | Int — low 62 bits of a 64-bit hash; a fingerprint, not crypto |
bytesum(s) | Int — low 62 bits of a byte-sum sketch |
sort(xs) | Result<List<String>, String> — Error on C OOM |
version(), capabilities() | String |
Native-only
The interpreter cannot call C, so every function here needs a native
build: tests run with louise test --native (the argon2 package's
pattern; ci.sh checks this package and tests it the same way). A fresh
build of the vendored library takes about two minutes (every SIMD tier at
-O2); the build directory caches it afterwards.
The pin
StringZilla v4.6.3, with one backported fix — see
vendor/stringzilla/PATCHES.md for the
patch (v4.6.3 ordered bytes as signed char, so non-ASCII sorted wrong;
the vendored copy carries v5's fix) and for why v4 rather than the v5
series (v5's build needs try-compile probes that [native.*] cannot
run). Bound through csrc/api.h, the bindgen-readable subset of the C
API; src/bindings.lse and csrc/bindings.c are generated:
louise bindgen csrc/api.h --lse src/bindings.lse --c csrc/bindings.c
Deferred
- Edit distance, streaming hashes, SHA-256, fingerprinting: since
StringZilla 4.0 these live in StringZillas, a C++17 library (with a
Fork Union dependency); a
kind = "c"native dependency builds C only. - UTF-8 segmenters (graphemes, words, sentences): bindable in principle (callbacks per segment), but no consumer asked yet.
- AVX-512 on the v5 series: see PATCHES.md.
Safety
No C here unwinds: the bound functions are leaf computations over
borrowed memory, allocation failure is a status (sort answers an
Error), and the shim's panics (out-of-range integers) are raised by the
generated wrappers with tl_panic_str, outside any C call. Externs
borrow their String arguments; results are owned. The sort's copies
live in a Resource (std/ffi) whose finalizer frees them; it is not
thread-safe, so the runtime pins it to its actor. The vendored library
builds with SZ_USE_MISALIGNED_LOADS=0 (aligned SWAR loads), and the
package's tests pass under AddressSanitizer and UBSan
(-fsanitize=address,undefined), as ci.sh's sanitizer lane runs the
qsort example.