Bug 293490 - New port: math/USearch Search Cluster Library
Summary: New port: math/USearch Search Cluster Library
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Gleb Popov
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2026-02-27 15:52 UTC by Markus Graf
Modified: 2026-03-03 17:38 UTC (History)
2 users (show)

See Also:


Attachments
New port USearch (4.49 KB, patch)
2026-02-27 15:52 UTC, Markus Graf
no flags Details | Diff
Patch SIMSIMD detect FreeBSD for runtime dynamic dispatch, x86-64 not v3 (7.29 KB, patch)
2026-02-27 21:01 UTC, Markus Graf
no flags Details | Diff
Revised patch (7.36 KB, patch)
2026-02-28 10:47 UTC, Gleb Popov
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Markus Graf 2026-02-27 15:52:55 UTC
Created attachment 268402 [details]
New port USearch

Thank you Gleb Popov for your time on the mailing list and your offer to help.

I will submit new port database/dtlvnative the native backend to datalevin soon which depends on this one.

Tested on FreeBSD 14.3 amd64 pourdriere.

I am new to submitting ports so I may need guidance on what is expected of me.  This is not just a port but me learning the system.  I mostly live in clojure land and browser, far away from C, C++, Java or Makefiles ;-)

I only use the C library myself via datalevin and dtlnative which also pass all tests.
I added the JNI because upstream made it possible, it passes its own tests.
Comment 1 Michael Osipov freebsd_committer freebsd_triage 2026-02-27 17:51:00 UTC
Question regarding:
.if ${ARCH} == amd64 # for simsimd
CMAKE_ARGS+=	-DCMAKE_CXX_FLAGS="${CXXFLAGS} -march=x86-64-v3"
.endif

There are also other ports which could use SIMD, but it is disabled (e.g. fluent-bit) because we cannot guarantee at build time that the target host will support it. WDYT?
Comment 2 Markus Graf 2026-02-27 18:45:06 UTC
I understood the concrete library in question, https://github.com/ashvardanian/SimSIMD as claiming dynamic dispatch https://github.com/ashvardanian/SimSIMD/blob/main/c/lib.c .
Am I wrong?

The -march=x86-64-v3" is an extra, conservative guard.

Dtlvnative, the parent port in my use case.  Uses this too in the precompiled, native .so, files they put into their java jars for linux, mac and windows, but not for FreeBSD, hence this port.  It seems not to have blown up there.

What do you suggest?  Maybe a switch?
When I researched it it seemed to me that this uses no features newer than Haswell, one Edge Case was caught with -march=x86-64-v3 to assume nothing newer than Haswell, or are you worried about arm, about which I know nothing?
Comment 3 Michael Osipov freebsd_committer freebsd_triage 2026-02-27 18:49:56 UTC
(In reply to Markus Graf from comment #2)

I cannot exactly tell what the right approach is, but wanted to point out that this could be a problem. Someone with more experience should give a reasonable advice.

It might be worth looking at:
osipovmi@deblndw011x:~/var/Projekte/freebsd/ports (main =)
$ grep -r --include=Makefile SIMD .
...
Comment 4 Markus Graf 2026-02-27 18:56:59 UTC
At least the homepage claims "binary with kernels that dynamically dispatch to the best available instructions for the CPU"

Marked Link here:
https://www.unum.cloud/usearch#:~:text=Each%20binary%20comes%20pre%2Dpackaged%20with%20hundreds%20of%20SIMD%2Doptimized%20kernels%20that%20dynamically%20dispatch%20to%20the%20best%20available%20instructions%20for%20your%20CPU%2E
Comment 5 Gleb Popov freebsd_committer freebsd_triage 2026-02-27 19:09:35 UTC
By default the port should target the least featured instruction set. So if the project automatically enables fancy SIMD stuff then we should pass -march=x86-64 here.
Comment 6 Markus Graf 2026-02-27 20:00:43 UTC
The code definitely dispatches dynamically at runtime eg. here https://github.com/ashvardanian/SimSIMD/blob/b311d17ab9e13ebddb3a4ebda9d74a78109a276d/include/simsimd/simsimd.h#L698

The selection what capabilities to choose from will however not recognize FreeBSD as it does hard code checks here https://github.com/ashvardanian/SimSIMD/blob/b311d17ab9e13ebddb3a4ebda9d74a78109a276d/c/lib.c#L23 

So only non chip dependent features remain.

What do we do to change -march=x86-64-v3 to -march=x86-64?  Do I upload a new patch?  What is the procedure here?
Comment 7 Markus Graf 2026-02-27 21:01:43 UTC
Created attachment 268406 [details]
Patch SIMSIMD detect FreeBSD for runtime dynamic dispatch, x86-64 not v3

I uploaded a new patch.

1. Patch SIMSIMD to add FreeBSD CPU detection for runtime dynamic dispatch.

2. -march=x86-64-v3 replaced by -march=x86-64
Comment 8 Gleb Popov freebsd_committer freebsd_triage 2026-02-28 10:47:16 UTC
Created attachment 268414 [details]
Revised patch

I took the patch and made changes that are mostly cosmetic. Markus, please let me know if you're OK with them.

I found a problem with the test suite on FreeBSD 13.5-RELEASE i386 - it crashes with a following backtrace:

* thread #1, name = 'test_cpp', stop reason = signal SIGABRT
    frame #0: 0x20927d5f libc.so.7`__sys_thr_kill + 7
    frame #1: 0x208a635b libc.so.7`__raise + 59
    frame #2: 0x2095119b libc.so.7`abort + 75
    frame #3: 0x207a0565 libcxxrt.so.1`std::terminate() + 53
    frame #4: 0x207a15fe libcxxrt.so.1`___lldb_unnamed_symbol477 + 78
    frame #5: 0x207a11f1 libcxxrt.so.1`___lldb_unnamed_symbol475 + 129
    frame #6: 0x0043f653 test_cpp`usearch_raise_runtime_error(message="C++ unit test failed at /wrkdirs/usr/ports/math/usearch/work/USearch-2.24.0/cpp/test.cpp:149") at index.hpp:169:5
    frame #7: 0x0043f5c5 test_cpp`__expect(must_be_true=false, file="/wrkdirs/usr/ports/math/usearch/work/USearch-2.24.0/cpp/test.cpp", line=149, message="C++ unit test failed") at test.cpp:39:5
    frame #8: 0x004c9a86 test_cpp`void __expect_eq<bool>(a=true, b=false, file="/wrkdirs/usr/ports/math/usearch/work/USearch-2.24.0/cpp/test.cpp", line=149, message=0x00000000) at test.cpp:44:5
  * frame #9: 0x0044696d test_cpp`test_uint40() at test.cpp:149:13
    frame #10: 0x0044bd2e test_cpp`main((null)=1, (null)=0xffffdbec) at test.cpp:1183:5
    frame #11: 0x0043f35e test_cpp`_start1(cleanup=0x20564cf0, argc=1, argv=0xffffdbec) at crt1_c.c:66:7
    frame #12: 0x0043f494 test_cpp`_start at crt1_s.S:42

Markus, do you have an idea what's wrong here? If the software is not even intended to be run on i386, then we can add NOT_FOR_ARCHS=i386 to the Makefile.
Comment 9 Markus Graf 2026-02-28 11:32:59 UTC
(In reply to Gleb Popov from comment #8)

Thank you for your changes!  I clearly missed some variables!

Let us go with NOT_FOR_ARCHS=i386 as I think the likelihood of anyone using this on i386 is to low to warrant the effort.
Comment 10 commit-hook freebsd_committer freebsd_triage 2026-03-01 16:24:13 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=0289e12ec3a7b736793e435454bbeeda59faedb6

commit 0289e12ec3a7b736793e435454bbeeda59faedb6
Author:     Markus Graf <markus.graf@markusgraf.net>
AuthorDate: 2026-02-27 20:48:41 +0000
Commit:     Gleb Popov <arrowd@FreeBSD.org>
CommitDate: 2026-03-01 16:24:01 +0000

    math/usearch: Search & Clustering engine library (C, C++ and Java JNI)

    PR:             293490
    Co-authored-by: Gleb Popov <arrowd@FreeBSD.org>

 math/Makefile                                  |  1 +
 math/usearch/Makefile (new)                    | 69 ++++++++++++++++++++++++++
 math/usearch/distinfo (new)                    |  9 ++++
 math/usearch/files/patch-simsimd_c_lib.c (new) | 47 ++++++++++++++++++
 math/usearch/pkg-descr (new)                   |  2 +
 math/usearch/pkg-plist (new)                   |  8 +++
 6 files changed, 136 insertions(+)
Comment 11 Gleb Popov freebsd_committer freebsd_triage 2026-03-01 16:27:04 UTC
Merged, thank you, Markus. I'd like to point out that your submission was of exceptional quality. For a Ports newcomer you did an astonishing job.
Comment 12 Markus Graf 2026-03-01 16:41:54 UTC
(In reply to Gleb Popov from comment #11)
Thank you for your guidance!  In the meantime I also realized that https://reviews.freebsd.org/differential/ is what you had intended me to use.  I will send the next ports there.
Comment 13 Gleb Popov freebsd_committer freebsd_triage 2026-03-01 16:42:39 UTC
You can also use GitHub to send pull requests to https://github.com/freebsd/freebsd-ports
Comment 14 Markus Graf 2026-03-03 16:59:05 UTC
The directory name changed from USearch to lower case usearch between the patches.  Sorry I missed that.
Comment 15 Gleb Popov freebsd_committer freebsd_triage 2026-03-03 17:00:14 UTC
(In reply to Markus Graf from comment #14)
Is this a problem? We tend to name port directories in lowercase module few exceptions.
Comment 16 Markus Graf 2026-03-03 17:38:05 UTC
(In reply to Gleb Popov from comment #15)
OK. I assumed a port name and directory should be the same.  Thank you!