This is a request of two exp-runs to test the second half of the SIMD libc enhancement patch set for correctness prior to inclusion into 15-CURRENT. The exp-runs aim to validate the contents of D41971, D41980, D42122, D42217, D42346, D42519, D42600, D42863, D42902, D42905, and D42925 by building all ports, once without, and once with the patch set applied. The results can then be compared to see if the patch set caused any new failures. The repository https://github.com/clausecker/freebsd-src.git has been prepared for this purpose. Tag <reference> points to the last commit prior to the llvm-17 import (which causes a bunch of new, unrelated failures). Tag <simd> points to the end of the exp-run branch branched off <reference> with the aforementioned DRs applied. Please run two exp-runs on amd64 based on these two source trees with a current ports tree (the same tree for both) and indicate if there were any changes in build failures between the two runs. This method of acceptance-testing has been proposed by mjg@, who is technical contact for this project.
There seems to be 1 new failure: https://pkg-status.freebsd.org/gohan04/data/mainamd64PR275785-default-foo/2023-12-19_10h40m10s/logs/errors/clearsilver-0.10.5_1.log
Thanks. I think there is another bug in strcspn() which I'll have to take care off.
The bug has been fixed in this commit: https://github.com/clausecker/freebsd-src/commit/80cc2d83340d447d4bb4588abe241b898842bfd5 Please redo the exp-run with tag <simd2> to see if we're now in the clear.
Exp-run seems fine.
Thank you for checking. I'll talk with mjg and will proceed with a commit if he agrees.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=fb197a4f7751bb4e116989e57ba7fb12a981895f commit fb197a4f7751bb4e116989e57ba7fb12a981895f Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-12-06 10:05:47 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 14:00:05 +0000 lib/libc/amd64/string: add memrchr() scalar, baseline implementation The scalar implementation is fairly simplistic and only performs slightly better than the generic C implementation. It could be improved by using the same algorithm as for memchr, but it would have been a lot more complicated. The baseline implementation is similar to timingsafe_memcmp. It's slightly slower than memchr() due to the more complicated main loop, but I don't think that can be significantly improved. Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42925 lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/memrchr.S (new) | 166 ++++++++++++++++++++++++++++++++++ 2 files changed, 167 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=5fa0fbf40b110e4699032b97a9f540db80997812 commit 5fa0fbf40b110e4699032b97a9f540db80997812 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-12-04 17:16:50 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:59:46 +0000 share/man/man7/simd.7: document simd-enhanced memccpy, strncat Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: HTTPS://reviews.freebsd.org/D42902 share/man/man7/simd.7 | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=fb9c25a91fcf4005141eeb81ce49f4399c425d32 commit fb9c25a91fcf4005141eeb81ce49f4399c425d32 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-12-06 13:47:03 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 14:00:08 +0000 share/man/man7/simd.7: document SIMD-enhanced memrchr implementation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 share/man/man7/simd.7 | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=691ff1832e09a6ccbc8f5b04c88cafc7452b3ce6 commit 691ff1832e09a6ccbc8f5b04c88cafc7452b3ce6 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-12-06 09:11:40 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:59:58 +0000 lib/libc/tests/string: add memrchr unit tests The "values" test case is specifically crafted to detect the off-by-one error previous discovered in the scalar strchrnul implementation. Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42925 lib/libc/tests/string/Makefile | 1 + lib/libc/tests/string/memrchr_test.c (new) | 116 +++++++++++++++++++++++++++++ 2 files changed, 117 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=fc0e38a7a67a6d43095efb00cf19ee5f95dcf710 commit fc0e38a7a67a6d43095efb00cf19ee5f95dcf710 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-12-02 12:28:05 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:59:42 +0000 lib/libc/amd64/string: add memccpy scalar, baseline implementation Based on the strlcpy code from D42863, this patch adds a SIMD-enhanced implementation of memccpy for amd64. A scalar implementation calling into memchr and memcpy to do the job is provided, too. Please note that this code does not behave exactly the same as the C implementation of memccpy for overlapping inputs. However, overlapping inputs are not allowed for this function by ISO/IEC 9899:1999 and neither has the C implementation any code to deal with the possibility. It just proceeds byte-by-byte, which may or may not do the expected thing for some overlaps. We do not document whether overlapping inputs are supported in memccpy(3). Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42902 lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/memccpy.S (new) | 259 ++++++++++++++++++++++++++++++++++ 2 files changed, 260 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=f7098b8659923873a7c60b64cb68182e470786f9 commit f7098b8659923873a7c60b64cb68182e470786f9 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-09 18:08:23 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:56:02 +0000 lib/libc/tests/string: add unit test for strlcpy A straightforward derivation from the stpncpy unit test. Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42863 lib/libc/tests/string/Makefile | 1 + lib/libc/tests/string/strlcpy_test.c (new) | 183 +++++++++++++++++++++++++++++ 2 files changed, 184 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=2b7b03b7ae179db465c1ef19a5007f729874916a commit 2b7b03b7ae179db465c1ef19a5007f729874916a Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-29 02:32:28 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:59:31 +0000 lib/libc/amd64/string: implement strlcat() through strlcpy() This should pick up our optimised memchr(), strlen(), and strlcpy() when strlcat() is called. Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42863 lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/strlcat.c (new) | 25 +++++++++++++++++++++++++ 2 files changed, 26 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=6fdcb85b1c10fb7a7f61323e52ee251cc48fd60e commit 6fdcb85b1c10fb7a7f61323e52ee251cc48fd60e Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-29 03:33:18 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:59:37 +0000 share/man/man7/simd.7: add forgotten aarch64 string functions I previously forgot to mention these as they are set up through contrib/arm-optimized/routines/string. Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 share/man/man7/simd.7 | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=74d6cfad54d676299ee5e4695139461876dfd757 commit 74d6cfad54d676299ee5e4695139461876dfd757 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-12 22:47:06 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:56:05 +0000 lib/libc/amd64/string: add strlcpy scalar, baseline implementation Somewhat similar to stpncpy, but different in that we need to compute the full source length even if the buffer is shorter than the source. strlcat is implemented as a simple wrapper around strlcpy. The scalar implementation of strlcpy just calls into strlen() and memcpy() to do the job. Perf-wise we're very close to stpncpy. The code is slightly slower as it needs to carry on with finding the source string length even if the buffer ends before the string. Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42863 lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/strlcpy.S (new) | 281 ++++++++++++++++++++++++++++++++++ 2 files changed, 282 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=e4b7b0bcbc741f0998640c2ba55ec00ba613bb75 commit e4b7b0bcbc741f0998640c2ba55ec00ba613bb75 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-12-03 11:42:17 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:59:40 +0000 lib/libc/tests/string: add unit tests for memccpy() Adapted from the strlcpy() unit tests. Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 lib/libc/tests/string/Makefile | 1 + lib/libc/tests/string/memccpy_test.c (new) | 205 +++++++++++++++++++++++++++++ 2 files changed, 206 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=ea7b13771cc9d45bf1bc6c6edad8d1b7bce12990 commit ea7b13771cc9d45bf1bc6c6edad8d1b7bce12990 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-12-04 17:32:49 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:59:48 +0000 lib/libc/amd64/string: implement strncat() by calling strlen(), memccpy() This picks up the accelerated implementation of memccpy(). Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42902 lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/strncat.c (new) | 29 +++++++++++++++++++++++++++++ 2 files changed, 30 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=aff9143a242c0012b0195b3666e03fa3b7cd33e8 commit aff9143a242c0012b0195b3666e03fa3b7cd33e8 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-14 18:09:08 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:55:53 +0000 lib/libc/amd64/string/strcat.S: enable use of SIMD strcat has a bespoke scalar assembly implementation we inherited from NetBSD. While it performs well, it is better to call into our SIMD implementations if any SIMD features are available at all. So do that and implement strcat() by calling into strlen() and strcpy() if these are available. Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Reviison: https://reviews.freebsd.org/D42600 lib/libc/amd64/string/strcat.S | 47 +++++++++++++++++++++++++++++++++++++----- 1 file changed, 42 insertions(+), 5 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=c983825a77f276ab71a34baee0250d55a84f59fd commit c983825a77f276ab71a34baee0250d55a84f59fd Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-14 18:26:21 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:55:59 +0000 share/man/man7/simd.7: document SIMD-enhanced strcat Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42600 share/man/man7/simd.7 | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=90253d49db09a9b1490c448d05314f3e4bbfa468 commit 90253d49db09a9b1490c448d05314f3e4bbfa468 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-10-30 03:15:46 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:55:42 +0000 lib/libc/amd64/string: add stpncpy scalar, baseline implementation This was surprisingly annoying to get right, despite being such a simple function. A scalar implementation is also provided, it just calls into our optimised memchr(), memcpy(), and memset() routines to carry out its job. I'm quite happy with the performance. glibc only beats us for very long strings, likely due to the use of AVX-512. The scalar implementation just calls into our optimised memchr(), memcpy(), and memset() routines, so it has a high overhead to begin with but then performs ok for the amount of effort that went into it. Still beats the old C code, except for very short strings. Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42519 lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/stpncpy.S (new) | 283 ++++++++++++++++++++++++++++++++++ 2 files changed, 284 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=75a9e2250656ff78ec1b0a124f282b925138ff51 commit 75a9e2250656ff78ec1b0a124f282b925138ff51 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-09 04:39:11 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:55:50 +0000 share/man/man7/simd.7: document simd-enhanced strncpy, stpncpy Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42519 share/man/man7/simd.7 | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=8b60e1fdbefab1ffbc1b4222ae0a67e1d455fb45 commit 8b60e1fdbefab1ffbc1b4222ae0a67e1d455fb45 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-09-25 21:45:26 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:55:04 +0000 share/man/man7/simd.7: document amd64 SIMD use for strpbrk() Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D41980 share/man/man7/simd.7 | 1 + 1 file changed, 1 insertion(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=f4fc317c364f2c81ad3d36763d8e5a60393ddbd1 commit f4fc317c364f2c81ad3d36763d8e5a60393ddbd1 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-09-25 21:43:12 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:54:58 +0000 lib/libc/amd64/string: implement strpbrk() through strcspn() This lets us use our optimised strcspn() routine for strpbrk() calls. Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D41980 lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/strcspn.S | 18 ++++++++------- lib/libc/amd64/string/strpbrk.c (new) | 43 +++++++++++++++++++++++++++++++++++ 3 files changed, 54 insertions(+), 8 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=bca25680b91b3bea7faef615765806a04634eb23 commit bca25680b91b3bea7faef615765806a04634eb23 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-09-16 05:29:39 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:54:33 +0000 lib/libc/amd64/string/strcmp.S: add baseline implementation This is the most complicated one so far. The basic idea is to process the bulk of the string in aligned blocks of 16 bytes such that one string runs ahead and the other runs behind. The string that runs ahead is checked for NUL bytes, the one that runs behind is compared with the corresponding chunk of the string that runs ahead. This trades an extra load per iteration for the very complicated block-reassembly needed in the other implementations (bionic, glibc). On the flip side, we need two code paths depending on the relative alignment of the two buffers. The initial part of the string is compared directly if it is known not to cross a page boundary. Otherwise, a complex slow path to avoid crossing into unmapped memory commences. Performance-wise we beat bionic for misaligned strings (i.e. the strings do not share an alignment offset) and reach comparable performance for aligned strings. glibc is a bit better as it has a special kernel for AVX-512, where this stuff is a bit easier to do. Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D41971 lib/libc/amd64/string/strcmp.S | 299 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 292 insertions(+), 7 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=75cb202698a65cb809eb2fea194a73e79c509d9c commit 75cb202698a65cb809eb2fea194a73e79c509d9c Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-10-24 03:57:56 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:55:34 +0000 share/man/man7/simd.7: document amd64 SIMD use for strsep() Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42346 share/man/man7/simd.7 | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=f5edd8450e5328c5b88977b4d00e780cdd744dea commit f5edd8450e5328c5b88977b4d00e780cdd744dea Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-10-08 02:46:36 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:55:18 +0000 share/man/man7/simd.7: document strncmp amd64 scalar, baseline implementations Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42122 share/man/man7/simd.7 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=459ddefcc9dcc010f6f7445285e61e2b6780379c commit 459ddefcc9dcc010f6f7445285e61e2b6780379c Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-09-28 20:40:20 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:55:08 +0000 lib/libc/tests/string: add unit tests for strncmp(3) These are patterned after the previously added (D41970) strcmp tests, but are extended to check for various length conditions. Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42122 lib/libc/tests/string/Makefile | 1 + lib/libc/tests/string/strncmp_test.c (new) | 165 +++++++++++++++++++++++++++++ 2 files changed, 166 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=e19d46c808267f53455e96a28ff7654211523d2c commit e19d46c808267f53455e96a28ff7654211523d2c Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-09 04:25:55 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:55:48 +0000 lib/libc/amd64/string: implement strncpy() by calling stpncpy() Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42519 lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/strncpy.c (new) | 41 +++++++++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=47adb1e012643b8f32f6cf8c84cd50eb58f0f17a commit 47adb1e012643b8f32f6cf8c84cd50eb58f0f17a Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-09-25 06:18:06 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:54:50 +0000 share/man/man7/simd.7: document new amd64 baseline strcmp() Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D41971 share/man/man7/simd.7 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=6fa9e7d8737548ef93c573387ce62402c368d486 commit 6fa9e7d8737548ef93c573387ce62402c368d486 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-05 04:02:00 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:55:37 +0000 lib/libc/tests/string/stpncpy_test.c: extend for upcoming SSE implementation This adds additional unit tests validating the function for All possible alignment offsets of source and destination. Also extend the test to allow testing of an external stpncpy implementation, which greatly simplifies the development of custom implementations. Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42519 lib/libc/tests/string/stpncpy_test.c | 99 +++++++++++++++++++++++++++++++----- 1 file changed, 85 insertions(+), 14 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=fd2ecd91aeeeab579c769c9a39f90b4bd4a493a9 commit fd2ecd91aeeeab579c769c9a39f90b4bd4a493a9 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-10-24 03:52:01 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:55:30 +0000 lib/libc/amd64/string: implement strsep() through strcspn() The strsep() function is basically strcspn() with extra steps. On amd64, we now have an optimised implementation of strcspn(), so instead of implementing the inner loop manually, just call into the optimised routine. Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42346 lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/strsep.c (new) | 57 ++++++++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=dd1c2e887c1fa087bf15cbf7f0bdc35b7875bdd6 commit dd1c2e887c1fa087bf15cbf7f0bdc35b7875bdd6 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-10-12 22:31:55 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:55:26 +0000 share/man/man7/simd.7: document strrchr scalar, baseline implementation Also mention missing rindex() entry, which is provided through strrchr(). Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42217 share/man/man7/simd.7 | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=2ed514a220edbac6ca5ec9f40a3e0b3f2804796d commit 2ed514a220edbac6ca5ec9f40a3e0b3f2804796d Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-10-12 05:37:41 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:55:22 +0000 lib/libc/amd64/string: add strrchr scalar, baseline implementation The baseline implementation is very straightforward, while the scalar implementation suffers from register pressure and the need to use SWAR techniques similar to those used for strchr(). Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42217 lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/strrchr.S (new) | 209 ++++++++++++++++++++++++++++++++++ 2 files changed, 210 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=14289e973f5c941e4502cc2b11265e4b3072839a commit 14289e973f5c941e4502cc2b11265e4b3072839a Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-09-27 22:46:04 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:55:13 +0000 lib/libc/amd64/string: add strncmp scalar, baseline implementation The scalar implementation is fairly straightforward and merely unrolled four times. The baseline implementation closely follows D41971 with appropriate extensions and extra code paths to pay attention to string length. Performance is quite good. We beat both glibc (except for very long strings, but they likely use AVX which we don't) and Bionic (except for medium-sized aligned strings, where we are still in the same ballpark). Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42122 lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/strncmp.S (new) | 488 ++++++++++++++++++++++++++++++++++ 2 files changed, 489 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=d4a5605d9fedcff34227707133a8d6694ccbfc36 commit d4a5605d9fedcff34227707133a8d6694ccbfc36 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-29 02:35:45 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2023-12-25 13:59:35 +0000 share/man/man7/simd.7: document scalar/baseline strlcpy, strlcat Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42863 share/man/man7/simd.7 | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=ddab9e646122df077570fc0dfb2af1516c098668 commit ddab9e646122df077570fc0dfb2af1516c098668 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-12-04 17:32:49 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:30 +0000 lib/libc/amd64/string: implement strncat() by calling strlen(), memccpy() This picks up the accelerated implementation of memccpy(). Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42902 (cherry picked from commit ea7b13771cc9d45bf1bc6c6edad8d1b7bce12990) lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/strncat.c (new) | 29 +++++++++++++++++++++++++++++ 2 files changed, 30 insertions(+)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=3f2c2dda263114c46d14c23b14f989118ab1a730 commit 3f2c2dda263114c46d14c23b14f989118ab1a730 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-12-06 13:47:03 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:31 +0000 share/man/man7/simd.7: document SIMD-enhanced memrchr implementation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 (cherry picked from commit fb9c25a91fcf4005141eeb81ce49f4399c425d32) share/man/man7/simd.7 | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=bd051ed3fed700693cf64961324b3e50102c897e commit bd051ed3fed700693cf64961324b3e50102c897e Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-12-04 17:16:50 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:30 +0000 share/man/man7/simd.7: document simd-enhanced memccpy, strncat Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: HTTPS://reviews.freebsd.org/D42902 (cherry picked from commit 5fa0fbf40b110e4699032b97a9f540db80997812) share/man/man7/simd.7 | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=acb47064d65813386dc15f6278037f1a76ec7eab commit acb47064d65813386dc15f6278037f1a76ec7eab Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-12-06 09:11:40 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:30 +0000 lib/libc/tests/string: add memrchr unit tests The "values" test case is specifically crafted to detect the off-by-one error previous discovered in the scalar strchrnul implementation. Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42925 (cherry picked from commit 691ff1832e09a6ccbc8f5b04c88cafc7452b3ce6) lib/libc/tests/string/Makefile | 1 + lib/libc/tests/string/memrchr_test.c (new) | 116 +++++++++++++++++++++++++++++ 2 files changed, 117 insertions(+)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=de2d155d2cac49025958d927aaaef66d99bcb537 commit de2d155d2cac49025958d927aaaef66d99bcb537 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-12-03 11:42:17 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:29 +0000 lib/libc/tests/string: add unit tests for memccpy() Adapted from the strlcpy() unit tests. Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 (cherry picked from commit e4b7b0bcbc741f0998640c2ba55ec00ba613bb75) lib/libc/tests/string/Makefile | 1 + lib/libc/tests/string/memccpy_test.c (new) | 205 +++++++++++++++++++++++++++++ 2 files changed, 206 insertions(+)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=7a605ba8f7996f38ba7b353a0120d84bae48da0f commit 7a605ba8f7996f38ba7b353a0120d84bae48da0f Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-14 18:09:08 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:28 +0000 lib/libc/amd64/string/strcat.S: enable use of SIMD strcat has a bespoke scalar assembly implementation we inherited from NetBSD. While it performs well, it is better to call into our SIMD implementations if any SIMD features are available at all. So do that and implement strcat() by calling into strlen() and strcpy() if these are available. Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Reviison: https://reviews.freebsd.org/D42600 (cherry picked from commit aff9143a242c0012b0195b3666e03fa3b7cd33e8) lib/libc/amd64/string/strcat.S | 47 +++++++++++++++++++++++++++++++++++++----- 1 file changed, 42 insertions(+), 5 deletions(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=a3ce82e5b8878dd8422b6121a529d489f42d37a2 commit a3ce82e5b8878dd8422b6121a529d489f42d37a2 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-12-02 12:28:05 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:30 +0000 lib/libc/amd64/string: add memccpy scalar, baseline implementation Based on the strlcpy code from D42863, this patch adds a SIMD-enhanced implementation of memccpy for amd64. A scalar implementation calling into memchr and memcpy to do the job is provided, too. Please note that this code does not behave exactly the same as the C implementation of memccpy for overlapping inputs. However, overlapping inputs are not allowed for this function by ISO/IEC 9899:1999 and neither has the C implementation any code to deal with the possibility. It just proceeds byte-by-byte, which may or may not do the expected thing for some overlaps. We do not document whether overlapping inputs are supported in memccpy(3). Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42902 (cherry picked from commit fc0e38a7a67a6d43095efb00cf19ee5f95dcf710) lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/memccpy.S (new) | 259 ++++++++++++++++++++++++++++++++++ 2 files changed, 260 insertions(+)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=0307a0095ed82a0127c4cc3191a980643ad5b0bc commit 0307a0095ed82a0127c4cc3191a980643ad5b0bc Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-29 03:33:18 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:29 +0000 share/man/man7/simd.7: add forgotten aarch64 string functions I previously forgot to mention these as they are set up through contrib/arm-optimized/routines/string. Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 (cherry picked from commit 6fdcb85b1c10fb7a7f61323e52ee251cc48fd60e) share/man/man7/simd.7 | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=5a37572b5ba6001534514992b2d9ac22ab347678 commit 5a37572b5ba6001534514992b2d9ac22ab347678 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-29 02:35:45 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:29 +0000 share/man/man7/simd.7: document scalar/baseline strlcpy, strlcat Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42863 (cherry picked from commit d4a5605d9fedcff34227707133a8d6694ccbfc36) share/man/man7/simd.7 | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=903cb811ffe2f3e68beccd8fef5208cf0e224770 commit 903cb811ffe2f3e68beccd8fef5208cf0e224770 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-12 22:47:06 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:28 +0000 lib/libc/amd64/string: add strlcpy scalar, baseline implementation Somewhat similar to stpncpy, but different in that we need to compute the full source length even if the buffer is shorter than the source. strlcat is implemented as a simple wrapper around strlcpy. The scalar implementation of strlcpy just calls into strlen() and memcpy() to do the job. Perf-wise we're very close to stpncpy. The code is slightly slower as it needs to carry on with finding the source string length even if the buffer ends before the string. Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42863 (cherry picked from commit 74d6cfad54d676299ee5e4695139461876dfd757) lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/strlcpy.S (new) | 281 ++++++++++++++++++++++++++++++++++ 2 files changed, 282 insertions(+)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=1f733be3cca4cac04ade21beb6aab04d66633d1e commit 1f733be3cca4cac04ade21beb6aab04d66633d1e Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-14 18:26:21 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:28 +0000 share/man/man7/simd.7: document SIMD-enhanced strcat Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42600 (cherry picked from commit c983825a77f276ab71a34baee0250d55a84f59fd) share/man/man7/simd.7 | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=ff7799e003110a8188bc129e663d23b54755531f commit ff7799e003110a8188bc129e663d23b54755531f Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-12-06 10:05:47 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:31 +0000 lib/libc/amd64/string: add memrchr() scalar, baseline implementation The scalar implementation is fairly simplistic and only performs slightly better than the generic C implementation. It could be improved by using the same algorithm as for memchr, but it would have been a lot more complicated. The baseline implementation is similar to timingsafe_memcmp. It's slightly slower than memchr() due to the more complicated main loop, but I don't think that can be significantly improved. Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42925 (cherry picked from commit fb197a4f7751bb4e116989e57ba7fb12a981895f) lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/memrchr.S (new) | 166 ++++++++++++++++++++++++++++++++++ 2 files changed, 167 insertions(+)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=3045c0f198a1113a02f44f77b161fcf79380ae63 commit 3045c0f198a1113a02f44f77b161fcf79380ae63 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-29 02:32:28 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:29 +0000 lib/libc/amd64/string: implement strlcat() through strlcpy() This should pick up our optimised memchr(), strlen(), and strlcpy() when strlcat() is called. Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42863 (cherry picked from commit 2b7b03b7ae179db465c1ef19a5007f729874916a) lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/strlcat.c (new) | 25 +++++++++++++++++++++++++ 2 files changed, 26 insertions(+)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=7527fecbfe0c7b8c9f8515ca92f9c936e687ec9d commit 7527fecbfe0c7b8c9f8515ca92f9c936e687ec9d Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-10-30 03:15:46 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:27 +0000 lib/libc/amd64/string: add stpncpy scalar, baseline implementation This was surprisingly annoying to get right, despite being such a simple function. A scalar implementation is also provided, it just calls into our optimised memchr(), memcpy(), and memset() routines to carry out its job. I'm quite happy with the performance. glibc only beats us for very long strings, likely due to the use of AVX-512. The scalar implementation just calls into our optimised memchr(), memcpy(), and memset() routines, so it has a high overhead to begin with but then performs ok for the amount of effort that went into it. Still beats the old C code, except for very short strings. Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42519 (cherry picked from commit 90253d49db09a9b1490c448d05314f3e4bbfa468) lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/stpncpy.S (new) | 283 ++++++++++++++++++++++++++++++++++ 2 files changed, 284 insertions(+)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=f42eca44bdfba36f4a3dc004593117e079af4d92 commit f42eca44bdfba36f4a3dc004593117e079af4d92 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-09-25 21:45:26 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:25 +0000 share/man/man7/simd.7: document amd64 SIMD use for strpbrk() Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D41980 (cherry picked from commit 8b60e1fdbefab1ffbc1b4222ae0a67e1d455fb45) share/man/man7/simd.7 | 1 + 1 file changed, 1 insertion(+)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=3a19fcb9fdbfd8bbc5955b14aba9bf481de2ad1a commit 3a19fcb9fdbfd8bbc5955b14aba9bf481de2ad1a Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-09-27 22:46:04 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:25 +0000 lib/libc/amd64/string: add strncmp scalar, baseline implementation The scalar implementation is fairly straightforward and merely unrolled four times. The baseline implementation closely follows D41971 with appropriate extensions and extra code paths to pay attention to string length. Performance is quite good. We beat both glibc (except for very long strings, but they likely use AVX which we don't) and Bionic (except for medium-sized aligned strings, where we are still in the same ballpark). Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42122 (cherry picked from commit 14289e973f5c941e4502cc2b11265e4b3072839a) lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/strncmp.S (new) | 488 ++++++++++++++++++++++++++++++++++ 2 files changed, 489 insertions(+)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=309b30ce8419058b23b285d1c73549890fc174f5 commit 309b30ce8419058b23b285d1c73549890fc174f5 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-09-25 21:43:12 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:24 +0000 lib/libc/amd64/string: implement strpbrk() through strcspn() This lets us use our optimised strcspn() routine for strpbrk() calls. Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D41980 (cherry picked from commit f4fc317c364f2c81ad3d36763d8e5a60393ddbd1) lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/strcspn.S | 18 ++++++++------- lib/libc/amd64/string/strpbrk.c (new) | 43 +++++++++++++++++++++++++++++++++++ 3 files changed, 54 insertions(+), 8 deletions(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=fa6d214aa16ed3f86b4f597fbdca9d606579e3b2 commit fa6d214aa16ed3f86b4f597fbdca9d606579e3b2 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-10-08 02:46:36 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:25 +0000 share/man/man7/simd.7: document strncmp amd64 scalar, baseline implementations Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42122 (cherry picked from commit f5edd8450e5328c5b88977b4d00e780cdd744dea) share/man/man7/simd.7 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=8dc0354823104385e8a0cb217264e07680e8fdfa commit 8dc0354823104385e8a0cb217264e07680e8fdfa Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-10-24 03:57:56 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:26 +0000 share/man/man7/simd.7: document amd64 SIMD use for strsep() Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42346 (cherry picked from commit 75cb202698a65cb809eb2fea194a73e79c509d9c) share/man/man7/simd.7 | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=438a1ff803a5d033b80b16880b2718f1402d904f commit 438a1ff803a5d033b80b16880b2718f1402d904f Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-05 04:02:00 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:27 +0000 lib/libc/tests/string/stpncpy_test.c: extend for upcoming SSE implementation This adds additional unit tests validating the function for All possible alignment offsets of source and destination. Also extend the test to allow testing of an external stpncpy implementation, which greatly simplifies the development of custom implementations. Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42519 (cherry picked from commit 6fa9e7d8737548ef93c573387ce62402c368d486) lib/libc/tests/string/stpncpy_test.c | 99 +++++++++++++++++++++++++++++++----- 1 file changed, 85 insertions(+), 14 deletions(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=f5d63ae20d653e12d3617704e6317e3c8c50b640 commit f5d63ae20d653e12d3617704e6317e3c8c50b640 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-09 18:08:23 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:28 +0000 lib/libc/tests/string: add unit test for strlcpy A straightforward derivation from the stpncpy unit test. Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42863 (cherry picked from commit f7098b8659923873a7c60b64cb68182e470786f9) lib/libc/tests/string/Makefile | 1 + lib/libc/tests/string/strlcpy_test.c (new) | 183 +++++++++++++++++++++++++++++ 2 files changed, 184 insertions(+)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=c105cd8426bb0e100a4023898635c66bcea50e1f commit c105cd8426bb0e100a4023898635c66bcea50e1f Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-10-12 22:31:55 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:26 +0000 share/man/man7/simd.7: document strrchr scalar, baseline implementation Also mention missing rindex() entry, which is provided through strrchr(). Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42217 (cherry picked from commit dd1c2e887c1fa087bf15cbf7f0bdc35b7875bdd6) share/man/man7/simd.7 | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=37728967ee5c845f5af9d22377be10ffb62c90aa commit 37728967ee5c845f5af9d22377be10ffb62c90aa Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-09-16 05:29:39 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:24 +0000 lib/libc/amd64/string/strcmp.S: add baseline implementation This is the most complicated one so far. The basic idea is to process the bulk of the string in aligned blocks of 16 bytes such that one string runs ahead and the other runs behind. The string that runs ahead is checked for NUL bytes, the one that runs behind is compared with the corresponding chunk of the string that runs ahead. This trades an extra load per iteration for the very complicated block-reassembly needed in the other implementations (bionic, glibc). On the flip side, we need two code paths depending on the relative alignment of the two buffers. The initial part of the string is compared directly if it is known not to cross a page boundary. Otherwise, a complex slow path to avoid crossing into unmapped memory commences. Performance-wise we beat bionic for misaligned strings (i.e. the strings do not share an alignment offset) and reach comparable performance for aligned strings. glibc is a bit better as it has a special kernel for AVX-512, where this stuff is a bit easier to do. Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D41971 (cherry picked from commit bca25680b91b3bea7faef615765806a04634eb23) lib/libc/amd64/string/strcmp.S | 299 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 292 insertions(+), 7 deletions(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=265fb89aba2a6504f0a055e47ce6a72b1eecacf1 commit 265fb89aba2a6504f0a055e47ce6a72b1eecacf1 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-10-24 03:52:01 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:26 +0000 lib/libc/amd64/string: implement strsep() through strcspn() The strsep() function is basically strcspn() with extra steps. On amd64, we now have an optimised implementation of strcspn(), so instead of implementing the inner loop manually, just call into the optimised routine. Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42346 (cherry picked from commit fd2ecd91aeeeab579c769c9a39f90b4bd4a493a9) lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/strsep.c (new) | 57 ++++++++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=9b1a851e1ed04d38736011c227e8f9494acec681 commit 9b1a851e1ed04d38736011c227e8f9494acec681 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-10-12 05:37:41 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:26 +0000 lib/libc/amd64/string: add strrchr scalar, baseline implementation The baseline implementation is very straightforward, while the scalar implementation suffers from register pressure and the need to use SWAR techniques similar to those used for strchr(). Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42217 (cherry picked from commit 2ed514a220edbac6ca5ec9f40a3e0b3f2804796d) lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/strrchr.S (new) | 209 ++++++++++++++++++++++++++++++++++ 2 files changed, 210 insertions(+)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=57da891ad354455a057eaec3d7860f2c462c3e16 commit 57da891ad354455a057eaec3d7860f2c462c3e16 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-09 04:39:11 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:27 +0000 share/man/man7/simd.7: document simd-enhanced strncpy, stpncpy Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42519 (cherry picked from commit 75a9e2250656ff78ec1b0a124f282b925138ff51) share/man/man7/simd.7 | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=d5782dbd011ca338a9da4c8d663c6897af7859e1 commit d5782dbd011ca338a9da4c8d663c6897af7859e1 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-09-25 06:18:06 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:24 +0000 share/man/man7/simd.7: document new amd64 baseline strcmp() Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D41971 (cherry picked from commit 47adb1e012643b8f32f6cf8c84cd50eb58f0f17a) share/man/man7/simd.7 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=76f9afcdcfa482d81d5f3017f29c087fe795bb4f commit 76f9afcdcfa482d81d5f3017f29c087fe795bb4f Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-11-09 04:25:55 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:27 +0000 lib/libc/amd64/string: implement strncpy() by calling stpncpy() Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42519 (cherry picked from commit e19d46c808267f53455e96a28ff7654211523d2c) lib/libc/amd64/string/Makefile.inc | 1 + lib/libc/amd64/string/strncpy.c (new) | 41 +++++++++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=098997c5f05e157f2fae2787ec9a8b372156d487 commit 098997c5f05e157f2fae2787ec9a8b372156d487 Author: Robert Clausecker <fuz@FreeBSD.org> AuthorDate: 2023-09-28 20:40:20 +0000 Commit: Robert Clausecker <fuz@FreeBSD.org> CommitDate: 2024-01-24 19:39:25 +0000 lib/libc/tests/string: add unit tests for strncmp(3) These are patterned after the previously added (D41970) strcmp tests, but are extended to check for various length conditions. Sponsored by: The FreeBSD Foundation Tested by: developers@, exp-run Approved by: mjg MFC after: 1 month MFC to: stable/14 PR: 275785 Differential Revision: https://reviews.freebsd.org/D42122 (cherry picked from commit 459ddefcc9dcc010f6f7445285e61e2b6780379c) lib/libc/tests/string/Makefile | 1 + lib/libc/tests/string/strncmp_test.c (new) | 165 +++++++++++++++++++++++++++++ 2 files changed, 166 insertions(+)