Bug 201243 - locate misses on too large paths
Summary: locate misses on too large paths
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Some People
Assignee: Wolfram Schneider
URL:
Keywords: patch
Depends on:
Blocks:
 
Reported: 2015-07-01 10:29 UTC by Willem Jan Withagen
Modified: 2022-01-31 17:38 UTC (History)
1 user (show)

See Also:


Attachments
Enlarge pathsize in locate and it subs (4.13 KB, patch)
2015-07-01 10:29 UTC, Willem Jan Withagen
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Willem Jan Withagen 2015-07-01 10:29:04 UTC
Created attachment 158216 [details]
Enlarge pathsize in locate and it subs

This is been bothering me for a while.

/etc/periodic/weekly/310.locate
---------
Rebuilding locate database:

locate: integer out of +-MAXPATHLEN (1024): 1027
locate: integer out of +-MAXPATHLEN (1024): 1027

Rebuilding whatis database:
----------

And enlarging the buffer in the locate programs does not impact the results....
The paths are returned by find and stored in a "database".
But the database does not care how large the path-field is.

So I removed MAXPATHLEN as buffersize, replaced it with LOCPATHLEN, and upted it to 2048.

Building a new database on a system with 15855810 files does do the job.

patch attached.

--WjW
Comment 1 Wolfram Schneider freebsd_committer freebsd_triage 2017-12-01 16:55:21 UTC
The limit for a path in FreeBSD is 1024 bytes. System calls as stat() cannot handle a larger path. This is the reason why locate use this limit.

I'm wondering how did you created these large path? Was it a script or did you did it manually?

In theory locate(1) should return the same results as find(1). A quick test show that find can returns lines longer than even 30,000 bytes.

I have no objections to increase the LOCPATHLEN value to 2k or higher if many people use large path names. But what should be the best value, 2k, 4k, or more?
Comment 2 commit-hook freebsd_committer freebsd_triage 2022-01-30 16:28:54 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=5260fbcebdfcf2c17f9575bfbe9a34c97d56ea0a

commit 5260fbcebdfcf2c17f9575bfbe9a34c97d56ea0a
Author:     Wolfram Schneider <wosch@FreeBSD.org>
AuthorDate: 2022-01-30 16:27:27 +0000
Commit:     Wolfram Schneider <wosch@FreeBSD.org>
CommitDate: 2022-01-30 16:27:27 +0000

    fix check for integer

    For historical reasons, the integer is stored with an offset of plus 14.
    That means, for a given max path length of 1024 the valid values
    are -1009 .. 1037 and not -1023 .. 1023

    PR: 201243

 usr.bin/locate/locate/util.c | 22 ++++++++++++++--------
 1 file changed, 14 insertions(+), 8 deletions(-)
Comment 3 commit-hook freebsd_committer freebsd_triage 2022-01-31 17:36:53 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=834a8fa1694db39a356f1732d78f39ff76da422c

commit 834a8fa1694db39a356f1732d78f39ff76da422c
Author:     Wolfram Schneider <wosch@FreeBSD.org>
AuthorDate: 2022-01-31 17:32:46 +0000
Commit:     Wolfram Schneider <wosch@FreeBSD.org>
CommitDate: 2022-01-31 17:32:46 +0000

    enable to configure the locate path length at compile time

    The length has not changed and is 1024 chars (equals PATH_MAX).

    PR: 201243
    Submitted by: Willem Jan Withagen <wjw@digiware.nl>

 usr.bin/locate/bigram/locate.bigram.c |  5 ++---
 usr.bin/locate/code/locate.code.c     |  6 +++---
 usr.bin/locate/locate/fastfind.c      | 14 +++++++-------
 usr.bin/locate/locate/locate.h        |  6 ++++++
 usr.bin/locate/locate/util.c          | 11 +++++------
 5 files changed, 23 insertions(+), 19 deletions(-)