Is there an UPDATING note that should be on Clamav 1.0.0,1 ?
Confirmed on FreeBSD 12.3-RELEASE using upgraded installation built from ports. First test, I re-used the previous daily.cld, which was acceptable to clamav 0.105. user@machine # freshclam ClamAV update process started at Tue Nov 29 20:33:52 2022 daily.cld database is up-to-date (version: 26735, sigs: 2013231, f-level: 90, builder: raynman) main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr) bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2) user@machine:/ # service clamav-clamd start Starting clamav_clamd. LibClamAV Error: cli_loadinfo: Incorrect digital signature LibClamAV Error: cli_loadinfo: Problem parsing database at line 25 LibClamAV Error: Can't load daily.info: Malformed database LibClamAV Error: cli_tgzload: Can't load daily.info LibClamAV Error: Can't load /var/db/clamav/daily.cld: Malformed database LibClamAV Error: cli_loaddbdir: error loading database /var/db/clamav/daily.cld ERROR: Malformed database /usr/local/etc/rc.d/clamav-clamd: WARNING: failed to start clamav_clamd I then moved the existing /var/db/clamav contents, and re-downloaded it. Fresh copy of the database is still has integrity issues: ClamAV update process started at Tue Nov 29 21:06:57 2022 daily database available for download (remote version: 26735) Time: 19.5s, ETA: 0.0s [========================>] 57.53MiB/57.53MiB ERROR: Verification: Can't verify database integrity Trying again in 5 secs... daily database available for download (remote version: 26735) Time: 27.0s, ETA: 0.0s [========================>] 57.53MiB/57.53MiB ERROR: Verification: Can't verify database integrity Trying again in 5 secs... daily database available for download (remote version: 26735) Time: 28.3s, ETA: 0.0s [========================>] 57.53MiB/57.53MiB ERROR: Verification: Can't verify database integrity Giving up on https://database.clamav.net... ERROR: Update failed for database: daily ERROR: Database update process failed: Invalid or corrupted CVD/CLD database ERROR: Update failed. I also re-built using rust 1.63 (from pkg), because I had some custom options. Still failed.
(In reply to fsbruva from comment #1) I tried to reproduce the problem with following steps. 1. Make clean install of 12.3-RELEASE amd64. 2. Build 0.105.1_5,1 of clamav and install it. 3. Add 'clamav_freshclam_enable="YES"' and 'clamav_clamd_enable="YES"' to /etc/rc.conf. 4. Start freshclam with `service clamav-freshclam start` and confirm virus data files are successfully downloaded. 5. Start clamd with `service clamav-clamd start` and confirm clamd successfully load virus data files. 6. Stop clamd with `service clamav-clamd stop`. 7. Stop freshclam with `service clamav-freshclam stop`. 8. Build 1.0.0,1 of clamav and upgrade from 0.105.1_5,1 to it. 9. Start freshclam with `service clamav-freshclam start` and confirm virus data files are up to date. 10. Start clamd with `service clamav-clamd start`. But after step 10 is finished, clamd successfully load virus data files and no error happens. So I can't reproduce the problem. > I also re-built using rust 1.63 (from pkg), because I had some custom options. Do you mean you build clamav with non defult option settings? If so please provide it.
(In reply to fsbruva from comment #1) I am seeing this exact same behavior on FreeBSD 13.1-RELEASE-p3. At first, I thought it might be an issue on the database server side and I was just receiving a bad DB. I have since tried too many times to pull a new DB so I am locked out (cool-down) of database.clamav.net. ClamAV hasn't worked for me for two days now. Not sure what to make of this. It's difficult to troubleshoot when I can't pull down the database(s).
(In reply to Yasuhiro Kimura from comment #2) > I tried to reproduce the problem with following steps. I followed those same steps in a FreeBSD 12.3 jail on amd64, with the following jail behavior options: allow.set_hostname = 0; allow.sysvipc = 0; allow.raw_sockets = 0; allow.chflags; devfs_ruleset = "4"; mount.devfs; sysvshm = "new"; sysvmsg = "new"; sysvsem = "new"; > Do you mean you build clamav with non defult option settings? If so please provide it. The first time I built clamav 1.0.0, I used rust 1.65, with all port options unset, and observed the error. Then, I reverted to rust 1.63 from pkg, which has SOURCES and WASM On. I then cleaned and rebuilt and reinstalled clamav 1.0.0, and observed the error. (In reply to ek@purplehat.org from comment #3) Are you running in a jail?
(In reply to fsbruva from comment #4) This particular ClamAV instance is running on the base host and not in a jail. Also, just an FYI, I'm also building against rust 1.65.0 with default port options and building ClamAV 1.0.0,1 with default port options as well.
(In reply to ek from comment #5) What version/edition of the database do you currently have?
(In reply to fsbruva from comment #6) According to the most recent output to freshclam.log, it appears to be using: daily.cvd version: 26737 main.cld version: 62 bytecode.cld version: 333
(In reply to ek from comment #7) Yep, mine, too. I reverted clamav to 0.105, and successfully obtained the latest updated definitions. I just updated my copy of the ports tree, and re-built and installed clamav 1.0.0. clamd still fails to start with the new database version, with same errors. I really don't feel like bisecting the changes since 0.105..... but it might come down to that....
(In reply to fsbruva from comment #8) Would you please provide the result of `sha256 /var/db/clamav/*` ?
(In reply to Yasuhiro Kimura from comment #9) My result: SHA256 (/var/db/clamav/bytecode.cvd) = aa5222822ba8b87cd319d9b00ef82e93b85f8c1c134b4465cad095368a0ab8b1 SHA256 (/var/db/clamav/daily.cvd) = f3ab5e274a8cdd7253cf55f381c7e4d6ae231d4a9fc66a210a12f8d1a980825e SHA256 (/var/db/clamav/freshclam.dat) = 5f769c181a808cf00df6ba51a646b5a17c00cf9ccc48002a075827145e6f1baf SHA256 (/var/db/clamav/main.cvd) = 9520cc7827ddc5267684f2c5c7044cf2df1f62606ca81402cbeae12183763a90
O.105.1-3 works 1.0.0 is broken
Same issue for me. I first looked for something in /usr/ports/UPDATING before ending here.
I am thinking this is some kind of latent dependency issue. I was able to install, update and run clamav 1.0.0 in another 12.3-RELEASE amd64 jail on the same machine. Differences are the successful jail never had clamav before, and different ports installed. I will look through the clamav build logs to see if I can spot differences. (In reply to Yasuhiro Kimura from comment #9) > Would you please provide the result of `sha256 /var/db/clamav/*` ? daily.cvd version: 26737 main.cld version: 62 bytecode.cld version: 333 SHA256 (/var/db/clamav/bytecode.cvd) = aa5222822ba8b87cd319d9b00ef82e93b85f8c1c134b4465cad095368a0ab8b1 SHA256 (/var/db/clamav/daily.cvd) = fc5e16e3862d39b9b219469cc902e7e7fbb0f70f02a86ac050ac2db08cbb1557 SHA256 (/var/db/clamav/freshclam.dat) = a6343045528f703d92f40b3d7f421d01db71c6252b1f4efb69ef51486804b82d SHA256 (/var/db/clamav/main.cvd) = 9520cc7827ddc5267684f2c5c7044cf2df1f62606ca81402cbeae12183763a90 user@machine:/var/db/clamav # service clamav-clamd start Starting clamav_clamd. LibClamAV Error: Can't load /var/db/clamav/daily.cvd: Can't verify database integrity LibClamAV Error: cli_loaddbdir: error loading database /var/db/clamav/daily.cvd ERROR: Can't verify database integrity /usr/local/etc/rc.d/clamav-clamd: WARNING: failed to start clamav_clamd I tried running freshclam 1.0.0, and got this: user@machine:/var/db/clamav # freshclam ClamAV update process started at Fri Dec 2 08:58:50 2022 daily database available for update (local version: 26737, remote version: 26738) Current database is 1 version behind. Downloading database patch # 26738... WARNING: [LibClamAV] CVD verification failed for: daily.cvd ERROR: mkdir_and_chdir_for_cdiff_tmp: Can't unpack daily.cvd into /var/db/clamav/tmp.f3e98ef3c4/clamav-2e2434f3e48000b9eac28f6f03a601ac.tmp The database server doesn't have the latest patch for the daily database (version 26738). The server will likely have updated if you check again in a few hours. main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr) bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2) Note the problem with unpacking daily.cvd. I have /var/db/clamav/ owned by clamav:clamav, with 755 permissions. I removed clamav 1.0.0, and re-installed 0.105.1_5. When I tried to start clamav 0.105.1 with these same files, I got: user@machine:/var/db/clamav # service clamav-clamd start Starting clamav_clamd. LibClamAV Error: Can't load /var/db/clamav/daily.cvd: Can't verify database integrity LibClamAV Error: cli_loaddbdir: error loading database /var/db/clamav/daily.cvd ERROR: Can't verify database integrity /usr/local/etc/rc.d/clamav-clamd: WARNING: failed to start clamav_clamd So I ran freshclam 0.105.1, and got this: root@condor_www:/var/db/clamav # freshclam ClamAV update process started at Fri Dec 2 09:05:31 2022 daily database available for update (local version: 26737, remote version: 26738) Current database is 1 version behind. Downloading database patch # 26738... Time: 0.3s, ETA: 0.0s [========================>] 8.52KiB/8.52KiB Testing database: '/var/db/clamav/tmp.6c1cb80df0/clamav-e65ae6b9ff22c449bf0ae6fd199bcf07.tmp-daily.cld' ... Database test passed. daily.cld updated (version: 26738, sigs: 2013306, f-level: 90, builder: raynman) main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr) bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2) File hashes are now: SHA256 (/var/db/clamav/bytecode.cvd) = aa5222822ba8b87cd319d9b00ef82e93b85f8c1c134b4465cad095368a0ab8b1 SHA256 (/var/db/clamav/daily.cld) = 0aefe922fbe12376d83d8b3c0e59dcdb75c77588bd790b03879de148ecfdae6f SHA256 (/var/db/clamav/freshclam.dat) = a6343045528f703d92f40b3d7f421d01db71c6252b1f4efb69ef51486804b82d SHA256 (/var/db/clamav/main.cvd) = 9520cc7827ddc5267684f2c5c7044cf2df1f62606ca81402cbeae12183763a90 And clamav 0.105.1 started successfully. Stop clamav service, deinstalled clamav 0.105, built clamav 1.0.0 from ports, installed, and saw same error with starting clamav-clamd service.
I've narrowed it down to a difference in the build dependency toolchain. I made a package of clamav 1.0.0 in the working jail, exported it to the jail with errors, installed it, and clamav 1.0.0 starts and functions properly. Therefore, it is likely down to a different in port options in one of the build-depends ports.
It is weird, have this problem since i built world to 13.1-STABLE FreeBSD 13.1-STABLE #0 abc542e34: Fri Dec 2 18:16:07 CET 2022 If i ldd clamav build from ports i get totally different libraries between another system running 13.1-RELEASE-p3 Having issue in STABLE [vdso] (0x7ffffffff5d0) libbrotlicommon.so.1 => /usr/local/lib/libbrotlicommon.so.1 (0xc8d49d44000) libbrotlidec.so.1 => /usr/local/lib/libbrotlidec.so.1 (0xc8d456e3000) libbz2.so.4 => /usr/lib/libbz2.so.4 (0xc8d3cefb000) libc++.so.1 => /usr/lib/libc++.so.1 (0xc8d46fe8000) libc.so.7 => /lib/libc.so.7 (0xc8d36f1f000) libclamav.so.11 => /usr/local/lib/libclamav.so.11 (0xc8d37a00000) libcrypto.so.11 => /usr/local/lib/libcrypto.so.11 (0xc8d40e22000) libcurl.so.4 => /usr/local/lib/libcurl.so.4 (0xc8d421e7000) libcxxrt.so.1 => /lib/libcxxrt.so.1 (0xc8d4671b000) libdevstat.so.7 => /lib/libdevstat.so.7 (0xc8d3c91e000) libelf.so.2 => /lib/libelf.so.2 (0xc8d4191c000) libexecinfo.so.1 => /usr/lib/libexecinfo.so.1 (0xc8d3623c000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xc8d363e3000) libicudata.so.72 => /usr/local/lib/libicudata.so.72 (0xc8d43c6d000) libicui18n.so.72 => /usr/local/lib/libicui18n.so.72 (0xc8d44031000) libicuuc.so.72 => /usr/local/lib/libicuuc.so.72 (0xc8d42dbb000) libidn2.so.0 => /usr/local/lib/libidn2.so.0 (0xc8d44f7d000) libjson-c.so.5 => /usr/local/lib/libjson-c.so.5 (0xc8d3f92e000) libkvm.so.7 => /lib/libkvm.so.7 (0xc8d3b374000) liblzma.so.5 => /usr/lib/liblzma.so.5 (0xc8d4288a000) libm.so.5 => /lib/libm.so.5 (0xc8d398d3000) libmd.so.6 => /lib/libmd.so.6 (0xc8d4586d000) libmspack.so.0 => /usr/local/lib/libmspack.so.0 (0xc8d35cfc000) libnghttp2.so.14 => /usr/local/lib/libnghttp2.so.14 (0xc8d4468b000) libpcre2-8.so.0 => /usr/local/lib/libpcre2-8.so.0 (0xc8d3deb3000) libprocstat.so.1 => /usr/lib/libprocstat.so.1 (0xc8d3b9fe000) librt.so.1 => /usr/lib/librt.so.1 (0xc8d38f81000) libssh2.so.1 => /usr/local/lib/libssh2.so.1 (0xc8d46975000) libssl.so.11 => /usr/local/lib/libssl.so.11 (0xc8d3f9ce000) libthr.so.3 => /lib/libthr.so.3 (0xc8d3f05f000) libunistring.so.5 => /usr/local/lib/libunistring.so.5 (0xc8d47c18000) libutil.so.9 => /lib/libutil.so.9 (0xc8d3a6c5000) libxml2.so.2 => /usr/local/lib/libxml2.so.2 (0xc8d3e516000) libz.so.6 => /lib/libz.so.6 (0xc8d40027000) Working RELEASE built with same settings: libasn1.so.11 => /usr/lib/libasn1.so.11 (0x801869000) libbz2.so.4 => /usr/lib/libbz2.so.4 (0x80032e000) libc.so.7 => /lib/libc.so.7 (0x800d2f000) libclamav.so.11 => /usr/local/lib/libclamav.so.11 (0x800400000) libcom_err.so.5 => /usr/lib/libcom_err.so.5 (0x8003fb000) libcrypt.so.5 => /lib/libcrypt.so.5 (0x801953000) libcrypto.so.111 => /lib/libcrypto.so.111 (0x801392000) libcurl.so.4 => /usr/local/lib/libcurl.so.4 (0x8016a2000) libdevstat.so.7 => /lib/libdevstat.so.7 (0x800325000) libelf.so.2 => /lib/libelf.so.2 (0x801742000) libexecinfo.so.1 => /usr/lib/libexecinfo.so.1 (0x800286000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x80028c000) libgssapi.so.10 => /usr/lib/libgssapi.so.10 (0x8019f6000) libgssapi_krb5.so.10 => /usr/lib/libgssapi_krb5.so.10 (0x801a02000) libheimbase.so.11 => /usr/lib/libheimbase.so.11 (0x801937000) libheimntlm.so.11 => /usr/lib/libheimntlm.so.11 (0x80180e000) libhx509.so.11 => /usr/lib/libhx509.so.11 (0x801818000) libidn2.so.0 => /usr/local/lib/libidn2.so.0 (0x801c2e000) libintl.so.8 => /usr/local/lib/libintl.so.8 (0x801a42000) libjson-c.so.5 => /usr/local/lib/libjson-c.so.5 (0x8012b9000) libkrb5.so.11 => /usr/lib/libkrb5.so.11 (0x801974000) libkvm.so.7 => /lib/libkvm.so.7 (0x800302000) liblzma.so.5 => /usr/lib/liblzma.so.5 (0x80175d000) libm.so.5 => /lib/libm.so.5 (0x8002a6000) libmd.so.6 => /lib/libmd.so.6 (0x801a24000) libmspack.so.0 => /usr/local/lib/libmspack.so.0 (0x80026d000) libnghttp2.so.14 => /usr/local/lib/libnghttp2.so.14 (0x801789000) libpcre2-8.so.0 => /usr/local/lib/libpcre2-8.so.0 (0x800344000) libprivateheimipcc.so.11 => /usr/lib/libprivateheimipcc.so.11 (0x801c62000) libprocstat.so.1 => /usr/lib/libprocstat.so.1 (0x800317000) libpsl.so.5 => /usr/local/lib/libpsl.so.5 (0x8017fa000) libroken.so.11 => /usr/lib/libroken.so.11 (0x80193e000) librt.so.1 => /usr/lib/librt.so.1 (0x8002e1000) libssh2.so.1 => /usr/local/lib/libssh2.so.1 (0x8017bb000) libssl.so.111 => /usr/lib/libssl.so.111 (0x8012fa000) libthr.so.3 => /lib/libthr.so.3 (0x8012cc000) libunistring.so.5 => /usr/local/lib/libunistring.so.5 (0x801a50000) libutil.so.9 => /lib/libutil.so.9 (0x8002ea000) libwind.so.11 => /usr/lib/libwind.so.11 (0x80190c000) libxml2.so.2 => /usr/local/lib/libxml2.so.2 (0x801139000) libz.so.6 => /lib/libz.so.6 (0x801685000)
Hello all. I noticed the same problem on 12.3-RELEASE FreeBSD 12.3-RELEASE releng/12.3-n234202-70cb68e7a00 GENERIC amd64 Maybe it is a problem with 'Decoded signature' look at: LibClamAV debug: MD5(.tar.gz) = 66662314e2576ce0f21b040490bdb5d6 LibClamAV debug: cli_versig: Decoded signature: 00000000000000000000000000000000 LibClamAV debug: cli_versig: Signature doesn't match. LibClamAV debug: cli_cvdverify: Digital signature verification error LibClamAV debug: Cleaning up phishcheck Reverted to port: security/clamav-lts, clamav-lts-0.103.7,1 and the problem was gone Here is a full debug output from freshclam: root@thirdeye:/var/db/clamav # freshclam -u vscan --debug -v Current working dir is /var/db/clamav/ Loaded freshclam.dat: version: 1 uuid: 2aeb987e-bdbd-4d28-97a6-e8c686158415 retry-after: 2022-12-02 21:40:44 ClamAV update process started at Sat Dec 3 11:49:47 2022 Current working dir is /var/db/clamav/ Querying current.cvd.clamav.net TTL: 1800 fc_dns_query_update_info: Software version from DNS: 0.103.7 WARNING: Cool-down expired, ok to try again. Saved freshclam.dat Current working dir is /var/db/clamav/ check_for_new_database_version: No local copy of "daily" database. query_remote_database_version: daily.cvd version from DNS: 26739 daily database available for download (remote version: 26739) Retrieving https://database.clamav.net/daily.cvd downloadFile: Download source: https://database.clamav.net/daily.cvd downloadFile: Download destination: /var/db/clamav/tmp.165ecb003d/clamav-bd3cfc7a9a3af6e708185426742b891f.tmp * Trying 104.16.219.84:443... * Connected to database.clamav.net (104.16.219.84) port 443 (#0) * ALPN: offers h2 * ALPN: offers http/1.1 * CAfile: /usr/local/share/certs/ca-root-nss.crt * CApath: none * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 * ALPN: server accepted h2 * Server certificate: * subject: C=US; ST=California; L=San Francisco; O=Cloudflare, Inc.; CN=sni.cloudflaressl.com * start date: Jun 14 00:00:00 2022 GMT * expire date: Jun 14 23:59:59 2023 GMT * subjectAltName: host "database.clamav.net" matched cert's "database.clamav.net" * issuer: C=US; O=Cloudflare, Inc.; CN=Cloudflare Inc ECC CA-3 * SSL certificate verify ok. * Using HTTP2, server supports multiplexing * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 * h2h3 [:method: GET] * h2h3 [:path: /daily.cvd] * h2h3 [:scheme: https] * h2h3 [:authority: database.clamav.net] * h2h3 [user-agent: ClamAV/1.0.0 (OS: FreeBSD, ARCH: amd64, CPU: amd64, UUID: 2aeb987e-bdbd-4d28-97a6-e8c686158415)] * h2h3 [accept: */*] * Using Stream ID: 1 (easy handle 0x801fd9000) > GET /daily.cvd HTTP/2 Host: database.clamav.net user-agent: ClamAV/1.0.0 (OS: FreeBSD, ARCH: amd64, CPU: amd64, UUID: 2aeb987e-bdbd-4d28-97a6-e8c686158415) accept: */* connection: close * old SSL session ID is stale, removing * Connection state changed (MAX_CONCURRENT_STREAMS == 256)! < HTTP/2 200 < date: Sat, 03 Dec 2022 10:49:48 GMT < content-type: application/octet-stream < content-length: 60333814 < last-modified: Sat, 03 Dec 2022 08:16:00 GMT < etag: "638b05c0-3989ef6" < expires: Sat, 03 Dec 2022 22:49:48 GMT < etag: "638b05c0-3989ef6" [82/1841] < expires: Sat, 03 Dec 2022 22:49:48 GMT < cache-control: public, max-age=43200 < cf-cache-status: HIT < age: 8995 < accept-ranges: bytes < server-timing: cf-q-config;dur=6.0000020312145e-06 < strict-transport-security: max-age=15552000 < x-content-type-options: nosniff < server: cloudflare < cf-ray: 773bbce0ecebbf65-WAW < Time: 4.9s, ETA: 0.0s [========================>] 57.54MiB/57.54MiB * Connection #0 to host database.clamav.net left intact LibClamAV debug: Initialized 1.0.0 engine LibClamAV debug: in cli_cvdload() LibClamAV debug: MD5(.tar.gz) = 66662314e2576ce0f21b040490bdb5d6 LibClamAV debug: cli_versig: Decoded signature: 00000000000000000000000000000000 LibClamAV debug: cli_versig: Signature doesn't match. LibClamAV debug: cli_cvdverify: Digital signature verification error LibClamAV debug: Cleaning up phishcheck LibClamAV debug: Phishcheck cleaned up ERROR: Verification: Can't verify database integrity Trying again in 5 secs... check_for_new_database_version: No local copy of "daily" database. query_remote_database_version: daily.cvd version from DNS: 26739 daily database available for download (remote version: 26739) Retrieving https://database.clamav.net/daily.cvd downloadFile: Download source: https://database.clamav.net/daily.cvd downloadFile: Download destination: /var/db/clamav/tmp.165ecb003d/clamav-71a990e570dd836b8a4a1002be6be9da.tmp * Trying 104.16.219.84:443... * Connected to database.clamav.net (104.16.219.84) port 443 (#0) * ALPN: offers h2 * ALPN: offers http/1.1 * CAfile: /usr/local/share/certs/ca-root-nss.crt * CApath: none * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 * ALPN: server accepted h2 * Server certificate: * subject: C=US; ST=California; L=San Francisco; O=Cloudflare, Inc.; CN=sni.cloudflaressl.com * start date: Jun 14 00:00:00 2022 GMT * expire date: Jun 14 23:59:59 2023 GMT * subjectAltName: host "database.clamav.net" matched cert's "database.clamav.net" * issuer: C=US; O=Cloudflare, Inc.; CN=Cloudflare Inc ECC CA-3 * SSL certificate verify ok. * Using HTTP2, server supports multiplexing * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 * h2h3 [:method: GET] * h2h3 [:path: /daily.cvd] * h2h3 [:scheme: https] * h2h3 [:authority: database.clamav.net] * h2h3 [user-agent: ClamAV/1.0.0 (OS: FreeBSD, ARCH: amd64, CPU: amd64, UUID: 2aeb987e-bdbd-4d28-97a6-e8c686158415)] * h2h3 [accept: */*] * Using Stream ID: 1 (easy handle 0x801fd9000) > GET /daily.cvd HTTP/2 Host: database.clamav.net user-agent: ClamAV/1.0.0 (OS: FreeBSD, ARCH: amd64, CPU: amd64, UUID: 2aeb987e-bdbd-4d28-97a6-e8c686158415) accept: */* connection: close * old SSL session ID is stale, removing * Connection state changed (MAX_CONCURRENT_STREAMS == 256)! < HTTP/2 200 < date: Sat, 03 Dec 2022 10:49:59 GMT < content-type: application/octet-stream < content-length: 60333814 < last-modified: Sat, 03 Dec 2022 08:16:00 GMT < etag: "638b05c0-3989ef6" < expires: Sat, 03 Dec 2022 22:49:59 GMT < cache-control: public, max-age=43200 < cf-cache-status: HIT < age: 9006 < accept-ranges: bytes < server-timing: cf-q-config;dur=6.9999987317715e-06 < strict-transport-security: max-age=15552000 < x-content-type-options: nosniff < server: cloudflare < cf-ray: 773bbd1fd9f8fbc6-WAW < Time: 4.2s, ETA: 0.0s [========================>] 57.54MiB/57.54MiB * Connection #0 to host database.clamav.net left intact LibClamAV debug: Initialized 1.0.0 engine LibClamAV debug: in cli_cvdload() LibClamAV debug: MD5(.tar.gz) = 66662314e2576ce0f21b040490bdb5d6 LibClamAV debug: cli_versig: Decoded signature: 00000000000000000000000000000000 LibClamAV debug: cli_versig: Signature doesn't match. LibClamAV debug: cli_cvdverify: Digital signature verification error LibClamAV debug: Cleaning up phishcheck LibClamAV debug: Phishcheck cleaned up ERROR: Verification: Can't verify database integrity Trying again in 5 secs... check_for_new_database_version: No local copy of "daily" database. query_remote_database_version: daily.cvd version from DNS: 26739 daily database available for download (remote version: 26739) Retrieving https://database.clamav.net/daily.cvd downloadFile: Download source: https://database.clamav.net/daily.cvd downloadFile: Download destination: /var/db/clamav/tmp.165ecb003d/clamav-fa17a57360dfb00dfa46cc38c9d74899.tmp * Trying 104.16.218.84:443... * Trying 2606:4700::6810:da54:443... * Immediate connect fail for 2606:4700::6810:da54: No route to host * Trying 2606:4700::6810:db54:443... * Immediate connect fail for 2606:4700::6810:db54: No route to host * Connected to database.clamav.net (104.16.218.84) port 443 (#0) * ALPN: offers h2 * ALPN: offers http/1.1 * CAfile: /usr/local/share/certs/ca-root-nss.crt * CApath: none * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 * ALPN: server accepted h2 * Server certificate: * subject: C=US; ST=California; L=San Francisco; O=Cloudflare, Inc.; CN=sni.cloudflaressl.com * start date: Jun 14 00:00:00 2022 GMT * expire date: Jun 14 23:59:59 2023 GMT * subjectAltName: host "database.clamav.net" matched cert's "database.clamav.net" * issuer: C=US; O=Cloudflare, Inc.; CN=Cloudflare Inc ECC CA-3 * SSL certificate verify ok. * Using HTTP2, server supports multiplexing * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 * h2h3 [:method: GET] * h2h3 [:path: /daily.cvd] * h2h3 [:scheme: https] * h2h3 [:authority: database.clamav.net] * h2h3 [user-agent: ClamAV/1.0.0 (OS: FreeBSD, ARCH: amd64, CPU: amd64, UUID: 2aeb987e-bdbd-4d28-97a6-e8c686158415)] * h2h3 [accept: */*] * Using Stream ID: 1 (easy handle 0x801fd9000) > GET /daily.cvd HTTP/2 Host: database.clamav.net user-agent: ClamAV/1.0.0 (OS: FreeBSD, ARCH: amd64, CPU: amd64, UUID: 2aeb987e-bdbd-4d28-97a6-e8c686158415) accept: */* connection: close * old SSL session ID is stale, removing * Connection state changed (MAX_CONCURRENT_STREAMS == 256)! < HTTP/2 200 < date: Sat, 03 Dec 2022 10:50:09 GMT < content-type: application/octet-stream < content-length: 60333814 < last-modified: Sat, 03 Dec 2022 08:16:00 GMT < etag: "638b05c0-3989ef6" < expires: Sat, 03 Dec 2022 22:50:09 GMT < cache-control: public, max-age=43200 < cf-cache-status: HIT < age: 9016 < accept-ranges: bytes < server-timing: cf-q-config;dur=5.0000016926788e-06 < strict-transport-security: max-age=15552000 < x-content-type-options: nosniff < server: cloudflare < cf-ray: 773bbd61ec01bfad-WAW < Time: 6.3s, ETA: 0.0s [========================>] 57.54MiB/57.54MiB * Connection #0 to host database.clamav.net left intact LibClamAV debug: Initialized 1.0.0 engine LibClamAV debug: in cli_cvdload() LibClamAV debug: MD5(.tar.gz) = 66662314e2576ce0f21b040490bdb5d6 LibClamAV debug: cli_versig: Decoded signature: 00000000000000000000000000000000 LibClamAV debug: cli_versig: Signature doesn't match. LibClamAV debug: cli_cvdverify: Digital signature verification error LibClamAV debug: Cleaning up phishcheck LibClamAV debug: Phishcheck cleaned up ERROR: Verification: Can't verify database integrity Giving up on https://database.clamav.net... ERROR: Update failed for database: daily ERROR: Database update process failed: Invalid or corrupted CVD/CLD database ERROR: Update failed.
Good morning. I have been tearing my hair out on this one. Here's the output when starting clamav 1.0.0 on the broken jail, using a known good (downloaded/verified by freshclam 1.0.0 in functioning jail) backup of /var/db/clamav: Attempting to start clamav-clamd 1.0.0 with known good database in broken jail: LibClamAV debug: Loading databases from /var/db/clamav LibClamAV debug: in cli_cvdload() LibClamAV debug: MD5(.tar.gz) = 8cbf2717c14dbd1406290693c0dcf014 LibClamAV debug: cli_versig: Decoded signature: 00000000000000000000000000000000 LibClamAV debug: cli_versig: Signature doesn't match. LibClamAV debug: cli_cvdverify: Digital signature verification error LibClamAV Error: Can't load /var/db/clamav/daily.cvd: Can't verify database integrity LibClamAV Error: cli_loaddbdir: error loading database /var/db/clamav/daily.cvd ERROR: Can't verify database integrity Closing the main socket. /usr/local/etc/rc.d/clamav-clamd: WARNING: failed to start clamav_clamd A similar error occurs when running freshclam 1.0.0 in the broken jail. You can see that the correct hash is present in the file.... user@machine:/var/db/clamav # hexdump -vC daily.cvd | head -n10 00000000 43 6c 61 6d 41 56 2d 56 44 42 3a 30 31 20 44 65 |ClamAV-VDB:01 De| 00000010 63 20 32 30 32 32 20 30 33 2d 32 32 20 2d 30 35 |c 2022 03-22 -05| 00000020 30 30 3a 32 36 37 33 37 3a 32 30 31 33 32 33 32 |00:26737:2013232| 00000030 3a 39 30 3a 38 63 62 66 32 37 31 37 63 31 34 64 |:90:8cbf2717c14d| 00000040 62 64 31 34 30 36 32 39 30 36 39 33 63 30 64 63 |bd1406290693c0dc| 00000050 66 30 31 34 3a 78 32 73 4f 65 6e 52 32 36 70 6a |f014:x2sOenR26pj| 00000060 39 44 36 30 4a 67 2f 79 44 44 78 53 64 47 6c 54 |9D60Jg/yDDxSdGlT| 00000070 79 45 78 48 35 4e 66 76 42 36 4a 30 2f 66 79 58 |yExH5NfvB6J0/fyX| 00000080 46 71 4f 41 59 50 6a 2f 37 74 34 52 76 34 66 43 |FqOAYPj/7t4Rv4fC| 00000090 34 65 47 42 4b 69 34 6b 56 2b 62 63 70 46 57 49 |4eGBKi4kV+bcpFWI| When that same file is used in the working jail, when clamav-clamd 1.0.0 is started, here's what you see... LibClamAV debug: Loading databases from /var/db/clamav LibClamAV debug: in cli_cvdload() LibClamAV debug: MD5(.tar.gz) = 8cbf2717c14dbd1406290693c0dcf014 LibClamAV debug: cli_versig: Decoded signature: 8cbf2717c14dbd1406290693c0dcf014 I have done the following thus far: 1. Installed all the ports that were unique to the functioning jail (using same OPTIONS) that into the non-working jail to try to solve dependencies issues. 2. Completely rebuilt entire chain and all dependencies in functioning jail, using portmaster -f security/clamav. Afterwards, resulting port install of clamav 1.0.0 still worked in the functioning jail. 3. Copied all ports OPTIONS from the functioning jail into non-working jail, and completely rebuilt entire chain in non-working jail, using portmaster -f security/clamav. Afterwards, resulting port install of clamav 1.0.0 still failed with the same error. 4. Looked at the diff between git releases 0.105 and 1.0. Nothing stood out in libclamav/dsig.c for changes to the behavior of cli_versig() or cli_decodesig() functions, or in libclamav/cvd.c within the cli_cvdload() or cli_cvdcertify() functions. Most of the code for those sections hasn't changed in like 3 years or more. It *might* be related to the upgrade of the TomsFastMath code from commit 375ecf6. But even if it is, I still can't figure why the jails are building differently. My next step is to progressively install the ports unique to the non-working jail into the functioning jail so I can try and detect when a failure occurs. I thought maybe the others seeing this issue could share the list of other ports they have installed, so help me confirm my outcomes. (In reply to doctor from comment #0) Can you please upload a file with the output of `pkg info`? (In reply to ek from comment #3) Can you please upload a file with the output of `pkg info`? (In reply to Arnaud de Prelle from comment #12) Can you please upload a file with the output of `pkg info`? (In reply to Sigi from comment #15) Can you please upload a file with the output of `pkg info`? (In reply to jasiu from comment #16) Can you please upload a file with the output of `pkg info`?
Ok. I think I have found the cause, and can reliably trigger the error. With the dependency change of clamav 1.0.0 to only use the bundled version of tomsfastmath, many of us who built from ports tree were left with an orphan installation of math/tomsfastmath that CMake finds and tries to include. Removing math/tomsfastmath enables clamav 1.0.0 to build and run successfully in the previously broken jail. (In reply to Yasuhiro Kimura from comment #2) In your example, did you build 0.105? Or you installed it? Were you building 0.105 or 1.0.0 with ccache enabled? Did you remove tomsfastmath before building 1.0.0? (In reply to doctor from comment #0) (In reply to ek from comment #3) (In reply to Arnaud de Prelle from comment #12) (In reply to Sigi from comment #15) (In reply to jasiu from comment #16) Can you confirm you have tomsfastmath? Can you de-install it, rebuild clamav and see if the error resolves?
Created attachment 238735 [details] clamav: patch to ignore math/tomsfastmath Here's my proposed patch to enable clamav 1.0.0 to be built and used, even if math/tomsfastmath is not removed.
Created attachment 238736 [details] clamav: patch to ignore math/tomsfastmath Here's my (correct) proposed patch to enable clamav 1.0.0 to be built and used, even if math/tomsfastmath is not removed.
(In reply to fsbruva from comment #18) Great find! I can confirm that removing math/tomsfastmath (pkg delete tomsfastmath-\*) and reinstalling security/clamav (portmaster -BCd security/clamav) did the trick. The freshclam process was able to pull down and verify all databases and the clamd daemon started up as expected. Thank you!
(In reply to fsbruva from comment #18) (In reply to ek from comment #21) Thank you very much for debugging this. I can as well confirm, that with removing math/tomsfastmath and then building security/clamav (1.0.0,1) and restarting clamd and freshclam worked fine as well on FreeBSD 12.3-RELEASE.
Also confirm that deleting tomsfastmath from ports unbreak clamav. Thanks for digging it out..
(In reply to fsbruva from comment #18) I tried following steps. 1. Make clean install of 13.1-RELEASE amd64. 2. Check out clean ports tree (ports bf399d8b9b25). 3. Install TomsFastMath with `cd /usr/ports/math/tomsfastmath; make install`. 4. Install ClamAV 1.0.0 with `cd /usr/ports/security/clamav; make instlal'. 5. Add 'clamav_clamd_enable="YES"" and 'clamav_freshclam_enable="YES"' to /etc/rc.conf. 6. Start freshclam with `service clamav-freshclam start`. 7. Start clamd with `service clamav-clamd start`. And both freshclam and clamd start without any error. Does anybody succeeds to reproduce the problem starting from cleanly installed system?
It's solved for me as well. Thanks for the tip ! # pkg remove tomsfastmath # portmaster clamav (...) # freshclam ClamAV update process started at Tue Dec 13 14:45:14 2022 daily.cld database is up-to-date (version: 26749, sigs: 2013718, f-level: 90, builder: raynman) main.cld database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr) bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2) # /usr/local/etc/rc.d/clamav-freshclam restart Stopping clamav_freshclam. Waiting for PIDS: 60994. Starting clamav_freshclam. # /usr/local/etc/rc.d/clamav-clamd restart Stopping clamav_clamd. Waiting for PIDS: 60896. Starting clamav_clamd. # /usr/local/etc/rc.d/clamav-milter restart Stopping clamav_milter. Waiting for PIDS: 61030. Waiting for clamd socket.. Starting clamav_milter. Waiting for clamav-milter socket.. # service postfix restart postfix/postfix-script: stopping the Postfix mail system postfix/postfix-script: starting the Postfix mail system
Thanks for sorting this out! Solved for me as well on 13.1. Great job.
(In reply to Yasuhiro Kimura from comment #24) Yes, but not with a pure stock system - I involved openssl from ports. I did the following: 1. Created clean 12.3-RELEASE jail 2. Obtained latest ports tree 3. Installed tomsfastmath as you did 4. Installed clamav with /etc/make.conf containing: OPTIONS_UNSET= X11 DOCS CUPS NLS DEFAULT_VERSIONS+= perl5=5.36 python=3.11 5. Started freshclam service 6. Started clamd service - *** SUCCESSFULLY STARTS *** 7. Shutdown clamd service 8. Modified /etc/make.conf to be: OPTIONS_UNSET= X11 DOCS CUPS NLS DEFAULT_VERSIONS+= perl5=5.36 python=3.11 ssl=openssl 9. Cleaned clamav (cd /usr/ports/security/clamav; make clean) 10. Re-installed clamav (cd /usr/ports/security/clamav; make reinstall), which also builds openssl, and then CMake uses libcrypto and libssl from /usr/local/lib when building clamav (instead of /usr/lib). 11. Started clamd service - FAILS TO START: LibClamAV debug: Loading databases from /var/db/clamav LibClamAV debug: in cli_cvdload() LibClamAV debug: MD5(.tar.gz) = f1820e08c2faed3d8e60382bcd9914f0 LibClamAV debug: cli_versig: Decoded signature: 00000000000000000000000000000000 12. Remove openssl from ports (pkg delete openssl) 13. Remove ssl=openssl from DEFAULT_VERSIONS in /etc/make.conf 14. Cleaned clamav (cd /usr/ports/security/clamav; make clean) 15. Re-installed clamav (cd /usr/ports/security/clamav; make reinstall) 16. Started clamd service - *** SUCCESSFULLY STARTS *** 17. Installed openssl from pkg (pkg install openssl) 18. Cleaned clamav (cd /usr/ports/security/clamav; make clean) 19. Re-installed clamav (cd /usr/ports/security/clamav; make reinstall) 20. Started clamd service - FAILS TO START, same error. 21. Remove tomsfastmath (pkg delete tomsfastmath) 22. Cleaned clamav (cd /usr/ports/security/clamav; make clean) 23. Re-installed clamav (cd /usr/ports/security/clamav; make reinstall) 24. Started clamd service - *** SUCCESSFULLY STARTS *** So it seems it's an interaction between security/openssl, math/tomsfastmath and security/clamav.
Created attachment 238795 [details] clamav: patch to avoid math/tomsfastmath incompatibility (In reply to Yasuhiro Kimura from comment #24) It is related to the header search location used by CMake when it detects security/openssl is installed. My previous patch was an error. I confirmed this a few ways. First method, I re-tried my submitted patch by itself, and it failed to fix the error. I hadn't tested it alone - I had tested it with a few other CMake flags, and obviously wasn't fully keeping track of which flag fixed it. Adding this (by itself), clamav builds correctly, but fails to start: -DCMAKE_IGNORE_PATH="/usr/local/include/tfm.h;/usr/local/lib/libtfm.a" Second method, I reverted the clamav Makefile, then developed a series of port patches that removed references to tfm.h and TOMSFASTMATH from clamav/bignum.h and clamav-config.cmake.in. Resulting clamav 1.0.0 builds successfully, but still errors out. Third method, I removed the patches that modified the clamav source code. I then moved the math/tomsfastmath header (tfm.h) out of /usr/local/include. Resulting clamav 1.0.0 builds and starts successfully. Fourth test, I re-installed the math/tomsfastmath header, and remove the math/tomsfastmath library from /usr/local/lib (libftm.a). Resulting clamav 1.0.0 builds successfully, but fails to start. Fifth method, I re-installed the math/tomsfastmath library, and then added the following to the clamav Makefile: -DOPENSSL_INCLUDE_DIR="/usr/local/include/openssl" Resulting clamav 1.0.0 builds successfully and starts successfully. My latest theory is the error is induced by CMake, when attempting to compile the bundled tomsfastmath in clamav, when security/openssl and math/tomsfastmath are both installed, because CMake adds this to the compile flags for the detected security/openssl: -I/usr/local/include. I cannot find a way to tell CMake to exclude the specific tfm.h header file that is confusing it.
(In reply to fsbruva from comment #27) Referencing your steps in comment #27 I succeeded to reproduce the problem with following steps. 1. Make clean install of 12.3-RELEASE amd64. 2. Checkout latest ports tree. 3. Add 'DEFAULT_VERSIONS+=ssl=openssl' to /etc/make.conf 4. Install TomsFastMath with `cd /usr/ports/math/tomsfastmath; make install`. 5. Install ClamAV with `cd /usr/ports/security/clamav; make install`. 6. Add 'clamav_freshclam_enable="YES"' to /etc/rc.conf. 7. Start freshclam with `service clamav-freshclam start`. So the problem happens when following 2 conditions are met. a. security/openssl ports is used as SSL library. b. math/tomsfastmath is installed before building ClamAV 1.0.0. As for the latter it is very likely as it was build dependency of 0.105.1. But the former surprises me a bit as only you mentioned it and I don't think a lot of user do it. But does this mean other users who faced the problem also use 'DEFAULT_VERSIONS+=ssl=openssl'? Anyway I'll chech your patch and commit it.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=689659cb4999c391f69400685fbacdafd943c1f7 commit 689659cb4999c391f69400685fbacdafd943c1f7 Author: Yasuhiro Kimura <yasu@FreeBSD.org> AuthorDate: 2022-12-15 05:02:04 +0000 Commit: Yasuhiro Kimura <yasu@FreeBSD.org> CommitDate: 2022-12-15 05:29:40 +0000 UPDATING: Add entry about possible runtime error of security/clamav and the solution PR: 268069 Reported by: many UPDATING | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
(In reply to fsbruva from comment #28) I tryied your latest patch and confirmed the problem doesn't happen. However, as I wrote in comment #29 it isn't clear to me if other reporters also use 'DEFAULT_VERSIONS+=ssl=openssl' and the problem never happens if OpenSSL in base system is used. On the other hand, it's almost certain that the problem never happens unless math/tomsfastmath is installed in the system that ClamAV is built. So rather than adding more flags to CMAKE_ARGS, I selected to adding entry to UPDATING how to fix the problem.
(In reply to Yasuhiro Kimura from comment #29) Yes, I also have 'DEFAULT_VERSIONS+=ssl=openssl' in /etc/make.conf. As you mention now in UPDATING, removing math/tomsfastmath seems to be the best solution so far.
(In reply to Yasuhiro Kimura from comment #31) That's a fair assessment. From my perspective, given the age of openssl in the 12.3-RELEASE system (1.1.1l, circa Aug 2021), it shouldn't be surprising that I and others are using the port version. The base openssl was built with numerous unsafe/obsolete ciphers, hashes and protocols, including: RC2, RC4, MD2, and MD4. This makes it unsuitable for a number of purposes, including as crypto engine in webserver. Upgrading to the latest in ports (1.1.1s, circa Nov 2022) closes the following vulnerabilities, among other bugs: CVE-2022-0778, CVE-2022-1292, CVE-2022-2068, CVE-2022-2097
Indeed, I'm also using OpenSSL from ports. # grep openssl /etc/make.conf DEFAULT_VERSIONS+=ssl=openssl