After port upgrade to 1.9.0 I can't connect to all of my Windows 10 machines (I don't have any other, so I can't check if this is only W10 issue). rdekstop core dumps with Assertion failed: ((len * 2) < size), function _utils_data_to_hex, file utils.c, line 499 1.8.6 version works, so I assume something is wrong with 1.9.0
Try net/freerdp. :-D
Created attachment 219734 [details] patch-utils.c Try the attached patch and see if that fixes the issue for you. Note this was also reported on the ports mailing list and this patch was initially developed by Yuri Pankov, with a slight tweak by me. If this fixes your problem please let me know and I'll have the patch committed.
Hi Ivan, have you had a chance to try the port with the attached patch? Does this resolve your issue?
Hello, Greg, I've applied your patch and looks noting has improved. I have the same assertion failure.
Can you let me know what the setup is of the host you're connecting to (OS, version, patch level, special configs, etc)?
OS Name: Microsoft Windows 10 Enterprise N LTSC OS Version: 10.0.17763 N/A Build 17763 OS Configuration: Standalone Workstation OS Build Type: Multiprocessor Free System Model: BHYVE BIOS Version: BHYVE 1.00, 3/14/2014 Windows Directory: C:\Windows System Directory: C:\Windows\system32 Boot Device: \Device\HarddiskVolume2 Total Physical Memory: 2,047 MB Available Physical Memory: 635 MB Virtual Memory: Max Size: 3,081 MB Virtual Memory: Available: 573 MB Virtual Memory: In Use: 2,508 MB Page File Location(s): C:\pagefile.sys Hotfix(s): 7 Hotfix(s) Installed. [01]: KB4533013 [02]: KB4465065 [03]: KB4470788 [04]: KB4487038 [05]: KB4516115 [06]: KB4523204 [07]: KB4530715 No special config, just vanilla Windows. I've installed xfreerdp and it works perfectly. I'll try to increase digest size further.
I suspect that this is not rdesktop bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252548 try rebuild security/libtasn1 with CFLAGS+=-O1
I've increased digest buffer to 512 and was able to connect to windows box. Also, I've tried net/freerdp and it works without any problems.
(In reply to Ivan from comment #8) I found in mail list that gnutls_x509_crt_get_fingerprint() fails with -72 for rdesktop. Typical usage for gnutls_x509_crt_get_fingerprint(): int error; uint8_t digest[1024]; size_t digest_size; digest_size = sizeof(digest); error = gnutls_x509_crt_get_fingerprint(crt_list[0], GNUTLS_DIG_MD5, digest, &digest_size); on error digest_size not changed. I am does not see rdesktop code, but suspect that it does not proper handle error for gnutls_x509_crt_get_fingerprint(). Probably more proper fix would be add error handler and set digest_size to zero: error = gnutls_x509_crt_get_fingerprint(crt_list[0], GNUTLS_DIG_MD5, digest, &digest_size); if (0 != error) { digest_size = 0; }
Interesting, I had been trying to reproduce this for a while, but my test environment was a bit older. Ivan, does the fix Roman committed for the libtasn1 bug solve this issue for you?
(In reply to Greg Veldman from comment #10) Yes, I recompiled everything, looks like rdesktop works with latest libtasn1. Thanks!
The last followup was that the libtasn1 fix works, and there have been no further reports here. Can this be closed out?
I think so. Thanks, Greg.