Bug 259235 - comms/py-esptool: 2.5.1 -> 3.1.
Summary: comms/py-esptool: 2.5.1 -> 3.1.
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: Fukang Chen
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-10-17 20:48 UTC by Tomasz "CeDeROM" CEDRO
Modified: 2021-10-20 04:30 UTC (History)
2 users (show)

See Also:
bugzilla: maintainer-feedback? (loader)


Attachments
port update patch (2.49 KB, patch)
2021-10-17 20:48 UTC, Tomasz "CeDeROM" CEDRO
no flags Details | Diff
updated port shar (2.52 KB, application/x-shellscript)
2021-10-17 20:49 UTC, Tomasz "CeDeROM" CEDRO
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Tomasz "CeDeROM" CEDRO 2021-10-17 20:48:15 UTC
Created attachment 228788 [details]
port update patch

Hello world :-)

Here goes the update to py-esptool 2.5.1 -> 3.1 :-)

Please note that although all install dependencies are met, some extras dependencies are missing on FreeBSD (these need to be added as separate ports):
* flake8-future-import
* unittest-xml-reporting


Here is the full dependency list:

https://github.com/espressif/esptool/blob/master/setup.py


One test does not pass because of erratic `import StringIO` instead `from io import StringIO`. I have reported that already at upstream:

https://github.com/espressif/esptool/issues/677


Above issues does not block using esptool :-)
Comment 1 Tomasz "CeDeROM" CEDRO 2021-10-17 20:49:04 UTC
Created attachment 228789 [details]
updated port shar
Comment 2 Fukang Chen freebsd_committer 2021-10-19 02:47:54 UTC
Thanks for the patch.

+BUILD_DEPENDS= ${PYTHON_PKGNAMEPREFIX}wheel>0:devel/py-wheel@${PY_FLAVOR}

The wheel dependency was needed for installing from pip, and devel/py-wheel
is not required when installing from ports.
https://github.com/espressif/esptool/commit/4b14df383c179d9b9bea48c269869e434de878fe



 RUN_DEPENDS=   ${PYTHON_PKGNAMEPREFIX}pyserial>=3.0:comms/py-pyserial@${PY_FLAVOR} \
                ${PYTHON_PKGNAMEPREFIX}pyaes>0:security/py-pyaes@${PY_FLAVOR} \
-               ${PYTHON_PKGNAMEPREFIX}ecdsa>0:security/py-ecdsa@${PY_FLAVOR}
-TEST_DEPENDS=  ${PYTHON_PKGNAMEPREFIX}pytest>0:devel/py-pytest@${PY_FLAVOR} \
-               ${PYTHON_PKGNAMEPREFIX}pyelftools>0:devel/py-pyelftools@${PY_FLAVOR}
+               ${PYTHON_PKGNAMEPREFIX}ecdsa>0:security/py-ecdsa@${PY_FLAVOR}\
+               ${PYTHON_PKGNAMEPREFIX}pyelftools>0:devel/py-pyelftools@${PY_FLAVOR}\
+               ${PYTHON_PKGNAMEPREFIX}flake8>3.2.0:devel/py-flake8@${PY_FLAVOR}\
+               ${PYTHON_PKGNAMEPREFIX}flake8-import-order>0:devel/py-flake8-import-order@${PY_FLAVOR}\
+               ${PYTHON_PKGNAMEPREFIX}coverage>0:devel/py-coverage@${PY_FLAVOR}\
+               ${PYTHON_PKGNAMEPREFIX}bitstring>=3.1.6:devel/py-bitstring@${PY_FLAVOR}\
+               ${PYTHON_PKGNAMEPREFIX}cryptography>=2.1.4:security/py-cryptography@${PY_FLAVOR}\
+               ${PYTHON_PKGNAMEPREFIX}ecdsa>=0.16.0:security/py-ecdsa@${PY_FLAVOR}\
+               ${PYTHON_PKGNAMEPREFIX}reedsolo>=1.5.3,<=1.5.4:devel/py-reedsolo@${PY_FLAVOR}
+TEST_DEPENDS=  ${PYTHON_PKGNAMEPREFIX}pytest>0:devel/py-pytest@${PY_FLAVOR}

These are actually TEST_DEPENDS, they are only needed for running unittests
https://github.com/espressif/esptool/blob/v3.1/setup.py#L114-L123
   114	    extras_require={
   115	        "dev": [
   116	            'flake8>=3.2.0',
   117	            'flake8-future-import',
   118	            'flake8-import-order',
   119	            'pyelftools',
   120	            'unittest-xml-reporting<=2.5.2',  # the replacement of the old xmlrunner package (Python 2 comp. version)
   121	            'coverage',
   122	        ],
   123	    },



 ESPTOOL_BAUDRATE?=     115200
-ESPTOOL_CHIP?=         esp8266
-ESPTOOL_SERIALPORT?=   /dev/ttyU0
+ESPTOOL_CHIP?=         esp32c3
+ESPTOOL_SERIALPORT?=   /dev/cuaU0

These variables can be overridden with command line arguments, for example:
# make ESPTOOL_CHIP=esp32c3 ESPTOOL_SERIALPORT=/dev/cuaU0 test



The old do-test target just calls pytest to automatically collect and run 
unittests, this may not work with some of the new 3.1 unittests. 
test_espefuse.py is not supposed to run on a physical connected chip,
let's just skip this one.

https://github.com/espressif/esptool/blob/v3.1/test/test_espefuse.py#L1-L10
     1	#!/usr/bin/env python
     2	#
     3	# Tests for espefuse.py on ESP32
     4	#
     5	# IMPORTANT These are not designed to be run on a real ESP32 chip.
     6	#
     7	# If you force them to run on a real chip, you will corrupt
     8	# the chip's efuses on the first run and not get any useful
     9	# test results either.
    10	#

I was waiting for a new release that contains this commit I submitted a few months ago,
https://github.com/espressif/esptool/commit/74c55a8ffce4347a11479b07d50ccca33e0c0b8d
unittest test_espefuse_host.py was blocked on FreeBSD. I will add this patch too.
Comment 3 commit-hook freebsd_committer 2021-10-19 02:59:00 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=304a720bf8c4bb80ba4caf986f48f012e1595080

commit 304a720bf8c4bb80ba4caf986f48f012e1595080
Author:     Fukang Chen <loader@FreeBSD.org>
AuthorDate: 2021-10-19 02:19:49 +0000
Commit:     Fukang Chen <loader@FreeBSD.org>
CommitDate: 2021-10-19 02:56:14 +0000

    comms/py-esptool: Update to 3.1

    Changelog:

        https://github.com/espressif/esptool/releases/tag/v3.1

    PR:             259235
    Submitted by:   Tomasz "CeDeROM" CEDRO <tomek@cedro.info> (based on)
    MFH:            No (feature release)

 comms/py-esptool/Makefile | 35 +++++++++++++++++++++++++++--------
 comms/py-esptool/distinfo | 10 +++++++---
 2 files changed, 34 insertions(+), 11 deletions(-)
Comment 4 Fukang Chen freebsd_committer 2021-10-19 03:06:53 UTC
Committed, thanks!
Comment 5 Tomasz "CeDeROM" CEDRO 2021-10-19 03:16:57 UTC
Thanks loader :-)

I would prefer to leave ESPTOOL_SERIALPORT=/dev/cuaU0 as this makes tool use chips on the port with no additional parameters and this is more convenient during development :-)

Beware that do-test target IS DESTRUCTIVE on real chips what I did not know! Luckily there was bug in that part and my ESP32-C3 stayed functional (I hope lol) :-)

https://github.com/espressif/esptool/issues/677
Comment 6 Tomasz "CeDeROM" CEDRO 2021-10-19 15:27:09 UTC
Can we please reopen and update:

Clear prompt with default abort on do-test that tests are destructive will mess witch chip fuses and should not be run on real hardware?

It is already done by selecting non-existent serial port?

:-)
Comment 7 Fukang Chen freebsd_committer 2021-10-20 02:15:25 UTC
(In reply to Tomasz "CeDeROM" CEDRO from comment #6)

There's only one unittest file (test/test_esptool.py) that requires
to run on a physical connected chip for both version 2.5.1 and 3.1
in the ports tree.

test/test_espefuse.py doesn't exist in 2.5.1 upstream, and port
'make test' doesn't run this unittest file for 3.1 either.

https://cgit.freebsd.org/ports/commit/?id=304a720bf8c4bb80ba4caf986f48f012e1595080
 do-test:
-	@cd ${TEST_WRKSRC} && ${SETENV} ${TEST_ENV} ${PYTHON_CMD} -m pytest \
-		--ignore test/test_esptool.py
+	@cd ${TEST_WRKSRC} && for test in \
+		test/test_imagegen.py \
+		test/test_espsecure.py \
+		test/test_merge_bin.py \
+		test/test_modules.py; \
+		do ${SETENV} ${TEST_ENV} ${PYTHON_CMD} $${test}; done
+	@cd ${TEST_WRKSRC} && for chip in \
+		esp32 esp32s2 esp32s3beta2 esp32s3beta3 esp32c3; \
+		do ${SETENV} ${TEST_ENV} ${PYTHON_CMD} \
+		test/test_espefuse_host.py $${chip}; done
 .if exists(${ESPTOOL_SERIALPORT})
 	@cd ${TEST_WRKSRC} && ${SETENV} ${TEST_ENV} \
+		PYTHONPATH=${STAGEDIR}${PYTHONPREFIX_SITELIBDIR} \
 		${PYTHON_CMD} test/test_esptool.py \
 		${ESPTOOL_SERIALPORT} ${ESPTOOL_CHIP} ${ESPTOOL_BAUDRATE}
 .endif
Comment 8 Tomasz "CeDeROM" CEDRO 2021-10-20 04:30:50 UTC
Roger that :-) Thank you! :-)