Bug 237211
| Summary: | [NEW PORT] devel/ghidra: Software reverse engineering (SRE) framework | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Ports & Packages | Reporter: | Tamas Szakaly <sghctoma> | ||||||||
| Component: | Individual Port(s) | Assignee: | Kurt Jaeger <pi> | ||||||||
| Status: | Closed FIXED | ||||||||||
| Severity: | Affects Only Me | CC: | koobs, pi, rgrimes, rhurlin, sghctoma, shawn.webb, yuri | ||||||||
| Priority: | --- | Keywords: | feature | ||||||||
| Version: | Latest | Flags: | pi:
maintainer-feedback+
|
||||||||
| Hardware: | amd64 | ||||||||||
| OS: | Any | ||||||||||
| Attachments: |
|
||||||||||
|
Description
Tamas Szakaly
2019-04-11 20:24:52 UTC
Did you test with poudriere ? It failed on 12.0amd64 early in the build, see https://people.freebsd.org/~pi/logs/devel__ghidra-120.txt This should go in CATEGORIES=security, with devel as a secondary category (In reply to Kurt Jaeger from comment #1) > Did you test with poudriere ? I've only tested it with Synth, but I realize now that it was a mistake, since it allows network connections after the fetch phase. I will look into how to fetch dependencies separately from building with Gradle. (In reply to Kubilay Kocak from comment #2) > This should go in CATEGORIES=security, with devel as a secondary category Yup, I also feel security would be a better category. I've used devel because radare2 is in devel, and wanted to be consistent. Tamas, Thank you for submitting this port. I have several questions/comments. What is the benefit of having the option selecting JDK11 vs. JDK12, why not just using once JDK? GH_PROJECT isn't needed when it is the same as PORTNAME. GH_TAGNAME= should be converted to DISTVERSIONPREFIX/DISTVERSION/DISTVERSIONSUFFIX It fails to build in poudriere. The immediate reason is: To honour the JVM settings for this build a new JVM will be forked. Please consider using the daemon: https://docs.gradle.org/5.0/userguide/gradle_daemon.html. Daemon will be stopped at the end of the build stopping after processing FAILURE: Build failed with an exception. * Where: Build file '/wrkdirs/usr/ports/devel/ghidra/work/ghidra-Ghidra_9.0.2_build/Ghidra/Features/Python/build.gradle' line: 28 * What went wrong: A problem occurred evaluating project ':Python'. > Could not resolve all files for configuration ':Python:jython'. > Could not resolve org.python:jython-standalone:2.7.1. Required by: project :Python > Could not resolve org.python:jython-standalone:2.7.1. > Could not get resource 'https://repo.maven.apache.org/maven2/org/python/jython-standalone/2.7.1/jython-standalone-2.7.1.pom'. > Could not GET 'https://repo.maven.apache.org/maven2/org/python/jython-standalone/2.7.1/jython-standalone-2.7.1.pom'. > repo.maven.apache.org > Could not resolve org.python:jython-standalone:2.7.1. > Could not get resource 'https://jcenter.bintray.com/org/python/jython-standalone/2.7.1/jython-standalone-2.7.1.pom'. > Could not GET 'https://jcenter.bintray.com/org/python/jython-standalone/2.7.1/jython-standalone-2.7.1.pom'. > jcenter.bintray.com The other concern that I have is that gradle should be called with --offline during build, otherwise it would be downloading things which isn't allowed. Please see biology/gatk as an example which uses gradle and is, IMO, a good gradle-based port implementation. (In reply to Yuri Victorovich from comment #4) Yuri, Thank you for the feedback, and sorry for the late response. I was, and still am swamped with work, but my next week is free, and I will finally have the time to fix these issues. To answer your questions: > What is the benefit of having the option selecting JDK11 vs. JDK12, why not just using once JDK? I didn't want to force JDK12 on somebody who might need JDK11 for whatever reason, and vice versa. > GH_PROJECT isn't needed when it is the same as PORTNAME. Yes, you are right. And I've just noticed I also have GH_TUPLE defined - I don't really remember, but probably wanted to switch to GH_TUPLE at one point, and forgot to the delete the other GH_ defines. > GH_TAGNAME= should be converted to DISTVERSIONPREFIX/DISTVERSION/DISTVERSIONSUFF Thanks, this makes the Makefile a bit tidier. > It fails to build in poudriere. ... > The other concern that I have is that gradle should be called with --offline during build, otherwise it would be downloading things which isn't allowed. Yes, the poudriere build fails because the port tries to download dependencies from Maven and JCentral during the build phase. I had some ideas (https://twitter.com/sghctoma/status/1123103954227929088) on how to resolve this, but somehow I didn't thought about the simple solution biology/gatk uses (host the dependencies as a separate tarball). One question though: would it be OK if I host the dependency tarball on my GitHub? Or is there a way to get such files hosted on the FreeBSD distfile cache sites? Regards, Toma (In reply to Tamas Szakaly from comment #5) Please see the port science/opsin for how to solve the dependencies issue. Yuri (In reply to Kubilay Kocak from comment #2) No, this should go in devel, all devel tools are usable for security purposes, not all security tools are developer tools. Reverse engineering tools are a class of developer tools, just because I am reverse engineering does not mean there is any secuirty concern at all. (In reply to Rodney W. Grimes from comment #7) I won't die in/on the CATEGORIES bikeshed/hill, but I'll just note that (ports) try to lean towards using devel as a category of last resort unless something is entirely and only development related, or there isn't somewhere more specific. There's plenty of development-only software in !devel categories. I also think its safe to assume that people would go looking for this in security as a first port of call, but the "development" class is also taken care of in my initial suggestion to use 'devel' it as a secondary category. But again, having said that, CATEGORIES=devel security is fine too, and this might be a good example of something that is 'more in devel' than it is 'security'. Feedback timeout expired: 20 days, more than the threshold of 14 days. @Tamas Please let us know that you 'can' provide an updated and tested patch within a reasonable timeframe (In reply to Kubilay Kocak from comment #10) What is the timeout policy for things like this? I thought that it is 14 days, which has expired. Yes, but some ports are more relevant than others. So having ghidra in the tree sounds like a valid reason to be liberal in feedback timeout values 8-} (In reply to Kurt Jaeger from comment #12) I was going to add it, but now I have to wait for another timeout to expire. On which timeout do you have to wait ? Please, just proceed. No need to be pedantic on those new ports. (In reply to Kurt Jaeger from comment #14) Ok, thanks! Hi, Sorry for my absence, life got in the way. I will upload a patch this weekend that fixes the issues mentioned here, and also updates the port to version 9.0.4. Created attachment 205307 [details]
ghidra-9.0.4.shar
I've uploaded a new .shar archive that addresses the issues mentioned in this thread, and also updates Ghidra to version 9.0.4.
To summarize the changes:
- Changed CATEGORIES to "devel security".
- Defined DISTVERSIONPREFIX and DISTVERSIONSUFFIX, which made GH_PROJECT/GH_TAGNAME unnecessary.
- Fixed the Poudriere build issue. I've copied relevant parts from science/opsin to achieve this. Both ports recommended here as examples (science/opsin and biology/gatk) use the LOCAL master site to store Maven/Gradle dependencies, which refers to the FreeBSD distfile cache sites. I don't know where and how to ask for a distfile to be uploaded there, so the port currently uses my GitHub for this purpose (line #72).
Thanks for your patience, and please let me know if any other changes are required!
testbuilds@work builds on 13-amd64, fails on 12.0 or 11.2/3-amd64. build error seems to be gradle-related ? See for example: https://people.freebsd.org/~pi/logs/devel__ghidra-120-1561360943.txt (In reply to Tamas Szakaly from comment #17) Can you tell which FreeBSD version was used in the poudriere test build ? (In reply to Kurt Jaeger from comment #20) The machine itself runs HardenedBSD 13-CURRENT, but the poudriere jail is 12.0-RELEASE-p6 amd64. I'll install a 12.0 VM to try to replicate this error. Created attachment 205345 [details]
ghidra-9.0.4_v2.shar
A closer look at the log file revealed that the error was caused by the Poudriere settings BUILD_AS_NON_ROOT=yes/PORTBUILD_USER=nobody. The build process tries to write a log file under the user's home directory (/nonexistent in case of nobody), which of course fails. I didn't run into this, because I've had CCACHE_DIR defined, which cancels BUILD_AS_NON_ROOT.
The solution was to override the "user.home" Java system variable in the Gradle build file. I've built this new version successfully in 11.2-RELEASE-p10, 12.0-RELEASE-p6 and 13.0-CURRENT jails with BUILD_AS_NON_ROOT enabled.
testbuilds@work I am swamped this week. Will try to look into it this weekend. testbuilds are fine on 11.2a, 11.3-rc3-a, 12.0, current 8-) Yuri, if it's OK, I'd commit it before Q3 ? Ok! Committed, thanks very much! Cool project! A commit references this bug: Author: pi Date: Sat Jun 29 00:17:52 UTC 2019 New revision: 505315 URL: https://svnweb.freebsd.org/changeset/ports/505315 Log: New port: devel/ghidra Ghidra is a software reverse engineering (SRE) framework created and maintained by the National Security Agency Research Directorate of the United States of America. This framework includes a suite of full-featured, high-end software analysis tools that enable users to analyze compiled code on a variety of platforms including Windows, macOS, and Linux. Capabilities include disassembly, assembly, decompilation, graphing, and scripting, along with hundreds of other features. Ghidra supports a wide variety of processor instruction sets and executable formats and can be run in both user-interactive and automated modes. Users may also develop their own Ghidra plug-in components and/or scripts using Java or Python. WWW: https://ghidra-sre.org/ PR: 237211 Submitted by: Tamas Szakaly <sghctoma@gmail.com> Reviewed by: koobs, yuri, rgrimes Changes: head/devel/Makefile head/devel/ghidra/ head/devel/ghidra/Makefile head/devel/ghidra/distinfo head/devel/ghidra/files/ head/devel/ghidra/files/patch-GPL_CabExtract_build.gradle head/devel/ghidra/files/patch-GPL_DemanglerGnu_build.gradle head/devel/ghidra/files/patch-GPL_build.gradle head/devel/ghidra/files/patch-GPL_nativeBuildProperties.gradle head/devel/ghidra/files/patch-GhidraBuild_Skeleton_certification.manifest head/devel/ghidra/files/patch-GhidraBuild_Skeleton_os_freebsd64_README.txt head/devel/ghidra/files/patch-Ghidra_Features_Decompiler_build.gradle head/devel/ghidra/files/patch-Ghidra_Features_Decompiler_src_decompile_cpp_Makefile head/devel/ghidra/files/patch-Ghidra_Features_Decompiler_src_decompile_cpp_loadimage__bfd.hh head/devel/ghidra/files/patch-Ghidra_Features_Decompiler_src_decompile_cpp_types.h head/devel/ghidra/files/patch-Ghidra_Features_FunctionID_build.gradle head/devel/ghidra/files/patch-Ghidra_Framework_Generic_src_main_java_ghidra_framework_Platform.java head/devel/ghidra/files/patch-Ghidra_Framework_Utility_src_main_java_ghidra_framework_OperatingSystem.java head/devel/ghidra/files/patch-Ghidra_RuntimeScripts_Linux_support_launch.sh head/devel/ghidra/files/patch-build.gradle head/devel/ghidra/files/patch-gradleScripts_distribution.gradle head/devel/ghidra/files/patch-gradleScripts_ip.gradle head/devel/ghidra/pkg-descr (In reply to Kurt Jaeger from comment #27) Thanks for committing, and thanks to everyone who helped! |
