After upgrading to FreeBSD 15, ctld behavior has changed compared to FreeBSD 12, 13. The same configuration works correctly on previous releases but fails on FreeBSD 15. Two issues are observed: ctld no longer allows multiple physical ports to be assigned to the same target. Frontend CAM target ports are not automatically set Online when ctld starts and must be enabled manually using ctladm. Steps to reproduce: Create the following /etc/ctl.conf: lun example_1 { path /dev/zvol/zroot/lun1 option naa 0x50015178f369f093 } target naa.50015178f369f092 { port isp0 port isp1 lun 0 example_1 } Start ctld with: service ctld onestart Actual result: ctld fails to start with the error: ctld: cannot set multiple physical ports for target "naa.50015178f369f092" ctld: configuration error; exiting If ctld is started with a single port, CAM target ports remain Offline until manually enabled with: ctladm port -o on Expected result: ctld should allow multiple physical ports (for example isp0 and isp1) to be assigned to a single target, as in FreeBSD 12–13. ctld should automatically bring frontend ports Online when the service starts, without requiring manual ctladm commands. Additional information: ctladm portlist -v output is identical on FreeBSD 12, 13, 14, and 15. The difference is only in runtime behavior on FreeBSD 15. The issue was reproduced with both QLogic ISP and Emulex ocs_fc drivers, indicating it is not driver-specific.
To help narrow down where a regression may have occurred, have you tested FreeBSD 14 as well? Or did you move from 13 directly to 15?
This was discussed outside of bugzilla, and mav believes that the regression was introduced in 969876fcee57ea1cb1c7b4d2ee757793cbfbe353 .
I have a patch to support multiple ports per target. It compiles, but I have no way to test it. The patch is available here: https://reviews.freebsd.org/D55767 The lack of enabling the port I'm a bit more puzzled by. In theory, conf::apply should call port::kernel_add which should bring the port up. My guess is that for some reason conf::apply thinks the port already exists in the old configuration. Can you please run ctld with syslog enabled to log all messages (including debug) for ctld to a log file? For example: ``` !ctld *.* /var/log/ctld.log ``` in /etc/syslog.d/ctld.log.
Hi, Multiple port issue seems fixed, however port online still there. as you said port::kernel_add() is only called for new ports, not existing ones. On first start, ports already exist in the kernel like this case not enabled. Also ctld reload case could be an issue in this situation. # ctld -d -f /etc/ctl.conf ctld: obtaining configuration from /etc/ctl.conf ctld: /etc/ctl.conf is world-readable ctld: auth-group "default" not defined; going with defaults ctld: portal-group "default" not defined; going with defaults ctld: transport-group "default" not defined; going with defaults ctld: opening pidfile /var/run/ctld.pid ctld: obtaining previously configured CTL luns from the kernel ctld: CTL port 0 "camsim" wasn't managed by ctld; ctld: CTL port 1 "ioctl" wasn't managed by ctld; ctld: CTL port 2 "tpc" wasn't managed by ctld; ctld: CTL port 3 "ocs_fc0" wasn't managed by ctld; ctld: CTL port 4 "ocs_fc1" wasn't managed by ctld; ctld: adding lun "example_1" ctld: adding port "default-naa.50015178f369f092" ctld: listening on 0.0.0.0, portal-group "default" ctld: listening on [::], portal-group "default" ctld: not listening on transport-group "default", not assigned to any target # ctladm portlist -v Port Online Frontend Name pp vp 0 NO camsim camsim 0 0 naa.5000000ccf2b8f01 Target: naa.5000000ccf2b8f00 All LUNs mapped port_type=8 1 YES ioctl ioctl 0 0 All LUNs mapped port_type=4 2 YES tpc tpc 0 0 All LUNs mapped port_type=8 3 NO camtgt ocs_fc0 0 0 All LUNs mapped port_type=1 4 NO camtgt ocs_fc1 0 0 All LUNs mapped port_type=1 5 YES iscsi iscsi 256 1 naa.50015178f369f092,t,0x0100 Target: naa.50015178f369f092 LUN 0: 0 port_type=16 cfiscsi_state=1 cfiscsi_target=naa.50015178f369f092 ctld_portal_group_name=default cfiscsi_portal_group_tag=256 # ctladm port -o on Front End Ports enabled # ctladm portlist -v Port Online Frontend Name pp vp 0 YES camsim camsim 0 0 naa.5000000ccf2b8f01 Target: naa.5000000ccf2b8f00 All LUNs mapped port_type=8 1 YES ioctl ioctl 0 0 All LUNs mapped port_type=4 2 YES tpc tpc 0 0 All LUNs mapped port_type=8 3 YES camtgt ocs_fc0 0 0 naa.100000109b223c61 Target: naa.200000109b223c61 All LUNs mapped port_type=1 4 YES camtgt ocs_fc1 0 0 naa.100000109b223c62 Target: naa.200000109b223c62 All LUNs mapped port_type=1 5 YES iscsi iscsi 256 1 naa.50015178f369f092,t,0x0100 Target: naa.50015178f369f092 LUN 0: 0 port_type=16 cfiscsi_state=1 cfiscsi_target=naa.50015178f369f092 ctld_portal_group_name=default cfiscsi_portal_group_tag=256
Just to help clarify, how are the physical ports being created? Is the driver adding them directly, are you using ctladm to create them? In particular, I assume the ports are already present before you start ctld the first time? ctld reload should be ok as the ports should in theory already be "up" so there's no need to do anything. Restarting ctld entirely won't remove/readd these ports though it seems, instead it seems like it just wants to do a disable/enable. My understanding of the code flow is that for these physical ports, conf_new_from_kernel() should add the port to `kports` without adding a corresponding port in the configuration returned by conf_new_from_kernel(). And we are seeing this log message: void add_iscsi_port(struct kports &kports, struct conf *conf, const struct cctl_port &port, std::string &name) { if (port.cfiscsi_target.empty()) { log_debugx("CTL port %u \"%s\" wasn't managed by ctld; ", port.port_id, name.c_str()); if (!kports.has_port(name)) { if (!kports.add_port(name, port.port_id)) { log_warnx("kports::add_port failed"); return; } } return; } Back in main(), this call should add ports to the `newconf`: if (!newconf->add_pports(kports)) log_errx(1, "Error associating physical ports; exiting"); Can you either use stepping with gdb/lldb or add some tracing to see if conf::add_pports() is actually adding ports? Something like: diff --git a/usr.sbin/ctld/ctld.cc b/usr.sbin/ctld/ctld.cc index 551a70b3fa8f..96e3885e92fe 100644 --- a/usr.sbin/ctld/ctld.cc +++ b/usr.sbin/ctld/ctld.cc @@ -2140,6 +2140,7 @@ conf::apply(struct conf *oldconf) port *newport = it->second.get(); if (newport->is_dummy()) { + log_debugx("skipping dummy port \"%s\"", name.c_str()); it++; continue; } @@ -2630,6 +2631,8 @@ conf::add_pports(struct kports &kports) "for %s", targ->label()); return (false); } + log_debugx("added ioctl port \"%s\" for %s", + pport.c_str(), targ->label()); continue; } @@ -2651,6 +2654,8 @@ conf::add_pports(struct kports &kports) pport.c_str(), targ->label()); return (false); } + log_debugx("added kernel port \"%s\" for %s", + pport.c_str(), targ->label()); } } return (true);
(In reply to John Baldwin from comment #5) I will try to capture with gdb next week. Meanwhile i realize that even ctl.conf target config port is ocs_fc, ctladm portlist show this target mapped to iscsi ! # cat /etc/ctl.conf lun example_1 { path /dev/zvol/zroot/lun1 option naa 0x50015178f369f093 } target naa.123456789 { port ocs_fc0 port ocs_fc1 lun 0 example_1 } # ctladm portlist -v Port Online Frontend Name pp vp 0 NO camsim camsim 0 0 naa.5000000a87f74f01 Target: naa.5000000a87f74f00 All LUNs mapped port_type=8 1 YES ioctl ioctl 0 0 All LUNs mapped port_type=4 2 YES tpc tpc 0 0 All LUNs mapped port_type=8 3 NO camtgt ocs_fc0 0 0 All LUNs mapped port_type=1 4 NO camtgt ocs_fc1 0 0 All LUNs mapped port_type=1 5 YES iscsi iscsi 256 1 naa.123456789,t,0x0100 Target: naa.123456789 LUN 0: 0 port_type=16 cfiscsi_state=1 cfiscsi_target=naa.123456789 ctld_portal_group_name=default cfiscsi_portal_group_tag=256
(In reply to Ken J. Thomson from comment #6) I do think that if ctld fails to create the actual internal ports for the physical ports, then any targets gets an "automatic" iscsi port created. So I think the issue is that for some reason the internal ports aren't being created for the physical ports. This would also explain why they aren't being enabled. I still don't know why they aren't being added though.
Reading symbols from ctld... Reading symbols from /usr/lib/debug//usr/sbin/ctld.debug... (gdb) break conf::apply Breakpoint 1 at 0x46965: file /usr/src/usr.sbin/ctld/ctld.cc, line 1967. (gdb) run -d -f /etc/ctl.conf Starting program: /usr/sbin/ctld -d -f /etc/ctl.conf ctld: obtaining configuration from /etc/ctl.conf ctld: /etc/ctl.conf is world-readable ctld: auth-group "default" not defined; going with defaults ctld: portal-group "default" not defined; going with defaults ctld: transport-group "default" not defined; going with defaults ctld: opening pidfile /var/run/ctld.pid ctld: obtaining previously configured CTL luns from the kernel ctld: CTL port 0 "camsim" wasn't managed by ctld; ctld: CTL port 1 "ioctl" wasn't managed by ctld; ctld: CTL port 2 "tpc" wasn't managed by ctld; ctld: CTL port 3 "ocs_fc0" wasn't managed by ctld; ctld: CTL port 4 "ocs_fc1" wasn't managed by ctld; ctld: added kernel port "ocs_fc0" for target "naa.123456789" ctld: added kernel port "ocs_fc1" for target "naa.123456789" Breakpoint 1, conf::apply (this=0x801a1f000, oldconf=0x801a1f1c0) at /usr/src/usr.sbin/ctld/ctld.cc:1967 1967 int cumulated_error = 0; (gdb) n 1969 if (oldconf->conf_debug != conf_debug) { (gdb) n 1982 if (!oldconf->conf_pidfile_path.empty() && (gdb) 2001 for (auto &kv : conf_portal_groups) { (gdb) n 2002 struct portal_group &newpg = *kv.second; (gdb) n 2004 if (newpg.tag() != 0) (gdb) n 2006 auto it = oldconf->conf_portal_groups.find(kv.first); (gdb) n 2007 if (it != oldconf->conf_portal_groups.end()) (gdb) n 2010 newpg.allocate_tag(); (gdb) n 2001 for (auto &kv : conf_portal_groups) { (gdb) n 2012 for (auto &kv : conf_transport_groups) { (gdb) n 2013 struct portal_group &newpg = *kv.second; (gdb) n 2015 if (newpg.tag() != 0) (gdb) n 2017 auto it = oldconf->conf_transport_groups.find(kv.first); (gdb) n 2018 if (it != oldconf->conf_transport_groups.end()) (gdb) n 2021 newpg.allocate_tag(); (gdb) n 2012 for (auto &kv : conf_transport_groups) { (gdb) n 2025 for (auto &kv : oldconf->conf_isns) { (gdb) n 2041 for (const auto &kv : oldconf->conf_ports) { (gdb) n 2065 for (auto it = oldconf->conf_luns.begin(); (gdb) n 2066 it != oldconf->conf_luns.end(); ) { (gdb) n 2065 for (auto it = oldconf->conf_luns.begin(); (gdb) n 2108 for (auto it = conf_luns.begin(); it != conf_luns.end(); ) { (gdb) n 2109 struct lun *newlun = it->second.get(); (gdb) n 2111 auto oldit = oldconf->conf_luns.find(it->first); (gdb) n 2112 if (oldit != oldconf->conf_luns.end()) { (gdb) n 2125 log_debugx("adding lun \"%s\"", newlun->name()); (gdb) n ctld: adding lun "example_1" 2126 if (!newlun->kernel_add()) { (gdb) n 2132 it++; (gdb) n 2108 for (auto it = conf_luns.begin(); it != conf_luns.end(); ) { (gdb) n 2138 for (auto it = conf_ports.begin(); it != conf_ports.end(); ) { (gdb) n 2139 const std::string &name = it->first; (gdb) print it->first.c_str() $1 = (const std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::value_type *) 0x801a2f161 "ocs_fc1-naa.123456789" (gdb) print it->second->is_dummy() $2 = true (gdb) print it->second->p_type There is no member or method named p_type. (gdb) break port::kernel_add Breakpoint 2 at 0x10a028e: file /usr/src/usr.sbin/ctld/kernel.cc, line 850. (gdb) continue Continuing. ctld: skipping dummy port "ocs_fc1-naa.123456789" ctld: skipping dummy port "ocs_fc0-naa.123456789" ctld: adding port "default-naa.123456789" Breakpoint 2, port::kernel_add (this=0x801a2f180) at /usr/src/usr.sbin/ctld/kernel.cc:850 850 struct target *targ = p_target;
Ah, try this fix (though I'm not currently sure why an iscsi port is created now): diff --git a/usr.sbin/ctld/ctld.hh b/usr.sbin/ctld/ctld.hh index 7ff6a0bc6d5f..7eb86e6cc535 100644 --- a/usr.sbin/ctld/ctld.hh +++ b/usr.sbin/ctld/ctld.hh @@ -269,7 +269,7 @@ struct port { virtual struct auth_group *auth_group() const { return nullptr; } virtual struct portal_group *portal_group() const { return nullptr; } - virtual bool is_dummy() const { return true; } + virtual bool is_dummy() const { return false; } virtual void clear_references(); Please let me see the output of ctladm portliest -v after trying this.
(In reply to John Baldwin from comment #9) As you said iscsi still creating the target. And ctld reload didnt enabled the ports. However ctld start now opened the ports after last patch. # ctladm portlist -v Port Online Frontend Name pp vp 0 NO camsim camsim 0 0 naa.5000000c7fee4b01 Target: naa.5000000c7fee4b00 All LUNs mapped port_type=8 1 YES ioctl ioctl 0 0 All LUNs mapped port_type=4 2 YES tpc tpc 0 0 All LUNs mapped port_type=8 3 NO camtgt ocs_fc0 0 0 All LUNs mapped port_type=1 4 NO camtgt ocs_fc1 0 0 All LUNs mapped port_type=1 # service ctld onestart Starting ctld. ctld: /etc/ctl.conf is world-readable [root@bsd15 ~]# ctladm portlist -v Port Online Frontend Name pp vp 0 NO camsim camsim 0 0 naa.5000000c7fee4b01 Target: naa.5000000c7fee4b00 All LUNs mapped port_type=8 1 YES ioctl ioctl 0 0 All LUNs mapped port_type=4 2 YES tpc tpc 0 0 All LUNs mapped port_type=8 3 YES camtgt ocs_fc0 0 0 naa.100000109b223c61 Target: naa.200000109b223c61 LUN 0: 0 port_type=1 4 NO camtgt ocs_fc1 0 0 All LUNs mapped port_type=1 5 YES iscsi iscsi 256 1 naa.123456789,t,0x0100 Target: naa.123456789 LUN 0: 0 port_type=16 cfiscsi_state=1 cfiscsi_target=naa.123456789 ctld_portal_group_name=default cfiscsi_portal_group_tag=25 #-- add second interface to target --# # cat /etc/ctl.conf lun example_1 { path /dev/zvol/zroot/lun1 option naa 0x50015178f369f093 } target naa.123456789 { port ocs_fc0 port ocs_fc1 lun 0 example_1 } root@bsd15 ~]# service ctld onereload [root@bsd15 ~]# ctladm portlist -v Port Online Frontend Name pp vp 0 NO camsim camsim 0 0 naa.5000000c7fee4b01 Target: naa.5000000c7fee4b00 All LUNs mapped port_type=8 1 YES ioctl ioctl 0 0 All LUNs mapped port_type=4 2 YES tpc tpc 0 0 All LUNs mapped port_type=8 3 YES camtgt ocs_fc0 0 0 naa.100000109b223c61 Target: naa.200000109b223c61 LUN 0: 0 port_type=1 4 NO camtgt ocs_fc1 0 0 All LUNs mapped port_type=1 5 YES iscsi iscsi 256 1 naa.123456789,t,0x0100 Target: naa.123456789 LUN 0: 0 port_type=16 cfiscsi_state=1 cfiscsi_target=naa.123456789 ctld_portal_group_name=default cfiscsi_portal_group_tag=256 # -- ctld reload do not enabled the port !!! [root@bsd15 ~]# service ctld onerestart Stopping ctld. Waiting for PIDS: 10307. Starting ctld. ctld: /etc/ctl.conf is world-readable [root@bsd15 ~]# ctladm portlist -v Port Online Frontend Name pp vp 0 NO camsim camsim 0 0 naa.5000000c7fee4b01 Target: naa.5000000c7fee4b00 All LUNs mapped port_type=8 1 YES ioctl ioctl 0 0 All LUNs mapped port_type=4 2 YES tpc tpc 0 0 All LUNs mapped port_type=8 3 YES camtgt ocs_fc0 0 0 naa.100000109b223c61 Target: naa.200000109b223c61 LUN 0: 0 port_type=1 4 YES camtgt ocs_fc1 0 0 naa.100000109b223c62 Target: naa.200000109b223c62 LUN 0: 0 port_type=1 5 YES iscsi iscsi 256 1 naa.123456789,t,0x0100 Target: naa.123456789 LUN 0: 0 port_type=16 cfiscsi_state=1 cfiscsi_target=naa.123456789 ctld_portal_group_name=default cfiscsi_portal_group_tag=256
I tried below approach, now iscsi didnt mapped to default portal group. target::verify() { if (t_auth_group == nullptr) { t_auth_group = t_conf->find_auth_group("default"); assert(t_auth_group != nullptr); } - if (t_ports.empty()){ + if (t_ports.empty() && t_pports.empty()) { struct portal_group *pg = default_portal_group(); assert(pg != NULL); t_conf->add_port(this, pg, nullptr); }
I tried below approach at ctld.cc, now iscsi didnt mapped to default portal group. target::verify() { if (t_auth_group == nullptr) { t_auth_group = t_conf->find_auth_group("default"); assert(t_auth_group != nullptr); } - if (t_ports.empty()){ + if (t_ports.empty() && t_pports.empty()) { struct portal_group *pg = default_portal_group(); assert(pg != NULL); t_conf->add_port(this, pg, nullptr); }
Created attachment 269544 [details] 4 patch to address current issues Can someone review them ? In my testing these are performed as expected. Thanks.
(In reply to Ken J. Thomson from comment #12) Yes, this is correct. I had thought this might be relevant and wanted to check when we added the ports relative to calling ::verify. I think your fix is the best way to handle this and will add it to my list of pending fixes.
So I had a branch with various fixes and some other refactors to cleanup things I thought were a bit dodgy I was hoping you could test. I think one of my refactors conflicts with your last fix which is around ignoring duplicate pp->link. I want to understand that case better though. I guess that is the case you are running into when reloading?
BTW, the branch of pending fixes is at https://github.com/bsdjhb/freebsd/tree/ctld_fixes
So I think your fix won't have the desired effect for reload. I think if you continue there, what will happen is that when re-parsing the configuration during reload, the existing ports will not be added in the "new" config, so conf::apply will end up removing the old ports (and thus disabling the physical ports). I think what we really want is to not have "linked" be a permanent property of the pport. Instead, inside of conf::add_pports we just want to catch duplicate assignments. I can actually use a local variable for this and provide a better error as I can then report the name of the first target when a port is claimed by multiple targets. This patch aims to fix that, but it does build on top my other refactor, so it might be easiest to pull my branch and test it instead as it won't apply directly to your tree: diff --git a/usr.sbin/ctld/ctld.cc b/usr.sbin/ctld/ctld.cc index b784c0e6f524..72001de6b5cb 100644 --- a/usr.sbin/ctld/ctld.cc +++ b/usr.sbin/ctld/ctld.cc @@ -1173,7 +1173,6 @@ conf::add_port(struct target *target, struct pport *pp) return (false); } - pp->link(); return (true); } @@ -2620,6 +2619,7 @@ conf_new_from_file(const char *path, bool ucl) bool conf::add_pports(struct kports &kports) { + std::unordered_map<struct pport *, struct target *> linked_ports; struct pport *pp; int ret, i_pp, i_vp; @@ -2633,11 +2633,13 @@ conf::add_pports(struct kports &kports) */ pp = kports.find_port(pport); if (pp != nullptr) { - if (pp->linked()) { + const auto &pair = linked_ports.try_emplace(pp, + targ); + if (!pair.second) { log_warnx("can't link port \"%s\" to " - "%s, port already linked to some " - "target", pport.c_str(), - targ->label()); + "%s, port already linked to %s", + pport.c_str(), targ->label(), + pair.first->second->label()); return (false); } diff --git a/usr.sbin/ctld/ctld.hh b/usr.sbin/ctld/ctld.hh index 7eb86e6cc535..b4d71b141fb2 100644 --- a/usr.sbin/ctld/ctld.hh +++ b/usr.sbin/ctld/ctld.hh @@ -569,13 +569,9 @@ struct pport { const char *name() const { return pp_name.c_str(); } uint32_t ctl_port() const { return pp_ctl_port; } - bool linked() const { return pp_linked; } - void link() { pp_linked = true; } - private: std::string pp_name; uint32_t pp_ctl_port; - bool pp_linked = false; }; struct kports {
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=7bb2b3801554a58039ed9d1fd05b65ce24c6c661 commit 7bb2b3801554a58039ed9d1fd05b65ce24c6c661 Author: John Baldwin <jhb@FreeBSD.org> AuthorDate: 2026-04-20 17:19:25 +0000 Commit: John Baldwin <jhb@FreeBSD.org> CommitDate: 2026-04-20 17:19:25 +0000 ctld: Support multiple physical ports in a target PR: 293076 Reported by: Ken J. Thomson <thomsonk@yandex.com> Reviewed by: asomers Fixes: 969876fcee57 ("ctld: parse config file independently of getting kernel info") Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D55767 usr.sbin/ctld/conf.cc | 4 +-- usr.sbin/ctld/conf.h | 2 +- usr.sbin/ctld/ctld.cc | 72 ++++++++++++++++++++++++----------------------- usr.sbin/ctld/ctld.hh | 11 ++++---- usr.sbin/ctld/parse.y | 2 +- usr.sbin/ctld/uclparse.cc | 2 +- 6 files changed, 47 insertions(+), 46 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=614ef718496eb6fd815ddcfde203bee8da1178b1 commit 614ef718496eb6fd815ddcfde203bee8da1178b1 Author: Ken J. Thomson <thomsonk@yandex.com> AuthorDate: 2026-04-23 19:15:48 +0000 Commit: John Baldwin <jhb@FreeBSD.org> CommitDate: 2026-04-23 19:15:48 +0000 ctld: Don't add an iscsi port for targets with only kernel ports PR: 293076 Reviewed by: asomers Fixes: 969876fcee57 ("ctld: parse config file independently of getting kernel info") Differential Revision: https://reviews.freebsd.org/D56523 usr.sbin/ctld/ctld.cc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=3df5cc33d894edd6b0ae87e51f0e35c3501fb907 commit 3df5cc33d894edd6b0ae87e51f0e35c3501fb907 Author: John Baldwin <jhb@FreeBSD.org> AuthorDate: 2026-04-23 19:16:08 +0000 Commit: John Baldwin <jhb@FreeBSD.org> CommitDate: 2026-04-23 19:16:08 +0000 ctld: Ports without a portal group are not dummy ports The default implementation of is_dummy should return false. Only portal group ports should possibly return true. PR: 293076 Reported by: Ken J. Thomson <thomsonk@yandex.com> Fixes: 6acc7afa34aa ("ctld: Convert struct port to a hierarchy of C++ classes") Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D56524 usr.sbin/ctld/ctld.hh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
I've merged several of the changes from this bug to main, but the last issue that remains I think is enabling a new port during a configuration reload. Can you please test two more patches? The first refactors some of the handling of kports and the second I think will fix this issue (and is the patch from comment 17)?
Created attachment 270072 [details] 0001-ctld-Refactor-ioctl-port-handling.patch
Created attachment 270073 [details] 0002-ctld-Only-check-physical-port-linking-within-a-singl.patch
There is a panic both in your repo and main freebsd repo, so didnt able to try your patch. panic: xpt_action: queued ccb and CAM_PRIORITY_NONE illegal. Reading symbols from /boot/kernel/kernel... Reading symbols from /usr/lib/debug//boot/kernel/kernel.debug... Reading symbols from /boot/kernel/ctl.ko... Reading symbols from /usr/lib/debug//boot/kernel/ctl.ko.debug... Reading symbols from /boot/kernel/zfs.ko... Reading symbols from /usr/lib/debug//boot/kernel/zfs.ko.debug... Reading symbols from /boot/kernel/ispfw.ko... Reading symbols from /usr/lib/debug//boot/kernel/ispfw.ko.debug... Reading symbols from /boot/kernel/hms.ko... Reading symbols from /usr/lib/debug//boot/kernel/hms.ko.debug... Reading symbols from /boot/kernel/hidmap.ko... Reading symbols from /usr/lib/debug//boot/kernel/hidmap.ko.debug... Reading symbols from /boot/kernel/ichsmb.ko... Reading symbols from /usr/lib/debug//boot/kernel/ichsmb.ko.debug... Reading symbols from /boot/kernel/smbus.ko... Reading symbols from /usr/lib/debug//boot/kernel/smbus.ko.debug... Reading symbols from /boot/kernel/ioat.ko... Reading symbols from /usr/lib/debug//boot/kernel/ioat.ko.debug... Reading symbols from /boot/kernel/pchtherm.ko... Reading symbols from /usr/lib/debug//boot/kernel/pchtherm.ko.debug... __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 57 __asm("movq %%gs:%c1,%0" : "=r" (td) (kgdb) #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 td = <optimized out> #1 doadump (textdump=textdump@entry=1) at /usr/src/sys/kern/kern_shutdown.c:399 error = 0 coredump = <optimized out> #2 0xffffffff80bdac30 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:519 once = 0 __pc = 0x0 #3 0xffffffff80bdb13d in vpanic (fmt=<optimized out>, ap=ap@entry=0xfffffe00cf5f2480) at /usr/src/sys/kern/kern_shutdown.c:974 buf = "xpt_action: queued ccb and CAM_PRIORITY_NONE illegal.", '\000' <repeats 202 times> __pc = 0x0 __pc = 0x0 __pc = 0x0 other_cpus = {__bits = {1099511595007, 0 <repeats 15 times>}} td = 0xfffff810a624f780 bootopt = <unavailable> newpanic = <optimized out> #4 0xffffffff80bdaf83 in panic (fmt=<unavailable>) at /usr/src/sys/kern/kern_shutdown.c:887 ap = {{gp_offset = 16, fp_offset = 48, overflow_arg_area = 0xfffffe00cf5f24b0, reg_save_area = 0xfffffe00cf5f2450}} #5 0xffffffff803a157b in xpt_action (start_ccb=<optimized out>) at /usr/src/sys/cam/cam_xpt.c:2545 No locals. #6 0xffffffff827605f0 in ctlferegister (periph=0xfffff80183e86000, arg=0xfffff8016a015280) at /usr/src/sys/cam/ctl/scsi_ctl.c:529 new_ccb = 0xfffff802959cf000 new_io = 0xfffffe032b590ac0 cmd_info = 0xfffff80183e4ac00
(In reply to Ken J. Thomson from comment #24) Ah, this appears to be fallout from an assertion added in 15.0-CURRENT. Please try this fix (also available for review at https://reviews.freebsd.org/D56995): diff --git a/sys/cam/ctl/scsi_ctl.c b/sys/cam/ctl/scsi_ctl.c index 68f1cabf6d07..6a55aba2669b 100644 --- a/sys/cam/ctl/scsi_ctl.c +++ b/sys/cam/ctl/scsi_ctl.c @@ -522,7 +522,8 @@ ctlferegister(struct cam_periph *periph, void *arg) new_ccb->ccb_h.io_ptr = new_io; LIST_INSERT_HEAD(&softc->atio_list, &new_ccb->ccb_h, periph_links.le); - xpt_setup_ccb(&new_ccb->ccb_h, periph->path, CAM_PRIORITY_NONE); + xpt_setup_ccb(&new_ccb->ccb_h, periph->path, + CAM_PRIORITY_NORMAL); new_ccb->ccb_h.func_code = XPT_ACCEPT_TARGET_IO; new_ccb->ccb_h.cbfcnp = ctlfedone; new_ccb->ccb_h.flags |= CAM_UNLOCKED; @@ -569,7 +570,8 @@ ctlferegister(struct cam_periph *periph, void *arg) new_ccb->ccb_h.io_ptr = new_io; LIST_INSERT_HEAD(&softc->inot_list, &new_ccb->ccb_h, periph_links.le); - xpt_setup_ccb(&new_ccb->ccb_h, periph->path, CAM_PRIORITY_NONE); + xpt_setup_ccb(&new_ccb->ccb_h, periph->path, + CAM_PRIORITY_NORMAL); new_ccb->ccb_h.func_code = XPT_IMMEDIATE_NOTIFY; new_ccb->ccb_h.cbfcnp = ctlfedone; new_ccb->ccb_h.flags |= CAM_UNLOCKED; @@ -1003,7 +1005,7 @@ ctlfe_requeue_ccb(struct cam_periph *periph, union ccb *ccb, int unlock) * target/lun. Reset the target and LUN fields back to the wildcard * values before we send them back down to the SIM. */ - xpt_setup_ccb_flags(&ccb->ccb_h, periph->path, CAM_PRIORITY_NONE, + xpt_setup_ccb_flags(&ccb->ccb_h, periph->path, CAM_PRIORITY_NORMAL, ccb->ccb_h.flags); xpt_action(ccb);
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=d1a8fa2e0f415f941e628f959fa0e70f23058fdb commit d1a8fa2e0f415f941e628f959fa0e70f23058fdb Author: John Baldwin <jhb@FreeBSD.org> AuthorDate: 2026-05-27 20:57:38 +0000 Commit: John Baldwin <jhb@FreeBSD.org> CommitDate: 2026-05-27 20:57:38 +0000 ctld: Only check physical port linking in a single configuration context Commit 969876fcee57 moved struct pport from being per-configuration to being a "global" object shared across multiple configurations. As a result, the check for duplicate ports actually spanned across configurations, such that reloading a configuration would now think that existing physical ports were already linked. The linking field in pport added in the C++-ification (commit 6acc7afa34aa) faithfully replicated this bug (albeit simpler as I had noticed that the TAILQ links weren't used after the earlier commit). To restore the desired behavior, remove the linking field from struct pport entirely and use a local unordered_map in conf::add_pports which tracks if a given pport is claimed by more than one target. PR: 293076 Reported by: Ken J. Thomson <thomsonk@yandex.com> Fixes: 969876fcee57 ("ctld: parse config file independently of getting kernel info") Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D57093 usr.sbin/ctld/ctld.cc | 12 +++++++----- usr.sbin/ctld/ctld.hh | 4 ---- 2 files changed, 7 insertions(+), 9 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=887841731be60a958e471b9fd79261169b67b7ad commit 887841731be60a958e471b9fd79261169b67b7ad Author: John Baldwin <jhb@FreeBSD.org> AuthorDate: 2026-05-27 20:58:05 +0000 Commit: John Baldwin <jhb@FreeBSD.org> CommitDate: 2026-05-27 20:58:05 +0000 ctl: Use CAM_PRIORITY_NORMAL for queued CCBs Previously this was using CAM_PRIORITY_NONE which tripped over the assertion added in b4b166b8c46b8. PR: 293076 Reported by: Ken J. Thomson <thomsonk@yandex.com> Reviewed by: imp Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D56995 sys/cam/ctl/scsi_ctl.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
I was able to do more testing today using ioctl ports and was able (I think) to reproduce the outstanding issue with the pport "linking" and verified that my patch to address that worked, so I've merged the various changes (as well as the kernel change to fix the panic) to main. I think if you pull from commit 887841731be60a958e471b9fd79261169b67b7ad that everything should work for you? The one thing I did still notice in my testing with ioctl ports is that when ctld exits, it always marks any pre-existing kernel port that was assigned to a target as offline as part of exiting, even if the port was online before ctld started. Possibly this is ok since listing a port in ctl.conf does imply that the user wants ctld to manage it? I do think ctld should be ignoring existing ports that are never mentioned in the config file.
I confirm the changes are fixed the issues, thanks.
A commit in branch stable/15 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=ec70458815ef5ccb3ef0b0abba2148192c8852fe commit ec70458815ef5ccb3ef0b0abba2148192c8852fe Author: Ken J. Thomson <thomsonk@yandex.com> AuthorDate: 2026-04-23 19:15:48 +0000 Commit: John Baldwin <jhb@FreeBSD.org> CommitDate: 2026-06-23 16:05:49 +0000 ctld: Don't add an iscsi port for targets with only kernel ports PR: 293076 Reviewed by: asomers Fixes: 969876fcee57 ("ctld: parse config file independently of getting kernel info") Differential Revision: https://reviews.freebsd.org/D56523 (cherry picked from commit 614ef718496eb6fd815ddcfde203bee8da1178b1) usr.sbin/ctld/ctld.cc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
A commit in branch stable/15 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=cb2d8649148f451ac7fc6f45870cd7d5aa78736a commit cb2d8649148f451ac7fc6f45870cd7d5aa78736a Author: John Baldwin <jhb@FreeBSD.org> AuthorDate: 2026-05-27 20:57:38 +0000 Commit: John Baldwin <jhb@FreeBSD.org> CommitDate: 2026-06-23 16:17:31 +0000 ctld: Only check physical port linking in a single configuration context Commit 969876fcee57 moved struct pport from being per-configuration to being a "global" object shared across multiple configurations. As a result, the check for duplicate ports actually spanned across configurations, such that reloading a configuration would now think that existing physical ports were already linked. The linking field in pport added in the C++-ification (commit 6acc7afa34aa) faithfully replicated this bug (albeit simpler as I had noticed that the TAILQ links weren't used after the earlier commit). To restore the desired behavior, remove the linking field from struct pport entirely and use a local unordered_map in conf::add_pports which tracks if a given pport is claimed by more than one target. PR: 293076 Reported by: Ken J. Thomson <thomsonk@yandex.com> Fixes: 969876fcee57 ("ctld: parse config file independently of getting kernel info") Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D57093 (cherry picked from commit d1a8fa2e0f415f941e628f959fa0e70f23058fdb) usr.sbin/ctld/ctld.cc | 12 +++++++----- usr.sbin/ctld/ctld.hh | 4 ---- 2 files changed, 7 insertions(+), 9 deletions(-)
A commit in branch stable/15 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=8071cdf83451f112ce17da3f429c8ae86699d7e5 commit 8071cdf83451f112ce17da3f429c8ae86699d7e5 Author: John Baldwin <jhb@FreeBSD.org> AuthorDate: 2026-04-20 17:19:25 +0000 Commit: John Baldwin <jhb@FreeBSD.org> CommitDate: 2026-06-23 16:05:37 +0000 ctld: Support multiple physical ports in a target PR: 293076 Reported by: Ken J. Thomson <thomsonk@yandex.com> Reviewed by: asomers Fixes: 969876fcee57 ("ctld: parse config file independently of getting kernel info") Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D55767 (cherry picked from commit 7bb2b3801554a58039ed9d1fd05b65ce24c6c661) usr.sbin/ctld/conf.cc | 4 +-- usr.sbin/ctld/conf.h | 2 +- usr.sbin/ctld/ctld.cc | 72 ++++++++++++++++++++++++----------------------- usr.sbin/ctld/ctld.hh | 11 ++++---- usr.sbin/ctld/parse.y | 2 +- usr.sbin/ctld/uclparse.cc | 2 +- 6 files changed, 47 insertions(+), 46 deletions(-)
A commit in branch stable/15 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=b6564c32ea988f738bdab11857b87f34b8189dd6 commit b6564c32ea988f738bdab11857b87f34b8189dd6 Author: John Baldwin <jhb@FreeBSD.org> AuthorDate: 2026-04-23 19:16:08 +0000 Commit: John Baldwin <jhb@FreeBSD.org> CommitDate: 2026-06-23 16:05:56 +0000 ctld: Ports without a portal group are not dummy ports The default implementation of is_dummy should return false. Only portal group ports should possibly return true. PR: 293076 Reported by: Ken J. Thomson <thomsonk@yandex.com> Fixes: 6acc7afa34aa ("ctld: Convert struct port to a hierarchy of C++ classes") Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D56524 (cherry picked from commit 3df5cc33d894edd6b0ae87e51f0e35c3501fb907) usr.sbin/ctld/ctld.hh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
A commit in branch stable/15 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=e61edb9eff065c2ec0c94e03f696e90c8454faa6 commit e61edb9eff065c2ec0c94e03f696e90c8454faa6 Author: John Baldwin <jhb@FreeBSD.org> AuthorDate: 2026-05-27 20:58:05 +0000 Commit: John Baldwin <jhb@FreeBSD.org> CommitDate: 2026-06-23 16:17:32 +0000 ctl: Use CAM_PRIORITY_NORMAL for queued CCBs Previously this was using CAM_PRIORITY_NONE which tripped over the assertion added in b4b166b8c46b8. PR: 293076 Reported by: Ken J. Thomson <thomsonk@yandex.com> Reviewed by: imp Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D56995 (cherry picked from commit 887841731be60a958e471b9fd79261169b67b7ad) sys/cam/ctl/scsi_ctl.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
All the fixes are merged to stable/15. They didn't make 15.1 but should be in 15.2.