Bug 222561 - sysutils/ansible 2.4.0.0 is broken
Summary: sysutils/ansible 2.4.0.0 is broken
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: Normal Affects Many People
Assignee: Nikolai Lifanov
URL:
Keywords: regression
Depends on:
Blocks:
 
Reported: 2017-09-24 16:28 UTC by gwright
Modified: 2017-09-25 22:50 UTC (History)
0 users

See Also:
bugzilla: maintainer-feedback? (lifanov)


Attachments
Proposed patch (494 bytes, patch)
2017-09-25 16:36 UTC, gwright
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description gwright 2017-09-24 16:28:08 UTC
The sysutils/ansible port is broken.  This happened when the version was bumped to 2.4.0.0; a four year old regression was introduced that broke ansible for almost every remote system type.

The error is in the stanza of the Makefile:

post-patch:
        ${FIND} ${WRKSRC} -type f | ${XARGS} ${REINPLACE_CMD} \
                -e 's|/usr/bin/python|${PYTHON_CMD}|g' \
                -e 's|/etc/ansible|${ETCDIR}|g' \
                -e 's|/usr/share/ansible|${DATADIR}|g'


We must not replace the interpreter path '/usr/bin/python' in any of the modules (which run on the remote system).  This has the effect of hardwiring the BSD python in path into the modules and prevents the 'anisble_python_interpreter' variable from specifying the python path on the remote.  (The 'ansible_python_interpreter' mechanism searches for the unedited, default interpreter path.  If it is changed, ansible will fail unless /usr/local/bin/python2.7 is present on the remote system.  This prevents, for example, configuring linux systems with ansible running on FreeBSD.

As I mentioned, this problem has bitten the BSDs before.  See the thread:

https://groups.google.com/forum/#!topic/ansible-project/oM2FNuumWmw

The 2.3.x series ports did not have this problem, but they did not mess with the python path.

Here is an example of the bug in action.  I am using vagrant with the ansible provisioner to set up an Ubuntu 14.04.5 environment.  Here is the invocation of the playbook (output of ansible -vvvvv ....):

    sc2-container-host-raw: Running ansible-playbook...
PYTHONUNBUFFERED=1 ANSIBLE_NOCOLOR=true ANSIBLE_HOST_KEY_CHECKING=false ANSIBLE_SSH_ARGS='-o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o ControlMaster=auto -o ControlPersist=60s' ansible-playbook --connection=ssh --timeout=30 --limit="sc2-container-host-raw" --inventory-file=/home/gitlab-runner/builds/e37c8767/0/spectrum-challenge/sc2-container-maker/.vagrant/provisioners/ansible/inventory -vvvv -e 'ansible_python_interpreter=/usr/bin/python' sc2-container-host.yml

The ansible_python_interpreter variable is set to /usr/bin/python (the path of python on the remote).  So modules should use that python as their interpreter on the remote.

However, the playbook fails the first time a python module tries to run on the remote because the BSD python path is hardwired:

fatal: [sc2-container-host-raw]: FAILED! => {
    "changed": false, 
    "failed": true, 
    "module_stderr": "OpenSSH_7.2p2, OpenSSL 1.0.2k-freebsd  26 Jan 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 93068\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to 127.0.0.1 closed.\r\n", 
    "module_stdout": "/bin/sh: 1: /usr/local/bin/python2.7: not found\r\n", 
    "msg": "MODULE FAILURE", 
    "rc": 0
}

If we must, the scripts that run on the local host (e.g., ansible and ansible-playbook) could have their python paths edited to the port/pkg installed python.  However, the modules must remain untouched.

Best Regards,
Greg Wright
Comment 1 Nikolai Lifanov freebsd_committer 2017-09-24 16:36:31 UTC
Thank you for the report and the background.
I'm going to take a look at it in a few hours.
Comment 2 gwright 2017-09-24 23:43:00 UTC
Bumped the importance to 'affects some people'.  I suspect I'm not the only one seeing this.
Comment 3 gwright 2017-09-25 16:36:21 UTC
Created attachment 186710 [details]
Proposed patch

Added proposed patch.  The patch simply drops the line which invoked sed to change the python path from "/usr/bin/python" to "/usr/local/bin/python2.7".  This change restores the pre-2.4.0.0 behavior.

It is important to understand that in ansible, python scripts are executed in two  places: on the "local" machine (the one doing the configuring) and on the "remote" machine (the one being configured).  The "ansible_python_interpreter" variable is substituted for the default "/usr/bin/python" path when a script is copied to the remote machine.  However, if the default path is changed (as it is in the 2.4.0.0_1 Makefile), the sed command that adjusts the remote python path fails.  Then invocation of the ansible module fails since a python interpreter is not found on the remote.

I have tested this patch with a quite complicated vagrant/ansible configuration and it appears to fix things.  The configuration completed normally.  Previously, ansible failed during the initial 'gathering facts' operation.
Comment 4 gwright 2017-09-25 16:41:59 UTC
I should add that in the test I referred to in comment #3, I was configuring a linux remote from a freebsd local machine.  The path problem may not show up when configuring a freebsd remote from a freebsd local, since the freebsd remote will typically have python at /usr/local/bin/python2.7.

This is the same case I was running when I originally saw the bug.
Comment 5 Nikolai Lifanov freebsd_committer 2017-09-25 21:53:57 UTC
I'm still working on a fix, but it's more complicated this time.
Comment 6 Nikolai Lifanov freebsd_committer 2017-09-25 22:18:48 UTC
The proposed patch by itself is not sufficient, but does part of the right thing.
Comment 7 Nikolai Lifanov freebsd_committer 2017-09-25 22:27:49 UTC
With this change, it works now:
https://reviews.freebsd.org/D12497
Comment 8 commit-hook freebsd_committer 2017-09-25 22:50:02 UTC
A commit references this bug:

Author: lifanov
Date: Mon Sep 25 22:49:41 UTC 2017
New revision: 450639
URL: https://svnweb.freebsd.org/changeset/ports/450639

Log:
  fix setting value for ansible_python_interpreter

  Thanks to mat for fixing qa.sh for this!

  PR:		222561
  Submitted by:	gwright@antiope.com
  Reported by:	gwright@antiope.com, Alex Mur (private email)

Changes:
  head/sysutils/ansible/Makefile
Comment 9 Nikolai Lifanov freebsd_committer 2017-09-25 22:50:25 UTC
Committed, thanks!