Bug 222561 - sysutils/ansible is broken
Summary: sysutils/ansible is broken
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: Normal Affects Many People
Assignee: Nikolai Lifanov
Keywords: regression
Depends on:
Reported: 2017-09-24 16:28 UTC by gwright
Modified: 2017-09-25 22:50 UTC (History)
0 users

See Also:
bugzilla: maintainer-feedback? (lifanov)

Proposed patch (494 bytes, patch)
2017-09-25 16:36 UTC, gwright
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description gwright 2017-09-24 16:28:08 UTC
The sysutils/ansible port is broken.  This happened when the version was bumped to; a four year old regression was introduced that broke ansible for almost every remote system type.

The error is in the stanza of the Makefile:

        ${FIND} ${WRKSRC} -type f | ${XARGS} ${REINPLACE_CMD} \
                -e 's|/usr/bin/python|${PYTHON_CMD}|g' \
                -e 's|/etc/ansible|${ETCDIR}|g' \
                -e 's|/usr/share/ansible|${DATADIR}|g'

We must not replace the interpreter path '/usr/bin/python' in any of the modules (which run on the remote system).  This has the effect of hardwiring the BSD python in path into the modules and prevents the 'anisble_python_interpreter' variable from specifying the python path on the remote.  (The 'ansible_python_interpreter' mechanism searches for the unedited, default interpreter path.  If it is changed, ansible will fail unless /usr/local/bin/python2.7 is present on the remote system.  This prevents, for example, configuring linux systems with ansible running on FreeBSD.

As I mentioned, this problem has bitten the BSDs before.  See the thread:


The 2.3.x series ports did not have this problem, but they did not mess with the python path.

Here is an example of the bug in action.  I am using vagrant with the ansible provisioner to set up an Ubuntu 14.04.5 environment.  Here is the invocation of the playbook (output of ansible -vvvvv ....):

    sc2-container-host-raw: Running ansible-playbook...
PYTHONUNBUFFERED=1 ANSIBLE_NOCOLOR=true ANSIBLE_HOST_KEY_CHECKING=false ANSIBLE_SSH_ARGS='-o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o ControlMaster=auto -o ControlPersist=60s' ansible-playbook --connection=ssh --timeout=30 --limit="sc2-container-host-raw" --inventory-file=/home/gitlab-runner/builds/e37c8767/0/spectrum-challenge/sc2-container-maker/.vagrant/provisioners/ansible/inventory -vvvv -e 'ansible_python_interpreter=/usr/bin/python' sc2-container-host.yml

The ansible_python_interpreter variable is set to /usr/bin/python (the path of python on the remote).  So modules should use that python as their interpreter on the remote.

However, the playbook fails the first time a python module tries to run on the remote because the BSD python path is hardwired:

fatal: [sc2-container-host-raw]: FAILED! => {
    "changed": false, 
    "failed": true, 
    "module_stderr": "OpenSSH_7.2p2, OpenSSL 1.0.2k-freebsd  26 Jan 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 93068\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to closed.\r\n", 
    "module_stdout": "/bin/sh: 1: /usr/local/bin/python2.7: not found\r\n", 
    "msg": "MODULE FAILURE", 
    "rc": 0

If we must, the scripts that run on the local host (e.g., ansible and ansible-playbook) could have their python paths edited to the port/pkg installed python.  However, the modules must remain untouched.

Best Regards,
Greg Wright
Comment 1 Nikolai Lifanov freebsd_committer 2017-09-24 16:36:31 UTC
Thank you for the report and the background.
I'm going to take a look at it in a few hours.
Comment 2 gwright 2017-09-24 23:43:00 UTC
Bumped the importance to 'affects some people'.  I suspect I'm not the only one seeing this.
Comment 3 gwright 2017-09-25 16:36:21 UTC
Created attachment 186710 [details]
Proposed patch

Added proposed patch.  The patch simply drops the line which invoked sed to change the python path from "/usr/bin/python" to "/usr/local/bin/python2.7".  This change restores the pre- behavior.

It is important to understand that in ansible, python scripts are executed in two  places: on the "local" machine (the one doing the configuring) and on the "remote" machine (the one being configured).  The "ansible_python_interpreter" variable is substituted for the default "/usr/bin/python" path when a script is copied to the remote machine.  However, if the default path is changed (as it is in the Makefile), the sed command that adjusts the remote python path fails.  Then invocation of the ansible module fails since a python interpreter is not found on the remote.

I have tested this patch with a quite complicated vagrant/ansible configuration and it appears to fix things.  The configuration completed normally.  Previously, ansible failed during the initial 'gathering facts' operation.
Comment 4 gwright 2017-09-25 16:41:59 UTC
I should add that in the test I referred to in comment #3, I was configuring a linux remote from a freebsd local machine.  The path problem may not show up when configuring a freebsd remote from a freebsd local, since the freebsd remote will typically have python at /usr/local/bin/python2.7.

This is the same case I was running when I originally saw the bug.
Comment 5 Nikolai Lifanov freebsd_committer 2017-09-25 21:53:57 UTC
I'm still working on a fix, but it's more complicated this time.
Comment 6 Nikolai Lifanov freebsd_committer 2017-09-25 22:18:48 UTC
The proposed patch by itself is not sufficient, but does part of the right thing.
Comment 7 Nikolai Lifanov freebsd_committer 2017-09-25 22:27:49 UTC
With this change, it works now:
Comment 8 commit-hook freebsd_committer 2017-09-25 22:50:02 UTC
A commit references this bug:

Author: lifanov
Date: Mon Sep 25 22:49:41 UTC 2017
New revision: 450639
URL: https://svnweb.freebsd.org/changeset/ports/450639

  fix setting value for ansible_python_interpreter

  Thanks to mat for fixing qa.sh for this!

  PR:		222561
  Submitted by:	gwright@antiope.com
  Reported by:	gwright@antiope.com, Alex Mur (private email)

Comment 9 Nikolai Lifanov freebsd_committer 2017-09-25 22:50:25 UTC
Committed, thanks!