Bug 242391 - sysutils/py-azure-cli: azcopy module doesn't grok FreeBSD
Summary: sysutils/py-azure-cli: azcopy module doesn't grok FreeBSD
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Danilo G. Baio
URL: https://github.com/Azure/azure-cli/is...
Keywords: needs-qa
Depends on:
Blocks:
 
Reported: 2019-12-02 21:27 UTC by Conrad Meyer
Modified: 2019-12-05 00:54 UTC (History)
0 users

See Also:
dbaio: maintainer-feedback+


Attachments
py-azure-cli-azcopy.patch (2.21 KB, patch)
2019-12-05 00:18 UTC, Danilo G. Baio
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Conrad Meyer freebsd_committer 2019-12-02 21:27:48 UTC
This small patch will at least permit the port to find azcopy in the same place as Linux.  Native Azcopy doesn't yet work on FreeBSD, but it probably just needs to be recognized as a weird variant of Linux.  The azcopy tool itself is MIT-licensed, with a variety of open source license dependencies.

Users could install the Linux azcopy binary in the same location and use the linuxulator to run it.  I don't know if that works.
Comment 1 Conrad Meyer freebsd_committer 2019-12-02 21:28:31 UTC
Um, forgot patch:

--- azure/cli/command_modules/storage/azcopy/util.py.orig        2019-12-02 13:03:52.425115000 -0800
+++ azure/cli/command_modules/storage/azcopy/util.py     2019-12-02 13:05:51.576481000 -0800
@@ -41,6 +41,8 @@
                 file_url = base_url.format('linux', 'tar.gz')
             elif self.system == 'Darwin':
                 file_url = base_url.format('darwin', 'zip')
+            elif self.system == 'FreeBSD':
+                raise CLIError('Azcopy ({}) binary not available, please build from https://github.com/Azure/azure-storage-azcopy and install in $HOME/bin/azcopy'.format(self.system))
             else:
                 raise CLIError('Azcopy ({}) does not exist.'.format(self.system))
             try:
@@ -163,7 +165,7 @@
         if not home_dir:
             return None
         install_location = os.path.join(home_dir, r'.azcopy\azcopy.exe')
-    elif system in ('Linux', 'Darwin'):
+    elif system in ('Linux', 'Darwin', 'FreeBSD'):
         install_location = os.path.expanduser(os.path.join('~', 'bin/azcopy'))
     else:
         install_location = None
Comment 2 Kubilay Kocak freebsd_committer freebsd_triage 2019-12-03 01:28:43 UTC
Thanks Conrad. Could you include your proposed patch as an attachment please. Also, does this change need to go upstream?
Comment 3 Conrad Meyer freebsd_committer 2019-12-03 01:33:37 UTC
(In reply to Kubilay Kocak from comment #2)
No and yes, respectively.
Comment 4 Conrad Meyer freebsd_committer 2019-12-03 01:34:33 UTC
(I set the URL to the upstream issue that happens to align with this bug; in addition, this PR to fix basic azcopy build on FreeBSD might be relevant to people following this bug: https://github.com/Azure/azure-storage-azcopy/pull/768 .)
Comment 5 Danilo G. Baio freebsd_committer 2019-12-04 01:04:38 UTC
(In reply to Conrad Meyer from comment #4)

Hi Conrad.

That is very nice, thank you for taking care of it.

I tried it on FreeBSD CURRENT and it's working well.

```
$ az storage blob sync -c {} --account-name {} -s {} -d {}
This command is in preview. It may be changed/removed in a future release.
Azcopy (FreeBSD) binary not available, please build from https://github.com/Azure/azure-storage-azcopy and install in $HOME/bin/azcopy
```

Then with it installed:

```
$ az storage blob sync -c {} --account-name {} -s {} -d {}
[...]
Job baf31108-xxxx-xxxx-xxxx-xxxxx Summary
Files Scanned at Source: 7
Files Scanned at Destination: 0
Elapsed Time (Minutes): 0.0334
Total Number Of Copy Transfers: 7
Number of Copy Transfers Completed: 7
Number of Copy Transfers Failed: 0
Number of Deletions at Destination: 0
Total Number of Bytes Transferred: 5465
Total Number of Bytes Enumerated: 5465
Final Job Status: Completed
```

Awesome.

About this patch on azure-cli, do you want to wait a few days for them to merge your azcopy PR?

what do you propose?

Maybe just point to your PR and forget the linuxulator thing.
```
Azcopy (FreeBSD) binary not available, please build from https://github.com/Azure/azure-storage-azcopy and install in $HOME/bin/azcopy
```

Regards.
Comment 6 Conrad Meyer freebsd_committer 2019-12-04 01:50:28 UTC
Hi Danilo,

The azcopy fix may be delayed for a while.  It's not clear to me when they plan to release 10.4.  I don't know if upstream azure-cli will take a version of the FreeBSD patch without buildable azcopy (I think there are reasonable tradeoffs either way).

IMO, it would be fine to patch this now and point to the azcopy PR for early adopters who are comfortable compiling themselves.  But it would be fine with me if you want to wait a little, too.

I'm not sure if Microsoft will integrate the freebsd azcopy binaries into their release process eventually or not.  It'd be great if they would, and the cli could just grab that like on other platforms.

Best,
Conrad
Comment 7 Danilo G. Baio freebsd_committer 2019-12-05 00:18:45 UTC
Created attachment 209707 [details]
py-azure-cli-azcopy.patch

I changed the instructions pointing users to a wiki page, feel free to change it.

https://wiki.freebsd.org/Ports/sysutils/py-azure-cli

I am ok in committing this, let me know if you want to change anything before.
Comment 8 Conrad Meyer freebsd_committer 2019-12-05 00:45:37 UTC
Looks perfect, thank you.
Comment 9 commit-hook freebsd_committer 2019-12-05 00:53:20 UTC
A commit references this bug:

Author: dbaio
Date: Thu Dec  5 00:52:53 UTC 2019
New revision: 519057
URL: https://svnweb.freebsd.org/changeset/ports/519057

Log:
  sysutils/py-azure-cli: Add experimental support for azcopy

  Azcopy binary is not available officially for FreeBSD and even if you have
  built it manually, azure-cli won't find/run it.

  This patch add FreeBSD as an available platform and now users can build
  and install azcopy manually to work with azure-cli.

  More info at https://wiki.freebsd.org/Ports/sysutils/py-azure-cli

  PR:		242391
  Submitted by:	cem

Changes:
  head/sysutils/py-azure-cli/Makefile
  head/sysutils/py-azure-cli/files/
  head/sysutils/py-azure-cli/files/patch-azure_cli_command__modules_storage_azcopy_util.py
Comment 10 Danilo G. Baio freebsd_committer 2019-12-05 00:54:27 UTC
(In reply to Conrad Meyer from comment #8)

Actually, thank you!

Regards.