Bug 216249 - [NEW PORT] textproc/py-elasticsearch2: Python client for Elasticsearch (2.x)
Summary: [NEW PORT] textproc/py-elasticsearch2: Python client for Elasticsearch (2.x)
Status: Closed Overcome By Events
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Tobias Kortkamp
URL:
Keywords: feature, needs-patch, needs-qa
Depends on:
Blocks: 222847
  Show dependency treegraph
 
Reported: 2017-01-18 23:41 UTC by James Elstone
Modified: 2019-02-27 07:36 UTC (History)
6 users (show)

See Also:
james: maintainer-feedback+


Attachments
Shell archive of textproc/py-elasticsearch2-py (1.97 KB, text/plain)
2017-01-18 23:41 UTC, James Elstone
no flags Details
Shell archive of textproc/py-elasticsearch2-py (updated dependencies) (2.08 KB, patch)
2017-01-26 09:14 UTC, James Elstone
no flags Details | Diff
Shell archive of textproc/py-elasticsearch2-py (corrected dependencies) (2.08 KB, text/plain)
2017-02-07 21:52 UTC, James Elstone
james: maintainer-approval+
Details
Python client for Elasticsearch (2.x) (1.95 KB, text/plain)
2017-10-07 13:24 UTC, James Elstone
james: maintainer-approval+
Details
Successful Poudriere log for py27-elasticsearch2 (36.94 KB, text/plain)
2017-10-07 13:26 UTC, James Elstone
no flags Details
Python client for Elasticsearch (2.x) (1.95 KB, text/plain)
2017-10-07 14:42 UTC, James Elstone
no flags Details
Python client for Elasticsearch (2.x) (1.95 KB, text/plain)
2017-10-07 14:43 UTC, James Elstone
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description James Elstone 2017-01-18 23:41:50 UTC
Created attachment 179057 [details]
Shell archive of textproc/py-elasticsearch2-py

An instance of version 2.4.1 of textproc/py-elasticsearch-py port.

This is to allow applications that rely on connectivity to installations using elasticsearch v2.x.y (textproc/elasticsearch2), as per the directions found on https://elasticsearch-py.readthedocs.io/en/master/

"For Elasticsearch 2.0 and later, use the major version 2 (2.x.y) of the library."

The version 2 (elasticsearch>=2.3.0,<3.0.0) of the py-elasticsearch-py port is still required by textproc/py-elasticsearch-curator-py/ as an installation dependancy.

Portlint -A: OK
poudriere:   OK

Confirmed working with latest version of textproc/py-elasticsearch-curator-py and other python packages that rely on elasticsearch>=2.3.0,<3.0.0.
Comment 1 bagas 2017-01-25 18:03:20 UTC
Py27-elasticsearch-py-5.0.0, this port is not being.
Without a specific server port py27-elasticsearch-py-5.0.0

# pkg version | grep "ela"
elasticsearch2-2.4.2               =
py27-elasticsearch-curator-py-3.5.1_1 >

# curator --help
Traceback (most recent call last):
  File "/usr/local/bin/curator", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3019, in <module>
    @_call_aside
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3003, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3032, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 655, in _build_master
    ws.require(__requires__)
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 963, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 849, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'elasticsearch<3.0.0,>=2.3.0' distribution was not found and is required by elasticsearch-curator

# uname -rms
FreeBSD 10.3-RELEASE-p16 amd64

# pkg version | grep "py"
py27-Babel-2.3.4                   =
py27-Jinja2-2.8                    =
py27-MarkupSafe-0.23               =
py27-alabaster-0.7.6               =
py27-certifi-2016.9.26             =
py27-cffi-1.7.0                    =
py27-click-6.7                     =
py27-cryptography-1.6              =
py27-docutils-0.13.1               =
py27-elasticsearch-curator-py-3.5.1_1 >
py27-enum34-1.1.6                  =
py27-idna-2.0                      =
py27-imagesize-0.7.1               =
py27-ipaddress-1.0.18              =
py27-openssl-16.0.0                =
py27-pyasn1-0.1.9                  =
py27-pycparser-2.10                =
py27-pygments-2.1.3                =
py27-pysocks-1.6.5                 =
py27-pystemmer-1.3.0_1             =
py27-pytz-2016.10,1                =
py27-setuptools27-32.1.0           =
py27-six-1.10.0                    =
py27-snowballstemmer-1.2.0_1       =
py27-sphinx-1.4.8,1                =
py27-sphinx_rtd_theme-0.1.9        =
py27-urllib3-1.20                  =
python2-2_3                        =
python27-2.7.13_1                  =
Comment 2 James Elstone 2017-01-26 09:14:24 UTC
Created attachment 179325 [details]
Shell archive of textproc/py-elasticsearch2-py (updated dependencies)

As implied by bagas in comment #1, added dependency for elasticsearch v2.4.x in updated shell archive patch.

While elasticsearch2-py technically supports v2.x.x, the dependency has been limited to the v2.4.x branch, as that was the target of the test cases performed by submitter, and matches available versions in the ports tree.

Maintainer also updated as moving away from original source port ethos of not specifying the elasticsearch dependency. Note: The issue raised in comment #1 also exists in textproc/py-elasticsearch-py.
Comment 3 bagas 2017-01-27 04:58:14 UTC
Hello.
I did not know, what to do to creator earned with elastic 2.4.2?
Comment 4 James Elstone 2017-01-28 10:07:14 UTC
(In reply to bagas from comment #3)
Hi Bagas,

From https://pypi.python.org/pypi/elasticsearch/5.1.0

"The library is compatible with all Elasticsearch versions since 0.90.x but you have to use a matching major version:

For Elasticsearch 5.0 and later, use the major version 5 (5.x.y) of the library.

For Elasticsearch 2.0 and later, use the major version 2 (2.x.y) of the library.

For Elasticsearch 1.0 and later, use the major version 1 (1.x.y) of the library.

For Elasticsearch 0.90.x, use a version from 0.4.x releases of the library."

The latest major version for an ES engine 2.x is version 2.4.1 of the python client.

The port is intended to only be used with ES v2.X hence the dependency being added.

The blocked patch ( curator) does not support 5.x in an official release yet, see my comments to bug #216250, comment #5.

Does this answer your question?
Comment 5 bagas 2017-01-29 16:33:14 UTC
Hmm.
For Elasticsearch 2.0 and later, use the major version 2 (2.x.y) of the library.
Where can I get this version of the port?

In the ports system, I can't see it (Elasticsearch 2.0 and later, use the major version 2 (2.x.y)).

# pkg version | grep "ela"
elasticsearch2-2.4.2               =
py27-elasticsearch-curator-py-3.5.1 =
py27-elasticsearch-py-5.1.0        =

Only py27-elasticsearch-py-5.1.0

Ports updated.

# make search name=elasticsearc | egrep 'Port|Moved'
Port:	elasticsearch-1.7.5
Port:	elasticsearch-plugin-head-2015.12.16
Port:	elasticsearch-plugin-hq-1.0.0_1,1
Port:	elasticsearch-plugin-marvel-1.3.1
Port:	elasticsearch-plugin-migration-1.19
Port:	elasticsearch-plugin-sql-1.4.9
Port:	elasticsearch-river-rabbitmq-2.6.0_1
Port:	elasticsearch2-2.4.2
Port:	elasticsearch2-plugin-hq-2.0.3
Port:	elasticsearch2-plugin-marvel-2.1.0_2
Port:	elasticsearch2-plugin-migration-2.0.3
Port:	elasticsearch2-plugin-sql-2.4.2.1
Port:	elasticsearch5-5.0.2
Port:	elasticsearch5-x-pack-5.0.2
Port:	p5-Search-Elasticsearch-5.01_1
Port:	p5-Search-Elasticsearch-Client-0_90-5.01_1
Port:	p5-Search-Elasticsearch-Client-1_0-5.01_1
Port:	p5-Search-Elasticsearch-Client-2_0-5.01_1
Port:	py27-elasticsearch-curator-py-3.5.1
Port:	py27-elasticsearch-dsl-py-0.0.8
Port:	py27-elasticsearch-py-5.1.0
Port:	py27-pyelasticsearch-1.4
Port:	rubygem-elasticsearch-5.0.1
Port:	rubygem-elasticsearch-api-5.0.1
Port:	rubygem-elasticsearch-transport-5.0.1
Port:	textproc/p5-ElasticSearch
Moved:	textproc/p5-ElasticSearch-legacy
Port:	textproc/p5-Elasticsearch
Moved:	textproc/p5-Search-Elasticsearch
Port:	textproc/elasticsearch-plugin-marvel2
Moved:	textproc/elasticsearch2-plugin-marvel
Port:	textproc/p5-ElasticSearch-SearchBuilder
Moved:	textproc/p5-Search-Elasticsearch
Port:	textproc/p5-ElasticSearch-legacy
Moved:	textproc/p5-Search-Elasticsearch
Comment 6 James Elstone 2017-01-30 00:24:51 UTC
(In reply to bagas from comment #5)
Hi Bagas,

This new port (shell archive attached to this bug report) is a Python client (v2.4.1) for Elasticsearch which now has a dependency on "textproc/elasticsearch2 (v2.4.2).  v2.4.2 of the Elasticsearch Python client was never released.

The existing "textproc/py-elasticsearch-py" (v5.1.0) has a dependency on "textproc/elasticsearch5" (v5.0.2).

You have "py27-elasticsearch-curator-py-3.5.1" installed which requires an Elasticsearch python client of version v3 or less. You also have Elasticsearch v2.4.1 installed which is good, but the v5.x.y version of the Python Elasticsearch client.

I would suggest you uninstall "py27-elasticsearch-py-5.1.0" and give the attached new port a go, but only you will know if this is applicable to your situation; your responsible for your own system.

If you do give the attached port a try, any feedback would be welcomed!

Kr,
James
Comment 7 bagas 2017-01-30 16:21:11 UTC
Deleted py27-elasticsearch-py-5.1.0

The situation has not changed.

Updated ports.
Rebuild the port /usr/ports/textproc/py-elasticsearch-curator-py

In dependencies installed on the port py27-elasticsearch-py-5.1.0

byte-compiling /usr/ports/textproc/py-elasticsearch-curator-py/work/stage/usr/local/lib/python2.7/site-packages/curator/cli/snapshot.py to snapshot.pyc
byte-compiling /usr/ports/textproc/py-elasticsearch-curator-py/work/stage/usr/local/lib/python2.7/site-packages/curator/cli/snapshot_selection.py to snapshot_selection.pyc
byte-compiling /usr/ports/textproc/py-elasticsearch-curator-py/work/stage/usr/local/lib/python2.7/site-packages/curator/cli/utils.py to utils.pyc
writing byte-compilation script '/tmp/tmpXeilZx.py'
/usr/local/bin/python2.7 -O /tmp/tmpXeilZx.py
removing /tmp/tmpXeilZx.py
running install_egg_info
Copying elasticsearch_curator.egg-info to /usr/ports/textproc/py-elasticsearch-curator-py/work/stage/usr/local/lib/python2.7/site-packages/elasticsearch_curator-3.5.1-py2.7.egg-info
running install_scripts
Installing curator script to /usr/ports/textproc/py-elasticsearch-curator-py/work/stage/usr/local/bin
Installing es_repo_mgr script to /usr/ports/textproc/py-elasticsearch-curator-py/work/stage/usr/local/bin
writing list of installed files to '/usr/ports/textproc/py-elasticsearch-curator-py/work/.PLIST.pymodtmp'
====> Compressing man pages (compress-man)
===>  Installing for py27-elasticsearch-curator-py-3.5.1
===>   Registering installation for py27-elasticsearch-curator-py-3.5.1
[demo_b2bmotion_ru] Installing py27-elasticsearch-curator-py-3.5.1...
===>  Cleaning for py27-elasticsearch-py-5.1.0
===>  Cleaning for py27-elasticsearch-curator-py-3.5.1
Comment 8 bagas 2017-01-30 16:27:22 UTC
Delete port py27-elasticsearch-py-5.1.0
The situation has not changed.

Updated ports.
Rebuild the port /usr/ports/textproc/py-elasticsearch-curator-py/

In dependencies installed py27-elasticsearch-py-5.1.0 port.

byte-compiling /usr/ports/textproc/py-elasticsearch-curator-py/work/stage/usr/local/lib/python2.7/site-packages/curator/cli/snapshot_selection.py to snapshot_selection.pyc
byte-compiling /usr/ports/textproc/py-elasticsearch-curator-py/work/stage/usr/local/lib/python2.7/site-packages/curator/cli/utils.py to utils.pyc
writing byte-compilation script '/tmp/tmpXeilZx.py'
/usr/local/bin/python2.7 -O /tmp/tmpXeilZx.py
removing /tmp/tmpXeilZx.py
running install_egg_info
Copying elasticsearch_curator.egg-info to /usr/ports/textproc/py-elasticsearch-curator-py/work/stage/usr/local/lib/python2.7/site-packages/elasticsearch_curator-3.5.1-py2.7.egg-info
running install_scripts
Installing curator script to /usr/ports/textproc/py-elasticsearch-curator-py/work/stage/usr/local/bin
Installing es_repo_mgr script to /usr/ports/textproc/py-elasticsearch-curator-py/work/stage/usr/local/bin
writing list of installed files to '/usr/ports/textproc/py-elasticsearch-curator-py/work/.PLIST.pymodtmp'
====> Compressing man pages (compress-man)
===>  Installing for py27-elasticsearch-curator-py-3.5.1
===>   Registering installation for py27-elasticsearch-curator-py-3.5.1
[demo_local] Installing py27-elasticsearch-curator-py-3.5.1...
===>  Cleaning for py27-elasticsearch-py-5.1.0
===>  Cleaning for py27-elasticsearch-curator-py-3.5.1
Comment 9 bagas 2017-02-04 07:16:09 UTC
I tried to insert (elasticsearch2=2.4.2:textproc/elasticsearch2), did not help.
I had to comment on this line (elasticsearch2=2.4.[0-9]:textproc/elasticsearch2).
Otherwise error.

Hello.
Thank you, did you have specified.
# make reinstall clean
===>  Staging for py27-elasticsearch2-py-2.4.1
===>   py27-elasticsearch2-py-2.4.1 depends on package: py27-urllib3>=1.8.2 - found
===>   py27-elasticsearch2-py-2.4.1 depends on package: elasticsearch2=2.4.[0-9] - not found
===>  Installing for elasticsearch2-2.4.2
===>   elasticsearch2-2.4.2 depends on file: /usr/local/openjdk8/bin/java - found
===>   Registering installation for elasticsearch2-2.4.2 as automatic
[demo_local_ru] Installing elasticsearch2-2.4.2...
===> Creating groups.
Using existing group 'elasticsearch'.
===> Creating users
Using existing user 'elasticsearch'.
======================================================================

Please see /usr/local/etc/elasticsearch for sample versions of
elasticsearch.yml and logging.yml.

======================================================================

===>   py27-elasticsearch2-py-2.4.1 depends on package: elasticsearch2=2.4.[0-9] - not found
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/textproc/py-elasticsearch2-py
*** Error code 1

Stop.
make: stopped in /usr/ports/textproc/py-elasticsearch2-py

Curator started.

# curator --version
curator, version 3.5.1

Let's test this crutch.
On the production dangerous conclusions.
When adding port in the ports tree?
Comment 10 James Elstone 2017-02-07 21:52:10 UTC
Created attachment 179728 [details]
Shell archive of textproc/py-elasticsearch2-py (corrected dependencies)

Updated Shell Archive containing the correct dependency.

Explicit version of ES 2.4.2 now specified; (Pattern matching in RUN_DEPENDS not possible).
Comment 11 James Elstone 2017-02-07 21:59:48 UTC
Hi Bagas,

The string I added in RUN_DEPENDS ("[0-9]") was in error; my bad.

Have updated to explicitly reference the latest version of ES v2 (2.4.2) and attached a new patch to try.

I would recommend testing this patch on a system that is not in a production environment first, but subject to your testing and usage, use it as you see fit.

Make sure textproc/py-elasticsearch-py is removed before using this patch.

You should not have to remove textproc/py-elasticsearch2-py for this to work.

Thanks for the feedback.

James
Comment 12 James Elstone 2017-02-07 22:01:56 UTC
Correction:

You should not have to remove or reinstall textproc/py-elasticsearch-curator-py for Curator to work once the attached patch is installed.
Comment 13 Dan Langille freebsd_committer 2017-02-09 16:28:36 UTC
I'm sorry, but my time is overloaded at present.  I am unable to do anything immediately.
Comment 14 Kubilay Kocak freebsd_committer freebsd_triage 2017-09-18 04:30:10 UTC
Thank you for submitting this port James. A few review items:

Python ports should be named exactly after their PyPI counterparts [1], could you please update this patch to name the port accordingly:

svn directory: py-elasticsearch2
PORTNAME: elasticsearch
PKGNAMEPREFIX=${PYTHON_PKGNAMEPREFIX}
PKGNAMESUFFIX=2

Note: The main py-elasticsearch-py port needs to be renamed (MOVED) as well to its correct PyPI name (py-elasticsearch), separately, at some point.

- Use CHEESESHOP (PyPI) as MASTER_SITES by default unless there is a compelling (temporary) reason to use an alternative source, such as if tests or test data files are missing from the PyPI sdist. This ensures upstream python packaging & distribution processes remains robust/well-tested.

- Update COMMENT to match setup.py:description with appropriate tweak for version, if necessary (in this case), like:

COMMENT=Python client for Elasticsearch (2.x)

[1] https://wiki.freebsd.org/Python/PortsPolicy
Comment 15 Kubilay Kocak freebsd_committer freebsd_triage 2017-09-18 04:45:47 UTC
Actually, I note upstream has distributed the package (the 2.x branch) under a specific elasticsearch2 PyPI name [1], so port should be named:

svn directory: py-elasticsearch2
PORTNAME: elasticsearch2
PKGNAMEPREFIX=${PYTHON_PKGNAMEPREFIX}

The port should also be updated to the latest version (2.5.0 at the time of writing)

[1] https://pypi.python.org/pypi/elasticsearch2
Comment 16 Dan Langille freebsd_committer 2017-09-18 21:00:46 UTC
Thank you for this.

I am AFK for several more days. I appreciate your help.
Comment 17 James Elstone 2017-10-07 13:23:02 UTC
(In reply to Kubilay Kocak from comment #14)

Hi Kubilay,

Learning a lot; thanks for your time - It is appreciated.

1)	Moved the port to py-elasticsearch2

2)	Corrected Prefix and Suffix options

3)	Now using CHEESESHOP as MASTER_SITES

4)	Updated COMMENT to match description. 

5)	Updated to version 2.5.0 (latest at submittal)

6)	Corrected observations seen by Portlint –ACN; Looks OK now.

7)	Poudriere logs attached; build successful against 11.1-RELEASE-p1 and head/451459

As the proposed maintainer for this new port, Can this be committed now please?

Kind regards,

James
Comment 18 James Elstone 2017-10-07 13:24:19 UTC
Created attachment 186975 [details]
Python client for Elasticsearch (2.x)
Comment 19 James Elstone 2017-10-07 13:26:41 UTC
Created attachment 186976 [details]
Successful Poudriere log for py27-elasticsearch2

Successful Poudriere log against head/451459 on 11.1-RELEASE-p1 for textproc/py-elasticsearch2.
Comment 20 James Elstone 2017-10-07 14:42:22 UTC
Created attachment 186980 [details]
Python client for Elasticsearch (2.x)
Comment 21 James Elstone 2017-10-07 14:43:06 UTC
Created attachment 186981 [details]
Python client for Elasticsearch (2.x)
Comment 22 Tobias Kortkamp freebsd_committer 2019-02-27 07:36:22 UTC
I'm closing this since textproc/elasticsearch2 was removed in ports r472497.