Bug 213769 - [New PORT] misc/py-glance: Openstack Image Service
Summary: [New PORT] misc/py-glance: Openstack Image Service
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Roman Bogorodskiy
Depends on:
Reported: 2016-10-25 13:55 UTC by Alexander Nusov
Modified: 2016-11-02 16:20 UTC (History)
1 user (show)

See Also:

glance dependencies (18.50 KB, patch)
2016-10-25 13:55 UTC, Alexander Nusov
no flags Details | Diff
glance service (11.30 KB, patch)
2016-10-25 13:56 UTC, Alexander Nusov
no flags Details | Diff
Add OPTIONs for memcached and mysql (725 bytes, patch)
2016-10-26 20:39 UTC, Alexander Nusov
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Nusov 2016-10-25 13:55:55 UTC
Created attachment 176141 [details]
glance dependencies

This patch adds the glance, Openstack Image Service from Newton release.

New packages:

new uids/gids:

new rc services:

Information how to install the glance you may find here
Comment 1 Alexander Nusov 2016-10-25 13:56:19 UTC
Created attachment 176142 [details]
glance service
Comment 2 Roman Bogorodskiy freebsd_committer 2016-10-26 16:49:07 UTC
(In reply to Alexander Nusov from comment #1)

Have you considered adding OPTIONs for mysql and memcached?
Comment 3 Alexander Nusov 2016-10-26 17:18:49 UTC
No, I haven't.
I don't see any reason to do so, memcached and mysql drivers are not direct dependencies.
Comment 4 Roman Bogorodskiy freebsd_committer 2016-10-26 18:39:17 UTC
(In reply to Alexander Nusov from comment #3)

I understand that those are not direct dependencies, but it doesn't seem to be the best user experience.

Imagine a user installing glance via port or package, going to configuration file, configuring database connection to mysql (or psql or whatever is supported) and getting a traceback somewhere. Then user spends some time to understand this error and then spends some more time to figure out what specific port/package is needed.

It's also a little harder to shoot in the foot when trying to remove some of the client libraries because with dependency it'll be obvious that it's still required by some other installed package.

Instead there could be just a radio option group to choose some of the supported DB clients.
Comment 5 Alexander Nusov 2016-10-26 19:37:34 UTC
Can you provide me an example of a port which has OPTIONs that just introduces new dependencies in RUN_DEPENDs without changing the build process please?

I agree that not including client libraries may impact the user experience but adding and enabling all possible options (which aren't listed in dependencies from the upstream) probably is not good idea too.
a few examples
1. devel/py-networkx port depends on numpy/scipy, all options are enabled.
Result: user should download a lot of packages or when building with -DBATCH wait more than an hour
2. devel/libvirt has options for QEMU, XEN but only BHYVE enabled.
Result: user may install xen from packages but libvirt without rebuilding and choosing  "right" options.
Comment 6 Alexander Nusov 2016-10-26 20:05:34 UTC
BTW, what do you think about adding the pkg-message that will tell the user to install mysql/memcached drivers if needed?

Glance packages from Ubuntu, RDO don't include pymysql driver and memcached one. debs/rpms are not ports but on Solaris I don't see these dependencies as well.
Comment 7 Alexander Nusov 2016-10-26 20:39:28 UTC
Created attachment 176198 [details]
Add OPTIONs for memcached and mysql
Comment 8 Alexander Nusov 2016-10-26 20:41:58 UTC
I'm fine with adding memcached and pymysql dependencies if such options will be disabled by default.

Comment 9 Alexander Nusov 2016-10-26 21:20:01 UTC
Before using the glance or keystone users should first populate the database and if on their system the DB/etc driver is not installed then exception will be raised (points to the module name)

Here is why I'm concerned:
1) openstack components may have multiple backends for persistent storage, in-memory, messaging bus (postgre, h2, mysql, percona, redis, memcached, rabbitmq, zeromq and many others)
2) listing all possible options is overkill
3) not listing something isn't okay and makes the port 'incomplete'
4) disabling all options won't solve the issue with installing from packages
5) enabling all options will lead to downloading a lot of unnecessary packages (extra traffic, time waste)
6) I have no idea what is the best practice for this case, any input would be appreciated.

I've added the patch with OPTIONs for backends I tested with. If you feel that it may improve the user experience I'll be happy to include that.
Comment 10 Alexander Nusov 2016-10-28 18:07:47 UTC
Please note that glance operates with persistent storage through sqlachemy and probably it's better to list supported backends there.

Proposed pkg-message:

Note that by default the installation includes only sqlite3 driver.
If you would like to use any other third party driver that is not included
to the dependencies from the upstream please consider these ports:

pymysq			databases/py-pymysql
pysycopg2		databases/py-psycopg2
python-memcached        databases/py-python-memcached
redis			databases/py-redis
pyzmq			net/py-pyzmq

Comment 11 commit-hook freebsd_committer 2016-10-29 12:02:50 UTC
A commit references this bug:

Author: novel
Date: Sat Oct 29 12:01:55 UTC 2016
New revision: 424902
URL: https://svnweb.freebsd.org/changeset/ports/424902

  Add py-automaton 1.4.0, friendly state machines for python.

  PR:		213769
  Submitted by:	Alexander Nusov <alexander.nusov@nfvexpress.com>

Comment 12 commit-hook freebsd_committer 2016-10-29 12:04:51 UTC
A commit references this bug:

Author: novel
Date: Sat Oct 29 12:04:37 UTC 2016
New revision: 424904
URL: https://svnweb.freebsd.org/changeset/ports/424904

  Add py-castellan 0.4.0, generic Key Manager interface for OpenStack.

  PR:		213769
  Submitted by:	Alexander Nusov <alexander.nusov@nfvexpress.com>

Comment 13 commit-hook freebsd_committer 2016-10-29 12:05:53 UTC
A commit references this bug:

Author: novel
Date: Sat Oct 29 12:05:16 UTC 2016
New revision: 424905
URL: https://svnweb.freebsd.org/changeset/ports/424905

  Add py-wsme 0.8.0, simplify the writing of REST APIs.

  PR:		213769
  Submitted by:	Alexander Nusov <alexander.nusov@nfvexpress.com>

Comment 14 commit-hook freebsd_committer 2016-10-29 12:06:54 UTC
A commit references this bug:

Author: novel
Date: Sat Oct 29 12:06:26 UTC 2016
New revision: 424906
URL: https://svnweb.freebsd.org/changeset/ports/424906

  Add py-glance-store 0.18.0, openStack Image Service Store Library.

  PR:		213769
  Submitted by:	Alexander Nusov <alexander.nusov@nfvexpress.com>

Comment 15 commit-hook freebsd_committer 2016-10-29 12:07:56 UTC
A commit references this bug:

Author: novel
Date: Sat Oct 29 12:07:04 UTC 2016
New revision: 424907
URL: https://svnweb.freebsd.org/changeset/ports/424907

  Add py-cursive 0.1.1, cursive implements OpenStack-specific validation
  of digital signatures.

  PR:		213769
  Submitted by:	Alexander Nusov <alexander.nusov@nfvexpress.com>

Comment 16 commit-hook freebsd_committer 2016-10-29 12:07:58 UTC
A commit references this bug:

Author: novel
Date: Sat Oct 29 12:07:45 UTC 2016
New revision: 424908
URL: https://svnweb.freebsd.org/changeset/ports/424908

  Add py-taskflow 2.6.0, taskflow structured state management library.

  PR:		213769
  Submitted by:	Alexander Nusov <alexander.nusov@nfvexpress.com>

Comment 17 Roman Bogorodskiy freebsd_committer 2016-10-29 17:10:01 UTC
(In reply to Alexander Nusov from comment #10)

Sorry, my bad, I didn't pay attention that sqlalchemy has dependencies for db clients already. In that case explicit dependencies are not needed.

Then I guess the only OPTION for now would be memcached support.
Comment 18 Alexander Nusov 2016-11-01 20:56:21 UTC
from the reference:
# Optionally specify a list of memcached server(s) to use for caching.
# If left undefined, tokens will instead be cached in-process. (list
# value)
# Deprecated group/name - [DEFAULT]/memcache_servers
#memcached_servers = <None>

since it's optional I think that memcached driver can be omitted from the glance and added to the keystomemiddleware later as an OPTION.

keystonemiddleware.auth_token [-] Using the in-process token cache is deprecated as of the 4.2.0 release and may be removed in the 5.0.0 releaseor the 'O' development cycle. The in-process cache causes inconsistent results and high memory usage. When the feature is removed the auth_token middleware will not cache tokens by default which may result in performance issues. It is recommended to use  memcache for the auth_token token cache by setting the memcached_servers option.
Comment 19 Roman Bogorodskiy freebsd_committer 2016-11-02 16:07:19 UTC
That sounds good to me. I'll commit the glance port as is then.
Comment 20 commit-hook freebsd_committer 2016-11-02 16:15:41 UTC
A commit references this bug:

Author: novel
Date: Wed Nov  2 16:14:43 UTC 2016
New revision: 425162
URL: https://svnweb.freebsd.org/changeset/ports/425162

  Add py-glance 13.0.0, OpenStack Image Service

  PR:		213769
  Submitted by:	Alexander Nusov <alexander.nusov@nfvexpress.com>

Comment 21 Roman Bogorodskiy freebsd_committer 2016-11-02 16:20:10 UTC
Everything committed, thanks!