Bug 94182 - [altq] [request] altq support for vlan driver
Summary: [altq] [request] altq support for vlan driver
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 5.4-STABLE
Hardware: Any Any
: Normal Affects Only Me
Assignee: freebsd-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-03-07 18:20 UTC by Emil Cazamir
Modified: 2019-10-09 21:49 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Emil Cazamir 2006-03-07 18:20:05 UTC
From altq(4):

"SUPPORTED DEVICES
     The driver modifications described in altq(9) are required to use a cer-
     tain network card with ALTQ.  They have been applied to the following
     hardware drivers: an(4), ath(4), awi(4), bfe(4), dc(4), de(4), ed(4),
     em(4), fxp(4), hme(4), lnc(4), re(4), rl(4), sf(4), sis(4), sk(4), vr(4),
     wi(4), and xl(4)."

At this time there is no support for altq using vlan interface. I use the following conficuration:
 physical interface: bge
    vlan1: vlan tag 1 vlandev bge0
    vlan2: vlan tag 2 vlandev bge0

I would like to use a setup like following:
- pf.conf -
  altq on vlan1 ...params...
  altq on vlan2 ...params...

Fix: 

perhaps some modifications to altq driver would do the thing, but i can't do them.
How-To-Repeat: enable pf and altq and try to use "altq on vlan" statements into /etc/pf.conf.
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2006-03-07 21:15:44 UTC
State Changed
From-To: open->suspended

Mark suspended awaiting someone to take an interest in this.
Comment 2 Gleb Smirnoff freebsd_committer freebsd_triage 2006-03-15 15:47:53 UTC
  Emil,

  if you search through mail archives, you will find, that the
question about ALTQ on vlan(4) was raised several times. There is
an opinion, that the idea itself is not correct at all.

-- 
Totus tuus, Glebius.
GLEBIUS-RIPN GLEB-RIPE
Comment 3 Emil Cazamir 2006-03-21 19:50:26 UTC
Gleb Smirnoff <glebius@FreeBSD.org> wrote:   Emil,

  if you search through mail archives, you will find, that the
question about ALTQ on vlan(4) was raised several times. There is
an opinion, that the idea itself is not correct at all.

-- 
Totus tuus, Glebius.
GLEBIUS-RIPN GLEB-RIPE

I'm not the person to decide if ALTQ and vlan(4) is a good or bad combination, but in my opinion it would be useful to specify each queueing strategy on his own interface, even if it is not a phisical one (such as tun(4)). A good example why vlan interfaces should be ALTQ-enabled is the following: 
- physical device:
   |- vlan1 - cbq on vlan1
   |- vlan2 - priq on vlan2
   `- vlan3 - hfsc on vlan3
At this time i know that altq can make use of only one traffic discipline on an interface, which makes the above case only a wish.
I think if ALTQ will not be implemented as standard feature in FreeBSD, it would be nice at least to be documented the fact that ALTQ works with vlan tagged frames somewhere in the man pages (no one officialy says that it works and I didn't tested that). At this time i don't find any reference to vlan tagged frames matching with ALTQ and pf, neither in altq(4) nor pf.conf(5) man pages. I'm using FreeBSD 5.5-PRERELEASE as of 2006, march 17.

Best regards,
Emil Cazamir


		
---------------------------------
Yahoo! Travel
 Find  great deals to the top 10 hottest destinations!
Comment 4 Gleb Smirnoff freebsd_committer freebsd_triage 2006-03-22 11:38:12 UTC
  Emil,

On Tue, Mar 21, 2006 at 11:50:26AM -0800, Emil Cazamir wrote:
E> I'm not the person to decide if ALTQ and vlan(4) is a good or bad combination, but in my opinion it would be useful to specify each queueing strategy on his own interface, even if it is not a phisical one (such as tun(4)). A good example why vlan interfaces should be ALTQ-enabled is the following: 
E> - physical device:
E>    |- vlan1 - cbq on vlan1
E>    |- vlan2 - priq on vlan2
E>    `- vlan3 - hfsc on vlan3

This may not work, unless you put a bandwidth limit on each vlan interface,
that is equal to limit of parent interface / number of vlans.

E> At this time i know that altq can make use of only one traffic discipline on an interface, which makes the above case only a wish.
E> I think if ALTQ will not be implemented as standard feature in FreeBSD, it would be nice at least to be documented the fact that ALTQ works with vlan tagged frames somewhere in the man pages (no one officialy says that it works and I didn't tested that). At this time i don't find any reference to vlan tagged frames matching with ALTQ and pf, neither in altq(4) nor pf.conf(5) man pages. I'm using FreeBSD 5.5-PRERELEASE as of 2006, march 17.

The first (theoretical) obstacle is that ALTQ is designed to shape traffic on
an interface, where exists a *bandwidth limit* and a *queue*. These two
things are property of a physical inteface. The vlan(4) interface doesn't
have any bandwidth limit. The current implementation has a queue. Packets
are temporarily queued, before they are sent to the underlying physical
driver. We have a plan to remove this queue, since it is a performance loss
for vlan(4) driver.

The second (practical) problem is that after removal of the intermediate
queue in the vlan(4) driver it will be difficult to add ALTQ support. No
queueing - no ALTernate Queueing.

-- 
Totus tuus, Glebius.
GLEBIUS-RIPN GLEB-RIPE
Comment 5 Emil Cazamir 2006-03-23 18:20:44 UTC
--0-980750068-1143138044=:59095
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit

Gleb Smirnoff <glebius@FreeBSD.org> wrote:   Emil,

On Tue, Mar 21, 2006 at 11:50:26AM -0800, Emil Cazamir wrote:
E> I'm not the person to decide if ALTQ and vlan(4) is a good or bad combination, but in my opinion it would be useful to specify each queueing strategy on his own interface, even if it is not a phisical one (such as tun(4)). A good example why vlan interfaces should be ALTQ-enabled is the following: 
E> - physical device:
E>    |- vlan1 - cbq on vlan1
E>    |- vlan2 - priq on vlan2
E>    `- vlan3 - hfsc on vlan3

This may not work, unless you put a bandwidth limit on each vlan interface,
that is equal to limit of parent interface / number of vlans.

E> At this time i know that altq can make use of only one traffic discipline on an interface, which makes the above case only a wish.
E> I think if ALTQ will not be implemented as standard feature in FreeBSD, it would be nice at least to be documented the fact that ALTQ works with vlan tagged frames somewhere in the man pages (no one officialy says that it works and I didn't tested that). At this time i don't find any reference to vlan tagged frames matching with ALTQ and pf, neither in altq(4) nor pf.conf(5) man pages. I'm using FreeBSD 5.5-PRERELEASE as of 2006, march 17.

The first (theoretical) obstacle is that ALTQ is designed to shape traffic on
an interface, where exists a *bandwidth limit* and a *queue*. These two
things are property of a physical inteface. The vlan(4) interface doesn't
have any bandwidth limit. The current implementation has a queue. Packets
are temporarily queued, before they are sent to the underlying physical
driver. We have a plan to remove this queue, since it is a performance loss
for vlan(4) driver.

The second (practical) problem is that after removal of the intermediate
queue in the vlan(4) driver it will be difficult to add ALTQ support. No
queueing - no ALTernate Queueing.

-- 
Totus tuus, Glebius.
GLEBIUS-RIPN GLEB-RIPE
I believe that this opinion (ALTQ + vlan is a bad idea) somehow attempts to minimize the benefits which vlan offers, or to make things more complicate than they really are. The reason for which I made the initial request is that I want to treat a vlan interface as an interface and nothing more, making use of some uniform resource configuration technique. Until recently, I used a PC with 3 physical (1xGbE + 2xFE) interfaces to connect a network to two upstream providers. At the same time, I do ip acconting  and traffic shaping with dummynet. Since recently, I can make use of vlans through one of the upstream providers, to reach another subnet under my administration. The main problem is that I'm limited only to use dummynet, and I want to use ALTQ in combination with dummynet to give to my customers more traffic options, while letting them to use more bandwidth to communicate between subnets at a higher speed.
Here's the big picture:
provider1 (vlan1)
 `- |             | vlan1-GbE0(myrouter)| lan#1
    |802.1q switch| vlan2-GbE0(myrouter)|
 ,- |             |   --- vlan3 ---     |
provider2
 `- |another_802.1q_sw|vlan2|another_router | lan#2
    |                 |   --- vlan3 ---     |

service type #1 (85% of all bandwidth): WF2Q+: dummynet with dynamic queues, use weighted fair all available bandwidth until they reach some byte counters.
service type #2 (15% of all bandwidth): ALTQ and cbq or hfsc with shared bandwidth between few customers with different needs.

My goal is to use dummynet and altq on vlan1, vlan2 and vlan3. Dummynet has the advantage of dynamic queues, which means that i need only one ipfw rule to do the traffic discipline with dummynet. I also have a small number of customers which want to use a service with maximum bandwidth much below to the bandwidth i offer with dummynet, but I can't use dynamic pipes with them because I won't be able to share bandwidth between them.
The main problem is that I don't have the budget to use a blade server chassis and server modules for this purpose, and I believe that until the number of customers will be below 1000 - 1500, I will be able to use a single computer for this purpose.

 
Best regards,
Emil Cazamir


		
---------------------------------
Yahoo! Messenger with Voice. Make PC-to-Phone Calls to the US (and 30+ countries) for 2ยข/min or less.
--0-980750068-1143138044=:59095
Content-Type: text/html; charset=iso-8859-1
Content-Transfer-Encoding: 8bit

<b style="font-family: courier;"><i>Gleb Smirnoff &lt;glebius@FreeBSD.org&gt;</i></b><span style="font-family: courier;"> wrote:</span><blockquote class="replbq" style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding-left: 5px; font-family: courier;">   Emil,<br><br>On Tue, Mar 21, 2006 at 11:50:26AM -0800, Emil Cazamir wrote:<br>E&gt; I'm not the person to decide if ALTQ and vlan(4) is a good or bad combination, but in my opinion it would be useful to specify each queueing strategy on h is own interface, even if it is not a phisical one (such as tun(4)). A good example why vlan interfaces should be ALTQ-enabled is the following: <br>E&gt; - physical device:<br>E&gt;    |- vlan1 - cbq on vlan1<br>E&gt;    |- vlan2 - priq on vlan2<br
Comment 6 Gleb Smirnoff freebsd_committer freebsd_triage 2006-03-27 08:11:20 UTC
On Sun, Mar 26, 2006 at 11:22:54AM -0800, Emil Cazamir wrote:
E> I see that the only way to work this out is the following:
E> - I upgrade the system to RELENG-6 or newer (altq has support for bge driver)
E> - I will attempt to use some undocumented option of pf: packet matching for any ethertype
E> - I make use of ipfw's packet matching code to enqueue traffic (ipfw can either see "from any to any via vlanX" or at least "from any to any mactype vlan"
E> - I will make more complicated my configuration scripts
E> 
E> Problems which will remain open:
E> - pf.conf manual page, which doesn't state that how to match packets with ethertype = 802.1Q (it only states that matching is "based on attributes of their layer 3 headers" is outdated. This makes pf integration to FreeBSD to appear as being in beta stage. 
E> - A simple packet matching and enqueueing rule such as "pass out on vlan1 inet from any to some.ip.add.ress queue vlan1_queue1_out" can be defined only with a stranger form such as: "pass out on vlan1 inet from any to some.ip.add.ress queue bge0_q1" and i see this a little weird. 

I don't see this weird.

E> - For most unexperienced users (and not only for them), an attempt to use a ruleset which looks  OK, and is working fine with physical-only interfaces, can lead to unexpected results. In the following situation: bge0 is the parent of vlan0:
E> pass out on vlan0 all queue q_egress_1
E> pass in   on vlan0 all
E> block on bge0 all
E> In the above situation a packet which should pass through vlan0 will be blocked by the rule below (not tested, but considering that pf inspects at layer3 address..), and with ipfw this won't happen (neither tested :)). This require a special section on pf.conf man page, to make the users aware of such situations.

I'm not sure, but I think, that the traffic will not be blocked in the above
ruleset. Please try it and see.

E> - there is no way to make use of different traffic classifiers on a physical interface with more vlan sub-interfaces
E> - there is only one default queue on a phisical interface, and it is useful to have one default queue per logical interface.

And this is correct. There is no reason to make a queue in a random place in the
kernel. The queue is made in a place where CPU is waiting for something to complete,
usually I/O. We can wait for NIC to transmit, or for HDD to write data. What for do
we wait in vlan(4) driver? Nothing. We enqueue and then dequeue, just wasting CPU
cycles.

E> Some workarounds for the performance impact you mentioned could be some definition in the kernel configuration file, which should be a trigger for "vlan enhanced performance code", and it's up to FreeBSD's core team whether this should be enabled by default or not. From my point of wiew, this  "vlan enhanced code" should be enabled for those who need it: (multi)GB_ethernet_routers_administrators, long_firewall_ruleset_users, etc.

-- 
Totus tuus, Glebius.
GLEBIUS-RIPN GLEB-RIPE
Comment 7 Eitan Adler freebsd_committer freebsd_triage 2018-05-20 23:57:13 UTC
For bugs matching the following conditions:
- Status == In Progress
- Assignee == "bugs@FreeBSD.org"
- Last Modified Year <= 2017

Do
- Set Status to "Open"
Comment 8 John-Mark Gurney freebsd_committer freebsd_triage 2019-10-09 21:34:38 UTC
With a bit of info from jhb, it was pointed out that the queues can be applied to the underlying interface.

example:
physical interface: igb0
vlan 10

pf.conf:
altq on igb0 cbq queue { def aq }
queue def bandwidth 90% cbq (default borrow)
queue aq bandwidth 10Mb cbq


pass in on igb0.10 proto udp all queue aq keep state

The reason this works is that the tags that are applied via the rules to mark which queue to use are attached to the mbuf, and when they finally go out the igb0 interface, they'll be processed there.

I'll try to look at updating some docs to let this be more visible.
Comment 9 commit-hook freebsd_committer freebsd_triage 2019-10-09 21:48:21 UTC
A commit references this bug:

Author: jmg
Date: Wed Oct  9 21:48:00 UTC 2019
New revision: 353374
URL: https://svnweb.freebsd.org/changeset/base/353374

Log:
  document how to apply altq to vlan interfaces w/ pf.

  Thanks to jhb for pointing out that this might possibly work.

  PR:		94182

Changes:
  head/share/man/man4/altq.4