Bug 87500

Summary: [patch] redo www/en/ports/index.html layout
Product: Documentation Reporter: edwin
Component: Books & ArticlesAssignee: freebsd-doc (Nobody) <doc>
Status: Closed FIXED    
Severity: Affects Only Me    
Priority: Normal    
Version: Latest   
Hardware: Any   
OS: Any   

Description edwin 2005-10-16 00:50:19 UTC 
- Add port groups as suggested by linimon@
  http://people.freebsd.org/~linimon/www/en/ports/index.html#logical_groups

- Cleanup portindex to be "use strict" and "-w" safe

- New output: 
  http://edwin.adsl.barnet.com.au/~edwin/freebsd-en/ports/index.html

Fix: 

And since the patch for portindex is hardly readable, here is the
full script:

#!/usr/bin/perl -w

# convert a ports INDEX file to HTML
#
# by John Fieber <jfieber@FreeBSD.org>
# Mon May 13 10:31:58 EST 1996
# $FreeBSD: www/en/ports/portindex,v 1.52 2005/10/04 19:43:49 hrs Exp $

############################################################

use strict;
use POSIX;
use IO::File;

my $base = "";
my $baseHTTP = "";
my $urlcgi = "";
my $packagesURL = "";
my $today = getdate();
my %p = ();

# Load local config file. You can override in portindex.conf
# the variables for the default web and the ftp server. This
# make it easy to maintain a local web mirror and let the
# URL point to yourself and not to the standard FreeBSD FTP server.

my $config = $0 . '.' . 'conf';
do $config if -f $config;

# This is the base of where we ftp stuff from
my $ftpserver;
if ($ENV{'MASTER_FTP_SERVER'}) {
    $ftpserver = $ENV{'MASTER_FTP_SERVER'};
} else {
    $ftpserver = 'ftp://ftp.FreeBSD.org' if !$ftpserver;
}
$baseHTTP = $base if !$baseHTTP;
my $baseFTP = "$ftpserver/pub/FreeBSD/ports/ports";
my $baseCVSWEB = 'http://www.FreeBSD.org/cgi/cvsweb.cgi/';
$urlcgi = 'http://www.FreeBSD.org/cgi/url.cgi' if !$urlcgi;
$packagesURL = "$ftpserver/pub/FreeBSD/ports/i386/packages-stable/All/"
    if !$packagesURL;
# support tar on the fly or gzip'ed tar on the fly
my $ftparchive = '';
$ftparchive = 'tar' if !defined $ftparchive;


# ports download sources script
my $pds = 'http://www.FreeBSD.org/cgi/pds.cgi';

# better layout and link to the sources
if ($urlcgi) {
    $baseHTTP = $urlcgi . '?' . $baseHTTP;
}

my %packages = ();
my %category_description = ();
my @category_groups = ();
my %category_groups = ();
packages_exist('packages.exists');
category_description(($ARGV[1] || '.') . '/categories' );
category_groups(($ARGV[1] || '.') . '/categories.descriptions' );
main();

sub getdate {
    return POSIX::strftime("Last modified: %d-%B-%Y",localtime);
}

sub header {
    my $fh = shift;
    my $htext = shift;
    #local($fh, $htext) = @_;
    print $fh <<EOF;
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" [
<!ENTITY base CDATA '..'>
<!ENTITY date  "<em>$today</em>">
<!ENTITY title '$htext'>
<!ENTITY blurb  SYSTEM "ports.inc">
<!ENTITY % navincludes SYSTEM "../includes.navdownload.sgml"> %navincludes;
<!ENTITY % includes SYSTEM "../includes.sgml">
<!ENTITY email 'ports'>
%includes;
]>
<html>&header;
EOF
}

sub footer {
    my $fh = shift;
    my $ftext = shift;
    #local ($fh, $ftext) = @_;
print $fh <<EOF;

$ftext
&footer;
</BODY>
</HTML>
EOF
}

sub packages_exist {
    my $file=shift;
    my %p;
    #local($file, *p) = @_;

    open(P, $file) || do {
	warn "open $file: $!\n";
	warn "Cannot create packages links\n";
	return 1;
    };

    while(<P>) {
	chop;
	$packages{$_} = 1;
    }
    close P;
}

sub category_description {
    my $file = shift;

    open(P, $file) || do {
	warn "open $file: $!\n";
	warn "Cannot find category description\n";
	return 1;
    };

    while(<P>) {
	# ignore comments
	next if /^\s*#/;
	if (/^\s*([^,]+),\s*"([^"]+)",([A-Z]+)/) {
	    $category_description{$1}{desc}=$2;
	    $category_description{$1}{group}=$3;
	}
    }
    close P;
}

sub category_groups {
    my $file = shift;

    open(P, $file) || do {
	warn "open $file: $!\n";
	warn "Cannot find category groups\n";
	return 1;
    };

    while(<P>) {
	# ignore comments
	next if /^\s*#/;
	if (/^\s*([^,]+),\s*(.+)/) {
	    $category_groups{$1}=$2;
	    push(@category_groups,$1);
	}
    }
    close P;
}

sub main {
    my @master=();
    my %stats;
    my %catkey;
    my %data;
    my $portnumber = 0;

    my $sep = "<B>:</B>";

    my $moutf = new IO::File;
    my $outf = new IO::File;
    my $mindex = new IO::File;

    open(INDEX, $ARGV[0]);
    while (<INDEX>) {
	chop;
	s/&/&amp;/g;
    	s/</&lt;/g;
    	s/>/&gt;/g;

    	# Read a record
	my ($name, $loc, $prefix, $desc, $ldesc, $owner, $cats, 
	    $bdep, $rdep, $www) = split('\|');

	# Check for double hyphens in the name (--).
	$name =~ s/--/-/g;

    	# Split the categories into an array
	my @cat = split("[ \t]+", $cats);

    	$catkey{$name} = $cat[0];

	my $sourcepath = $loc;
	$sourcepath =~ s%/usr/%%;
	
	foreach my $i (sort(@cat)) {
	    $stats{$i}++;

	    # figure out the FTP url
	    $loc =~ s/\/usr\//$baseCVSWEB/;
	    $ldesc =~ s/\/usr\//$baseHTTP/;

	    # The name description and maintainer
	    $data{$i} .= "<DT><B><A NAME=\"$name\"></A><A HREF=\"$loc$ftparchive\">$name</A></B> ";
     

	    $data{$i} .= "<DD>$desc<BR><A HREF=\"$ldesc\">Long description</A>";
	    if ($packages{"$name.tbz"}) { 
		$data{$i} .= qq{ | <a href="$packagesURL$name.tbz">Package</a>};
	    } elsif ($packages{"$name.tgz"}) { 
		$data{$i} .= qq{ | <a href="$packagesURL$name.tgz">Package</a>};
	    }

	    $data{$i} .= qq{ | <A HREF="$pds?$sourcepath">Sources</A>};

	    if ($www ne "") {
		$data{$i} .= qq{ | <a href="$www">Main Web Site</a>};
	    }

	    my $ownerurl = $owner;
	    $ownerurl =~ s/&lt;/</g;
	    $ownerurl =~ s/&gt;/>/g;
	    $data{$i} .= 
		"<BR><I>Maintained by:</I> <A HREF=\"mailto:$ownerurl\">$owner</A>";

	    # If there are any dependencies, list them
	    if ($bdep ne "" || $rdep ne "") {
		$data{$i} .= "<BR><I>Requires:</I> ";
		my @dep = split(/ /, "$bdep $rdep");
		my $last = '';
		foreach my $j (sort @dep) {
		    next if $j eq $last;
		    $last = $j;
		    $data{$i} .= " <A HREF=\"##$j##.html#$j\">$j</A>,";
		}
		# remove the trailing comma
		chop $data{$i};
	    }

	    # If the port is listed in more than one category, throw
	    # in some cross references
	    if ($#cat > 0) {
		$data{$i} .= "<BR><EM>Also listed in:</EM> ";
		foreach my $j (@cat) {
		    if ($j ne $i) {
			if ($j eq $cat[0]) {
			    $data{$i} .= " <STRONG><A HREF=\"$j.html#$name\">\u$j</A></STRONG>,";
			}
			else {
			    $data{$i} .= " <A HREF=\"$j.html#$name\">\u$j</A>,";
			}
		    }
		}
		# remove the trailing comma
		chop($data{$i});
	    }
	    $data{$i} .= "<P></P></DD>\n"
	}

    	# Add an entry to the master index

	# workaround for SGML bug, `--' is not allowed in comments
	my $sname = $name; 
	$sname =~ s/--/-=/g;
    	$master[$portnumber] = 
    	    "<!-- $sname --><STRONG><A HREF=\"$cat[0].html#$name\">$name</A></STRONG> " .
    	    " -- <EM>$desc</EM><BR>\n";
	$portnumber++;
    }

    $moutf->open(">index.sgml");

    header($moutf, "FreeBSD Ports");
#   print $moutf "<!--#include virtual=\"./ports.inc\" -->\n";
    print $moutf "&blurb;";
    print $moutf <<EOF;
<HR>
<P>There are currently $portnumber ports in the FreeBSD Ports Collection.<BR>
You may download a
<A HREF="$baseFTP/ports.tar.gz">gzip'd tar file of all ports</A>
(about 30 megabytes), view the
<A HREF=\"master-index.html\">alphabetical list of all ports</A>,
or browse the ports in the following categories:
<A NAME=\"categories\"></A>
EOF

    foreach my $cg (@category_groups) {
	print $moutf "<H3>",$category_groups{$cg},"</H3>\n";
	print $moutf "<UL>\n";
	foreach my $key (sort(keys(%stats))) {
	    next if ($category_description{$key}{group} ne $cg);
	    # For the master file...
	    print $moutf 
		"<LI><A HREF=\"$key.html\">\u$key</A> <em>($stats{$key})</em>";
	    if ($category_description{$key}{desc}) {
		print $moutf " -- " . $category_description{$key}{desc};
	    }

	    # Someone forgot to add a category to the description file
	    # or there is a typo in the category field.
	    else {
		warn "No description found for category: ``$key''!\n";
		warn "Please fix me or send an E-Mail to doc\@FreeBSD.org\a\n";
		sleep(3);
	    }
	    print $moutf "</LI>\n";

	    # Create the category file
	    $outf->open(">$key.sgml");
	    header($outf, "FreeBSD Ports: \u$key");
	    if ($category_description{$key}{desc}) {
		print $outf "<h3>", $category_description{$key}{desc}, "</h3>\n";
	    }
	    print $outf "<DL>\n";
	    my $d = join("\n", sort(split(/\n/, $data{$key})));
	    $d =~ s/##([^#]*)##/$catkey{$1}/g;
	    print $outf $d;
	    print $outf "</DL>\n";
	    footer($outf, "<p></p><A HREF=\"index.html\">Port Categories</A>" .
		" -- <A HREF=\"master-index.html\">Index</A>");
	    $outf->close;

	}
	print $moutf "</UL>\n";
    }

    print $moutf 
    	"<UL><LI><A HREF=\"master-index.html\">Alphabetical list of all $portnumber ports</A></LI></UL>\n";
    footer($moutf, "");
    $moutf->close;

    # Create the master index file
    $mindex->open(">master-index.sgml");
    header($mindex, "FreeBSD Ports Collection Index");
    print $mindex "<P>\n";
    print $mindex sort @master;
    print $mindex "</P>";
    footer($mindex, "<A HREF=\"index.html\">Port Categories</A>");
    $mindex->close;
    close(INDEX);
}--V7CzrjOae2UIihm8arNyA776BZZrmpi07Oe1rckwF1yvNf4U
Content-Type: text/plain; name="file.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="file.diff"

Index: Makefile
===================================================================
RCS file: /home/dcvs/www/en/ports/Makefile,v
retrieving revision 1.39
diff -u -r1.39 Makefile
--- Makefile	13 Nov 2004 12:23:18 -0000	1.39
+++ Makefile	15 Oct 2005 23:40:42 -0000
@@ -77,7 +77,7 @@
 beforeinstall:
 .else
 beforeinstall:
-	${PERL} -pi -e 's/href=\"growth/href=\"http:\/\/www.FreeBSD.org\/ports\/growth/' index.html
+	${SED} -i "" -e 's,HREF="growth,href="http://www.FreeBSD.org/ports/growth,' index.html
 .endif
 
 .include "${WEB_PREFIX}/share/mk/web.site.mk"
Index: categories
===================================================================
RCS file: /home/dcvs/www/en/ports/categories,v
retrieving revision 1.27
diff -u -r1.27 categories
--- categories	19 Dec 2004 02:39:16 -0000	1.27
+++ categories	15 Oct 2005 23:40:42 -0000
@@ -1,92 +1,95 @@
-# Originally from src/release/sysinstall/index.c,v 1.57 1998/10/15 
-# $FreeBSD: www/en/ports/categories,v 1.27 2004/12/19 02:39:16 linimon Exp $
+#Originally from src/release/sysinstall/index.c,v 1.57 1998/10/15 
+#$FreeBSD: www/en/ports/categories,v 1.27 2004/12/19 02:39:16 linimon Exp $
 
-    "accessibility", "Ports to help disabled users.",
-    "afterstep", "Ports to support the AfterStep window manager.",
-    "arabic", "Ported software for the Arabic market.",
-    "archivers", "Utilities for archiving and unarchiving data.",
-    "astro", "Applications related to astronomy.",
-    "audio", "Audio utilities - most require a supported sound card.",
-    "benchmarks", "Utilities for measuring system performance.",
-    "biology", "Software related to biology.",
-    "cad", "Computer Aided Design utilities.",
-    "chinese", "Ported software for the Chinese market.",
-    "comms", "Communications utilities.",
-    "converters", "Format conversion utilities.",
-    "databases", "Database software.",
-    "deskutils", "Various Desktop utilities.",
-    "devel", "Software development utilities and libraries.",
-    "dns", "DNS client and server utilities.",
-    "editors", "Common text editors.",
-    "elisp", "Things related to Emacs Lisp.",
-    "emulators", "Utilities for emulating other OS types.",
-    "finance", "Monetary, financial and related applications.",
-    "french", "Ported software for French countries.",
-    "ftp", "FTP client and server utilities.",
-    "games", "Various and sundry amusements.",
-    "german", "Ported software for Germanic countries.",
-    "gnome", "Components of the Gnome Desktop environment.",
-    "graphics", "Graphics libraries and utilities.",
-    "haskell", "Software related to the Haskell language.",
-    "hebrew", "Ported software for Hebrew language.",
-    "hungarian", "Ported software for the Hungarian market.",
-    "ipv6", "IPv6 related software.",
-    "irc", "Internet Relay Chat utilities.",
-    "japanese", "Ported software for the Japanese market.",
-    "java", "Java language support.",
-    "kde", "Software for the K Desktop Environment.",
-    "korean", "Ported software for the Korean market.",
-    "lang", "Computer languages.",
-    "linux", "Linux programs that can be run under binary compatibility.",
-    "lisp", "Things related to pure lisp.",
-    "mail", "Electronic mail packages and utilities.",
-    "math", "Mathematical computation software.",
-    "mbone", "Applications and utilities for the MBONE.",
-    "misc", "Miscellaneous utilities.",
-    "multimedia", "Multimedia software.",
-    "net", "Networking utilities.",
-    "net-mgmt", "Network management utilities.",
-    "news", "USENET News support software.",
-    "offix", "An office automation suite of sorts.",
-    "palm", "Software support for the Palm(tm) series.",
-    "parallel", "Applications dealing with parallelism in computing.",
-    "pear", "Utilities/modules that fall into the PEAR system.",
-    "perl5", "Utilities/modules for the PERL5 language.",
-    "plan9", "Software from the Plan9 operating system.",
-    "polish", "Ported software for the Polish market.",
-    "portuguese", "Ported software for the Portuguese market.",
-    "print", "Utilities for dealing with printing.",
-    "python", "Software related to the Python language.",
-    "ruby", "Software related to the Ruby language.",
-    "russian", "Ported software for the Russian market.",
-    "scheme", "Software related to the Scheme language.",
-    "science", "Scientific software.",
-    "security", "System security software.",
-    "shells", "Various shells (tcsh, bash, etc).",
-    "sysutils", "Various system utilities.",
-    "tcl80", "TCL v8.0 and packages which depend on it.",
-    "tcl81", "TCL v8.1 and packages which depend on it.",
-    "tcl82", "TCL v8.2 and packages which depend on it.",
-    "tcl83", "TCL v8.3 and packages which depend on it.",
-    "tcl84", "TCL v8.4 and packages which depend on it.",
-    "textproc", "Text processing/search utilities.",
-    "tk42", "Tk4.2 and packages which depend on it.",
-    "tk80", "Tk8.0 and packages which depend on it.",
-    "tk82", "Tk8.2 and packages which depend on it.",
-    "tk83", "Tk8.3 and packages which depend on it.",
-    "tk84", "Tk8.4 and packages which depend on it.",
-    "tkstep80", "tkstep wm and packages which depend on it.",
-    "ukrainian", "Ported software for the Ukrainian market.",
-    "vietnamese", "Ported software for the Vietnamese market.",
-    "windowmaker", "Ports to support the WindowMaker window manager.",
-    "www", "WEB utilities (browsers, HTTP servers, etc).",
-    "x11", "X Window System based utilities.",
-    "x11-clocks", "X Window System based clocks.",
-    "x11-fm", "X Window System based file managers.",
-    "x11-fonts", "X Window System fonts and font utilities.",
-    "x11-servers", "X Window System servers.",
-    "x11-themes", "X Window System themes.",
-    "x11-toolkits", "X Window System based development toolkits.",
-    "x11-wm", "X Window System window managers.",
-    "xfce", "Ports to support the Xfce desktop environment.",
-    "zope", "Software related to the Zope platform.",
+# See categories.descriptions for the meaning of the abbrevations at the
+# end of each category.
+
+accessibility,"Ports to help disabled users.",EU
+afterstep,"Ports to support the AfterStep window manager.",VC
+arabic,"Ported software for the Arabic market.",HL
+archivers,"Utilities for archiving and unarchiving data.",SAAD
+astro,"Applications related to astronomy.",SAE
+audio,"Audio utilities - most require a supported sound card.",EU
+benchmarks,"Utilities for measuring system performance.",AOP
+biology,"Software related to biology.",SAE
+cad,"Computer Aided Design utilities.",SAE
+chinese,"Ported software for the Chinese market.",HL
+comms,"Communications utilities.",CCATI
+converters,"Format conversion utilities.",AOP
+databases,"Database software.",SAAD
+deskutils,"Various Desktop utilities.",EU
+devel,"Software development utilities and libraries.",SAAD
+dns,"DNS client and server utilities.",CCATI
+editors,"Common text editors.",EU
+elisp,"Things related to Emacs Lisp.",VC
+emulators,"Utilities for emulating other OS types.",SAAD
+finance,"Monetary, financial and related applications.",EU
+french,"Ported software for French countries.",HL
+ftp,"FTP client and server utilities.",CCATI
+games,"Various and sundry amusements.",EU
+german,"Ported software for Germanic countries.",HL
+gnome,"Components of the Gnome Desktop environment.",VC
+graphics,"Graphics libraries and utilities.",EU
+haskell,"Software related to the Haskell language.",VC
+hebrew,"Ported software for Hebrew language.",HL
+hungarian,"Ported software for the Hungarian market.",HL
+ipv6,"IPv6 related software.",VC
+irc,"Internet Relay Chat utilities.",CCATI
+japanese,"Ported software for the Japanese market.",HL
+java,"Java language support.",SAAD
+kde,"Software for the K Desktop Environment.",VC
+korean,"Ported software for the Korean market.",HL
+lang,"Computer languages.",SAAD
+linux,"Linux programs that can be run under binary compatibility.",VC
+lisp,"Things related to pure lisp.",VC
+mail,"Electronic mail packages and utilities.",CCATI
+math,"Mathematical computation software.",SAE
+mbone,"Applications and utilities for the MBONE.",CCATI
+misc,"Miscellaneous utilities.",AOP
+multimedia,"Multimedia software.",EU
+net,"Networking utilities.",CCATI
+net-mgmt,"Network management utilities.",CCATI
+news,"USENET News support software.",CCATI
+offix,"An office automation suite of sorts.",VC
+palm,"Software support for the Palm(tm) series.",AOP
+parallel,"Applications dealing with parallelism in computing.",VC
+pear,"Utilities/modules that fall into the PEAR system.",VC
+perl5,"Utilities/modules for the PERL5 language.",VC
+plan9,"Software from the Plan9 operating system.",VC
+polish,"Ported software for the Polish market.",HL
+portuguese,"Ported software for the Portuguese market.",HL
+print,"Utilities for dealing with printing.",EU
+python,"Software related to the Python language.",VC
+ruby,"Software related to the Ruby language.",VC
+russian,"Ported software for the Russian market.",HL
+scheme,"Software related to the Scheme language.",VC
+science,"Scientific software.",SAE
+security,"System security software.",SAAD
+shells,"Various shells (tcsh, bash, etc).",EU
+sysutils,"Various system utilities.",SAAD
+tcl80,"TCL v8.0 and packages which depend on it.",VC
+tcl81,"TCL v8.1 and packages which depend on it.",VC
+tcl82,"TCL v8.2 and packages which depend on it.",VC
+tcl83,"TCL v8.3 and packages which depend on it.",VC
+tcl84,"TCL v8.4 and packages which depend on it.",VC
+textproc,"Text processing/search utilities.",EU
+tk42,"Tk4.2 and packages which depend on it.",VC
+tk80,"Tk8.0 and packages which depend on it.",VC
+tk82,"Tk8.2 and packages which depend on it.",VC
+tk83,"Tk8.3 and packages which depend on it.",VC
+tk84,"Tk8.4 and packages which depend on it.",VC
+tkstep80,"tkstep wm and packages which depend on it.",VC
+ukrainian,"Ported software for the Ukrainian market.",HL
+vietnamese,"Ported software for the Vietnamese market.",HL
+windowmaker,"Ports to support the WindowMaker window manager.",VC
+www,"WEB utilities (browsers, HTTP servers, etc).",CCATI
+x11,"X Window System based utilities.",XWS
+x11-clocks,"X Window System based clocks.",XWS
+x11-fm,"X Window System based file managers.",XWS
+x11-fonts,"X Window System fonts and font utilities.",XWS
+x11-servers,"X Window System servers.",XWS
+x11-themes,"X Window System themes.",XWS
+x11-toolkits,"X Window System based development toolkits.",XWS
+x11-wm,"X Window System window managers.",XWS
+xfce,"Ports to support the Xfce desktop environment.",VC
+zope,"Software related to the Zope platform.",VC
Index: categories.descriptions
===================================================================
RCS file: categories.descriptions
diff -N categories.descriptions
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ categories.descriptions	15 Oct 2005 23:40:42 -0000
@@ -0,0 +1,12 @@
+# $FreeBSD$
+#
+# Abbrevations which are used in the categories file.
+
+EU,Ports primarily for end-users
+SAAD,Ports primarily for system administrators and developers
+CCATI,Ports relating to computer communications and the Internet
+XWS,Ports relating to the X Window System
+SAE,Ports relating to science and engineering
+HL,Ports relating to human languages
+VC,Ports in virtual categories (secondary groupings to above)
+AOP,All other ports
Index: portindex
===================================================================
RCS file: /home/dcvs/www/en/ports/portindex,v
retrieving revision 1.52
diff -u -r1.52 portindex
--- portindex	4 Oct 2005 19:43:49 -0000	1.52
+++ portindex	15 Oct 2005 23:40:42 -0000
@@ -1,4 +1,4 @@
-#!/usr/bin/perl
+#!/usr/bin/perl -w
 
 # convert a ports INDEX file to HTML
 #
@@ -8,77 +8,100 @@
 
 ############################################################
 
+use strict;
+use POSIX;
+use IO::File;
+
+my $base = "";
+my $baseHTTP = "";
+my $urlcgi = "";
+my $packagesURL = "";
+my $today = getdate();
+my %p = ();
+
 # Load local config file. You can override in portindex.conf
 # the variables for the default web and the ftp server. This
 # make it easy to maintain a local web mirror and let the
 # URL point to yourself and not to the standard FreeBSD FTP server.
-$config = $0 . '.' . 'conf';
+
+my $config = $0 . '.' . 'conf';
 do $config if -f $config;
 
 # This is the base of where we ftp stuff from
+my $ftpserver;
 if ($ENV{'MASTER_FTP_SERVER'}) {
     $ftpserver = $ENV{'MASTER_FTP_SERVER'};
 } else {
     $ftpserver = 'ftp://ftp.FreeBSD.org' if !$ftpserver;
 }
-$base = "" if !$base;
 $baseHTTP = $base if !$baseHTTP;
-$baseFTP = "$ftpserver/pub/FreeBSD/ports/ports";
-$baseCVSWEB = 'http://www.FreeBSD.org/cgi/cvsweb.cgi/';
+my $baseFTP = "$ftpserver/pub/FreeBSD/ports/ports";
+my $baseCVSWEB = 'http://www.FreeBSD.org/cgi/cvsweb.cgi/';
 $urlcgi = 'http://www.FreeBSD.org/cgi/url.cgi' if !$urlcgi;
 $packagesURL = "$ftpserver/pub/FreeBSD/ports/i386/packages-stable/All/"
     if !$packagesURL;
 # support tar on the fly or gzip'ed tar on the fly
-$ftparchive = '';
+my $ftparchive = '';
 $ftparchive = 'tar' if !defined $ftparchive;
 
 
 # ports download sources script
-$pds = 'http://www.FreeBSD.org/cgi/pds.cgi';
+my $pds = 'http://www.FreeBSD.org/cgi/pds.cgi';
 
 # better layout and link to the sources
 if ($urlcgi) {
     $baseHTTP = $urlcgi . '?' . $baseHTTP;
 }
 
-$today = &getdate;
-
-&packages_exist('packages.exists', *packages);
-&category_description(($ARGV[1] || '.') . '/categories', *category_description);
-&main;
+my %packages = ();
+my %category_description = ();
+my @category_groups = ();
+my %category_groups = ();
+packages_exist('packages.exists');
+category_description(($ARGV[1] || '.') . '/categories' );
+category_groups(($ARGV[1] || '.') . '/categories.descriptions' );
+main();
 
 sub getdate {
-    @months = ("January", "February", "March", "April", "May","June",
-	       "July", "August", "September", "October", "November", "December");
-    ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);
-    $year += 1900;
-    return "Last modified: $mday-$months[$mon]-$year";
+    return POSIX::strftime("Last modified: %d-%B-%Y",localtime);
 }
 
 sub header {
-    local ($fh, $htext) = @_;
-    print $fh "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" [\n";
-    print $fh "<!ENTITY base CDATA '..'>\n";
-    print $fh "<!ENTITY date  \"<em>$today</em>\">\n";
-    print $fh "<!ENTITY title '$htext'>\n";
-    print $fh "<!ENTITY blurb  SYSTEM \"ports.inc\">\n";
-    print $fh "<!ENTITY % navincludes SYSTEM \"../includes.navdownload.sgml\"> %navincludes;\n";
-    print $fh "<!ENTITY % includes SYSTEM \"../includes.sgml\">\n";
-    print $fh "<!ENTITY email 'ports'>\n";
-    print $fh "%includes;\n";
-    print $fh "]>\n";
-    print $fh "<html>&header;\n";
+    my $fh = shift;
+    my $htext = shift;
+    #local($fh, $htext) = @_;
+    print $fh <<EOF;
+<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" [
+<!ENTITY base CDATA '..'>
+<!ENTITY date  "<em>$today</em>">
+<!ENTITY title '$htext'>
+<!ENTITY blurb  SYSTEM "ports.inc">
+<!ENTITY % navincludes SYSTEM "../includes.navdownload.sgml"> %navincludes;
+<!ENTITY % includes SYSTEM "../includes.sgml">
+<!ENTITY email 'ports'>
+%includes;
+]>
+<html>&header;
+EOF
 }
 
 sub footer {
-    local ($fh, $ftext) = @_;
-    print $fh "\n$ftext\n";
-    print $fh "&footer;\n";
-    print $fh "</BODY>\n</HTML>\n";
+    my $fh = shift;
+    my $ftext = shift;
+    #local ($fh, $ftext) = @_;
+print $fh <<EOF;
+
+$ftext
+&footer;
+</BODY>
+</HTML>
+EOF
 }
 
 sub packages_exist {
-    local($file, *p) = @_;
+    my $file=shift;
+    my %p;
+    #local($file, *p) = @_;
 
     open(P, $file) || do {
 	warn "open $file: $!\n";
@@ -88,14 +111,13 @@
 
     while(<P>) {
 	chop;
-	$p{$_} = 1;
+	$packages{$_} = 1;
     }
     close P;
-    return 0;
 }
 
 sub category_description {
-    local($file, *p) = @_;
+    my $file = shift;
 
     open(P, $file) || do {
 	warn "open $file: $!\n";
@@ -103,21 +125,49 @@
 	return 1;
     };
 
-    local($category, $description);
     while(<P>) {
 	# ignore comments
 	next if /^\s*#/;
+	if (/^\s*([^,]+),\s*"([^"]+)",([A-Z]+)/) {
+	    $category_description{$1}{desc}=$2;
+	    $category_description{$1}{group}=$3;
+	}
+    }
+    close P;
+}
+
+sub category_groups {
+    my $file = shift;
 
-       ($category, $description) = /^\s*"([^"]+)",\s*"([^"]+)/;
-	$p{$category} = $description;
+    open(P, $file) || do {
+	warn "open $file: $!\n";
+	warn "Cannot find category groups\n";
+	return 1;
+    };
+
+    while(<P>) {
+	# ignore comments
+	next if /^\s*#/;
+	if (/^\s*([^,]+),\s*(.+)/) {
+	    $category_groups{$1}=$2 if (/^\s*([^,]+),\s*(.+)/);
+	    push(@category_groups,$1);
+	}
     }
     close P;
-    return 0;
 }
 
 sub main {
-
-    $sep = "<B>:</B>";
+    my @master=();
+    my %stats;
+    my %catkey;
+    my %data;
+    my $portnumber = 0;
+
+    my $sep = "<B>:</B>";
+
+    my $moutf = new IO::File;
+    my $outf = new IO::File;
+    my $mindex = new IO::File;
 
     open(INDEX, $ARGV[0]);
     while (<INDEX>) {
@@ -127,29 +175,28 @@
     	s/>/&gt;/g;
 
     	# Read a record
-	($name, $loc, $prefix, $desc, $ldesc, $owner, $cats, 
+	my ($name, $loc, $prefix, $desc, $ldesc, $owner, $cats, 
 	    $bdep, $rdep, $www) = split('\|');
 
 	# Check for double hyphens in the name (--).
 	$name =~ s/--/-/g;
 
     	# Split the categories into an array
-	@cat = split("[ \t]+", $cats);
+	my @cat = split("[ \t]+", $cats);
 
     	$catkey{$name} = $cat[0];
 
-	local($sourcepath) = $loc;
+	my $sourcepath = $loc;
 	$sourcepath =~ s%/usr/%%;
 	
-	foreach $i (@cat) {
-
+	foreach my $i (sort(@cat)) {
 	    $stats{$i}++;
 
-    	    # figure out the FTP url
+	    # figure out the FTP url
 	    $loc =~ s/\/usr\//$baseCVSWEB/;
 	    $ldesc =~ s/\/usr\//$baseHTTP/;
 
-    	    # The name description and maintainer
+	    # The name description and maintainer
 	    $data{$i} .= "<DT><B><A NAME=\"$name\"></A><A HREF=\"$loc$ftparchive\">$name</A></B> ";
      
 
@@ -162,54 +209,54 @@
 
 	    $data{$i} .= qq{ | <A HREF="$pds?$sourcepath">Sources</A>};
 
-            if ($www ne "") {
+	    if ($www ne "") {
 		$data{$i} .= qq{ | <a href="$www">Main Web Site</a>};
-            }
+	    }
 
-    	    $ownerurl = $owner;
-    	    $ownerurl =~ s/&lt;/</g;
-    	    $owenrurl =~ s/&gt;/>/g;
+	    my $ownerurl = $owner;
+	    $ownerurl =~ s/&lt;/</g;
+	    $ownerurl =~ s/&gt;/>/g;
 	    $data{$i} .= 
 		"<BR><I>Maintained by:</I> <A HREF=\"mailto:$ownerurl\">$owner</A>";
 
-    	    # If there are any dependencies, list them
-    	    if ($bdep ne "" || $rdep ne "") {
-    	    	$data{$i} .= "<BR><I>Requires:</I> ";
-    	    	@dep = split(/ /, "$bdep $rdep");
-		local($last) = '';
-    	    	foreach $j (sort @dep) {
+	    # If there are any dependencies, list them
+	    if ($bdep ne "" || $rdep ne "") {
+		$data{$i} .= "<BR><I>Requires:</I> ";
+		my @dep = split(/ /, "$bdep $rdep");
+		my $last = '';
+		foreach my $j (sort @dep) {
 		    next if $j eq $last;
 		    $last = $j;
-    	    	    $data{$i} .= " <A HREF=\"##$j##.html#$j\">$j</A>,";
-    	    	}
-    	    	# remove the trailing comma
-    	    	chop $data{$i};
-    	    }
-
-    	    # If the port is listed in more than one category, throw
-    	    # in some cross references
-    	    if ($#cat > 0) {
-    	    	$data{$i} .= "<BR><EM>Also listed in:</EM> ";
-    	    	foreach $j (@cat) {
-    	    	    if ($j ne $i) {
+		    $data{$i} .= " <A HREF=\"##$j##.html#$j\">$j</A>,";
+		}
+		# remove the trailing comma
+		chop $data{$i};
+	    }
+
+	    # If the port is listed in more than one category, throw
+	    # in some cross references
+	    if ($#cat > 0) {
+		$data{$i} .= "<BR><EM>Also listed in:</EM> ";
+		foreach my $j (@cat) {
+		    if ($j ne $i) {
 			if ($j eq $cat[0]) {
 			    $data{$i} .= " <STRONG><A HREF=\"$j.html#$name\">\u$j</A></STRONG>,";
 			}
 			else {
 			    $data{$i} .= " <A HREF=\"$j.html#$name\">\u$j</A>,";
 			}
-    	    	    }
-    	    	}
-    	    	# remove the trailing comma
-    	    	chop($data{$i});
-    	    }
-    	    $data{$i} .= "<P></P></DD>\n"
+		    }
+		}
+		# remove the trailing comma
+		chop($data{$i});
+	    }
+	    $data{$i} .= "<P></P></DD>\n"
 	}
 
     	# Add an entry to the master index
 
 	# workaround for SGML bug, `--' is not allowed in comments
-	local ($sname) = $name; 
+	my $sname = $name; 
 	$sname =~ s/--/-=/g;
     	$master[$portnumber] = 
     	    "<!-- $sname --><STRONG><A HREF=\"$cat[0].html#$name\">$name</A></STRONG> " .
@@ -217,65 +264,74 @@
 	$portnumber++;
     }
 
-    open(MOUTF, ">index.sgml");
+    $moutf->open(">index.sgml");
 
-    &header(MOUTF, "FreeBSD Ports");
-#    print MOUTF "<!--#include virtual=\"./ports.inc\" -->\n";
-    print MOUTF "&blurb;";
-    print MOUTF "<hr><P>There are currently $portnumber ports in the
-FreeBSD Ports Collection.<br> <A HREF=\"$baseFTP/ports.tar.gz\">Download
-a gzip'd tar file of all $portnumber ports</A> (about 25 megabytes) or
-browse the following categories:\n";
-
-    print MOUTF "<UL>\n";
-
-    @foos = sort(keys %stats);
-    foreach $key  (@foos) {
-	# For the master file...
-	print MOUTF 
-	    "<LI><A HREF=\"$key.html\">\u$key</A> <em>($stats{$key})</em>";
-	if ($category_description{$key}) {
-	    print MOUTF " -- " . $category_description{$key};
-	}
+    header($moutf, "FreeBSD Ports");
+#   print $moutf "<!--#include virtual=\"./ports.inc\" -->\n";
+    print $moutf "&blurb;";
+    print $moutf <<EOF;
+<HR>
+<P>There are currently $portnumber ports in the FreeBSD Ports Collection.<BR>
+You may download a
+<A HREF="$baseFTP/ports.tar.gz">gzip'd tar file of all ports</A>
+(about 30 megabytes), view the
+<A HREF=\"master-index.html\">alphabetical list of all ports</A>,
+or browse the ports in the following categories:
+<A NAME=\"categories\"></A>
+EOF
+
+    foreach my $cg (@category_groups) {
+	print $moutf "<H3>",$category_groups{$cg},"</H3>\n";
+	print $moutf "<UL>\n";
+	foreach my $key (sort(keys(%stats))) {
+	    next if ($category_description{$key}{group} ne $cg);
+	    # For the master file...
+	    print $moutf 
+		"<LI><A HREF=\"$key.html\">\u$key</A> <em>($stats{$key})</em>";
+	    if ($category_description{$key}{desc}) {
+		print $moutf " -- " . $category_description{$key}{desc};
+	    }
 
-	# Someone forgot to add a category to the description file
-	# or there is a typo in the category field.
-	else {
-	    warn "No description found for category: ``$key''!\n";
-	    warn "Please fix me or send an E-Mail to doc\@FreeBSD.org\a\n";
-	    sleep(3);
-	}
-	print MOUTF "</LI>\n";
+	    # Someone forgot to add a category to the description file
+	    # or there is a typo in the category field.
+	    else {
+		warn "No description found for category: ``$key''!\n";
+		warn "Please fix me or send an E-Mail to doc\@FreeBSD.org\a\n";
+		sleep(3);
+	    }
+	    print $moutf "</LI>\n";
 
-	# Create the category file
-	open(OUTF, ">$key.sgml");
-	&header(OUTF, "FreeBSD Ports: \u$key");
-	if ($category_description{$key}) {
-	    print OUTF "<h3>", $category_description{$key}, "</h3>\n";
-	}
-	print OUTF "<DL>\n";
-	$d = join("\n", sort(split(/\n/, $data{$key})));
-    	$d =~ s/##([^#]*)##/$catkey{$1}/g;
-	print OUTF $d;
-	print OUTF "</DL>\n";
-	&footer(OUTF, "<p></p><A HREF=\"index.html\">Port Categories</A>" .
-	    " -- <A HREF=\"master-index.html\">Index</A>");
-	close(OUTF);
+	    # Create the category file
+	    $outf->open(">$key.sgml");
+	    header($outf, "FreeBSD Ports: \u$key");
+	    if ($category_description{$key}{desc}) {
+		print $outf "<h3>", $category_description{$key}{desc}, "</h3>\n";
+	    }
+	    print $outf "<DL>\n";
+	    my $d = join("\n", sort(split(/\n/, $data{$key})));
+	    $d =~ s/##([^#]*)##/$catkey{$1}/g;
+	    print $outf $d;
+	    print $outf "</DL>\n";
+	    footer($outf, "<p></p><A HREF=\"index.html\">Port Categories</A>" .
+		" -- <A HREF=\"master-index.html\">Index</A>");
+	    $outf->close;
 
+	}
+	print $moutf "</UL>\n";
     }
-    print MOUTF "</UL>\n";
-    print MOUTF 
+
+    print $moutf 
     	"<UL><LI><A HREF=\"master-index.html\">Alphabetical list of all $portnumber ports</A></LI></UL>\n";
-    &footer(MOUTF, "");
-    close(MOUTF);
+    footer($moutf, "");
+    $moutf->close;
 
     # Create the master index file
-    open(MINDEX, ">master-index.sgml");
-    &header(MINDEX, "FreeBSD Ports Collection Index");
-    print MINDEX "<P>\n";
-    print MINDEX sort @master;
-    print MINDEX "</P>";
-    &footer(MINDEX, "<A HREF=\"index.html\">Port Categories</A>");
-    close(MINDEX);
+    $mindex->open(">master-index.sgml");
+    header($mindex, "FreeBSD Ports Collection Index");
+    print $mindex "<P>\n";
+    print $mindex sort @master;
+    print $mindex "</P>";
+    footer($mindex, "<A HREF=\"index.html\">Port Categories</A>");
+    $mindex->close;
     close(INDEX);
 }
Comment 1 Mark Linimon 2005-10-16 04:45:36 UTC 
A prototype of these changes merged with my updated changes in www/45169
is at http://people.FreeBSD.org/~linimon/www/en/ports/index.html.

Applying the patch in 87500 followed by 45169 gives you one conflict which
can then be resolved by applying the following:
Comment 2 Simon L. B. Nielsen freebsd_committer freebsd_triage 2005-10-16 12:29:17 UTC 
On 2005.10.16 09:45:36 +1000, Edwin Groothuis wrote:
> 
> >Description:
> 
> - Add port groups as suggested by linimon@
>   http://people.freebsd.org/~linimon/www/en/ports/index.html#logical_groups
> 
> - Cleanup portindex to be "use strict" and "-w" safe


Which perl versions have you tested this with?  More specificly, does
it work with perl 5.0 ?

-- 
Simon L. Nielsen
Comment 3 edwin 2005-10-16 13:26:12 UTC 
On Sun, Oct 16, 2005 at 01:29:17PM +0200, Simon L. Nielsen wrote:
> On 2005.10.16 09:45:36 +1000, Edwin Groothuis wrote:
> > 
> > >Description:
> > 
> > - Add port groups as suggested by linimon@
> >   http://people.freebsd.org/~linimon/www/en/ports/index.html#logical_groups
> > 
> > - Cleanup portindex to be "use strict" and "-w" safe
> 
> Which perl versions have you tested this with?  More specificly, does
> it work with perl 5.0 ?

I've tested it on a 5.2.1 machine with perl 5.8.6 and on a 4.8
machine with perl 5.005_03.

Edwin

-- 
Edwin Groothuis      |            Personal website: http://www.mavetju.org
edwin@mavetju.org    |          Weblog: http://weblog.barnet.com.au/edwin/
Comment 4 Simon L. B. Nielsen freebsd_committer freebsd_triage 2005-10-16 21:15:35 UTC 
On 2005.10.16 22:26:12 +1000, Edwin Groothuis wrote:
> On Sun, Oct 16, 2005 at 01:29:17PM +0200, Simon L. Nielsen wrote:
> > On 2005.10.16 09:45:36 +1000, Edwin Groothuis wrote:
> > > 
> > > >Description:
> > > 
> > > - Add port groups as suggested by linimon@
> > >   http://people.freebsd.org/~linimon/www/en/ports/index.html#logical_groups
> > > 
> > > - Cleanup portindex to be "use strict" and "-w" safe
> > 
> > Which perl versions have you tested this with?  More specificly, does
> > it work with perl 5.0 ?
> 
> I've tested it on a 5.2.1 machine with perl 5.8.6 and on a 4.8
> machine with perl 5.005_03.


OK, good.  Just want to make sure, since www.freebsd.org still use
perl 5.0.

-- 
Simon L. Nielsen
Comment 5 Mark Linimon freebsd_committer freebsd_triage 2005-10-18 04:45:42 UTC 
State Changed
From-To: open->closed

Committed, thanks.