|
Lines 1-694
Link Here
|
| 1 |
<?xml version="1.0" encoding="iso-8859-1"?> |
1 |
<?xml version="1.0" encoding="iso-8859-1"?> |
| 2 |
<!DOCTYPE article PUBLIC "-//FreeBSD//DTD DocBook XML V5.0-Based Extension//EN" |
2 |
<!DOCTYPE article PUBLIC "-//FreeBSD//DTD DocBook XML V5.0-Based Extension//EN" |
| 3 |
"http://www.FreeBSD.org/XML/share/xml/freebsd50.dtd"> |
3 |
"http://www.FreeBSD.org/XML/share/xml/freebsd50.dtd"> |
| 4 |
<article xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en"> |
4 |
<article xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en"> |
| 5 |
<info><title>Mirroring FreeBSD</title> |
5 |
<info><title>Mirroring FreeBSD</title> |
| 6 |
|
6 |
|
| 7 |
<authorgroup> |
7 |
<authorgroup> |
| 8 |
<author><personname><firstname>Jun</firstname><surname>Kuriyama</surname></personname><affiliation> |
8 |
<author><personname><firstname>Jun</firstname><surname>Kuriyama</surname></personname><affiliation> |
| 9 |
<address><email>kuriyama@FreeBSD.org</email></address> |
9 |
<address><email>kuriyama@FreeBSD.org</email></address> |
| 10 |
</affiliation></author> |
10 |
</affiliation></author> |
| 11 |
<author><personname><firstname>Valentino</firstname><surname>Vaschetto</surname></personname><affiliation> |
11 |
<author><personname><firstname>Valentino</firstname><surname>Vaschetto</surname></personname><affiliation> |
| 12 |
<address><email>logo@FreeBSD.org</email></address> |
12 |
<address><email>logo@FreeBSD.org</email></address> |
| 13 |
</affiliation></author> |
13 |
</affiliation></author> |
| 14 |
<author><personname><firstname>Daniel</firstname><surname>Lang</surname></personname><affiliation> |
14 |
<author><personname><firstname>Daniel</firstname><surname>Lang</surname></personname><affiliation> |
| 15 |
<address><email>dl@leo.org</email></address> |
15 |
<address><email>dl@leo.org</email></address> |
| 16 |
</affiliation></author> |
16 |
</affiliation></author> |
| 17 |
<author><personname><firstname>Ken</firstname><surname>Smith</surname></personname><affiliation> |
17 |
<author><personname><firstname>Ken</firstname><surname>Smith</surname></personname><affiliation> |
| 18 |
<address><email>kensmith@FreeBSD.org</email></address> |
18 |
<address><email>kensmith@FreeBSD.org</email></address> |
| 19 |
</affiliation></author> |
19 |
</affiliation></author> |
| 20 |
</authorgroup> |
20 |
</authorgroup> |
| 21 |
|
21 |
|
| 22 |
<legalnotice xml:id="trademarks" role="trademarks"> |
22 |
<legalnotice xml:id="trademarks" role="trademarks"> |
| 23 |
&tm-attrib.freebsd; |
23 |
&tm-attrib.freebsd; |
| 24 |
&tm-attrib.general; |
24 |
&tm-attrib.general; |
| 25 |
</legalnotice> |
25 |
</legalnotice> |
| 26 |
|
26 |
|
| 27 |
<pubdate>$FreeBSD$</pubdate> |
27 |
<pubdate>$FreeBSD$</pubdate> |
| 28 |
|
28 |
|
| 29 |
<releaseinfo>$FreeBSD$</releaseinfo> |
29 |
<releaseinfo>$FreeBSD$</releaseinfo> |
| 30 |
|
30 |
|
| 31 |
<abstract> |
31 |
<abstract> |
| 32 |
<para>An in-progress article on how to mirror FreeBSD, aimed at |
32 |
<para>An in-progress article on how to mirror FreeBSD, aimed at |
| 33 |
hub administrators.</para> |
33 |
hub administrators.</para> |
| 34 |
</abstract> |
34 |
</abstract> |
| 35 |
</info> |
35 |
</info> |
| 36 |
|
36 |
|
| 37 |
<note> |
37 |
<note> |
| 38 |
<para>We are not accepting new mirrors at this time.</para> |
38 |
<para>We are not accepting new mirrors at this time.</para> |
| 39 |
</note> |
39 |
</note> |
| 40 |
|
40 |
|
| 41 |
<sect1 xml:id="mirror-contact"> |
41 |
<sect1 xml:id="mirror-contact"> |
| 42 |
<title>Contact Information</title> |
42 |
<title>Contact Information</title> |
| 43 |
|
43 |
|
| 44 |
<para>The Mirror System Coordinators can be reached through email |
44 |
<para>The Mirror System Coordinators can be reached through email |
| 45 |
at <email>mirror-admin@FreeBSD.org</email>. There is also |
45 |
at <email>mirror-admin@FreeBSD.org</email>. There is also |
| 46 |
a &a.hubs;.</para> |
46 |
a &a.hubs;.</para> |
| 47 |
</sect1> |
47 |
</sect1> |
| 48 |
|
48 |
|
| 49 |
<sect1 xml:id="mirror-requirements"> |
49 |
<sect1 xml:id="mirror-requirements"> |
| 50 |
<title>Requirements for FreeBSD mirrors</title> |
50 |
<title>Requirements for FreeBSD mirrors</title> |
| 51 |
<sect2 xml:id="mirror-diskspace"> |
51 |
<sect2 xml:id="mirror-diskspace"> |
| 52 |
<title>Disk Space</title> |
52 |
<title>Disk Space</title> |
| 53 |
<para> |
53 |
<para> |
| 54 |
Disk space is one of the most important requirements. |
54 |
Disk space is one of the most important requirements. |
| 55 |
Depending on the set of releases, architectures, |
55 |
Depending on the set of releases, architectures, |
| 56 |
and degree of completeness you want to mirror, a huge |
56 |
and degree of completeness you want to mirror, a huge |
| 57 |
amount of disk space may be consumed. Also keep in mind |
57 |
amount of disk space may be consumed. Also keep in mind |
| 58 |
that <emphasis>official</emphasis> mirrors are probably required to be |
58 |
that <emphasis>official</emphasis> mirrors are probably required to be |
| 59 |
complete. The web pages should |
59 |
complete. The web pages should |
| 60 |
always be mirrored completely. Also note that the |
60 |
always be mirrored completely. Also note that the |
| 61 |
numbers stated here are reflecting the current |
61 |
numbers stated here are reflecting the current |
| 62 |
state (at &rel2.current;-RELEASE/&rel.current;-RELEASE). Further development and |
62 |
state (at &rel2.current;-RELEASE/&rel.current;-RELEASE). Further development and |
| 63 |
releases will only increase the required amount. |
63 |
releases will only increase the required amount. |
| 64 |
Also make sure to keep some (ca. 10-20%) extra space |
64 |
Also make sure to keep some (ca. 10-20%) extra space |
| 65 |
around just to be sure. |
65 |
around just to be sure. |
| 66 |
Here are some approximate figures: |
66 |
Here are some approximate figures: |
| 67 |
</para> |
67 |
</para> |
| 68 |
<itemizedlist> |
68 |
<itemizedlist> |
| 69 |
<listitem><para>Full FTP Distribution: 1.4 TB</para></listitem> |
69 |
<listitem><para>Full FTP Distribution: 1.4 TB</para></listitem> |
| 70 |
<listitem><para>CTM deltas: 10 GB</para></listitem> |
70 |
<listitem><para>CTM deltas: 10 GB</para></listitem> |
| 71 |
<listitem><para>Web pages: 1GB</para></listitem> |
71 |
<listitem><para>Web pages: 1GB</para></listitem> |
| 72 |
</itemizedlist> |
72 |
</itemizedlist> |
| 73 |
<para> |
73 |
<para> |
| 74 |
The current disk usage of FTP Distribution can be found at |
74 |
The current disk usage of FTP Distribution can be found at |
| 75 |
<link xlink:href="ftp://ftp.FreeBSD.org/pub/FreeBSD/dir.sizes">ftp://ftp.FreeBSD.org/pub/FreeBSD/dir.sizes</link>. |
75 |
<link xlink:href="ftp://ftp.FreeBSD.org/pub/FreeBSD/dir.sizes">ftp://ftp.FreeBSD.org/pub/FreeBSD/dir.sizes</link>. |
| 76 |
</para> |
76 |
</para> |
| 77 |
</sect2> |
77 |
</sect2> |
| 78 |
<sect2 xml:id="mirror-bandwidth"> |
78 |
<sect2 xml:id="mirror-bandwidth"> |
| 79 |
<title>Network Connection/Bandwidth</title> |
79 |
<title>Network Connection/Bandwidth</title> |
| 80 |
<para> |
80 |
<para> |
| 81 |
Of course, you need to be connected to the Internet. |
81 |
Of course, you need to be connected to the Internet. |
| 82 |
The required bandwidth depends on your intended use |
82 |
The required bandwidth depends on your intended use |
| 83 |
of the mirror. If you just want to mirror some |
83 |
of the mirror. If you just want to mirror some |
| 84 |
parts of FreeBSD for local use at your site/intranet, |
84 |
parts of FreeBSD for local use at your site/intranet, |
| 85 |
the demand may be much smaller than if you want to |
85 |
the demand may be much smaller than if you want to |
| 86 |
make the files publicly available. If you intend |
86 |
make the files publicly available. If you intend |
| 87 |
to become an official mirror, the bandwidth required will be even higher. We can only give rough |
87 |
to become an official mirror, the bandwidth required will be even higher. We can only give rough |
| 88 |
estimates here: |
88 |
estimates here: |
| 89 |
</para> |
89 |
</para> |
| 90 |
<itemizedlist> |
90 |
<itemizedlist> |
| 91 |
<listitem><para>Local site, no public access: basically no minimum, |
91 |
<listitem><para>Local site, no public access: basically no minimum, |
| 92 |
but < 2 Mbps could make syncing too slow.</para></listitem> |
92 |
but < 2 Mbps could make syncing too slow.</para></listitem> |
| 93 |
<listitem><para>Unofficial public site: 34 Mbps is probably a good start.</para></listitem> |
93 |
<listitem><para>Unofficial public site: 34 Mbps is probably a good start.</para></listitem> |
| 94 |
<listitem><para>Official site: > 100 Mbps is recommended, and your host |
94 |
<listitem><para>Official site: > 100 Mbps is recommended, and your host |
| 95 |
should be connected as close as possible to your border router.</para></listitem> |
95 |
should be connected as close as possible to your border router.</para></listitem> |
| 96 |
</itemizedlist> |
96 |
</itemizedlist> |
| 97 |
</sect2> |
97 |
</sect2> |
| 98 |
<sect2 xml:id="mirror-system"> |
98 |
<sect2 xml:id="mirror-system"> |
| 99 |
<title>System Requirements, CPU, RAM</title> |
99 |
<title>System Requirements, CPU, RAM</title> |
| 100 |
<para> |
100 |
<para> |
| 101 |
One thing this depends on the expected number of clients, |
101 |
One thing this depends on the expected number of clients, |
| 102 |
which is determined by the server's policy. It is |
102 |
which is determined by the server's policy. It is |
| 103 |
also affected by the types of services you want to offer. |
103 |
also affected by the types of services you want to offer. |
| 104 |
Plain FTP or HTTP services may not require a huge |
104 |
Plain FTP or HTTP services may not require a huge |
| 105 |
amount of resources. Watch out if you provide |
105 |
amount of resources. Watch out if you provide |
| 106 |
rsync. This can have a huge |
106 |
rsync. This can have a huge |
| 107 |
impact on CPU and memory requirements as it is |
107 |
impact on CPU and memory requirements as it is |
| 108 |
considered a memory hog. |
108 |
considered a memory hog. |
| 109 |
The following |
109 |
The following |
| 110 |
are just examples to give you a very rough hint. |
110 |
are just examples to give you a very rough hint. |
| 111 |
</para> |
111 |
</para> |
| 112 |
<para> |
112 |
<para> |
| 113 |
For a moderately visited site that offers |
113 |
For a moderately visited site that offers |
| 114 |
<application>rsync</application>, you might |
114 |
<application>rsync</application>, you might |
| 115 |
consider a current CPU with around 800MHz - 1 GHz, |
115 |
consider a current CPU with around 800MHz - 1 GHz, |
| 116 |
and at least 512MB RAM. This is probably the |
116 |
and at least 512MB RAM. This is probably the |
| 117 |
minimum you want for an <emphasis>official</emphasis> |
117 |
minimum you want for an <emphasis>official</emphasis> |
| 118 |
site. |
118 |
site. |
| 119 |
</para> |
119 |
</para> |
| 120 |
<para> |
120 |
<para> |
| 121 |
For a frequently used site you definitely need |
121 |
For a frequently used site you definitely need |
| 122 |
more RAM (consider 2GB as a good start) |
122 |
more RAM (consider 2GB as a good start) |
| 123 |
and possibly more CPU, which could also mean |
123 |
and possibly more CPU, which could also mean |
| 124 |
that you need to go for a SMP system. |
124 |
that you need to go for a SMP system. |
| 125 |
</para> |
125 |
</para> |
| 126 |
<para> |
126 |
<para> |
| 127 |
You also want to consider a fast disk subsystem. |
127 |
You also want to consider a fast disk subsystem. |
| 128 |
Operations on the SVN repository require a fast |
128 |
Operations on the SVN repository require a fast |
| 129 |
disk subsystem (RAID is highly advised). A SCSI |
129 |
disk subsystem (RAID is highly advised). A SCSI |
| 130 |
controller that has a cache of its own can also |
130 |
controller that has a cache of its own can also |
| 131 |
speed up things since most of these services incur a |
131 |
speed up things since most of these services incur a |
| 132 |
large number of small modifications to the disk. |
132 |
large number of small modifications to the disk. |
| 133 |
</para> |
133 |
</para> |
| 134 |
</sect2> |
134 |
</sect2> |
| 135 |
<sect2 xml:id="mirror-services"> |
135 |
<sect2 xml:id="mirror-services"> |
| 136 |
<title>Services to offer</title> |
136 |
<title>Services to offer</title> |
| 137 |
<para> |
137 |
<para> |
| 138 |
Every mirror site is required to have a set of core services |
138 |
Every mirror site is required to have a set of core services |
| 139 |
available. In addition to these required services, there are |
139 |
available. In addition to these required services, there are |
| 140 |
a number of optional services that |
140 |
a number of optional services that |
| 141 |
server administrators may choose to offer. This section explains |
141 |
server administrators may choose to offer. This section explains |
| 142 |
which services you can provide and how to go about implementing them. |
142 |
which services you can provide and how to go about implementing them. |
| 143 |
</para> |
143 |
</para> |
| 144 |
<sect3 xml:id="mirror-serv-ftp"> |
144 |
<sect3 xml:id="mirror-serv-ftp"> |
| 145 |
<title>FTP (required for FTP fileset)</title> |
145 |
<title>FTP (required for FTP fileset)</title> |
| 146 |
<para> |
146 |
<para> |
| 147 |
This is one of the most basic services, and |
147 |
This is one of the most basic services, and |
| 148 |
it is required for each mirror offering public |
148 |
it is required for each mirror offering public |
| 149 |
FTP distributions. FTP access must be |
149 |
FTP distributions. FTP access must be |
| 150 |
anonymous, and no upload/download ratios |
150 |
anonymous, and no upload/download ratios |
| 151 |
are allowed (a ridiculous thing anyway). |
151 |
are allowed (a ridiculous thing anyway). |
| 152 |
Upload capability is not required (and <emphasis>must</emphasis> |
152 |
Upload capability is not required (and <emphasis>must</emphasis> |
| 153 |
never be allowed for the FreeBSD file space). |
153 |
never be allowed for the FreeBSD file space). |
| 154 |
Also the FreeBSD archive should be available under |
154 |
Also the FreeBSD archive should be available under |
| 155 |
the path <filename>/pub/FreeBSD</filename>. |
155 |
the path <filename>/pub/FreeBSD</filename>. |
| 156 |
</para> |
156 |
</para> |
| 157 |
<para> |
157 |
<para> |
| 158 |
There is a lot of software available which |
158 |
There is a lot of software available which |
| 159 |
can be set up to allow anonymous FTP |
159 |
can be set up to allow anonymous FTP |
| 160 |
(in alphabetical order).</para> |
160 |
(in alphabetical order).</para> |
| 161 |
<itemizedlist> |
161 |
<itemizedlist> |
| 162 |
<listitem><para><command>/usr/libexec/ftpd</command>: FreeBSD's own ftpd |
162 |
<listitem><para><command>/usr/libexec/ftpd</command>: FreeBSD's own ftpd |
| 163 |
can be used. Be sure to read &man.ftpd.8;.</para> |
163 |
can be used. Be sure to read &man.ftpd.8;.</para> |
| 164 |
</listitem> |
164 |
</listitem> |
| 165 |
<listitem> |
165 |
<listitem> |
| 166 |
<para><package>ftp/ncftpd</package>: A commercial package, |
166 |
<para><package>ftp/ncftpd</package>: A commercial package, |
| 167 |
free for educational use.</para> |
167 |
free for educational use.</para> |
| 168 |
</listitem> |
168 |
</listitem> |
| 169 |
<listitem> |
169 |
<listitem> |
| 170 |
<para><package>ftp/oftpd</package>: An ftpd designed with |
170 |
<para><package>ftp/oftpd</package>: An ftpd designed with |
| 171 |
security as a main focus.</para> |
171 |
security as a main focus.</para> |
| 172 |
</listitem> |
172 |
</listitem> |
| 173 |
<listitem> |
173 |
<listitem> |
| 174 |
<para><package>ftp/proftpd</package>: A modular and very flexible ftpd.</para> |
174 |
<para><package>ftp/proftpd</package>: A modular and very flexible ftpd.</para> |
| 175 |
</listitem> |
175 |
</listitem> |
| 176 |
<listitem> |
176 |
<listitem> |
| 177 |
<para><package>ftp/pure-ftpd</package>: Another ftpd developed with |
177 |
<para><package>ftp/pure-ftpd</package>: Another ftpd developed with |
| 178 |
security in mind.</para> |
178 |
security in mind.</para> |
| 179 |
</listitem> |
179 |
</listitem> |
| 180 |
<listitem><para><package>ftp/twoftpd</package>: As above.</para></listitem> |
180 |
<listitem><para><package>ftp/twoftpd</package>: As above.</para></listitem> |
| 181 |
<listitem><para><package>ftp/vsftpd</package>: The <quote>very secure</quote> ftpd.</para></listitem> |
181 |
<listitem><para><package>ftp/vsftpd</package>: The <quote>very secure</quote> ftpd.</para></listitem> |
| 182 |
</itemizedlist> |
182 |
</itemizedlist> |
| 183 |
<para>FreeBSD's <application>ftpd</application>, <application>proftpd</application> |
183 |
<para>FreeBSD's <application>ftpd</application>, <application>proftpd</application> |
| 184 |
and maybe <application>ncftpd</application> |
184 |
and maybe <application>ncftpd</application> |
| 185 |
are among the most commonly used FTPds. |
185 |
are among the most commonly used FTPds. |
| 186 |
The others do not have a large userbase among mirror sites. One |
186 |
The others do not have a large userbase among mirror sites. One |
| 187 |
thing to consider is that you may need flexibility in limiting |
187 |
thing to consider is that you may need flexibility in limiting |
| 188 |
how many simultaneous connections are allowed, thus limiting how |
188 |
how many simultaneous connections are allowed, thus limiting how |
| 189 |
much network bandwidth and system resources are consumed. |
189 |
much network bandwidth and system resources are consumed. |
| 190 |
</para> |
190 |
</para> |
| 191 |
</sect3> |
191 |
</sect3> |
| 192 |
<sect3 xml:id="mirror-serv-rsync"> |
192 |
<sect3 xml:id="mirror-serv-rsync"> |
| 193 |
<title>Rsync (optional for FTP fileset)</title> |
193 |
<title>Rsync (optional for FTP fileset)</title> |
| 194 |
<para> |
194 |
<para> |
| 195 |
<application>Rsync</application> is often offered for access to the |
195 |
<application>Rsync</application> is often offered for access to the |
| 196 |
contents of the FTP area of FreeBSD, so other mirror sites can use your system as their source. The |
196 |
contents of the FTP area of FreeBSD, so other mirror sites can use your system as their source. The |
| 197 |
protocol is different from FTP in many ways. |
197 |
protocol is different from FTP in many ways. |
| 198 |
It is much more |
198 |
It is much more |
| 199 |
bandwidth friendly, as only differences between files |
199 |
bandwidth friendly, as only differences between files |
| 200 |
are transferred instead of whole files when they change. |
200 |
are transferred instead of whole files when they change. |
| 201 |
<application>Rsync</application> does require a significant amount of memory for |
201 |
<application>Rsync</application> does require a significant amount of memory for |
| 202 |
each instance. The size depends on the size of |
202 |
each instance. The size depends on the size of |
| 203 |
the synced module in terms of the number of directories and |
203 |
the synced module in terms of the number of directories and |
| 204 |
files. <application>Rsync</application> can use <command>rsh</command> and |
204 |
files. <application>Rsync</application> can use <command>rsh</command> and |
| 205 |
<command>ssh</command> (now default) as a transport, |
205 |
<command>ssh</command> (now default) as a transport, |
| 206 |
or use its own protocol for stand-alone access |
206 |
or use its own protocol for stand-alone access |
| 207 |
(this is the preferred method for public rsync servers). |
207 |
(this is the preferred method for public rsync servers). |
| 208 |
Authentication, connection limits, and other restrictions |
208 |
Authentication, connection limits, and other restrictions |
| 209 |
may be applied. There is just one software package |
209 |
may be applied. There is just one software package |
| 210 |
available:</para> |
210 |
available:</para> |
| 211 |
<itemizedlist> |
211 |
<itemizedlist> |
| 212 |
<listitem><para><package>net/rsync</package></para></listitem> |
212 |
<listitem><para><package>net/rsync</package></para></listitem> |
| 213 |
</itemizedlist> |
213 |
</itemizedlist> |
| 214 |
</sect3> |
214 |
</sect3> |
| 215 |
<sect3 xml:id="mirror-serv-http"> |
215 |
<sect3 xml:id="mirror-serv-http"> |
| 216 |
<title>HTTP (required for web pages, optional for FTP fileset)</title> |
216 |
<title>HTTP (required for web pages, optional for FTP fileset)</title> |
| 217 |
<para> |
217 |
<para> |
| 218 |
If you want to offer the FreeBSD web pages, you will need |
218 |
If you want to offer the FreeBSD web pages, you will need |
| 219 |
to install a web server. |
219 |
to install a web server. |
| 220 |
You may optionally offer the FTP fileset via HTTP. |
220 |
You may optionally offer the FTP fileset via HTTP. |
| 221 |
The choice of web server software is left up to the mirror administrator. |
221 |
The choice of web server software is left up to the mirror administrator. |
| 222 |
Some of the most popular choices are:</para> |
222 |
Some of the most popular choices are:</para> |
| 223 |
|
223 |
|
| 224 |
<itemizedlist> |
224 |
<itemizedlist> |
| 225 |
<listitem> |
225 |
<listitem> |
| 226 |
<para><package>www/apache22</package>: |
226 |
<para><package>www/apache24</package>: |
| 227 |
<application>Apache</application> is the most widely |
227 |
<application>Apache</application> is still one of the most widely |
| 228 |
deployed web server on the Internet. It is used |
228 |
deployed web servers on the Internet. It is used |
| 229 |
extensively by the FreeBSD Project.</para> |
229 |
extensively by the FreeBSD Project.</para> |
| 230 |
</listitem> |
230 |
</listitem> |
| 231 |
|
231 |
|
| 232 |
<listitem> |
232 |
<listitem> |
| 233 |
<para><package>www/thttpd</package>: |
233 |
<para><package>www/boa</package>: |
| 234 |
If you are going to be serving a large amount of static content |
234 |
<application>Boa</application> is a single-tasking HTTP server. |
| 235 |
you may find that using an application such as thttpd is more |
235 |
Unlike traditional web servers, it does not fork for each incoming |
| 236 |
efficient than <application>Apache</application>. It is |
236 |
connection, nor does it fork many copies of itself to handle multiple |
| 237 |
optimized for excellent performance on FreeBSD.</para> |
237 |
connections. Although, it should provide considerably great |
|
|
238 |
performance for purely static content.</para> |
| 238 |
</listitem> |
239 |
</listitem> |
| 239 |
|
240 |
|
| 240 |
<listitem> |
241 |
<listitem> |
| 241 |
<para><package>www/boa</package>: |
242 |
<para><package>www/cherokee</package>: |
| 242 |
<application>Boa</application> is another alternative to |
243 |
<application>>Cherokee</application> is a very fast, flexible and |
| 243 |
<application>thttpd</application> and |
244 |
easy to configure web server. It supports the widespread technologies |
| 244 |
<application>Apache</application>. It should provide |
245 |
nowadays: FastCGI, SCGI, PHP, CGI, SSL/TLS encrypted connections, |
| 245 |
considerably better performance than |
246 |
vhosts, users authentication, on the fly encoding and load balancing. |
| 246 |
<application>Apache</application> for purely static |
247 |
It also generates <application>Apache</application> compatible log |
| 247 |
content. It does not, at the time of this writing, |
248 |
files.</para> |
| 248 |
contain the same set of optimizations for FreeBSD that |
|
|
| 249 |
are found in <application>thttpd</application>.</para> |
| 250 |
</listitem> |
249 |
</listitem> |
| 251 |
|
250 |
|
| 252 |
<listitem> |
251 |
<listitem> |
|
|
252 |
<para><package>www/lighttpd</package>: |
| 253 |
<application>lighttpd</application> is a secure, fast, compliant and |
| 254 |
very flexible web server which has been optimized for high-performance |
| 255 |
environments. It has a very low memory footprint compared to other web |
| 256 |
servers and takes care of cpu-load.</para> |
| 257 |
</listitem> |
| 258 |
|
| 259 |
<listitem> |
| 253 |
<para><package>www/nginx</package>: |
260 |
<para><package>www/nginx</package>: |
| 254 |
<application>Nginx</application> is a high performance edge web |
261 |
<application>nginx</application> is a high performance edge web |
| 255 |
server with a low memory footprint and key features to build |
262 |
server with a low memory footprint and key features to build |
| 256 |
a modern and efficient web infrastructure. Features include |
263 |
a modern and efficient web infrastructure. Features include |
| 257 |
a HTTP server, HTTP and mail reverse proxy, caching, load |
264 |
a HTTP server, HTTP and mail reverse proxy, caching, load |
| 258 |
balancing, compression, request throttling, connection |
265 |
balancing, compression, request throttling, connection |
| 259 |
multiplexing and reuse, SSL offload and HTTP media |
266 |
multiplexing and reuse, SSL offload and HTTP media |
| 260 |
streaming.</para> |
267 |
streaming.</para> |
|
|
268 |
</listitem> |
| 269 |
|
| 270 |
<listitem> |
| 271 |
<para><package>www/thttpd</package>: |
| 272 |
If you are going to be serving a large amount of static content |
| 273 |
you may find that using an application such as |
| 274 |
<application>thttpd<application> is more efficient than others. |
| 275 |
It is also optimized for excellent performance on FreeBSD.</para> |
| 261 |
</listitem> |
276 |
</listitem> |
| 262 |
</itemizedlist> |
277 |
</itemizedlist> |
| 263 |
</sect3> |
278 |
</sect3> |
| 264 |
</sect2> |
279 |
</sect2> |
| 265 |
</sect1> |
280 |
</sect1> |
| 266 |
<sect1 xml:id="mirror-howto"> |
281 |
<sect1 xml:id="mirror-howto"> |
| 267 |
<title>How to Mirror FreeBSD</title> |
282 |
<title>How to Mirror FreeBSD</title> |
| 268 |
<para> |
283 |
<para> |
| 269 |
Ok, now you know the requirements and how to offer |
284 |
Ok, now you know the requirements and how to offer |
| 270 |
the services, but not how to get it. :-) |
285 |
the services, but not how to get it. :-) |
| 271 |
This section explains how to actually mirror |
286 |
This section explains how to actually mirror |
| 272 |
the various parts of FreeBSD, what tools to use, |
287 |
the various parts of FreeBSD, what tools to use, |
| 273 |
and where to mirror from. |
288 |
and where to mirror from. |
| 274 |
</para> |
289 |
</para> |
| 275 |
<sect2 xml:id="mirror-ftp-rsync"> |
290 |
<sect2 xml:id="mirror-ftp-rsync"> |
| 276 |
<title>Mirroring the FTP site</title> |
291 |
<title>Mirroring the FTP site</title> |
| 277 |
<para> |
292 |
<para> |
| 278 |
The FTP area is the largest amount of data that |
293 |
The FTP area is the largest amount of data that |
| 279 |
needs to be mirrored. It includes the <emphasis>distribution |
294 |
needs to be mirrored. It includes the <emphasis>distribution |
| 280 |
sets</emphasis> required for network installation, the |
295 |
sets</emphasis> required for network installation, the |
| 281 |
<emphasis>branches</emphasis> which are actually snapshots |
296 |
<emphasis>branches</emphasis> which are actually snapshots |
| 282 |
of checked-out source trees, the <emphasis>ISO Images</emphasis> |
297 |
of checked-out source trees, the <emphasis>ISO Images</emphasis> |
| 283 |
to write CD-ROMs with the installation distribution, |
298 |
to write CD-ROMs with the installation distribution, |
| 284 |
a live file system, and a snapshot of the ports tree. All of |
299 |
a live file system, and a snapshot of the ports tree. All of |
| 285 |
course for various FreeBSD versions, and various architectures. |
300 |
course for various FreeBSD versions, and various architectures. |
| 286 |
</para> |
301 |
</para> |
| 287 |
<para> |
302 |
<para> |
| 288 |
The best way to mirror the FTP area is <application>rsync</application>. |
303 |
The best way to mirror the FTP area is <application>rsync</application>. |
| 289 |
You can install the port <package>net/rsync</package> and then use |
304 |
You can install the port <package>net/rsync</package> and then use |
| 290 |
rsync to sync with your upstream host. |
305 |
rsync to sync with your upstream host. |
| 291 |
<application>rsync</application> is already mentioned |
306 |
<application>rsync</application> is already mentioned |
| 292 |
in <xref linkend="mirror-serv-rsync"/>. |
307 |
in <xref linkend="mirror-serv-rsync"/>. |
| 293 |
Since <application>rsync</application> access is not |
308 |
Since <application>rsync</application> access is not |
| 294 |
required, your preferred upstream site may not allow it. |
309 |
required, your preferred upstream site may not allow it. |
| 295 |
You may need to hunt around a little bit to find a site |
310 |
You may need to hunt around a little bit to find a site |
| 296 |
that allows <application>rsync</application> access.</para> |
311 |
that allows <application>rsync</application> access.</para> |
| 297 |
<note> |
312 |
<note> |
| 298 |
<para> |
313 |
<para> |
| 299 |
Since the number of <application>rsync</application> |
314 |
Since the number of <application>rsync</application> |
| 300 |
clients will have a significant impact on the server |
315 |
clients will have a significant impact on the server |
| 301 |
machine, most admins impose limitations on their |
316 |
machine, most admins impose limitations on their |
| 302 |
server. For a mirror, you should ask the site maintainer |
317 |
server. For a mirror, you should ask the site maintainer |
| 303 |
you are syncing from about their policy, and maybe |
318 |
you are syncing from about their policy, and maybe |
| 304 |
an exception for your host (since you are a mirror). |
319 |
an exception for your host (since you are a mirror). |
| 305 |
</para> |
320 |
</para> |
| 306 |
</note> |
321 |
</note> |
| 307 |
<para>A command line to mirror FreeBSD might look like:</para> |
322 |
<para>A command line to mirror FreeBSD might look like:</para> |
| 308 |
<screen>&prompt.user; <userinput>rsync -vaHz --delete rsync://ftp4.de.FreeBSD.org/FreeBSD/ /pub/FreeBSD/</userinput></screen> |
323 |
<screen>&prompt.user; <userinput>rsync -vaHz --delete rsync://ftp4.de.FreeBSD.org/FreeBSD/ /pub/FreeBSD/</userinput></screen> |
| 309 |
<para>Consult the documentation for <application>rsync</application>, |
324 |
<para>Consult the documentation for <application>rsync</application>, |
| 310 |
which is also available at |
325 |
which is also available at |
| 311 |
<link xlink:href="http://rsync.samba.org/">http://rsync.samba.org/</link>, |
326 |
<link xlink:href="http://rsync.samba.org/">http://rsync.samba.org/</link>, |
| 312 |
about the various options to be used with rsync. |
327 |
about the various options to be used with rsync. |
| 313 |
If you sync the whole module (unlike subdirectories), |
328 |
If you sync the whole module (unlike subdirectories), |
| 314 |
be aware that the module-directory (here "FreeBSD") |
329 |
be aware that the module-directory (here "FreeBSD") |
| 315 |
will not be created, so you cannot omit the target directory. |
330 |
will not be created, so you cannot omit the target directory. |
| 316 |
Also you might |
331 |
Also you might |
| 317 |
want to set up a script framework that calls such a command |
332 |
want to set up a script framework that calls such a command |
| 318 |
via &man.cron.8;. |
333 |
via &man.cron.8;. |
| 319 |
</para> |
334 |
</para> |
| 320 |
</sect2> |
335 |
</sect2> |
| 321 |
<sect2 xml:id="mirror-www"> |
336 |
<sect2 xml:id="mirror-www"> |
| 322 |
<title>Mirroring the WWW pages</title> |
337 |
<title>Mirroring the WWW pages</title> |
| 323 |
<para> |
338 |
<para> |
| 324 |
The FreeBSD website should only be mirrored via |
339 |
The FreeBSD website should only be mirrored via |
| 325 |
<application>rsync</application>.</para> |
340 |
<application>rsync</application>.</para> |
| 326 |
<para>A command line to mirror the FreeBSD web site might look like:</para> |
341 |
<para>A command line to mirror the FreeBSD web site might look like:</para> |
| 327 |
<screen>&prompt.user; <userinput>rsync -vaHz --delete rsync://bit0.us-west.freebsd.org/FreeBSD-www-data/ /usr/local/www/</userinput></screen> |
342 |
<screen>&prompt.user; <userinput>rsync -vaHz --delete rsync://bit0.us-west.freebsd.org/FreeBSD-www-data/ /usr/local/www/</userinput></screen> |
| 328 |
</sect2> |
343 |
</sect2> |
| 329 |
<sect2 xml:id="mirror-pkgs"> |
344 |
<sect2 xml:id="mirror-pkgs"> |
| 330 |
<title>Mirroring Packages</title> |
345 |
<title>Mirroring Packages</title> |
| 331 |
<para>Due to very high requirements of bandwidth, storage and |
346 |
<para>Due to very high requirements of bandwidth, storage and |
| 332 |
adminstration the &os; Project has decided not to allow public |
347 |
adminstration the &os; Project has decided not to allow public |
| 333 |
mirrors of packages. For sites with lots of machines, it might |
348 |
mirrors of packages. For sites with lots of machines, it might |
| 334 |
be advantagous to run a caching HTTP proxy for the &man.pkg.8; |
349 |
be advantagous to run a caching HTTP proxy for the &man.pkg.8; |
| 335 |
process. Alternatively specific packages and their dependencies |
350 |
process. Alternatively specific packages and their dependencies |
| 336 |
can be fetched by running something like the following:</para> |
351 |
can be fetched by running something like the following:</para> |
| 337 |
|
352 |
|
| 338 |
<screen>&prompt.user; <userinput>pkg fetch -d -o <replaceable>/usr/local/mirror</replaceable> <replaceable>vim</replaceable></userinput></screen> |
353 |
<screen>&prompt.user; <userinput>pkg fetch -d -o <replaceable>/usr/local/mirror</replaceable> <replaceable>vim</replaceable></userinput></screen> |
| 339 |
|
354 |
|
| 340 |
<para>Once those packages have been fetched, the repository metadata must be generated by running:</para> |
355 |
<para>Once those packages have been fetched, the repository metadata must be generated by running:</para> |
| 341 |
|
356 |
|
| 342 |
<screen>&prompt.user; <userinput>pkg repo <replaceable>/usr/local/mirror</replaceable></userinput></screen> |
357 |
<screen>&prompt.user; <userinput>pkg repo <replaceable>/usr/local/mirror</replaceable></userinput></screen> |
| 343 |
|
358 |
|
| 344 |
<para>Once the packages have been fetched and the metadata for the |
359 |
<para>Once the packages have been fetched and the metadata for the |
| 345 |
repository has been generated, serve the packages up to the |
360 |
repository has been generated, serve the packages up to the |
| 346 |
client machines via HTTP. For additional information see the |
361 |
client machines via HTTP. For additional information see the |
| 347 |
man pages for &man.pkg.8;, specifically the &man.pkg-repo.8; page. |
362 |
man pages for &man.pkg.8;, specifically the &man.pkg-repo.8; page. |
| 348 |
</para> |
363 |
</para> |
| 349 |
</sect2> |
364 |
</sect2> |
| 350 |
<sect2 xml:id="mirror-how-often"> |
365 |
<sect2 xml:id="mirror-how-often"> |
| 351 |
<title>How often should I mirror?</title> |
366 |
<title>How often should I mirror?</title> |
| 352 |
<para> |
367 |
<para> |
| 353 |
Every mirror should be updated at a minimum of once per day. |
368 |
Every mirror should be updated at a minimum of once per day. |
| 354 |
Certainly a script with locking to prevent multiple runs |
369 |
Certainly a script with locking to prevent multiple runs |
| 355 |
happening at the same time will be needed to run from |
370 |
happening at the same time will be needed to run from |
| 356 |
&man.cron.8;. Since nearly every admin does this in their own |
371 |
&man.cron.8;. Since nearly every admin does this in their own |
| 357 |
way, specific instructions cannot be provided. It could work |
372 |
way, specific instructions cannot be provided. It could work |
| 358 |
something like this: |
373 |
something like this: |
| 359 |
</para> |
374 |
</para> |
| 360 |
<procedure> |
375 |
<procedure> |
| 361 |
<step> |
376 |
<step> |
| 362 |
<para> |
377 |
<para> |
| 363 |
Put the command to run your mirroring application |
378 |
Put the command to run your mirroring application |
| 364 |
in a script. Use of a plain <command>/bin/sh</command> |
379 |
in a script. Use of a plain <command>/bin/sh</command> |
| 365 |
script is recommended. |
380 |
script is recommended. |
| 366 |
</para> |
381 |
</para> |
| 367 |
</step> |
382 |
</step> |
| 368 |
<step> |
383 |
<step> |
| 369 |
<para> |
384 |
<para> |
| 370 |
Add some output redirections so diagnostic |
385 |
Add some output redirections so diagnostic |
| 371 |
messages are logged to a file. |
386 |
messages are logged to a file. |
| 372 |
</para> |
387 |
</para> |
| 373 |
</step> |
388 |
</step> |
| 374 |
<step> |
389 |
<step> |
| 375 |
<para> |
390 |
<para> |
| 376 |
Test if your script works. Check the logs. |
391 |
Test if your script works. Check the logs. |
| 377 |
</para> |
392 |
</para> |
| 378 |
</step> |
393 |
</step> |
| 379 |
<step> |
394 |
<step> |
| 380 |
<para> |
395 |
<para> |
| 381 |
Use &man.crontab.1; to add the script to the |
396 |
Use &man.crontab.1; to add the script to the |
| 382 |
appropriate user's &man.crontab.5;. This should be a |
397 |
appropriate user's &man.crontab.5;. This should be a |
| 383 |
different user than what your FTP daemon runs as so that |
398 |
different user than what your FTP daemon runs as so that |
| 384 |
if file permissions inside your FTP area are not |
399 |
if file permissions inside your FTP area are not |
| 385 |
world-readable those files can not be accessed by anonymous |
400 |
world-readable those files can not be accessed by anonymous |
| 386 |
FTP. This is used to <quote>stage</quote> releases — |
401 |
FTP. This is used to <quote>stage</quote> releases — |
| 387 |
making sure all of the official mirror sites have all of the |
402 |
making sure all of the official mirror sites have all of the |
| 388 |
necessary release files on release day. |
403 |
necessary release files on release day. |
| 389 |
</para> |
404 |
</para> |
| 390 |
</step> |
405 |
</step> |
| 391 |
</procedure> |
406 |
</procedure> |
| 392 |
<para> |
407 |
<para> |
| 393 |
Here are some recommended schedules:</para> |
408 |
Here are some recommended schedules:</para> |
| 394 |
<itemizedlist> |
409 |
<itemizedlist> |
| 395 |
<listitem><para>FTP fileset: daily</para></listitem> |
410 |
<listitem><para>FTP fileset: daily</para></listitem> |
| 396 |
<listitem><para>WWW pages: daily</para></listitem> |
411 |
<listitem><para>WWW pages: daily</para></listitem> |
| 397 |
</itemizedlist> |
412 |
</itemizedlist> |
| 398 |
</sect2> |
413 |
</sect2> |
| 399 |
</sect1> |
414 |
</sect1> |
| 400 |
<sect1 xml:id="mirror-where"> |
415 |
<sect1 xml:id="mirror-where"> |
| 401 |
<title>Where to mirror from</title> |
416 |
<title>Where to mirror from</title> |
| 402 |
<para> |
417 |
<para> |
| 403 |
This is an important issue. So this section will |
418 |
This is an important issue. So this section will |
| 404 |
spend some effort to explain the backgrounds. We will say this |
419 |
spend some effort to explain the backgrounds. We will say this |
| 405 |
several times: under no circumstances should you mirror from |
420 |
several times: under no circumstances should you mirror from |
| 406 |
<systemitem class="fqdomainname">ftp.FreeBSD.org</systemitem>. |
421 |
<systemitem class="fqdomainname">ftp.FreeBSD.org</systemitem>. |
| 407 |
</para> |
422 |
</para> |
| 408 |
<sect2 xml:id="mirror-where-organization"> |
423 |
<sect2 xml:id="mirror-where-organization"> |
| 409 |
<title>A few words about the organization</title> |
424 |
<title>A few words about the organization</title> |
| 410 |
<para> |
425 |
<para> |
| 411 |
Mirrors are organized by country. All |
426 |
Mirrors are organized by country. All |
| 412 |
official mirrors have a DNS entry of the form |
427 |
official mirrors have a DNS entry of the form |
| 413 |
<systemitem class="fqdomainname">ftpN.CC.FreeBSD.org</systemitem>. |
428 |
<systemitem class="fqdomainname">ftpN.CC.FreeBSD.org</systemitem>. |
| 414 |
<emphasis>CC</emphasis> (i.e. country code) is the |
429 |
<emphasis>CC</emphasis> (i.e. country code) is the |
| 415 |
<emphasis>top level domain</emphasis> (TLD) |
430 |
<emphasis>top level domain</emphasis> (TLD) |
| 416 |
of the country where this mirror is located. |
431 |
of the country where this mirror is located. |
| 417 |
<emphasis>N</emphasis> is a number, |
432 |
<emphasis>N</emphasis> is a number, |
| 418 |
telling that the host would be the <emphasis>Nth</emphasis> |
433 |
telling that the host would be the <emphasis>Nth</emphasis> |
| 419 |
mirror in that country. |
434 |
mirror in that country. |
| 420 |
(Same applies to |
435 |
(Same applies to |
| 421 |
<systemitem>wwwN.CC.FreeBSD.org</systemitem>, etc.) |
436 |
<systemitem>wwwN.CC.FreeBSD.org</systemitem>, etc.) |
| 422 |
There are mirrors with no <emphasis>CC</emphasis> part. |
437 |
There are mirrors with no <emphasis>CC</emphasis> part. |
| 423 |
These are the mirror sites that are very well connected and |
438 |
These are the mirror sites that are very well connected and |
| 424 |
allow a large number of concurrent users. |
439 |
allow a large number of concurrent users. |
| 425 |
<systemitem class="fqdomainname">ftp.FreeBSD.org</systemitem> is actually two machines, one currently |
440 |
<systemitem class="fqdomainname">ftp.FreeBSD.org</systemitem> is actually two machines, one currently |
| 426 |
located in Denmark and the other in the United States. |
441 |
located in Denmark and the other in the United States. |
| 427 |
It is <emphasis>NOT</emphasis> a master site and should never be |
442 |
It is <emphasis>NOT</emphasis> a master site and should never be |
| 428 |
used to mirror from. Lots of online documentation leads |
443 |
used to mirror from. Lots of online documentation leads |
| 429 |
<quote>interactive</quote>users to |
444 |
<quote>interactive</quote>users to |
| 430 |
<systemitem class="fqdomainname">ftp.FreeBSD.org</systemitem> so automated mirroring |
445 |
<systemitem class="fqdomainname">ftp.FreeBSD.org</systemitem> so automated mirroring |
| 431 |
systems should find a different machine to mirror from. |
446 |
systems should find a different machine to mirror from. |
| 432 |
</para> |
447 |
</para> |
| 433 |
<para> |
448 |
<para> |
| 434 |
Additionally there exists a hierarchy of mirrors, which |
449 |
Additionally there exists a hierarchy of mirrors, which |
| 435 |
is described in terms of <emphasis>tiers</emphasis>. |
450 |
is described in terms of <emphasis>tiers</emphasis>. |
| 436 |
The master sites are not referred to but can be |
451 |
The master sites are not referred to but can be |
| 437 |
described as <emphasis>Tier-0</emphasis>. Mirrors |
452 |
described as <emphasis>Tier-0</emphasis>. Mirrors |
| 438 |
that mirror from these sites can be considered |
453 |
that mirror from these sites can be considered |
| 439 |
<emphasis>Tier-1</emphasis>, mirrors of <emphasis>Tier-1</emphasis>-mirrors, |
454 |
<emphasis>Tier-1</emphasis>, mirrors of <emphasis>Tier-1</emphasis>-mirrors, |
| 440 |
are <emphasis>Tier-2</emphasis>, etc. |
455 |
are <emphasis>Tier-2</emphasis>, etc. |
| 441 |
Official sites are encouraged to be of a low <emphasis>tier</emphasis>, |
456 |
Official sites are encouraged to be of a low <emphasis>tier</emphasis>, |
| 442 |
but the lower the tier the higher the requirements in |
457 |
but the lower the tier the higher the requirements in |
| 443 |
terms as described in <xref linkend="mirror-requirements"/>. |
458 |
terms as described in <xref linkend="mirror-requirements"/>. |
| 444 |
Also access to low-tier-mirrors may be restricted, and |
459 |
Also access to low-tier-mirrors may be restricted, and |
| 445 |
access to master sites is definitely restricted. |
460 |
access to master sites is definitely restricted. |
| 446 |
The <emphasis>tier</emphasis>-hierarchy is not reflected |
461 |
The <emphasis>tier</emphasis>-hierarchy is not reflected |
| 447 |
by DNS and generally not documented anywhere except |
462 |
by DNS and generally not documented anywhere except |
| 448 |
for the master sites. However, official mirrors with low numbers |
463 |
for the master sites. However, official mirrors with low numbers |
| 449 |
like 1-4, are usually <emphasis>Tier-1</emphasis> |
464 |
like 1-4, are usually <emphasis>Tier-1</emphasis> |
| 450 |
(this is just a rough hint, and there is no rule). |
465 |
(this is just a rough hint, and there is no rule). |
| 451 |
</para> |
466 |
</para> |
| 452 |
</sect2> |
467 |
</sect2> |
| 453 |
<sect2 xml:id="mirror-where-where"> |
468 |
<sect2 xml:id="mirror-where-where"> |
| 454 |
<title>Ok, but where should I get the stuff now?</title> |
469 |
<title>Ok, but where should I get the stuff now?</title> |
| 455 |
<para> |
470 |
<para> |
| 456 |
Under no circumstances should you mirror from <systemitem class="fqdomainname">ftp.FreeBSD.org</systemitem>. |
471 |
Under no circumstances should you mirror from <systemitem class="fqdomainname">ftp.FreeBSD.org</systemitem>. |
| 457 |
The short answer is: from the |
472 |
The short answer is: from the |
| 458 |
site that is closest to you in Internet terms, or gives you |
473 |
site that is closest to you in Internet terms, or gives you |
| 459 |
the fastest access. |
474 |
the fastest access. |
| 460 |
</para> |
475 |
</para> |
| 461 |
<sect3 xml:id="mirror-where-simple"> |
476 |
<sect3 xml:id="mirror-where-simple"> |
| 462 |
<title>I just want to mirror from somewhere!</title> |
477 |
<title>I just want to mirror from somewhere!</title> |
| 463 |
<para> |
478 |
<para> |
| 464 |
If you have no special intentions or |
479 |
If you have no special intentions or |
| 465 |
requirements, the statement in <xref linkend="mirror-where-where"/> |
480 |
requirements, the statement in <xref linkend="mirror-where-where"/> |
| 466 |
applies. This means: |
481 |
applies. This means: |
| 467 |
</para> |
482 |
</para> |
| 468 |
<procedure> |
483 |
<procedure> |
| 469 |
<step> |
484 |
<step> |
| 470 |
<para> |
485 |
<para> |
| 471 |
Check for those which provide fastest access |
486 |
Check for those which provide fastest access |
| 472 |
(number of hops, round-trip-times) |
487 |
(number of hops, round-trip-times) |
| 473 |
and offer the services you intend to |
488 |
and offer the services you intend to |
| 474 |
use (like <application>rsync</application>). |
489 |
use (like <application>rsync</application>). |
| 475 |
</para> |
490 |
</para> |
| 476 |
</step> |
491 |
</step> |
| 477 |
<step> |
492 |
<step> |
| 478 |
<para> |
493 |
<para> |
| 479 |
Contact the administrators of your chosen site stating your |
494 |
Contact the administrators of your chosen site stating your |
| 480 |
request, and asking about their terms and |
495 |
request, and asking about their terms and |
| 481 |
policies. |
496 |
policies. |
| 482 |
</para> |
497 |
</para> |
| 483 |
</step> |
498 |
</step> |
| 484 |
<step> |
499 |
<step> |
| 485 |
<para> |
500 |
<para> |
| 486 |
Set up your mirror as described above. |
501 |
Set up your mirror as described above. |
| 487 |
</para> |
502 |
</para> |
| 488 |
</step> |
503 |
</step> |
| 489 |
</procedure> |
504 |
</procedure> |
| 490 |
</sect3> |
505 |
</sect3> |
| 491 |
<sect3 xml:id="mirror-where-official"> |
506 |
<sect3 xml:id="mirror-where-official"> |
| 492 |
<title>I am an official mirror, what is the right site for me?</title> |
507 |
<title>I am an official mirror, what is the right site for me?</title> |
| 493 |
<para> |
508 |
<para> |
| 494 |
In general the description in <xref linkend="mirror-where-simple"/> |
509 |
In general the description in <xref linkend="mirror-where-simple"/> |
| 495 |
still applies. Of course you may want to put some |
510 |
still applies. Of course you may want to put some |
| 496 |
weight on the fact that your upstream should be of |
511 |
weight on the fact that your upstream should be of |
| 497 |
a low tier. |
512 |
a low tier. |
| 498 |
There are some other considerations about <emphasis>official</emphasis> |
513 |
There are some other considerations about <emphasis>official</emphasis> |
| 499 |
mirrors that are described in <xref linkend="mirror-official"/>. |
514 |
mirrors that are described in <xref linkend="mirror-official"/>. |
| 500 |
</para> |
515 |
</para> |
| 501 |
</sect3> |
516 |
</sect3> |
| 502 |
<sect3 xml:id="mirror-where-master"> |
517 |
<sect3 xml:id="mirror-where-master"> |
| 503 |
<title>I want to access the master sites!</title> |
518 |
<title>I want to access the master sites!</title> |
| 504 |
<para> |
519 |
<para> |
| 505 |
If you have good reasons and good prerequisites, |
520 |
If you have good reasons and good prerequisites, |
| 506 |
you may want and get access to one of the |
521 |
you may want and get access to one of the |
| 507 |
master sites. Access to these sites is |
522 |
master sites. Access to these sites is |
| 508 |
generally restricted, and there are special policies |
523 |
generally restricted, and there are special policies |
| 509 |
for access. If you are already an <emphasis>official</emphasis> |
524 |
for access. If you are already an <emphasis>official</emphasis> |
| 510 |
mirror, this certainly helps you getting access. |
525 |
mirror, this certainly helps you getting access. |
| 511 |
In any other case make sure your country really needs another mirror. |
526 |
In any other case make sure your country really needs another mirror. |
| 512 |
If it already has three or more, ask the <quote>zone administrator</quote> (<email>hostmaster@CC.FreeBSD.org</email>) or &a.hubs; first.</para> |
527 |
If it already has three or more, ask the <quote>zone administrator</quote> (<email>hostmaster@CC.FreeBSD.org</email>) or &a.hubs; first.</para> |
| 513 |
|
528 |
|
| 514 |
<para> |
529 |
<para> |
| 515 |
Whoever helped you become, an <emphasis>official</emphasis> |
530 |
Whoever helped you become, an <emphasis>official</emphasis> |
| 516 |
should have helped you gain access to an appropriate upstream |
531 |
should have helped you gain access to an appropriate upstream |
| 517 |
host, either one of the master sites or a suitable Tier-1 |
532 |
host, either one of the master sites or a suitable Tier-1 |
| 518 |
site. If not, you can send email to |
533 |
site. If not, you can send email to |
| 519 |
<email>mirror-admin@FreeBSD.org</email> to request help with |
534 |
<email>mirror-admin@FreeBSD.org</email> to request help with |
| 520 |
that. |
535 |
that. |
| 521 |
</para> |
536 |
</para> |
| 522 |
<para> |
537 |
<para> |
| 523 |
There is one master site for the FTP fileset. |
538 |
There is one master site for the FTP fileset. |
| 524 |
</para> |
539 |
</para> |
| 525 |
<sect4 xml:id="mirror-where-master-ftp"> |
540 |
<sect4 xml:id="mirror-where-master-ftp"> |
| 526 |
<title>ftp-master.FreeBSD.org</title> |
541 |
<title>ftp-master.FreeBSD.org</title> |
| 527 |
<para> |
542 |
<para> |
| 528 |
This is the master site for the FTP fileset. |
543 |
This is the master site for the FTP fileset. |
| 529 |
</para> |
544 |
</para> |
| 530 |
<para> |
545 |
<para> |
| 531 |
<systemitem>ftp-master.FreeBSD.org</systemitem> provides |
546 |
<systemitem>ftp-master.FreeBSD.org</systemitem> provides |
| 532 |
<application>rsync</application> |
547 |
<application>rsync</application> |
| 533 |
access, in addition to FTP. |
548 |
access, in addition to FTP. |
| 534 |
Refer to <xref linkend="mirror-ftp-rsync"/>. |
549 |
Refer to <xref linkend="mirror-ftp-rsync"/>. |
| 535 |
</para> |
550 |
</para> |
| 536 |
<para> |
551 |
<para> |
| 537 |
Mirrors are also encouraged to allow <application>rsync</application> |
552 |
Mirrors are also encouraged to allow <application>rsync</application> |
| 538 |
access for the FTP contents, since they are |
553 |
access for the FTP contents, since they are |
| 539 |
<emphasis>Tier-1</emphasis>-mirrors. |
554 |
<emphasis>Tier-1</emphasis>-mirrors. |
| 540 |
</para> |
555 |
</para> |
| 541 |
</sect4> |
556 |
</sect4> |
| 542 |
</sect3> |
557 |
</sect3> |
| 543 |
</sect2> |
558 |
</sect2> |
| 544 |
</sect1> |
559 |
</sect1> |
| 545 |
<sect1 xml:id="mirror-official"> |
560 |
<sect1 xml:id="mirror-official"> |
| 546 |
<title>Official Mirrors</title> |
561 |
<title>Official Mirrors</title> |
| 547 |
<para> |
562 |
<para> |
| 548 |
Official mirrors are mirrors that</para> |
563 |
Official mirrors are mirrors that</para> |
| 549 |
<itemizedlist> |
564 |
<itemizedlist> |
| 550 |
<listitem> |
565 |
<listitem> |
| 551 |
<para> |
566 |
<para> |
| 552 |
a) have a <systemitem>FreeBSD.org</systemitem> DNS entry |
567 |
a) have a <systemitem>FreeBSD.org</systemitem> DNS entry |
| 553 |
(usually a CNAME). |
568 |
(usually a CNAME). |
| 554 |
</para> |
569 |
</para> |
| 555 |
</listitem> |
570 |
</listitem> |
| 556 |
<listitem> |
571 |
<listitem> |
| 557 |
<para> |
572 |
<para> |
| 558 |
b) are listed as an official mirror in the FreeBSD |
573 |
b) are listed as an official mirror in the FreeBSD |
| 559 |
documentation (like handbook). |
574 |
documentation (like handbook). |
| 560 |
</para> |
575 |
</para> |
| 561 |
</listitem> |
576 |
</listitem> |
| 562 |
</itemizedlist> |
577 |
</itemizedlist> |
| 563 |
|
578 |
|
| 564 |
<para>So far to distinguish official mirrors. |
579 |
<para>So far to distinguish official mirrors. |
| 565 |
Official mirrors are not necessarily <emphasis>Tier-1</emphasis>-mirrors. |
580 |
Official mirrors are not necessarily <emphasis>Tier-1</emphasis>-mirrors. |
| 566 |
However you probably will not find a <emphasis>Tier-1</emphasis>-mirror, |
581 |
However you probably will not find a <emphasis>Tier-1</emphasis>-mirror, |
| 567 |
that is not also official. |
582 |
that is not also official. |
| 568 |
</para> |
583 |
</para> |
| 569 |
<sect2 xml:id="mirror-official-requirements"> |
584 |
<sect2 xml:id="mirror-official-requirements"> |
| 570 |
<title>Special Requirements for official (tier-1) mirrors</title> |
585 |
<title>Special Requirements for official (tier-1) mirrors</title> |
| 571 |
<para> |
586 |
<para> |
| 572 |
It is not so easy to state requirements for all |
587 |
It is not so easy to state requirements for all |
| 573 |
official mirrors, since the project is sort of |
588 |
official mirrors, since the project is sort of |
| 574 |
tolerant here. It is more easy to say, |
589 |
tolerant here. It is more easy to say, |
| 575 |
what <emphasis>official tier-1 mirrors</emphasis> |
590 |
what <emphasis>official tier-1 mirrors</emphasis> |
| 576 |
are required to. All other official mirrors |
591 |
are required to. All other official mirrors |
| 577 |
can consider this a big <emphasis>should</emphasis>.</para> |
592 |
can consider this a big <emphasis>should</emphasis>.</para> |
| 578 |
<para> |
593 |
<para> |
| 579 |
Tier-1 mirrors are required to:</para> |
594 |
Tier-1 mirrors are required to:</para> |
| 580 |
<itemizedlist> |
595 |
<itemizedlist> |
| 581 |
<listitem><para>carry the complete fileset</para></listitem> |
596 |
<listitem><para>carry the complete fileset</para></listitem> |
| 582 |
<listitem><para>allow access to other mirror sites</para></listitem> |
597 |
<listitem><para>allow access to other mirror sites</para></listitem> |
| 583 |
<listitem><para>provide <application>FTP</application> and |
598 |
<listitem><para>provide <application>FTP</application> and |
| 584 |
<application>rsync</application> access</para></listitem> |
599 |
<application>rsync</application> access</para></listitem> |
| 585 |
</itemizedlist> |
600 |
</itemizedlist> |
| 586 |
|
601 |
|
| 587 |
<para>Furthermore, admins should be subscribed to the &a.hubs;. |
602 |
<para>Furthermore, admins should be subscribed to the &a.hubs;. |
| 588 |
See <link xlink:href="https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/eresources.html#ERESOURCES-MAIL">this link</link> for details, how to subscribe. |
603 |
See <link xlink:href="https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/eresources.html#ERESOURCES-MAIL">this link</link> for details, how to subscribe. |
| 589 |
</para> |
604 |
</para> |
| 590 |
<important> |
605 |
<important> |
| 591 |
<para>It is <emphasis>very</emphasis> important for a hub administrator, especially |
606 |
<para>It is <emphasis>very</emphasis> important for a hub administrator, especially |
| 592 |
Tier-1 hub admins, to check the |
607 |
Tier-1 hub admins, to check the |
| 593 |
<link xlink:href="https://www.FreeBSD.org/releng/">release schedule</link> |
608 |
<link xlink:href="https://www.FreeBSD.org/releng/">release schedule</link> |
| 594 |
for the next FreeBSD release. This is important because it will tell you when the |
609 |
for the next FreeBSD release. This is important because it will tell you when the |
| 595 |
next release is scheduled |
610 |
next release is scheduled |
| 596 |
to come out, and thus giving you time to prepare for the big spike of traffic which follows it. |
611 |
to come out, and thus giving you time to prepare for the big spike of traffic which follows it. |
| 597 |
</para> |
612 |
</para> |
| 598 |
<para> |
613 |
<para> |
| 599 |
It is also important that hub administrators try to keep their mirrors as up-to-date as |
614 |
It is also important that hub administrators try to keep their mirrors as up-to-date as |
| 600 |
possible (again, even more crucial for Tier-1 mirrors). If Mirror1 does not update for a |
615 |
possible (again, even more crucial for Tier-1 mirrors). If Mirror1 does not update for a |
| 601 |
while, lower tier mirrors will begin to mirror old data from Mirror1 and thus begins |
616 |
while, lower tier mirrors will begin to mirror old data from Mirror1 and thus begins |
| 602 |
a downward spiral... Keep your mirrors up to date! |
617 |
a downward spiral... Keep your mirrors up to date! |
| 603 |
</para> |
618 |
</para> |
| 604 |
</important> |
619 |
</important> |
| 605 |
</sect2> |
620 |
</sect2> |
| 606 |
<sect2 xml:id="mirror-official-become"> |
621 |
<sect2 xml:id="mirror-official-become"> |
| 607 |
<title>How to become official then?</title> |
622 |
<title>How to become official then?</title> |
| 608 |
<!-- |
623 |
<!-- |
| 609 |
<para> |
624 |
<para> |
| 610 |
An interesting question, especially, since the state |
625 |
An interesting question, especially, since the state |
| 611 |
of being official comes with some benefits, like a much |
626 |
of being official comes with some benefits, like a much |
| 612 |
higher bill from your ISP as more people will be using |
627 |
higher bill from your ISP as more people will be using |
| 613 |
your site. Also it may be a key requirement to get access |
628 |
your site. Also it may be a key requirement to get access |
| 614 |
to a master site. |
629 |
to a master site. |
| 615 |
</para> |
630 |
</para> |
| 616 |
<para> |
631 |
<para> |
| 617 |
Before applying, please consider (again) if |
632 |
Before applying, please consider (again) if |
| 618 |
another official mirror is really needed for |
633 |
another official mirror is really needed for |
| 619 |
your region. Check first with your zone administrator (<email>hostmaster@CC.FreeBSD.org</email>) or, if that fails, ask on the &a.hubs;. |
634 |
your region. Check first with your zone administrator (<email>hostmaster@CC.FreeBSD.org</email>) or, if that fails, ask on the &a.hubs;. |
| 620 |
</para> |
635 |
</para> |
| 621 |
<para>Ok, here is how to do it:</para> |
636 |
<para>Ok, here is how to do it:</para> |
| 622 |
<procedure> |
637 |
<procedure> |
| 623 |
<step> |
638 |
<step> |
| 624 |
<para> |
639 |
<para> |
| 625 |
Get the mirror running in first place (maybe not |
640 |
Get the mirror running in first place (maybe not |
| 626 |
using a master site, yet). |
641 |
using a master site, yet). |
| 627 |
</para> |
642 |
</para> |
| 628 |
</step> |
643 |
</step> |
| 629 |
<step> |
644 |
<step> |
| 630 |
<para> |
645 |
<para> |
| 631 |
<ulink url="https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/eresources.html#ERESOURCES-MAIL">Subscribe</ulink> to the &a.hubs;. |
646 |
<ulink url="https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/eresources.html#ERESOURCES-MAIL">Subscribe</ulink> to the &a.hubs;. |
| 632 |
</para> |
647 |
</para> |
| 633 |
</step> |
648 |
</step> |
| 634 |
<step> |
649 |
<step> |
| 635 |
<para> |
650 |
<para> |
| 636 |
If everything works so far, contact the DNS administrator responsible |
651 |
If everything works so far, contact the DNS administrator responsible |
| 637 |
for your region/country, and ask for a DNS entry for your |
652 |
for your region/country, and ask for a DNS entry for your |
| 638 |
site. The admin should able to be contacted via |
653 |
site. The admin should able to be contacted via |
| 639 |
<email>hostmaster@CC.FreeBSD.org</email>, where |
654 |
<email>hostmaster@CC.FreeBSD.org</email>, where |
| 640 |
<emphasis>CC</emphasis> is your country code/TLD. |
655 |
<emphasis>CC</emphasis> is your country code/TLD. |
| 641 |
Your DNS entry will be as described |
656 |
Your DNS entry will be as described |
| 642 |
in <xref linkend="mirror-where-organization"/>. |
657 |
in <xref linkend="mirror-where-organization"/>. |
| 643 |
</para> |
658 |
</para> |
| 644 |
<para> |
659 |
<para> |
| 645 |
If there is no subdomain set up for your |
660 |
If there is no subdomain set up for your |
| 646 |
country yet, you should contact |
661 |
country yet, you should contact |
| 647 |
<email>mirror-admin@FreeBSD.org</email>, |
662 |
<email>mirror-admin@FreeBSD.org</email>, |
| 648 |
or you can try the &a.hubs; first. |
663 |
or you can try the &a.hubs; first. |
| 649 |
</para> |
664 |
</para> |
| 650 |
</step> |
665 |
</step> |
| 651 |
<step> |
666 |
<step> |
| 652 |
<para> |
667 |
<para> |
| 653 |
Whoever helps you get an official name should send email |
668 |
Whoever helps you get an official name should send email |
| 654 |
to <email>mirror-admin@FreeBSD.org</email> so your site will be |
669 |
to <email>mirror-admin@FreeBSD.org</email> so your site will be |
| 655 |
added to the mirror list in the |
670 |
added to the mirror list in the |
| 656 |
<ulink url="https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook">FreeBSD |
671 |
<ulink url="https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook">FreeBSD |
| 657 |
Handbook</ulink>. |
672 |
Handbook</ulink>. |
| 658 |
</para> |
673 |
</para> |
| 659 |
</step> |
674 |
</step> |
| 660 |
</procedure> |
675 |
</procedure> |
| 661 |
<para>That is it.</para> |
676 |
<para>That is it.</para> |
| 662 |
--> |
677 |
--> |
| 663 |
<para> |
678 |
<para> |
| 664 |
We are not accepting any new mirrors at this time. |
679 |
We are not accepting any new mirrors at this time. |
| 665 |
</para> |
680 |
</para> |
| 666 |
</sect2> |
681 |
</sect2> |
| 667 |
</sect1> |
682 |
</sect1> |
| 668 |
<sect1 xml:id="mirror-statpages"> |
683 |
<sect1 xml:id="mirror-statpages"> |
| 669 |
<title>Some statistics from mirror sites</title> |
684 |
<title>Some statistics from mirror sites</title> |
| 670 |
<para> |
685 |
<para> |
| 671 |
Here are links to the stat pages of your favorite mirrors |
686 |
Here are links to the stat pages of your favorite mirrors |
| 672 |
(a.k.a. the only ones who feel like providing stats). |
687 |
(a.k.a. the only ones who feel like providing stats). |
| 673 |
</para> |
688 |
</para> |
| 674 |
<sect2 xml:id="mirror-statpagesftp"> |
689 |
<sect2 xml:id="mirror-statpagesftp"> |
| 675 |
<title>FTP site statistics</title> |
690 |
<title>FTP site statistics</title> |
| 676 |
<itemizedlist> |
691 |
<itemizedlist> |
| 677 |
<listitem> |
692 |
<listitem> |
| 678 |
<para>ftp.is.FreeBSD.org - <email>hostmaster@is.FreeBSD.org</email> - |
693 |
<para>ftp.is.FreeBSD.org - <email>hostmaster@is.FreeBSD.org</email> - |
| 679 |
<link xlink:href="http://www.rhnet.is/status/draupnir/draupnir.html"> |
694 |
<link xlink:href="http://www.rhnet.is/status/draupnir/draupnir.html"> |
| 680 |
(Bandwidth)</link> <link xlink:href="http://www.rhnet.is/status/ftp/ftp-notendur.html">(FTP |
695 |
(Bandwidth)</link> <link xlink:href="http://www.rhnet.is/status/ftp/ftp-notendur.html">(FTP |
| 681 |
processes)</link> <link xlink:href="http://www.rhnet.is/status/ftp/http-notendur.html">(HTTP processes) |
696 |
processes)</link> <link xlink:href="http://www.rhnet.is/status/ftp/http-notendur.html">(HTTP processes) |
| 682 |
</link> |
697 |
</link> |
| 683 |
</para> |
698 |
</para> |
| 684 |
</listitem> |
699 |
</listitem> |
| 685 |
<listitem> |
700 |
<listitem> |
| 686 |
<para>ftp2.ru.FreeBSD.org - <email>mirror@macomnet.ru</email> - |
701 |
<para>ftp2.ru.FreeBSD.org - <email>mirror@macomnet.ru</email> - |
| 687 |
<link xlink:href="http://mirror.macomnet.net/mrtg/mirror.macomnet.net_195.128.64.25.html">(Bandwidth)</link> |
702 |
<link xlink:href="http://mirror.macomnet.net/mrtg/mirror.macomnet.net_195.128.64.25.html">(Bandwidth)</link> |
| 688 |
<link xlink:href="http://mirror.macomnet.net/mrtg/mirror.macomnet.net_proc.html">(HTTP and FTP users)</link> |
703 |
<link xlink:href="http://mirror.macomnet.net/mrtg/mirror.macomnet.net_proc.html">(HTTP and FTP users)</link> |
| 689 |
</para> |
704 |
</para> |
| 690 |
</listitem> |
705 |
</listitem> |
| 691 |
</itemizedlist> |
706 |
</itemizedlist> |
| 692 |
</sect2> |
707 |
</sect2> |
| 693 |
</sect1> |
708 |
</sect1> |
| 694 |
</article> |
709 |
</article> |