|
Lines 26-32
Link Here
|
| 26 |
<firstname>Valentino</firstname> |
26 |
<firstname>Valentino</firstname> |
| 27 |
<surname>Vaschetto</surname> |
27 |
<surname>Vaschetto</surname> |
| 28 |
<affiliation> |
28 |
<affiliation> |
| 29 |
<address><email>logo@FreeBSD.org</email></address> |
29 |
<address><email>logo@FreeBSD.org</email></address> |
| 30 |
</affiliation> |
30 |
</affiliation> |
| 31 |
</author> |
31 |
</author> |
| 32 |
<author> |
32 |
<author> |
|
Lines 36-48
Link Here
|
| 36 |
<address><email>dl@leo.org</email></address> |
36 |
<address><email>dl@leo.org</email></address> |
| 37 |
</affiliation> |
37 |
</affiliation> |
| 38 |
</author> |
38 |
</author> |
|
|
39 |
<author> |
| 40 |
<firstname>Ken</firstname> |
| 41 |
<surname>Smith</surname> |
| 42 |
<affiliation> |
| 43 |
<address><email>kensmith@cse.buffalo.edu</email></address> |
| 44 |
</affiliation> |
| 45 |
</author> |
| 39 |
</authorgroup> |
46 |
</authorgroup> |
| 40 |
<abstract> |
47 |
<abstract> |
| 41 |
<para>An in-progress article on how to mirror FreeBSD, aimed at |
48 |
<para>An in-progress article on how to mirror FreeBSD, aimed at |
| 42 |
hub administrators</para> |
49 |
hub administrators.</para> |
| 43 |
</abstract> |
50 |
</abstract> |
| 44 |
</articleinfo> |
51 |
</articleinfo> |
| 45 |
|
52 |
|
|
|
53 |
<sect1 id="mirror-contact"> |
| 54 |
<title>Contact Information</title> |
| 55 |
<para> |
| 56 |
The Mirror System Coordinators can be reached through email |
| 57 |
at <email>mirror-admin@FreeBSD.org</email>. There is also |
| 58 |
a &a.hubs;. |
| 59 |
</para> |
| 60 |
</sect1> |
| 46 |
<sect1 id="mirror-requirements"> |
61 |
<sect1 id="mirror-requirements"> |
| 47 |
<title>Requirements for FreeBSD mirrors</title> |
62 |
<title>Requirements for FreeBSD mirrors</title> |
| 48 |
<sect2 id="mirror-diskspace"> |
63 |
<sect2 id="mirror-diskspace"> |
|
Lines 50-71
Link Here
|
| 50 |
<para> |
65 |
<para> |
| 51 |
Disk space is one of the most important requirements. |
66 |
Disk space is one of the most important requirements. |
| 52 |
Depending on the set of releases, architectures, |
67 |
Depending on the set of releases, architectures, |
| 53 |
and degree of completeness you want to mirror, a huge |
68 |
and degree of completeness you want to mirror a huge |
| 54 |
amount of disk space may be consumed. Also keep in mind, |
69 |
amount of disk space may be consumed. Also keep in mind |
| 55 |
that <emphasis>official</emphasis> mirrors are probably required to be |
70 |
that <emphasis>official</emphasis> mirrors are probably required to be |
| 56 |
complete. The CVS repository and the web pages should |
71 |
complete. The CVS repository and the web pages should |
| 57 |
always be mirrored completely. Also note, that the |
72 |
always be mirrored completely. Also note that the |
| 58 |
numbers stated here, are reflecting the current |
73 |
numbers stated here are reflecting the current |
| 59 |
state (at 4.5-RELEASE). Further development and |
74 |
state (at 4.8-RELEASE/5.1-RELEASE). Further development and |
| 60 |
releases will only increase the required amount. |
75 |
releases will only increase the required amount. |
| 61 |
Also make sure, to keep some (ca. 10-20%) extra space |
76 |
Also make sure to keep some (ca. 10-20%) extra space |
| 62 |
around, just to be sure. |
77 |
around just to be sure. |
| 63 |
Here are some approximate figures: |
78 |
Here are some approximate figures: |
| 64 |
</para> |
79 |
</para> |
| 65 |
<itemizedlist> |
80 |
<itemizedlist> |
| 66 |
<listitem><para>Full FTP Distribution: 100 GB</para></listitem> |
81 |
<listitem><para>Full FTP Distribution: 126 GB</para></listitem> |
| 67 |
<listitem><para>CVS repository: 2 GB</para></listitem> |
82 |
<listitem><para>CVS repository: 2.7 GB</para></listitem> |
| 68 |
<listitem><para>CTM deltas: 1.5 GB</para></listitem> |
83 |
<listitem><para>CTM deltas: 1.8 GB</para></listitem> |
| 69 |
<listitem><para>Webpages: 300 MB</para></listitem> |
84 |
<listitem><para>Webpages: 300 MB</para></listitem> |
| 70 |
</itemizedlist> |
85 |
</itemizedlist> |
| 71 |
</sect2> |
86 |
</sect2> |
|
Lines 75-102
Link Here
|
| 75 |
Of course, you need to be connected to the Internet. |
90 |
Of course, you need to be connected to the Internet. |
| 76 |
The required bandwidth depends on your intended use |
91 |
The required bandwidth depends on your intended use |
| 77 |
of the mirror. If you just want to mirror some |
92 |
of the mirror. If you just want to mirror some |
| 78 |
parts of FreeBSD for local use at your site/intranet, |
93 |
parts of FreeBSD for local use at your site/intranet |
| 79 |
the demand may be much smaller, than if you want to |
94 |
the demand may be much smaller than if you want to |
| 80 |
make the files publicly available or even if you intend |
95 |
make the files publicly available. If you intend |
| 81 |
to become an official mirror. We can only give rough |
96 |
to become an official mirror the bandwidth required will |
|
|
97 |
be even higher. We can only give rough |
| 82 |
estimates here: |
98 |
estimates here: |
| 83 |
</para> |
99 |
</para> |
| 84 |
<itemizedlist> |
100 |
<itemizedlist> |
| 85 |
<listitem><para>Local site, no public access: basically no minimum, |
101 |
<listitem><para>Local site, no public access: basically no minimum, |
| 86 |
but < 2 Mbps could make syncing a pain.</para></listitem> |
102 |
but < 2 Mbps could make syncing too slow.</para></listitem> |
| 87 |
<listitem><para>Unofficial public site: 34 Mbps is probably a good start.</para></listitem> |
103 |
<listitem><para>Unofficial public site: 34 Mbps is probably a good start.</para></listitem> |
| 88 |
<listitem><para>Official site: > 100 Mbps is recommended, also your host |
104 |
<listitem><para>Official site: > 100 Mbps is recommended, and your host |
| 89 |
should be connected as close as possible to your border router.</para></listitem> |
105 |
should be connected as close as possible to your border router.</para></listitem> |
| 90 |
</itemizedlist> |
106 |
</itemizedlist> |
| 91 |
</sect2> |
107 |
</sect2> |
| 92 |
<sect2 id="mirror-system"> |
108 |
<sect2 id="mirror-system"> |
| 93 |
<title>System Requirements, CPU, RAM</title> |
109 |
<title>System Requirements, CPU, RAM</title> |
| 94 |
<para> |
110 |
<para> |
| 95 |
This also depends on the expected amount of clients, |
111 |
One thing this depends on the expected number of clients, |
| 96 |
which is determined by the servers policy. It is |
112 |
which is determined by the server's policy. It is |
| 97 |
also affected by the types of services you want to offer. |
113 |
also affected by the types of services you want to offer. |
| 98 |
Plain FTP or HTTP services may not require a huge |
114 |
Plain FTP or HTTP services may not require a huge |
| 99 |
amount of resources. Watch out, if you provide |
115 |
amount of resources. Watch out if you provide |
| 100 |
CVSup, rsync or even AnonCVS. This can have a huge |
116 |
CVSup, rsync or even AnonCVS. This can have a huge |
| 101 |
impact on CPU and memory requirements. Especially |
117 |
impact on CPU and memory requirements. Especially |
| 102 |
rsync is considered a memory hog, and CVSup does |
118 |
rsync is considered a memory hog, and CVSup does |
|
Lines 107-132
Link Here
|
| 107 |
are just examples to give you a very rough hint. |
123 |
are just examples to give you a very rough hint. |
| 108 |
</para> |
124 |
</para> |
| 109 |
<para> |
125 |
<para> |
| 110 |
For a moderately visited site, that offers |
126 |
For a moderately visited site that offers |
| 111 |
<application>rsync</application>, you might |
127 |
<application>rsync</application> you might |
| 112 |
consider a current CPU with around 800MHz - 1 GHz, |
128 |
consider a current CPU with around 800MHz - 1 GHz, |
| 113 |
and at least 512MB RAM. This is probably the |
129 |
and at least 512MB RAM. This is probably the |
| 114 |
minimum you want for an <emphasis>official</emphasis> |
130 |
minimum you want for an <emphasis>official</emphasis> |
| 115 |
site. |
131 |
site. |
| 116 |
</para> |
132 |
</para> |
| 117 |
<para> |
133 |
<para> |
| 118 |
For a frequently used site you need definitely |
134 |
For a frequently used site you definitely need |
| 119 |
more RAM (consider 2GB as a good start), |
135 |
more RAM (consider 2GB as a good start) |
| 120 |
and possibly more CPU, which could also mean, |
136 |
and possibly more CPU, which could mean |
| 121 |
that you need to go for a SMP system. |
137 |
that you need to go for a SMP system. |
| 122 |
</para> |
138 |
</para> |
| 123 |
<para> |
139 |
<para> |
| 124 |
You also want to consider a fast disk subsystem. |
140 |
You also want to consider a fast disk subsystem. |
| 125 |
Operations on the CVS repository require a fast |
141 |
Operations on the CVS repository require a fast |
| 126 |
disk subsystem (RAID is greatly advised). A SCSI |
142 |
disk subsystem (RAID is highly advised). A SCSI |
| 127 |
controller that has a cache of its own can also |
143 |
controller that has a cache of its own can also |
| 128 |
speed up things, since most of these services incur a |
144 |
speed up things since most of these services incur a |
| 129 |
very large number of small modifications to the disk. |
145 |
large number of small modifications to the disk. |
| 130 |
</para> |
146 |
</para> |
| 131 |
<para>You can also experiment with enlarging the portion |
147 |
<para>You can also experiment with enlarging the portion |
| 132 |
of system memory which is used for the file system buffer cache. |
148 |
of system memory which is used for the file system buffer cache. |
|
Lines 139-146
Link Here
|
| 139 |
<title>Services to offer</title> |
155 |
<title>Services to offer</title> |
| 140 |
<para> |
156 |
<para> |
| 141 |
Every mirror site is required to have a set of core services |
157 |
Every mirror site is required to have a set of core services |
| 142 |
available. In addition to these basic services, which mirrors are |
158 |
available. In addition to these required services there are a |
| 143 |
required to provide, there is a number of optional services that |
159 |
number of optional services that |
| 144 |
server administrators may choose to offer. This section explains |
160 |
server administrators may choose to offer. This section explains |
| 145 |
which services you can provide and how to go about implementing them. |
161 |
which services you can provide and how to go about implementing them. |
| 146 |
</para> |
162 |
</para> |
|
Lines 148-154
Link Here
|
| 148 |
<title>FTP (required for FTP fileset)</title> |
164 |
<title>FTP (required for FTP fileset)</title> |
| 149 |
<para> |
165 |
<para> |
| 150 |
This is one of the most basic services, and |
166 |
This is one of the most basic services, and |
| 151 |
it is required for each mirror, offering public |
167 |
it is required for each mirror offering public |
| 152 |
FTP distributions. FTP access must be |
168 |
FTP distributions. FTP access must be |
| 153 |
anonymous, and no upload/download ratios |
169 |
anonymous, and no upload/download ratios |
| 154 |
are allowed (a ridiculous thing anyway). |
170 |
are allowed (a ridiculous thing anyway). |
|
Lines 194-214
Link Here
|
| 194 |
FreeBSD's <application>ftpd</application>, <application>proftpd</application>, |
210 |
FreeBSD's <application>ftpd</application>, <application>proftpd</application>, |
| 195 |
<application>wu-ftpd</application> and maybe <application>ncftpd</application> |
211 |
<application>wu-ftpd</application> and maybe <application>ncftpd</application> |
| 196 |
are among the most commonly ones. |
212 |
are among the most commonly ones. |
| 197 |
The others do not have a large userbase among mirror sites. |
213 |
The others do not have a large userbase among mirror sites. One |
|
|
214 |
thing to consider is that you may need flexibility in limiting |
| 215 |
how many simultaneous connections are allowed, thus limiting how |
| 216 |
much network bandwidth and system resources are consumed. |
| 198 |
</para> |
217 |
</para> |
| 199 |
</sect3> |
218 |
</sect3> |
| 200 |
<sect3 id="mirror-serv-rsync"> |
219 |
<sect3 id="mirror-serv-rsync"> |
| 201 |
<title>RSYNC (optional for FTP fileset)</title> |
220 |
<title>RSYNC (optional for FTP fileset)</title> |
| 202 |
<para> |
221 |
<para> |
| 203 |
<application>Rsync</application> is often also offered for convenience, for the |
222 |
<application>Rsync</application> is often offered for access to the |
| 204 |
contents of the FTP area of FreeBSD. The |
223 |
contents of the FTP area of FreeBSD so other Mirror Sites can use |
| 205 |
protocol is different from FTP in many ways, and |
224 |
your system as their source. The |
| 206 |
overall, it can be stated, that it is much more |
225 |
protocol is different from FTP in many ways. |
|
|
226 |
It is much more |
| 207 |
bandwidth friendly, as only differences between files |
227 |
bandwidth friendly, as only differences between files |
| 208 |
are transferred, not whole files. |
228 |
are transferred instead of whole files when they change. |
| 209 |
<application>Rsync</application> does require significant amount of memory for |
229 |
<application>Rsync</application> does require a significant |
|
|
230 |
amount of memory for |
| 210 |
each instance. The size depends on the size of |
231 |
each instance. The size depends on the size of |
| 211 |
the synced module in terms of number of directories and |
232 |
the synced module in terms of the number of directories and |
| 212 |
files. <application>Rsync</application> can use <command>rsh</command> and |
233 |
files. <application>Rsync</application> can use <command>rsh</command> and |
| 213 |
<command>ssh</command> (now default) as a transport, |
234 |
<command>ssh</command> (now default) as a transport, |
| 214 |
or use it's own protocol for stand-alone access |
235 |
or use it's own protocol for stand-alone access |
|
Lines 242-248
Link Here
|
| 242 |
|
263 |
|
| 243 |
<listitem> |
264 |
<listitem> |
| 244 |
<para><filename role="package">www/thttpd</filename>: |
265 |
<para><filename role="package">www/thttpd</filename>: |
| 245 |
If you are going to be serving a lot amount of static content |
266 |
If you are going to be serving a large amount of static content |
| 246 |
you may find that using an application such as tHttpd is more |
267 |
you may find that using an application such as tHttpd is more |
| 247 |
efficient than Apache. It is optimized for excellent performance |
268 |
efficient than Apache. It is optimized for excellent performance |
| 248 |
on FreeBSD.</para> |
269 |
on FreeBSD.</para> |
|
Lines 263-272
Link Here
|
| 263 |
<title>CVSup (desired for CVS repository)</title> |
284 |
<title>CVSup (desired for CVS repository)</title> |
| 264 |
<para> |
285 |
<para> |
| 265 |
<application>CVSup</application> is a very efficient way of distributing files. |
286 |
<application>CVSup</application> is a very efficient way of distributing files. |
| 266 |
It works similar as rsync, but was specially designed for |
287 |
It works similar to rsync, but was specially designed for |
| 267 |
the use with CVS repositories. If you want to offer the |
288 |
use with CVS repositories. If you want to offer the |
| 268 |
FreeBSD CVS repository, you really want to consider |
289 |
FreeBSD CVS repository you really should consider |
| 269 |
offering it via <application>CVSup</application>. Still it is possible to offer |
290 |
offering it via <application>CVSup</application>. It is possible to offer |
| 270 |
the CVS repository via <application>AnonCVS</application>, FTP, |
291 |
the CVS repository via <application>AnonCVS</application>, FTP, |
| 271 |
<application>Rsync</application> or HTTP, but |
292 |
<application>Rsync</application> or HTTP, but |
| 272 |
people would benefit much more from <application>CVSup</application> access. |
293 |
people would benefit much more from <application>CVSup</application> access. |
|
Lines 274-280
Link Here
|
| 274 |
It is a bit tricky to install on non-FreeBSD platforms, |
295 |
It is a bit tricky to install on non-FreeBSD platforms, |
| 275 |
since it is written in Modula-3 and therefore requires |
296 |
since it is written in Modula-3 and therefore requires |
| 276 |
a Modula-3 environment. John Polstra has built a |
297 |
a Modula-3 environment. John Polstra has built a |
| 277 |
stripped down version of M3, that is sufficient to |
298 |
stripped down version of M3 that is sufficient to |
| 278 |
run <application>CVSup</application>, and can be installed much easier. |
299 |
run <application>CVSup</application>, and can be installed much easier. |
| 279 |
See <ulink url="http://www.polstra.com/projects/freeware/ezm3/">Ezm3</ulink> |
300 |
See <ulink url="http://www.polstra.com/projects/freeware/ezm3/">Ezm3</ulink> |
| 280 |
for details. Related ports are: |
301 |
for details. Related ports are: |
|
Lines 286-293
Link Here
|
| 286 |
</listitem> |
307 |
</listitem> |
| 287 |
<listitem> |
308 |
<listitem> |
| 288 |
<para><filename role="package">net/cvsup-mirror</filename>: The CVSup mirror kit, which requires |
309 |
<para><filename role="package">net/cvsup-mirror</filename>: The CVSup mirror kit, which requires |
| 289 |
<filename role="package">net/cvsup</filename>, and configures it mirror-ready. Some |
310 |
<filename role="package">net/cvsup</filename>,and configures it mirror-ready. Some |
| 290 |
site administrators may want a different setup, though. |
311 |
site administrators may want a different setup though. |
| 291 |
</para> |
312 |
</para> |
| 292 |
</listitem> |
313 |
</listitem> |
| 293 |
</itemizedlist> |
314 |
</itemizedlist> |
|
Lines 296-325
Link Here
|
| 296 |
<filename role="package">net/cvsup-without-gui</filename> you might want to have |
317 |
<filename role="package">net/cvsup-without-gui</filename> you might want to have |
| 297 |
a look at. If you prefer a static binary package, take a look |
318 |
a look at. If you prefer a static binary package, take a look |
| 298 |
<ulink url="http://people.FreeBSD.org/~jdp/s1g/">here</ulink>. |
319 |
<ulink url="http://people.FreeBSD.org/~jdp/s1g/">here</ulink>. |
| 299 |
This page still refers to the S1G bug, that was present |
320 |
This page still refers to the S1G bug that was present |
| 300 |
in <application>CVSup</application>. Maybe |
321 |
in <application>CVSup</application>. Maybe |
| 301 |
John will setup a generic download-site to get |
322 |
John will setup a generic download-site to get |
| 302 |
static binaries for various platforms. |
323 |
static binaries for various platforms. |
| 303 |
</para> |
324 |
</para> |
| 304 |
<para> |
325 |
<para> |
| 305 |
It is possible to use CVSup to offer |
326 |
It is possible to use CVSup to offer |
| 306 |
any kind of fileset, not just CVS repositories, |
327 |
any kind of fileset, not just CVS repositories. |
| 307 |
but configuration can be complex. |
328 |
But configuration can be complex. |
| 308 |
CVSup is known to eat some CPU on the server as on the |
329 |
CVSup is known to eat some CPU on both the server and the |
| 309 |
client, since it needs to compare lots of files. |
330 |
client, since it needs to compare lots of files. |
| 310 |
</para> |
331 |
</para> |
| 311 |
</sect3> |
332 |
</sect3> |
| 312 |
<sect3 id="mirror-anoncvs"> |
333 |
<sect3 id="mirror-anoncvs"> |
| 313 |
<title>AnonCVS (optional for CVS repository)</title> |
334 |
<title>AnonCVS (optional for CVS repository)</title> |
| 314 |
<para> |
335 |
<para> |
| 315 |
If you have the CVS repository, you may want to offer |
336 |
If you have the CVS repository you may want to offer |
| 316 |
anonymous CVS access. A short warning first: |
337 |
anonymous CVS access. A short warning first: |
| 317 |
There is not that much demand for it, |
338 |
There is not much demand for it, |
| 318 |
and it requires some experience and you need to know, |
339 |
it requires some experience, and you need to know |
| 319 |
what you are doing. |
340 |
what you are doing. |
| 320 |
</para> |
341 |
</para> |
| 321 |
<para> |
342 |
<para> |
| 322 |
Generally there are two ways, how |
343 |
Generally there are two ways |
| 323 |
to access a CVS repository remotely: via |
344 |
to access a CVS repository remotely: via |
| 324 |
<emphasis>pserver</emphasis> or via <command>ssh</command> |
345 |
<emphasis>pserver</emphasis> or via <command>ssh</command> |
| 325 |
(we don't consider <command>rsh</command>). |
346 |
(we don't consider <command>rsh</command>). |
|
Lines 330-336
Link Here
|
| 330 |
in the CVS repository, to be used as a login-shell for the |
351 |
in the CVS repository, to be used as a login-shell for the |
| 331 |
anonymous ssh account. It does a chroot, and therefore |
352 |
anonymous ssh account. It does a chroot, and therefore |
| 332 |
requires the CVS repository to be available under the |
353 |
requires the CVS repository to be available under the |
| 333 |
anonymous user's home-directory, which may not be possible |
354 |
anonymous user's home directory. This may not be possible |
| 334 |
for all sites. If you just offer <emphasis>pserver</emphasis> |
355 |
for all sites. If you just offer <emphasis>pserver</emphasis> |
| 335 |
this restriction does not apply, but you may run with |
356 |
this restriction does not apply, but you may run with |
| 336 |
more security risks. You don't need to install any special |
357 |
more security risks. You don't need to install any special |
|
Lines 341-359
Link Here
|
| 341 |
<programlisting> |
362 |
<programlisting> |
| 342 |
cvspserver stream tcp nowait root /usr/bin/cvs cvs -f -l -R -T /anoncvstmp --allow-root=/home/ncvs pserver |
363 |
cvspserver stream tcp nowait root /usr/bin/cvs cvs -f -l -R -T /anoncvstmp --allow-root=/home/ncvs pserver |
| 343 |
</programlisting> |
364 |
</programlisting> |
| 344 |
See the manpage for details of the options. See also the cvs <emphasis>info</emphasis> |
365 |
See the manpage for details of the options. Also see the cvs <emphasis>info</emphasis> |
| 345 |
page, about additional ways to make sure, access is read-only. |
366 |
page about additional ways to make sure access is read-only. |
| 346 |
It is advisable, that you create an unprivileged account, |
367 |
It is advised that you create an unprivileged account, |
| 347 |
preferably called <username>anoncvs</username>. |
368 |
preferably called <username>anoncvs</username>. |
| 348 |
Also you need to create a file <filename>passwd</filename> |
369 |
Also you need to create a file <filename>passwd</filename> |
| 349 |
in your <filename>/home/ncvs/CVSROOT</filename> and assign a |
370 |
in your <filename>/home/ncvs/CVSROOT</filename> and assign a |
| 350 |
CVS password (empty or <literal>anoncvs</literal>) to that user. |
371 |
CVS password (empty or <literal>anoncvs</literal>) to that user. |
| 351 |
The directory <filename>/anoncvstmp</filename> is a special |
372 |
The directory <filename>/anoncvstmp</filename> is a special |
| 352 |
purpose memory based file system. It is not required but |
373 |
purpose memory based file system. It is not required but |
| 353 |
advised, since &man.cvs.1; creates a shadow directory |
374 |
advised since &man.cvs.1; creates a shadow directory |
| 354 |
structure in your <filename>/tmp</filename> which is |
375 |
structure in your <filename>/tmp</filename> which is |
| 355 |
not used after the operation, but slows things |
376 |
not used after the operation but slows things |
| 356 |
dramatically, if real disk operations are required. |
377 |
dramatically if real disk operations are required. |
| 357 |
Here is an excerpt from <filename>/etc/fstab</filename>, |
378 |
Here is an excerpt from <filename>/etc/fstab</filename>, |
| 358 |
how to set up such a MFS: |
379 |
how to set up such a MFS: |
| 359 |
<programlisting> |
380 |
<programlisting> |
|
Lines 365-373
Link Here
|
| 365 |
</sect2> |
386 |
</sect2> |
| 366 |
</sect1> |
387 |
</sect1> |
| 367 |
<sect1 id="mirror-howto"> |
388 |
<sect1 id="mirror-howto"> |
| 368 |
<title>How to mirror FreeBSD</title> |
389 |
<title>How To Mirror FreeBSD</title> |
| 369 |
<para> |
390 |
<para> |
| 370 |
Ok now, you know the requirements, and how to offer |
391 |
Ok, now you know the requirements and how to offer |
| 371 |
the services, but not how to get it. :-) |
392 |
the services, but not how to get it. :-) |
| 372 |
This section explains how to actually mirror |
393 |
This section explains how to actually mirror |
| 373 |
the various parts of FreeBSD, what tools to use, |
394 |
the various parts of FreeBSD, what tools to use, |
|
Lines 376-398
Link Here
|
| 376 |
<sect2 id="mirror-ftp"> |
397 |
<sect2 id="mirror-ftp"> |
| 377 |
<title>FTP</title> |
398 |
<title>FTP</title> |
| 378 |
<para> |
399 |
<para> |
| 379 |
The FTP area is the largest amount of data, that |
400 |
The FTP area is the largest amount of data that |
| 380 |
needs to be mirrored. It includes the <emphasis>distributions |
401 |
needs to be mirrored. It includes the <emphasis>distribution |
| 381 |
sets</emphasis>, required for network installation, the |
402 |
sets</emphasis> required for network installation, the |
| 382 |
<emphasis>branches</emphasis>, that are actually snapshots |
403 |
<emphasis>branches</emphasis> which are actually snapshots |
| 383 |
of checked-out source trees, the <emphasis>ISO Images</emphasis> |
404 |
of checked-out source trees, the <emphasis>ISO Images</emphasis> |
| 384 |
to write CD-ROMs with the installation distribution, |
405 |
to write CD-ROMs with the installation distribution, |
| 385 |
a live file system, and lots of packages, the ports tree, |
406 |
a live file system, lots of packages, the ports tree, |
| 386 |
distfiles and a huge amount of packages. All of course |
407 |
distfiles, and a huge amount of packages. All of course |
| 387 |
for various FreeBSD versions, and <emphasis>i386</emphasis> |
408 |
for various FreeBSD versions, and various architectures. |
| 388 |
and <emphasis>alpha</emphasis> architecture. |
|
|
| 389 |
</para> |
409 |
</para> |
| 390 |
<sect3 id="mirror-ftp-ftp"> |
410 |
<sect3 id="mirror-ftp-ftp"> |
| 391 |
<title>With FTP mirror</title> |
411 |
<title>With FTP mirror</title> |
| 392 |
<para> |
412 |
<para> |
| 393 |
You can use a <application>FTP mirror</application> |
413 |
You can use a <application>FTP mirror</application> |
| 394 |
program, to get the files. There are a lot around, and |
414 |
program to get the files. There are a lot around and |
| 395 |
widely used, like: |
415 |
widely used, for example: |
| 396 |
<itemizedlist> |
416 |
<itemizedlist> |
| 397 |
<listitem><para><filename role="package">ftp/mirror</filename></para></listitem> |
417 |
<listitem><para><filename role="package">ftp/mirror</filename></para></listitem> |
| 398 |
<listitem><para><filename role="package">ftp/ftpmirror</filename></para></listitem> |
418 |
<listitem><para><filename role="package">ftp/ftpmirror</filename></para></listitem> |
|
Lines 404-443
Link Here
|
| 404 |
|
424 |
|
| 405 |
<filename role="package">ftp/mirror</filename> was very popular, but seemed |
425 |
<filename role="package">ftp/mirror</filename> was very popular, but seemed |
| 406 |
to have some drawbacks, as it is written in &man.perl.1;, |
426 |
to have some drawbacks, as it is written in &man.perl.1;, |
| 407 |
and did have real problems on mirroring large |
427 |
and had real problems with mirroring large |
| 408 |
directories like a FreeBSD site. There are rumors, that |
428 |
directories like a FreeBSD site. There are rumors that |
| 409 |
the current version has fixed this, by allowing |
429 |
the current version has fixed this by allowing |
| 410 |
to specify a different algorithm for comparing |
430 |
a different algorithm for comparing |
| 411 |
the directory structure. |
431 |
the directory structure to be specified. |
| 412 |
</para> |
432 |
</para> |
| 413 |
<para> |
433 |
<para> |
| 414 |
In general FTP is not really good for mirroring, since it transfers |
434 |
In general FTP is not really good for mirroring. It transfers |
| 415 |
each whole file, if it has changed, and does |
435 |
the whole file if it has changed, and does |
| 416 |
not create a single data stream, that will benefit from |
436 |
not create a single data stream which would benefit from |
| 417 |
a large TCP congestion window. |
437 |
a large TCP congestion window. |
| 418 |
</para> |
438 |
</para> |
| 419 |
</sect3> |
439 |
</sect3> |
| 420 |
<sect3 id="mirror-ftp-rsync"> |
440 |
<sect3 id="mirror-ftp-rsync"> |
| 421 |
<title>With RSYNC</title> |
441 |
<title>With RSYNC</title> |
| 422 |
<para> |
442 |
<para> |
| 423 |
A better way, to mirror the FTP area is <application>rsync</application>. |
443 |
A better way to mirror the FTP area is <application>rsync</application>. |
| 424 |
You can install the port <filename role="package">net/rsync</filename> and then use |
444 |
You can install the port <filename role="package">net/rsync</filename> and then use |
| 425 |
rsync to sync with your upstream host. |
445 |
rsync to sync with your upstream host. |
| 426 |
<application>rsync</application> is already mentioned |
446 |
<application>rsync</application> is already mentioned |
| 427 |
in <xref linkend="mirror-serv-rsync">. |
447 |
in <xref linkend="mirror-serv-rsync">. |
| 428 |
Since <application>rsync</application> access is not |
448 |
Since <application>rsync</application> access is not |
| 429 |
required, your preferred upstream site may not allow it. |
449 |
required your preferred upstream site may not allow it. |
| 430 |
Since it is quite common, though, chances are small, that |
450 |
You may need to hunt around a little bit to find a site |
| 431 |
you cannot use it. You can always consider using an |
451 |
that allows <application>rsync</application> access. |
| 432 |
upstream server, that offers it, just for the benefits |
|
|
| 433 |
of rsync. |
| 434 |
<note> |
452 |
<note> |
| 435 |
<para> |
453 |
<para> |
| 436 |
Since the number of <application>rsync</application> |
454 |
Since the number of <application>rsync</application> |
| 437 |
clients will have a significant impact on the server |
455 |
clients will have a significant impact on the server |
| 438 |
machine, most admins impose limitations on their |
456 |
machine, most admins impose limitations on their |
| 439 |
server. For a mirror, you should ask the site maintainer |
457 |
server. For a mirror you should ask the site maintainer |
| 440 |
you are syncing from, about their policy, and maybe |
458 |
you are syncing from about their policy, and maybe make |
| 441 |
an exception for your host (since you are a mirror). |
459 |
an exception for your host (since you are a mirror). |
| 442 |
</para> |
460 |
</para> |
| 443 |
</note> |
461 |
</note> |
|
Lines 452-458
Link Here
|
| 452 |
be aware that the module-directory (here "FreeBSD") |
470 |
be aware that the module-directory (here "FreeBSD") |
| 453 |
will not be created, so you cannot omit the target directory. |
471 |
will not be created, so you cannot omit the target directory. |
| 454 |
Also you might |
472 |
Also you might |
| 455 |
want to set up a script framework, that calls such a command |
473 |
want to set up a script framework that calls such a command |
| 456 |
via &man.cron.8;. |
474 |
via &man.cron.8;. |
| 457 |
</para> |
475 |
</para> |
| 458 |
</sect3> |
476 |
</sect3> |
|
Lines 462-470
Link Here
|
| 462 |
A few sites, including the one-and-only <hostid>ftp-master.FreeBSD.org</hostid> |
480 |
A few sites, including the one-and-only <hostid>ftp-master.FreeBSD.org</hostid> |
| 463 |
even offer <application>CVSup</application> to mirror the contents of |
481 |
even offer <application>CVSup</application> to mirror the contents of |
| 464 |
the FTP space. You need to install a <application>cvsup</application> |
482 |
the FTP space. You need to install a <application>cvsup</application> |
| 465 |
client, preferably from the port: <filename role="package">net/cvsup</filename>. |
483 |
client, preferably from the port <filename role="package">net/cvsup</filename>. |
| 466 |
(Also reread <xref linkend="mirror-serv-cvsup">.) |
484 |
(Also reread <xref linkend="mirror-serv-cvsup">.) |
| 467 |
A sample supfile, suitable for <hostid>ftp-master.FreeBSD.org</hostid> |
485 |
A sample supfile suitable for <hostid>ftp-master.FreeBSD.org</hostid> |
| 468 |
looks like this: |
486 |
looks like this: |
| 469 |
<programlisting> |
487 |
<programlisting> |
| 470 |
# |
488 |
# |
|
Lines 484-492
Link Here
|
| 484 |
</programlisting> |
502 |
</programlisting> |
| 485 |
|
503 |
|
| 486 |
It seems <application>CVSup</application> would be the best |
504 |
It seems <application>CVSup</application> would be the best |
| 487 |
way to mirror the archive, in terms of efficiency, but |
505 |
way to mirror the archive in terms of efficiency, but |
| 488 |
it is only available from few sites. In fact I just know |
506 |
it is only available from a few sites. |
| 489 |
<hostid>ftp-master.FreeBSD.org</hostid> for sure. |
|
|
| 490 |
<note id="mirror-cvsup-s-option"> |
507 |
<note id="mirror-cvsup-s-option"> |
| 491 |
<para> |
508 |
<para> |
| 492 |
Please have look at the <application>CVSup</application> documentation |
509 |
Please have look at the <application>CVSup</application> documentation |
|
Lines 501-508
Link Here
|
| 501 |
<sect2 id="mirror-cvs"> |
518 |
<sect2 id="mirror-cvs"> |
| 502 |
<title>Mirroring the CVS repository</title> |
519 |
<title>Mirroring the CVS repository</title> |
| 503 |
<para> |
520 |
<para> |
| 504 |
Again you have various possibilities, but the most |
521 |
Again you have various possibilities but the most |
| 505 |
recommended one, is to use <link linkend="mirror-cvs-cvsup">CVSup</link>. |
522 |
recommended one is to use <link linkend="mirror-cvs-cvsup">CVSup</link>. |
| 506 |
</para> |
523 |
</para> |
| 507 |
<sect3 id="mirror-cvs-cvsup"> |
524 |
<sect3 id="mirror-cvs-cvsup"> |
| 508 |
<title>Using CVSup</title> |
525 |
<title>Using CVSup</title> |
|
Lines 533-539
Link Here
|
| 533 |
</para> |
550 |
</para> |
| 534 |
<note> |
551 |
<note> |
| 535 |
<para> |
552 |
<para> |
| 536 |
Please don't forget to consider the hint, |
553 |
Please don't forget to consider the hint |
| 537 |
mentioned in <link linkend="mirror-cvsup-s-option">this note</link> |
554 |
mentioned in <link linkend="mirror-cvsup-s-option">this note</link> |
| 538 |
above. |
555 |
above. |
| 539 |
</para> |
556 |
</para> |
|
Lines 554-569
Link Here
|
| 554 |
<listitem><para>maybe even <application>HTTP</application></para></listitem> |
571 |
<listitem><para>maybe even <application>HTTP</application></para></listitem> |
| 555 |
</itemizedlist> |
572 |
</itemizedlist> |
| 556 |
|
573 |
|
| 557 |
If you find a site, that supports it, you could use |
574 |
If you find a site that supports it you could use |
| 558 |
<filename role="package">net/sup</filename>, but it is inferior to <application>CVSup</application> |
575 |
<filename role="package">net/sup</filename>. But it is inferior to <application>CVSup</application> |
| 559 |
and it's deficiencies caused John Polstra to develop |
576 |
and it's deficiencies caused John Polstra to develop |
| 560 |
<application>CVSup</application> in the first place, so |
577 |
<application>CVSup</application> in the first place so |
| 561 |
it is clearly not recommended. |
578 |
it is clearly not recommended. |
| 562 |
|
579 |
|
| 563 |
<important> |
580 |
<important> |
| 564 |
<para> |
581 |
<para> |
| 565 |
You can <emphasis>NOT</emphasis> use AnonCVS to |
582 |
You can <emphasis>NOT</emphasis> use AnonCVS to |
| 566 |
mirror the CVS repository, since CVS does not allow |
583 |
mirror the CVS repository since CVS does not allow |
| 567 |
you to access the repository itself, but only checked |
584 |
you to access the repository itself, but only checked |
| 568 |
out versions of the modules. |
585 |
out versions of the modules. |
| 569 |
</para> |
586 |
</para> |
|
Lines 574-586
Link Here
|
| 574 |
<sect2 id="mirror-www"> |
591 |
<sect2 id="mirror-www"> |
| 575 |
<title>Mirroring the WWW pages</title> |
592 |
<title>Mirroring the WWW pages</title> |
| 576 |
<para> |
593 |
<para> |
| 577 |
The best way is, to check out the <emphasis>www</emphasis> |
594 |
The best way is to check out the <emphasis>www</emphasis> |
| 578 |
distribution from CVS. If you have a local mirror of the |
595 |
distribution from CVS. If you have a local mirror of the |
| 579 |
CVS repository, it is probably as easy as: |
596 |
CVS repository it is probably as easy as: |
| 580 |
<screen>&prompt.user; <userinput>cvs -d /home/ncvs co www</userinput></screen> |
597 |
<screen>&prompt.user; <userinput>cvs -d /home/ncvs co www</userinput></screen> |
| 581 |
and a <emphasis>cronjob</emphasis>, that calls <command>cvs up -d -P</command> |
598 |
and a <emphasis>cronjob</emphasis>, that calls <command>cvs up -d -P</command> |
| 582 |
on a regular basis, maybe just after your repository was updated. |
599 |
on a regular basis, maybe just after your repository was updated. |
| 583 |
Of course, the files need to remain in a directory, available |
600 |
Of course, the files need to remain in a directory available |
| 584 |
for public WWW access. The installation and configuration of a |
601 |
for public WWW access. The installation and configuration of a |
| 585 |
webserver is not discussed here. |
602 |
webserver is not discussed here. |
| 586 |
</para> |
603 |
</para> |
|
Lines 588-594
Link Here
|
| 588 |
<note><para>For the website to be visible, users must execute the &man.make.1; |
605 |
<note><para>For the website to be visible, users must execute the &man.make.1; |
| 589 |
command in the main <filename>www</filename> directory. This command |
606 |
command in the main <filename>www</filename> directory. This command |
| 590 |
will create the standard <filename>*.html</filename> files for web |
607 |
will create the standard <filename>*.html</filename> files for web |
| 591 |
viewing. For this to work, however, the |
608 |
viewing. For this to work however, the |
| 592 |
<filename role="package">textproc/docproj</filename> port must be |
609 |
<filename role="package">textproc/docproj</filename> port must be |
| 593 |
installed.</para></note> |
610 |
installed.</para></note> |
| 594 |
<para> |
611 |
<para> |
|
Lines 615-630
Link Here
|
| 615 |
</programlisting> |
632 |
</programlisting> |
| 616 |
</para> |
633 |
</para> |
| 617 |
<para> |
634 |
<para> |
| 618 |
Using <filename role="package">ftp/wget</filename> or other web-mirror tools, is |
635 |
Using <filename role="package">ftp/wget</filename> or other web-mirror tools is |
| 619 |
probably not recommended. |
636 |
probably not recommended. |
| 620 |
</para> |
637 |
</para> |
| 621 |
<sect3 id="mirror-www-doc"> |
638 |
<sect3 id="mirror-www-doc"> |
| 622 |
<title>Mirroring the FreeBSD documentation</title> |
639 |
<title>Mirroring the FreeBSD documentation</title> |
| 623 |
<para> |
640 |
<para> |
| 624 |
As the documentation is referenced a lot from the |
641 |
Since the documentation is referenced a lot from the |
| 625 |
webpages, it is recommended, that you mirror the |
642 |
webpages it is recommended that you mirror the |
| 626 |
FreeBSD documentation as well. However, this is not |
643 |
FreeBSD documentation as well. However, this is not |
| 627 |
so trivial as the www-pages alone. |
644 |
as trivial as the www-pages alone. |
| 628 |
</para> |
645 |
</para> |
| 629 |
<para> |
646 |
<para> |
| 630 |
First of all, you should get the doc sources, |
647 |
First of all, you should get the doc sources, |
|
Lines 650-676
Link Here
|
| 650 |
</para> |
667 |
</para> |
| 651 |
<para> |
668 |
<para> |
| 652 |
Then you need to install a couple of ports. |
669 |
Then you need to install a couple of ports. |
| 653 |
You are lucky, that there is a meta-port: |
670 |
You are lucky, there is a meta-port: |
| 654 |
<filename role="package">textproc/docproj</filename> to do the work |
671 |
<filename role="package">textproc/docproj</filename> to do the work |
| 655 |
for you. You need to setup some |
672 |
for you. You need to set up some |
| 656 |
environment variables, like |
673 |
environment variables, like |
| 657 |
<literal>SGML_CATALOG_FILES</literal>, |
674 |
<literal>SGML_CATALOG_FILES</literal>. |
| 658 |
also have a look at your <filename>/etc/make.conf</filename> |
675 |
Also have a look at your <filename>/etc/make.conf</filename> |
| 659 |
(copy <filename>/etc/defaults/make.conf</filename> if |
676 |
(copy <filename>/etc/defaults/make.conf</filename> if |
| 660 |
you don't have one), and look at the |
677 |
you don't have one) and look at the |
| 661 |
<literal>DOC_LANG</literal> variable. |
678 |
<literal>DOC_LANG</literal> variable. |
| 662 |
Now you are probably ready to run <command>make</command> |
679 |
Now you are probably ready to run <command>make</command> |
| 663 |
in you doc directory (<filename>/usr/share/doc</filename> |
680 |
in you doc directory (<filename>/usr/share/doc</filename> |
| 664 |
by default) and build the documentation. |
681 |
by default) and build the documentation. |
| 665 |
Again you need to make it accessible for your webserver |
682 |
Again you need to make it accessible for your webserver |
| 666 |
and make sure, the links point to the right location. |
683 |
and make sure the links point to the right location. |
| 667 |
<important> |
684 |
<important> |
| 668 |
<para> |
685 |
<para> |
| 669 |
The building of the documentation, as well as lots |
686 |
The building of the documentation, as well as lots |
| 670 |
of side issues, is documented itself in: |
687 |
of side issues, is documented itself in: |
| 671 |
<ulink url="http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/fdp-primer/">fdp-primer</ulink>. |
688 |
<ulink url="http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/fdp-primer/">fdp-primer</ulink>. |
| 672 |
Please read this piece of documentation, especially if you |
689 |
Please read this piece of documentation, especially if you |
| 673 |
have problems, building the documentation. |
690 |
have problems building the documentation. |
| 674 |
</para> |
691 |
</para> |
| 675 |
</important> |
692 |
</important> |
| 676 |
<note> |
693 |
<note> |
|
Lines 686-692
Link Here
|
| 686 |
<para> |
703 |
<para> |
| 687 |
Every mirror should be updated on a regular |
704 |
Every mirror should be updated on a regular |
| 688 |
basis. You will certainly need some script |
705 |
basis. You will certainly need some script |
| 689 |
framework for it, that will be called by |
706 |
framework for it that will be called by |
| 690 |
&man.cron.8;. Since nearly every admin |
707 |
&man.cron.8;. Since nearly every admin |
| 691 |
does this his own way, we cannot give |
708 |
does this his own way, we cannot give |
| 692 |
specific instructions. It could work |
709 |
specific instructions. It could work |
|
Lines 702-720
Link Here
|
| 702 |
</step> |
719 |
</step> |
| 703 |
<step> |
720 |
<step> |
| 704 |
<para> |
721 |
<para> |
| 705 |
Add some output redirections, so diagnostic |
722 |
Add some output redirections so diagnostic |
| 706 |
messages are logged to a file. |
723 |
messages are logged to a file. |
| 707 |
</para> |
724 |
</para> |
| 708 |
</step> |
725 |
</step> |
| 709 |
<step> |
726 |
<step> |
| 710 |
<para> |
727 |
<para> |
| 711 |
Test, if your script works. Check the logs. |
728 |
Test if your script works. Check the logs. |
| 712 |
</para> |
729 |
</para> |
| 713 |
</step> |
730 |
</step> |
| 714 |
<step> |
731 |
<step> |
| 715 |
<para> |
732 |
<para> |
| 716 |
Use &man.crontab.1; to add the script the |
733 |
Use &man.crontab.1; to add the script to the |
| 717 |
appropriate user's &man.crontab.5;. |
734 |
appropriate user's &man.crontab.5;. This should be a |
|
|
735 |
different user than what your FTP daemon runs as so that |
| 736 |
if file permissions inside your FTP area are not world-readable |
| 737 |
those files can not be accessed by anonymous FTP. This is used |
| 738 |
to "stage" releases - making sure all of the Official Mirror |
| 739 |
sites have all of the necessary Release files on Release Day. |
| 718 |
</para> |
740 |
</para> |
| 719 |
</step> |
741 |
</step> |
| 720 |
</procedure> |
742 |
</procedure> |
|
Lines 732-784
Link Here
|
| 732 |
<title>Where to mirror from</title> |
754 |
<title>Where to mirror from</title> |
| 733 |
<para> |
755 |
<para> |
| 734 |
This is an important issue. So this section will |
756 |
This is an important issue. So this section will |
| 735 |
spend some effort to explain the backgrounds. |
757 |
spend some effort to explain the backgrounds. We will say this |
|
|
758 |
several times : Under no circumstances should you mirror from |
| 759 |
<hostid>ftp.FreeBSD.org</hostid>. |
| 736 |
</para> |
760 |
</para> |
| 737 |
<sect2 id="mirror-where-organization"> |
761 |
<sect2 id="mirror-where-organization"> |
| 738 |
<title>A few words about the organization</title> |
762 |
<title>A few words about the organization</title> |
| 739 |
<para> |
763 |
<para> |
| 740 |
Mirrors are organized by country. All |
764 |
Mirrors are organized by country. All |
| 741 |
official mirrors have a DNS entry of the form |
765 |
official mirrors have a DNS entry of the form |
| 742 |
<hostid>ftpN.CC.FreeBSD.org</hostid>. With |
766 |
<hostid>ftpN.CC.FreeBSD.org</hostid>. <emphasis>CC</emphasis> |
| 743 |
<emphasis>CC</emphasis> (i.e. country code) being the |
767 |
(i.e. country code) is the |
| 744 |
<emphasis>top level domain</emphasis> |
768 |
<emphasis>top level domain</emphasis> (TLD) |
| 745 |
of the country, where this mirror is located; |
769 |
of the country where this mirror is located. |
| 746 |
and <emphasis>N</emphasis> is a number, |
770 |
<emphasis>N</emphasis> is a number, |
| 747 |
telling that the host would be the <emphasis>Nth</emphasis> |
771 |
telling that the host would be the <emphasis>Nth</emphasis> |
| 748 |
mirror in that country. |
772 |
mirror in that country. |
| 749 |
(Same applies to <hostid>cvsupN.CC.FreeBSD.org</hostid>, |
773 |
(Same applies to <hostid>cvsupN.CC.FreeBSD.org</hostid>, |
| 750 |
<hostid>wwwN.CC.FreeBSD.org</hostid>, etc.) |
774 |
<hostid>wwwN.CC.FreeBSD.org</hostid>, etc.) |
| 751 |
There are mirrors with no <emphasis>CC</emphasis> part. |
775 |
There are mirrors with no <emphasis>CC</emphasis> part. |
| 752 |
These are usually located in the US, but don't need to. |
776 |
These are the mirror sites that are very well connected and |
| 753 |
<hostid>ftp.FreeBSD.org</hostid> is currently |
777 |
allow a large number of concurrent users. |
| 754 |
located in Denmark and just another mirror |
778 |
<hostid>ftp.FreeBSD.org</hostid> is actually two machines, one currently |
| 755 |
(i.e. it is <emphasis>NO</emphasis> master site). |
779 |
located in Denmark and the other in the United States. |
|
|
780 |
It is <emphasis>NOT</emphasis> a master site and should never be |
| 781 |
used to mirror from. Lots of online documentation leads "interactive" |
| 782 |
users to <hostid>ftp.FreeBSD.org</hostid> so automated mirroring |
| 783 |
systems should find a different machine to mirror from. |
| 756 |
</para> |
784 |
</para> |
| 757 |
<para> |
785 |
<para> |
| 758 |
Additionally there exists a hierarchy of mirrors, which |
786 |
Additionally there exists a hierarchy of mirrors, which |
| 759 |
is described terms of <emphasis>tiers</emphasis>. |
787 |
is described in terms of <emphasis>tiers</emphasis>. |
| 760 |
The master sites are not referred to, but can be |
788 |
The master sites are not referred to but can be |
| 761 |
described as <emphasis>Tier-0</emphasis>. Mirrors |
789 |
described as <emphasis>Tier-0</emphasis>. Mirrors |
| 762 |
that mirror from these sites can be considered |
790 |
that mirror from these sites can be considered |
| 763 |
<emphasis>Tier-1</emphasis>, mirrors of <emphasis>Tier-1</emphasis>-mirrors, |
791 |
<emphasis>Tier-1</emphasis>, mirrors of <emphasis>Tier-1</emphasis>-mirrors, |
| 764 |
are <emphasis>Tier-2</emphasis>, etc. |
792 |
are <emphasis>Tier-2</emphasis>, etc. |
| 765 |
Official sites are encouraged to be of a low <emphasis>tier</emphasis>, |
793 |
Official sites are encouraged to be of a low <emphasis>tier</emphasis>, |
| 766 |
but the lower the tier, the higher the requirements in |
794 |
but the lower the tier the higher the requirements in |
| 767 |
terms as described in <xref linkend="mirror-requirements">. |
795 |
terms as described in <xref linkend="mirror-requirements">. |
| 768 |
Also access to low-tier-mirrors may be restricted, and |
796 |
Also access to low-tier-mirrors may be restricted and |
| 769 |
access to master sites is definitely restricted. |
797 |
access to master sites is definitely restricted. |
| 770 |
The <emphasis>tier</emphasis>-hierarchy is not reflected |
798 |
The <emphasis>tier</emphasis>-hierarchy is not reflected |
| 771 |
by DNS and generally not documented anywhere, except |
799 |
by DNS and generally not documented anywhere except |
| 772 |
for the master sites. However, official mirrors with low numbers, |
800 |
for the master sites. However, official mirrors with low numbers |
| 773 |
like 1-4, are usually <emphasis>Tier-1</emphasis> |
801 |
like 1-4, are usually <emphasis>Tier-1</emphasis> |
| 774 |
(this is just a rough hint, and there's no rule). |
802 |
(this is just a rough hint, and there is no rule). |
| 775 |
</para> |
803 |
</para> |
| 776 |
</sect2> |
804 |
</sect2> |
| 777 |
<sect2 id="mirror-where-where"> |
805 |
<sect2 id="mirror-where-where"> |
| 778 |
<title>Ok, but where should I get the stuff now?</title> |
806 |
<title>Ok, but where should I get the stuff now?</title> |
| 779 |
<para> |
807 |
<para> |
|
|
808 |
Under no circumstances should you mirror from <hostid>ftp.FreeBSD.org |
| 809 |
</hostid>. |
| 780 |
The short answer is: from the |
810 |
The short answer is: from the |
| 781 |
site, that is closest to you in Internet terms, or gives you |
811 |
site that is closest to you in Internet terms, or gives you |
| 782 |
the fastest access. |
812 |
the fastest access. |
| 783 |
</para> |
813 |
</para> |
| 784 |
<sect3 id="mirror-where-simple"> |
814 |
<sect3 id="mirror-where-simple"> |
|
Lines 798-804
Link Here
|
| 798 |
</step> |
828 |
</step> |
| 799 |
<step> |
829 |
<step> |
| 800 |
<para> |
830 |
<para> |
| 801 |
Check roughly those, which provide fastest access |
831 |
Check for those which provide fastest access |
| 802 |
(number of hops, round-trip-times) |
832 |
(number of hops, round-trip-times) |
| 803 |
and offer the services you intend to |
833 |
and offer the services you intend to |
| 804 |
use (like <application>rsync</application> |
834 |
use (like <application>rsync</application> |
|
Lines 807-814
Link Here
|
| 807 |
</step> |
837 |
</step> |
| 808 |
<step> |
838 |
<step> |
| 809 |
<para> |
839 |
<para> |
| 810 |
Contact the admins of your chosen site, stating your |
840 |
Contact the admins of your chosen site stating your |
| 811 |
request, and asking about their terms and |
841 |
request and asking about their terms and |
| 812 |
policies. |
842 |
policies. |
| 813 |
</para> |
843 |
</para> |
| 814 |
</step> |
844 |
</step> |
|
Lines 824-851
Link Here
|
| 824 |
<para> |
854 |
<para> |
| 825 |
In general the description in <xref linkend="mirror-where-simple"> |
855 |
In general the description in <xref linkend="mirror-where-simple"> |
| 826 |
still applies. Of course you may want to put some |
856 |
still applies. Of course you may want to put some |
| 827 |
weight on the fact, that your upstream should be of |
857 |
weight on the fact that your upstream should be of |
| 828 |
a low tier. |
858 |
a low tier. |
| 829 |
There are some other considerations about <emphasis>official</emphasis> |
859 |
There are some other considerations about <emphasis>official</emphasis> |
| 830 |
mirrors, that are described in <xref linkend="mirror-official">. |
860 |
mirrors that are described in <xref linkend="mirror-official">. |
| 831 |
</para> |
861 |
</para> |
| 832 |
</sect3> |
862 |
</sect3> |
| 833 |
<sect3 id="mirror-where-master"> |
863 |
<sect3 id="mirror-where-master"> |
| 834 |
<title>I want to access the master sites!</title> |
864 |
<title>I want to access the master sites!</title> |
| 835 |
<para> |
865 |
<para> |
| 836 |
If you have good reasons, and good prerequisites, |
866 |
If you have good reasons and good prerequisites |
| 837 |
you may want and get access to one of the |
867 |
you may want and get access to one of the |
| 838 |
master sites. Access to these sites is |
868 |
master sites. Access to these sites is |
| 839 |
generally restricted, and there are special policies |
869 |
generally restricted, and there are special policies |
| 840 |
for access. If you are already an <emphasis>official</emphasis> |
870 |
for access. If you are already an <emphasis>official</emphasis> |
| 841 |
mirror, this certainly helps you getting access. |
871 |
mirror this certainly helps you getting access. |
| 842 |
In any other case make sure your country really needs another mirror. |
872 |
In any other case make sure your country really needs another mirror. |
| 843 |
If it already has three or more, ask the &a.hubs; first. |
873 |
If it already has three or more, ask the Zone Administrator |
|
|
874 |
(<email>hostmaster@CC.freebsd.org</email>) or &a.hubs; first. |
| 875 |
</para> |
| 876 |
<para> |
| 877 |
Whoever helped you become an <emphasis>official</emphasis> should |
| 878 |
have helped you gain access to an appropriate upstream host, either |
| 879 |
one of the master sites or a suitable Tier-1 site. If not you can |
| 880 |
send email to <email>mirror-admin@FreeBSD.org</email> to request |
| 881 |
help with that. |
| 844 |
</para> |
882 |
</para> |
| 845 |
<para> |
883 |
<para> |
| 846 |
There are just two master sites, one for the FTP fileset and |
884 |
There are three master sites for the FTP fileset and |
| 847 |
one for the CVS repository (the webpages and docs are |
885 |
one for the CVS repository (the webpages and docs are |
| 848 |
obtained from CVS, so there is no need for it). |
886 |
obtained from CVS, so there is no need for a master). |
| 849 |
</para> |
887 |
</para> |
| 850 |
<sect4 id="mirror-where-master-ftp"> |
888 |
<sect4 id="mirror-where-master-ftp"> |
| 851 |
<title>ftp-master.FreeBSD.org</title> |
889 |
<title>ftp-master.FreeBSD.org</title> |
|
Lines 865-874
Link Here
|
| 865 |
access for the FTP contents, since they are |
903 |
access for the FTP contents, since they are |
| 866 |
<emphasis>Tier-1</emphasis>-mirrors. |
904 |
<emphasis>Tier-1</emphasis>-mirrors. |
| 867 |
</para> |
905 |
</para> |
| 868 |
<para> |
|
|
| 869 |
To get access to <hostid>ftp-master.FreeBSD.org</hostid>, |
| 870 |
you need to contact &a.ftp-master;. |
| 871 |
</para> |
| 872 |
</sect4> |
906 |
</sect4> |
| 873 |
<sect4 id="mirror-where-master-cvsup"> |
907 |
<sect4 id="mirror-where-master-cvsup"> |
| 874 |
<title>cvsup-master.FreeBSD.org</title> |
908 |
<title>cvsup-master.FreeBSD.org</title> |
|
Lines 956-968
Link Here
|
| 956 |
<important> |
990 |
<important> |
| 957 |
<para>It is <emphasis>very</emphasis> important for a hub administrator, especially |
991 |
<para>It is <emphasis>very</emphasis> important for a hub administrator, especially |
| 958 |
Tier-1 hub admins, to check the |
992 |
Tier-1 hub admins, to check the |
| 959 |
<ulink url="http://www.FreeBSD.org/releases/4.6R/schedule.html">release schedule</ulink> |
993 |
<ulink url="http://www.FreeBSD.org/releng/">release schedule</ulink> |
| 960 |
for the next FreeBSD release. This is important because it will tell you when the |
994 |
for the next FreeBSD release. This is important because it will tell you when the |
| 961 |
next release is scheduled |
995 |
next release is scheduled |
| 962 |
to come out, and thus giving you time to prepare for the big spike of traffic which follows it. |
996 |
to come out, and thus giving you time to prepare for the big spike of traffic which follows it. |
| 963 |
</para> |
997 |
</para> |
| 964 |
<para> |
998 |
<para> |
| 965 |
It is also eminent that hub administrators try to keep their mirrors as up-to-date as |
999 |
It is also important that hub administrators try to keep their mirrors as up-to-date as |
| 966 |
possible (again, even more crucial for Tier-1 mirrors). If Mirror1 doesn't update for a |
1000 |
possible (again, even more crucial for Tier-1 mirrors). If Mirror1 doesn't update for a |
| 967 |
while, lower tier mirrors will begin to mirror old data from Mirror1 and thus begins |
1001 |
while, lower tier mirrors will begin to mirror old data from Mirror1 and thus begins |
| 968 |
a downward spiral... Keep your mirrors up to date! |
1002 |
a downward spiral... Keep your mirrors up to date! |
|
Lines 974-987
Link Here
|
| 974 |
<para> |
1008 |
<para> |
| 975 |
An interesting question, especially, since the state |
1009 |
An interesting question, especially, since the state |
| 976 |
of being official comes with some benefits, like a much |
1010 |
of being official comes with some benefits, like a much |
| 977 |
higher bill from your ISP, as more people will be using |
1011 |
higher bill from your ISP as more people will be using |
| 978 |
your site. Also it may be a key requirement, to get access |
1012 |
your site. Also it may be a key requirement to get access |
| 979 |
to a master site. |
1013 |
to a master site. |
| 980 |
</para> |
1014 |
</para> |
| 981 |
<para> |
1015 |
<para> |
| 982 |
Before applying, please consider (again) if |
1016 |
Before applying please consider (again) if |
| 983 |
another official mirror is really needed for |
1017 |
another official mirror is really needed for |
| 984 |
your region. Ask on the &a.hubs;, if in doubt. |
1018 |
your region. Check first with your Zone Administrator |
|
|
1019 |
(<email>hostmaster@CC.FreeBSD.org</email>) or, if that |
| 1020 |
fails, ask on the &a.hubs;. |
| 985 |
</para> |
1021 |
</para> |
| 986 |
<para>Ok, here is how to do it:</para> |
1022 |
<para>Ok, here is how to do it:</para> |
| 987 |
<procedure> |
1023 |
<procedure> |
|
Lines 998-1029
Link Here
|
| 998 |
</step> |
1034 |
</step> |
| 999 |
<step> |
1035 |
<step> |
| 1000 |
<para> |
1036 |
<para> |
| 1001 |
If everything works so far, contact the DNS admin, responsible |
1037 |
If everything works so far, contact the DNS admin responsible |
| 1002 |
for your region/country, and ask for a DNS entry for your |
1038 |
for your region/country and ask for a DNS entry for your |
| 1003 |
site. The admin should able to be contacted via |
1039 |
site. The admin should able to be contacted via |
| 1004 |
<email>hostmaster@cc.FreeBSD.org</email>, which |
1040 |
<email>hostmaster@CC.FreeBSD.org</email>, where |
| 1005 |
<emphasis>cc</emphasis> being your country code/TLD |
1041 |
<emphasis>CC</emphasis> is your country code/TLD. |
| 1006 |
again. Your DNS entry will look like described |
1042 |
Your DNS entry will be as described |
| 1007 |
in <xref linkend="mirror-where-organization">. |
1043 |
in <xref linkend="mirror-where-organization">. |
| 1008 |
</para> |
1044 |
</para> |
| 1009 |
<para> |
1045 |
<para> |
| 1010 |
If there is no subdomain delegated, yet, for your |
1046 |
If there is no subdomain set up for your |
| 1011 |
country, you probably need to contact |
1047 |
country yet, you should contact |
| 1012 |
<email>hostmaster@FreeBSD.org</email>, |
1048 |
<email>mirror-admin@FreeBSD.org</email> |
| 1013 |
however, you can try the &a.hubs; first. |
1049 |
or you can try the &a.hubs; first. |
| 1014 |
</para> |
1050 |
</para> |
| 1015 |
</step> |
1051 |
</step> |
| 1016 |
<step> |
1052 |
<step> |
| 1017 |
<para> |
1053 |
<para> |
| 1018 |
Then you can ask the &a.doc; or the &a.hubs; |
1054 |
Whoever helps you get an official name should send email |
| 1019 |
to add your mirror site to the mirror list in the |
1055 |
to <email>mirror-admin@FreeBSD.org</email> so your site will be |
|
|
1056 |
added to the mirror list in the |
| 1020 |
<ulink url="http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook">FreeBSD |
1057 |
<ulink url="http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook">FreeBSD |
| 1021 |
Handbook</ulink>. Make sure you tell them the email address, |
1058 |
Handbook</ulink>. |
| 1022 |
to list as the maintainer of the site. |
|
|
| 1023 |
</para> |
1059 |
</para> |
| 1024 |
</step> |
1060 |
</step> |
| 1025 |
</procedure> |
1061 |
</procedure> |
| 1026 |
<para>This is it.</para> |
1062 |
<para>That is it.</para> |
| 1027 |
</sect2> |
1063 |
</sect2> |
| 1028 |
</sect1> |
1064 |
</sect1> |
| 1029 |
<sect1 id="mirror-statpages"> |
1065 |
<sect1 id="mirror-statpages"> |