Attachment #234786 for bug #264771

View | Details | Raw Unified | Return to bug 264771
Collapse All | Expand All




---
title: A project model for the FreeBSD Project
authors:
  - author: Niklas Saers
copyright: Copyright © 2002-2005 Niklas Saers
description: A formal study of the organization of the FreeBSD project
trademarks: ["freebsd", "ibm", "ieee", "adobe", "intel", "linux", "microsoft", "opengroup", "sun", "netbsd", "general"]
bookOrder: 45
tags: ["model", "project model", "FreeBSD"]
---

[[overview]]
== Overview
A project model is a means to reduce the communications overhead in a project.
As shown by [<<brooks, Brooks, 1995>>], increasing the number of project participants increases the communication in the project exponentionally.
FreeBSD has during the past few years increased both its mass of active users and committers, and the communication in the project has risen accordingly.
This project model will serve to reduce this overhead by providing an up-to-date description of the project.

During the Core elections in 2002, Mark Murray stated "I am opposed to a long rule-book, as that satisfies lawyer-tendencies, and is counter to the technocentricity that the project so badly needs."
[<<bsd-election2002, FreeBSD, 2002B>>].
This project model is not meant to be a tool to justify creating impositions for developers, but as a tool to facilitate coordination.
It is meant as a description of the project, with an overview of how the different processes are executed.
It is an introduction to how the FreeBSD project works.

The FreeBSD project model will be described as of July 1st, 2004.
It is based on the Niels Jørgensen's paper [<<jorgensen2001, Jørgensen, 2001>>], FreeBSD's official documents, discussions on FreeBSD mailing lists and interviews with developers.

After providing definitions of terms used, this document will outline the organisational structure (including role descriptions and communication lines), discuss the methodology model and after presenting the tools used for process control, it will present the defined processes.
Finally it will outline major sub-projects of the FreeBSD project.

[<<freebsd-developer-handbook, FreeBSD, 2002A>>] Section 1.2 and 1.3 give the vision and the architectural guidelines for the project.
The vision is "To produce the best UNIX-like operating system package possible, with due respect to the original software tools ideology as well as usability, performance and stability."
The architectural guidelines help determine whether a problem that someone wants to be solved is within the scope of the project

[[definitions]]
== Definitions


An "activity" is an element of work performed during the course of a project [<<ref-pmbok, PMI, 2000>>].
It has an output and leads towards an outcome.
Such an output can either be an input to another activity or a part of the process' delivery.

[[def-process]]
=== Process

A "process" is a series of activities that lead towards a particular outcome.
A process can consist of one or more sub-processes.
An example of a process is software design.

[[ref-hat]]
=== Hat

A "hat" is synonymous with role.
A hat has certain responsibilities in a process and for the process outcome.
The hat executes activities.
It is well defined what issues the hat should be contacted about by the project members and people outside the project.

[[ref-outcome]]
=== Outcome

An "outcome" is the final output of the process.
This is synonymous with deliverable, that is defined as "any measurable, tangible, verifiable outcome, result or item that must be produced to complete a project or part of a project.
Often used more narrowly in reference to an external deliverable, which is a deliverable that is subject to approval by the project sponsor or customer" by [<<ref-pmbok, PMI, 2000>>].
Examples of outcomes are a piece of software, a decision made or a report written.

[[ref-freebsd]]
=== FreeBSD

[[model-orgstruct]]
== Organisational structure

While no-one takes ownership of FreeBSD, the FreeBSD organisation is divided into core, committers and contributors and is part of the FreeBSD community that lives around it.

The FreeBSD Project's structure (in order of descending authority)

[.informaltable]
[cols="1,1", options="header"]

|9

|Committers
|317

|Contributors
|~3000
|===

Number of committers has been determined by going through CVS logs from January 1st, 2004 to December 31st, 2004 and contributors by going through the list of contributions and problem reports.

The main resource in the FreeBSD community is its developers: the committers and contributors.
It is with their contributions that the project can move forward.
Regular developers are referred to as contributors.
As of January 1st, 2003, there are an estimated 5500 contributors on the project.

Committers are developers with the privilege of being able to commit changes.
These are usually the most active developers who are willing to spend their time not only integrating their own code but integrating code submitted by the developers who do not have this privilege.
They are also the developers who elect the core team, and they have access to closed discussions.

The project can be grouped into four distinct separate parts, and most developers will focus their involvement in one part of FreeBSD.
The four parts are kernel development, userland development, ports and documentation.
When referring to the base system, both kernel and userland is meant.

This split changes our table to look like this:

The FreeBSD Project's structure with active committers in categories

[.informaltable]
[cols="1,1,1", options="header"]

|9

|Committers
|Base
|164

|
|Userland
|50

|
|Docs
|44

|
|Ports
|164

|
|Total
|372

|Contributors
|
|~3000
|===

Number of committers per area has been determined by going through access file of git repos.
Note that many committers work in multiple areas, making the total number lower than numerical total.
The total number of active unique committers on June 2022 was 317.

Committers fall into three groups: committers who are only concerned with one area of the project (for instance file systems), committers who are involved only with one sub-project, and committers who commit to different parts of the code, including sub-projects.
Because some committers work on different parts, the total number in the committers section of the table is higher than in the above table.

The kernel is the main building block of FreeBSD.
While the userland applications are protected against faults in other userland applications, the entire system is vulnerable to errors in the kernel.
This, combined with the vast amount of dependencies in the kernel and that it is not easy to see all the consequences of a kernel change, demands developers with a relative full understanding of the kernel.
Multiple development efforts in the kernel also require a closer coordination than userland applications do.

The core utilities, known as userland, provide the interface that identifies FreeBSD, both user interface, shared libraries and external interfaces to connecting clients.
Currently, 162 people are involved in userland development and maintenance, many being maintainers for their own part of the code.
Maintainership will be discussed in the <<role-maintainer>> section.

Documentation is handled by <<sub-project-documentation>> and includes all documents surrounding the FreeBSD project, including the web pages.
There were during 2004 101 people making commits to the FreeBSD Documentation Project.

Ports is the collection of meta-data that is needed to make software packages build correctly on FreeBSD.
An example of a port is the port for the web-browser Mozilla.
It contains information about where to fetch the source, what patches to apply and how, and how the package should be installed on the system.
This allows automated tools to fetch, build and install the package.
As of this writing, there are more than 12600 ports available.
footnote:[Statistics are generated by counting the number of entries in the file fetched by portsdb by April 1st, 2005. portsdb is a part of the port sysutils/portupgrade.] , ranging from web servers to games, programming languages and most of the application types that are in use on modern computers.
Ports will be discussed further in the section <<sub-project-ports>>.

[[methodology-model]]
== Methodology model

=== Development model

There is no defined model for how people write code in FreeBSD.
However, Niels Jørgenssen has suggested a model of how written code is integrated into the project.

Jørgenssen's model for change integration

[.informaltable]
[cols="1,1,1", options="header"]

|code
|===

The "development release" is the FreeBSD-CURRENT ("-CURRENT") branch and the "production release" is the FreeBSD-STABLE branch ("-STABLE") [<<jorgensen2001, Jørgensen, 2001>>].

This is a model for one change, and shows that after coding, developers seek community review and try integrating it with their own systems.
After integrating the change into the development release, called FreeBSD-CURRENT, it is tested by many users and developers in the FreeBSD community.
After it has gone through enough testing, it is merged into the production release, called FreeBSD-STABLE.
Unless each stage is finished successfully, the developer needs to go back and make modifications in the code and restart the process.
To integrate a change with either -CURRENT or -STABLE is called making a commit.

Jørgensen found that most FreeBSD developers work individually, meaning that this model is used in parallel by many developers on the different ongoing development efforts.
A developer can also be working on multiple changes, so that while they are waiting for review or people to test one or more of their changes, they may be writing another change.

As each commit represents an increment, this is a massively incremental model.
The commits are in fact so frequent that during one year footnote:[The period from January 1st, 2004 to December 31st, 2004 was examined to find this number.] , 85427 commits were made, making a daily average of 233 commits.

Within the "code" bracket in Jørgensen's model, each programmer has their own working style and follows their own development models.
The bracket could very well have been called "development" as it includes requirements gathering and analysis, system and detailed design, implementation and verification.
However, the only output from these stages is the source code or system documentation.

From a stepwise model's perspective (such as the waterfall model), the other brackets can be seen as further verification and system integration.
This system integration is also important to see if a change is accepted by the community.
Up until the code is committed, the developer is free to choose how much to communicate about it to the rest of the project.
In order for -CURRENT to work as a buffer (so that bright ideas that had some undiscovered drawbacks can be backed out) the minimum time a commit should be in -CURRENT before merging it to -STABLE is 3 days.
Such a merge is referred to as an MFC (Merge From Current).

It is important to notice the word "change".
Most commits do not contain radical new features, but are maintenance updates.

The only exceptions from this model are security fixes and changes to features that are deprecated in the -CURRENT branch.
In these cases, changes can be committed directly to the -STABLE branch.

In addition to many people working on the project, there are many related projects to the FreeBSD Project.
These are either projects developing brand new features, sub-projects or projects whose outcome is incorporated into FreeBSD footnote:[For instance, the development of the Bluetooth stack started as a sub-project until it was deemed stable enough to be merged into the -CURRENT branch. Now it is a part of the core FreeBSD system.].
These projects fit into the FreeBSD Project just like regular development efforts: they produce code that is integrated with the FreeBSD Project.
However, some of them (like Ports and Documentation) have the privilege of being applicable to both branches or commit directly to both -CURRENT and -STABLE.

There is no standards to how design should be done, nor is design collected in a centralised repository.
The main design is that of 4.4BSD.
footnote:[According to Kirk McKusick, after 20 years of developing UNIX operating systems, the interfaces are for the most part figured out. There is therefore no need for much design. However, new applications of the system and new hardware leads to some implementations being more beneficial than those that used to be preferred. One example is the introduction of web browsing that made the normal TCP/IP connection a short burst of data rather than a steady stream over a longer period of time.]
As design is a part of the "Code" bracket in Jørgenssen's model, it is up to every developer or sub-project how this should be done.
Even if the design should be stored in a central repository, the output from the design stages would be of limited use as the differences of methodologies would make them poorly if at all interoperable.
For the overall design of the project, the project relies on the sub-projects to negotiate fit interfaces between each other rather than to dictate interfacing.

[[release-branches]]
=== Release branches

The releases of FreeBSD are best illustrated by a tree with many branches where each major branch represents a major version.
Minor versions are represented by branches of the major branches.

In the following release tree, arrows that follow one-another in a particular direction represent a branch.
Boxes with full lines and diamonds represent official releases.
Boxes with dotted lines represent the development branch at that time.
Security branches are represented by ovals.
Diamonds differ from boxes in that they represent a fork, meaning a place where a branch splits into two branches where one of the branches becomes a sub-branch.
For example, at 4.0-RELEASE the 4.0-CURRENT branch split into 4-STABLE and 5.0-CURRENT. At 4.5-RELEASE, the branch forked off a security branch called RELENG_4_5.

.The FreeBSD release tree
image::branches.png[Refer to table below for a screen-reader friendly version.]


|3.0 Current (development branch)
|
|Releng 3 branches: 3.0 Release to 3.5 Release, leading to 3.5.1 Release and the subsequent 3 Stable branch

|4.0 Current (development branch)
|3.1 Release
|Releng 4 branches: 4.1 Release to 4.6 Release (and 4.6.2 Release), then 4.7 Release to 4.11 Release (all starting at 4.3 Release also leading to a Releng_4_n branch), and the subsequent 4 Release branch

|5.0 Current (development branch)
|4.0 Release
|Releng 5 branches: 5.0 Release to 5.4 Release (all except 5.0 and 5.3 also leading to a Releng_5_n branch), and the subsequent 5 Release branch

|6.0 Current (development branch)
|5.3 Release

An example of this is that following 3.0-RELEASE, 3.1-RELEASE was also a continuation of the -CURRENT-branch, and -CURRENT did not become a true development branch until this version was released and the 3-STABLE branch was forked.
When -CURRENT returns to becoming a development branch, it can only be followed by a major release.
5-STABLE is predicted to be forked off 5.0-CURRENT at around 5.3-RELEASE.
It is not until 5-STABLE is forked that the development branch will be branded 6.0-CURRENT.

A "minor release" is made from the -CURRENT branch following a major release, or from the -STABLE branch.

Following and including, 4.3-RELEASEfootnote:[The first release this actually happened for was 4.5-RELEASE, but security branches were at the same time created for 4.3-RELEASE and 4.4-RELEASE.], when a minor release has been made, it becomes a "security branch".
This is meant for organisations that do not want to follow the -STABLE branch and the potential new/changed features it offers, but instead require an absolutely stable environment, only updating to implement security updates. footnote:[There is a terminology overlap with respect to the word "stable", which leads to some confusion. The -STABLE branch is still a development branch, whose goal is to be useful for most people. If it is never acceptable for a system to get changes that are not announced at the time it is deployed, that system should run a security branch.]

Each update to a security branch is called a "patchlevel".
For every security enhancement that is done, the patchlevel number is increased, making it easy for people tracking the branch to see what security enhancements they have implemented.
In cases where there have been especially serious security flaws, an entire new release can be made from a security branch.
An example of this is 4.6.2-RELEASE.

[[model-summary]]
=== Model summary

To summarise, the development model of FreeBSD can be seen as the following tree:

.The overall development model
image::freebsd-code-model.png[Refer to paragraphs below for a screen-reader friendly version.]

The tree of the FreeBSD development with ongoing development efforts and continuous integration.

The tree symbolises the release versions with major versions spawning new main branches and minor versions being versions of the main branch.
The top branch is the -CURRENT branch where all new development is integrated, and the -STABLE branch is the branch directly below it.
Below the -STABLE branch are old, unsupported versions.

Clouds of development efforts hang over the project where developers use the development models they see fit.
The product of their work is then integrated into -CURRENT where it undergoes parallel debugging and is finally merged from -CURRENT into -STABLE.
Security fixes are merged from -STABLE to the security branches.

Many committers have a special area of responsibility.
These roles are called hats.

Many of these hats are not formalised.
Formalised hats have a charter stating the exact purpose of the hat along with its privileges and responsibilities.
The writing of such charters is a new part of the project, and has thus yet to be completed for all hats.
These hat descriptions are not such a formalisation, rather a summary of the role with links to the charter where available and contact addresses.

[[sect-hats]]
== Hats

[[role-committer]]
==== Committer

A person who has the required privileges to add their code or documentation to the repository.
A committer has made a commit within the past 12 months.
[<<freebsd-developer-handbook, FreeBSD, 2000A>>] An active committer is a committer who has made an average of one commit per month during that time.

It is worth noting that there are no technical barriers to prevent someone, once having gained commit privileges to the main- or a sub-project, to make commits in parts of that project's source the committer did not specifically get permission to modify.
However, when wanting to make modifications to parts a committer has not been involved in before, they should read the logs to see what has happened in this area before, and also read the MAINTAINERS file to see if the maintainer of this part has any special requests on how changes in the code should be made.

[[role-core]]
==== Core Team

The core team is elected by the committers from the pool of committers and serves as the board of directors of the FreeBSD project.
It promotes active contributors to committers, assigns people to well-defined hats, and is the final arbiter of decisions involving which way the project should be heading.
As of July 1st, 2004, core consisted of 9 members.
Elections are held every two years.

[[role-maintainer]]
==== Maintainership

Maintainership means that that person is responsible for what is allowed to go into that area of the code and has the final say should disagreements over the code occur.
This involves proactive work aimed at stimulating contributions and reactive work in reviewing commits.

With the FreeBSD source comes the MAINTAINERS file that contains a one-line summary of how each maintainer would like contributions to be made.
Having this notice and contact information enables developers to focus on the development effort rather than being stuck in a slow correspondence should the maintainer be unavailable for some time.

If the maintainer is unavailable for an unreasonably long period of time, and other people do a significant amount of work, maintainership may be switched without the maintainer's approval.
This is based on the stance that maintainership should be demonstrated, not declared.

Maintainership of a particular piece of code is a hat that is not held as a group.

[[official-hats]]
=== Official Hats

The official hats in the FreeBSD Project are hats that are more or less formalised and mainly administrative roles.
They have the authority and responsibility for their area.
The following list shows the responsibility lines and gives a description of each hat, including who it is held by.

[[role-doc-manager]]
==== Documentation project manager

<<sub-project-documentation>> architect is responsible for defining and following up documentation goals for the committers in the Documentation project, which they supervise.

Hat held by: The DocEng team mailto:doceng@FreeBSD.org[doceng@FreeBSD.org].
The https://www.freebsd.org/internal/doceng/[ DocEng Charter].

[[role-postmaster]]
==== Postmaster

The Postmaster is responsible for mail being correctly delivered to the committers' email address.
They are also responsible for ensuring that the mailing lists work and should take measures against possible disruptions of mail such as having troll-, spam- and virus-filters.

Hat currently held by: the Postmaster Team mailto:postmaster@FreeBSD.org[postmaster@FreeBSD.org].

[[role-release-coordination]]
==== Release Coordination

The responsibilities of the Release Engineering Team are

* Setting, publishing and following a release schedule for official releases
* Documenting and formalising release engineering procedures
* Creation and maintenance of code branches
* Coordinating with the Ports and Documentation teams to have an updated set of packages and documentation released with the new releases
* Coordinating with the Security team so that pending releases are not affected by recently disclosed vulnerabilities.

Further information about the development process is available in the <<process-release-engineering>> section.

[[role-releng]]
Hat held by: the Release Engineering team mailto:re@FreeBSD.org[re@FreeBSD.org].
The https://www.freebsd.org/releng/charter/[ Release Engineering Charter].

[[role-pr-cr]]
==== Public Relations & Corporate Liaison

The Public Relations & Corporate Liaison's responsibilities are:

* Making press statements when happenings that are important to the FreeBSD Project happen.
* Being the official contact person for corporations that are working close with the FreeBSD Project.
* Take steps to promote FreeBSD within both the Open Source community and the corporate world.
* Handle the "freebsd-advocacy" mailing list.

This hat is currently not occupied.

[[role-security-officer]]
==== Security Officer

The Security Officer's main responsibility is to coordinate information exchange with others in the security community and in the FreeBSD project.
The Security Officer is also responsible for taking action when security problems are reported and promoting proactive development behavior when it comes to security.

Because of the fear that information about vulnerabilities may leak out to people with malicious intent before a patch is available, only the Security Officer, consisting of an officer, a deputy and two <<role-core>> members, receive sensitive information about security issues.
However, to create or implement a patch, the Security Officer has the Security Officer Team mailto:security-team@FreeBSD.org[security-team@FreeBSD.org] to help do the work.

[[role-repo-manager]]
==== Source Repository Manager

The Source Repository Manager is the only one who is allowed to directly modify the repository without using the <<tool-svn>> tool.
It is their responsibility to ensure that technical problems that arise in the repository are resolved quickly.
The source repository manager has the authority to back out commits if this is necessary to resolve a SVN technical problem.

Hat held by: the Source Repository Manager mailto:clusteradm@FreeBSD.org[clusteradm@FreeBSD.org].

[[role-election-manager]]
==== Election Manager

The manager is responsible for running and maintaining the election system, and is the final authority should minor unforeseen events happen in the election process.
Major unforeseen events have to be discussed with the <<role-core>>

Hat held only during elections.

[[role-webmaster]]
==== Web site Management

The Web site Management hat is responsible for coordinating the rollout of updated web pages on mirrors around the world, for the overall structure of the primary web site and the system it is running upon.
The management needs to coordinate the content with <<sub-project-documentation>> and acts as maintainer for the "www" tree.

Hat held by: the FreeBSD Webmasters mailto:www@FreeBSD.org[www@FreeBSD.org].

[[role-ports-manager]]
==== Ports Manager

The Ports Manager acts as a liaison between <<sub-project-ports>> and the core project, and all requests from the project should go to the ports manager.

Hat held by: the Ports Management Team mailto:portmgr@FreeBSD.org[portmgr@FreeBSD.org].
The https://www.freebsd.org/portmgr/charter/[Portmgr charter].

[[role-standards]]
==== Standards

The Standards hat is responsible for ensuring that FreeBSD complies with the standards it is committed to , keeping up to date on the development of these standards and notifying FreeBSD developers of important changes that allows them to take a proactive role and decrease the time between a standards update and FreeBSD's compliancy.

Hat currently held by: Garrett Wollman mailto:wollman@FreeBSD.org[wollman@FreeBSD.org].

[[role-core-secretary]]
==== Core Secretary

The Core Secretary's main responsibility is to write drafts to and publish the final Core Reports.
The secretary also keeps the core agenda, thus ensuring that no balls are dropped unresolved.

Hat currently held by: {bofh}.

[[role-bugmeister]]
==== Bugmeister

The Bugmeister is responsible for ensuring that the maintenance database is in working order, that the entries are correctly categorised and that there are no invalid entries.
They supervise bugbusters.

Hat currently held by: the Bugmeister Team mailto:bugmeister@FreeBSD.org[bugmeister@FreeBSD.org].

[[role-donations]]
==== Donations Liaison Officer

The task of the donations liaison officer is to match the developers with needs with people or organisations willing to make a donation.

Hat held by: the Donations Liaison Office mailto:donations@FreeBSD.org[donations@FreeBSD.org].
The https://www.freebsd.org/donations/[ Donations Liaison Charter].

[[role-admin]]
==== Admin

(Also called "FreeBSD Cluster Admin")

The admin team consists of the people responsible for administrating the computers that the project relies on for its distributed work and communication to be synchronised.
It consists mainly of those people who have physical access to the servers.

Hat held by: the Admin team mailto:admin@FreeBSD.org[admin@FreeBSD.org].

[[proc-depend-hats]]
=== Process dependent hats

[[role-problem-originator]]
==== Report originator

The person originally responsible for filing a Problem Report.

[[role-bugbuster]]
==== Bugbuster

A person who will either find the right person to solve the problem, or close the PR if it is a duplicate or otherwise not an interesting one.

[[role-mentor]]
==== Mentor

A mentor is a committer who takes it upon them to introduce a new committer to the project, both in terms of ensuring the new committer's setup is valid, that the new committer knows the available tools required in their work, and that the new committer knows what is expected of them in terms of behavior.

[[role-vendor]]
==== Vendor

The person(s) or organisation whom external code comes from and whom patches are sent to.

[[role-reviewer]]
==== Reviewers

People on the mailing list where the request for review is posted.

The following section will describe the defined project processes.
Issues that are not handled by these processes happen on an ad-hoc basis based on what has been customary to do in similar cases.


The Core team has the responsibility of giving and removing commit privileges to contributors.
This can only be done through a vote on the core mailing list.
The ports and documentation sub-projects can give commit privileges to people working on these projects, but have to date not removed such privileges.

Normally a contributor is recommended to core by a committer.
For contributors or outsiders to contact core asking to be a committer is not well thought of and is usually rejected.

If the area of particular interest for the developer potentially overlaps with other committers' area of maintainership, the opinion of those maintainers is sought.
However, it is frequently this committer that recommends the developer.

When a contributor is given committer status, they are assigned a mentor.
The committer who recommended the new committer will, in the general case, take it upon themselves to be the new committers mentor.

When a contributor is given their commit bit, a <<tool-pgp>>-signed email is sent from either <<role-core-secretary>>, <<role-ports-manager>>, or nik@freebsd.org to both admins@freebsd.org, the assigned mentor, the new committer, and core confirming the approval of a new account.
The mentor then gathers a password line, <<tool-ssh2>> public key, and PGP key from the new committer and sends them to <<role-admin>>.
When the new account is created, the mentor activates the commit bit and guides the new committer through the rest of the initial process.

.Process summary: adding a new committer
image::proc-add-committer.png[Refer to paragraph below for a screen-reader friendly version.]

If they recommend this to core, core will vote on this recommendation.
If the vote is in favour, a mentor is assigned the new committer and the new committer has to email their details to the administrators for an account to be created.
After this, the new committer is all set to make their first commit.
By tradition, this is by adding their name to the committers list.

Recall that a committer is considered to be someone who has committed code during the past 12 months.
However, it is not until after 18 months of inactivity have passed that commit privileges are eligible to be revoked.
[<<freebsd-expiration-policy, FreeBSD, 2002H>>]
There are, however, no automatic procedures for doing this.
For reactions concerning commit privileges not triggered by time, see <<process-reactions,section 1.5.8>>.

.Process summary: removing a committer
image::proc-rm-committer.png[Refer to paragraph below for a screen-reader friendly version.]

When Core decides to clean up the committers list, they check who has not made a commit for the past 18 months.
Committers who have not done so have their commit bits revoked and their account removed by the administrators.

It is also possible for committers to request that their commit bit be retired if for some reason they are no longer going to be actively committing to the project.
In this case, it can also be restored at a later time by core, should the committer ask.

Roles in this process:

. <<role-core>>
. <<role-contributor>>

[[committing]]
=== Committing code

The committing of new or modified code is one of the most frequent processes in the FreeBSD project and will usually happen many times a day.
Committing of code can only be done by a "committer".
Committers commit either code written by themselves, code submitted to them, or code submitted through a <<model-pr,problem report>>.

When code is written by the developer that is non-trivial, they should seek a code review from the community.
This is done by sending mail to the relevant list asking for review.
Before submitting the code for review, they should ensure it compiles correctly with the entire tree and that all relevant tests run.
This is called "pre-commit test".
When contributed code is received, it should be reviewed by the committer and tested the same way.

When a change is committed to a part of the source that has been contributed from an outside <<role-vendor>>, the maintainer should ensure that the patch is contributed back to the vendor.
This is in line with the open source philosophy and makes it easier to stay in sync with outside projects as the patches do not have to be reapplied every time a new release is made.

After the code has been available for review and no further changes are necessary, the code is committed into the development branch, -CURRENT.
If the change applies for the -STABLE branch or the other branches as well, a "Merge From Current" ("MFC") countdown is set by the committer.
After the number of days the committer chose when setting the MFC have passed, an email will automatically be sent to the committer reminding them to commit it to the -STABLE branch (and possibly security branches as well).
Only security critical changes should be merged to security branches.

Delaying the commit to -STABLE and other branches allows for "parallel debugging" where the committed code is tested on a wide range of configurations.
This makes changes to -STABLE to contain fewer faults and thus giving the branch its name.

.Process summary: A committer commits code
image::proc-commit.png[Refer to paragraph below for a screen-reader friendly version.]

If the code is contributed by an outside vendor, the maintainer should create a patch that is sent back to the vendor.
The code is then committed and then deployed by the users.
Should they find problems with the code, this will be reported and the committer can go back to writing a patch.
If a vendor is affected, they can choose to implement or ignore the patch.

.Process summary: A contributor commits code
image::proc-contrib.png[Refer to paragraphs below and above for a screen-reader friendly version.]

The difference when a contributor makes a code contribution is that they submit the code through the Bugzilla interface.
This report is picked up by the maintainer who reviews the code and commits it.

Hats included in this process are:

. <<role-committer>>
. <<role-contributor>>
. <<role-vendor>>
. <<role-reviewer>>

[<<freebsd-committer, FreeBSD, 2001>>] [<<jorgensen2001, Jørgensen, 2001>>]

[[process-core-election]]
=== Core election

Core elections are held at least every two years. footnote:[The first Core election was held September 2000] Nine core members are elected.
New elections are held if the number of core members drops below seven.
New elections can also be held should at least 1/3 of the active committers demand this.

When an election is to take place, core announces this at least 6 weeks in advance, and appoints an election manager to run the elections.

Only committers can be elected into core.
The candidates need to submit their candidacy at least one week before the election starts, but can refine their statements until the voting starts.
They are presented in the http://election.uk.freebsd.org/candidates.html[candidates list].
When writing their election statements, the candidates must answer a few standard questions submitted by the election manager.

During elections, the rule that a committer must have committed during the 12 past months is followed strictly.
Only these committers are eligible to vote.

When voting, the committer may vote once in support of up to nine nominees.
The voting is done over a period of four weeks with reminders being posted on "developers" mailing list that is available to all committers.

The election results are released one week after the election ends, and the new core team takes office one week after the results have been posted.

Should there be a voting tie, this will be resolved by the new, unambiguously elected core members.

Votes and candidate statements are archived, but the archives are not publicly available.

.Process summary: Core elections
image::proc-elections.png[Refer to paragraph below for a screen-reader friendly version.]

Core announces the election and selects an election manager who prepares the elections, and when ready, candidates can announce their candidacies through submitting their statements.
The committers then vote.
After the vote is over, the election results are announced and the new core team takes office.

Hats in core elections are:

* <<role-core>>
* <<role-committer>>
* <<role-election-manager>>

[<<freebsd-bylaws, FreeBSD, 2000A>>] [<<bsd-election2002, FreeBSD, 2002B>>] [<<freebsd-election, FreeBSD, 2002G>>]

[[new-features]]
=== Development of new features

These projects are generally done by one person [<<jorgensen2001, Jørgensen, 2001>>].
Every project is free to organise development as it sees fit.
However, when the project is merged to the -CURRENT branch it must follow the project guidelines.
When the code has been well tested in the -CURRENT branch and deemed stable enough and relevant to the -STABLE branch, it is merged to the -STABLE branch.

The requirements of the project are given by developer wishes, requests from the community in terms of direct requests by mail, Problem Reports, commercial funding for the development of features, or contributions by the scientific community.
The wishes that come within the responsibility of a developer are given to that developer who prioritises their time between the request and their wishes.
A common way to do this is maintain a TODO-list maintained by the project.
Items that do not come within someone's responsibility are collected on TODO-lists unless someone volunteers to take the responsibility.
All requests, their distribution and follow-up are handled by the <<tool-bugzilla>> tool.

Requirements analysis happens in two ways.
The requests that come in are discussed on mailing lists, both within the main project and in the sub-project that the request belongs to or is spawned by the request.
Furthermore, individual developers on the sub-project will evaluate the feasibility of the requests and determine the prioritisation between them.
Other than archives of the discussions that have taken place, no outcome is created by this phase that is merged into the main project.

As the requests are prioritised by the individual developers on the basis of doing what they find interesting, necessary, or are funded to do, there is no overall strategy or prioritisation of what requests to regard as requirements and following up their correct implementation.
However, most developers have some shared vision of what issues are more important, and they can ask for guidelines from the release engineering team.

The verification phase of the project is two-fold.
Before committing code to the current-branch, developers request their code to be reviewed by their peers.

Some parts of the code have designated maintainers. Others have de-facto maintainers, and some parts of the system do not have maintainers.
The maintainer is usually a person from the sub-project that wrote and integrated the code, or someone who has ported it from the platform it was written for.
footnote:[sendmail and named are examples of code that has been merged from other platforms.]
The maintainer's job is to make sure the code is in sync with the project the code comes from if it is contributed code, and apply patches submitted by the community or write fixes to issues that are discovered.

The main bulk of work that is put into the FreeBSD project is maintenance.
[<<jorgensen2001, Jørgensen, 2001>>] has made a figure showing the life cycle of changes.

Jørgenssen's model for change integration

[.informaltable]
[cols="1,1,1", options="header"]

|===

Here "development release" refers to the -CURRENT branch while "production release" refers to the -STABLE branch.
The "pre-commit test" is the functional testing by peer developers when asked to do so or trying out the code to determine the status of the sub-project.
"Parallel debugging" is the functional testing that can trigger more review, and debugging when the code is included in the -CURRENT branch.

As of this writing, there were 269 committers in the project.
When they commit a change to a branch, that constitutes a new release.
It is very common for users in the community to track a particular branch.
The immediate existence of a new release makes the changes widely available right away and allows for rapid feedback from the community.
This also gives the community the response time they expect on issues that are of importance to them.
This makes the community more engaged, and thus allows for more and better feedback that again spurs more maintenance and ultimately should create a better product.

Before making changes to code in parts of the tree that has a history unknown to the committer, the committer is required to read the commit logs to see why certain features are implemented the way they are in order not to make mistakes that have previously either been thought through or resolved.

[[model-pr]]
=== Problem reporting

Before FreeBSD 10, FreeBSD included a problem reporting tool called `send-pr`.
Problems include bug reports, feature requests, feature enhancements and notices of new versions of external software that are included in the project.
Although `send-pr` is available, users and developers are encouraged to submit issues using our https://bugs.freebsd.org/submit/[ problem report form].

Problem reports are sent to an email address where it is inserted into the Problem Reports maintenance database.
A <<role-bugbuster>> classifies the problem and sends it to the correct group or maintainer within the project.

Finally, the working patch is integrated into the project, and documented if applicable.
It there goes through the regular maintenance cycle as described in section <<model-maintenance>>.
These are the states a problem report can be in: open, analyzed, feedback, patched, suspended and closed.
The suspended state is for when further progress is not possible due to the lack of information or for when the task would require so much work that nobody is working on it at the moment.

.Process summary: problem reporting
image::proc-pr.png[Refer to paragraph below for a screen-reader friendly version.]

A problem is reported by the report originator.
It is then classified by a bugbuster and handed to the correct maintainer.
They verify the problem and discuss the problem with the originator until they have enough information to create a working patch.
This patch is then committed and the problem report is closed.

The roles included in this process are:

. <<role-problem-originator>>
. <<role-maintainer>>
. <<role-bugbuster>>

[<<freebsd-handle-pr, FreeBSD, 2002C>>]. [<<freebsd-send-pr, FreeBSD, 2002D>>]

[[process-reactions]]
=== Reacting to misbehavior

[<<freebsd-committer, FreeBSD, 2001>>] has a number of rules that committers should follow.
However, it happens that these rules are broken.
The following rules exist in order to be able to react to misbehavior.
They specify what actions will result in how long a suspension of the committer's commit privileges.

* Committing during code freezes without the approval of the Release Engineering team - 2 days
* Committing to a security branch without approval - 2 days
* Commit wars - 5 days to all participating parties
* Impolite or inappropriate behavior - 5 days

[<<ref-freebsd-trenches, Lehey, 2002>>]

For the suspensions to be efficient, any single core member can implement a suspension before discussing it on the "core" mailing list.
Repeat offenders can, with a 2/3 vote by core, receive harsher penalties, including permanent removal of commit privileges.
(However, the latter is always viewed as a last resort, due to its inherent tendency to create controversy.)
All suspensions are posted to the "developers" mailing list, a list available to committers only.

It is important that you cannot be suspended for making technical errors.
All penalties come from breaking social etiquette.

Hats involved in this process:

* <<role-core>>
* <<role-committer>>

The FreeBSD project has a Release Engineering  team with a principal release engineer that is responsible for creating releases of FreeBSD that can be brought out to the user community via the net or sold in retail outlets.
Since FreeBSD is available on multiple platforms and releases for the different architectures are made available at the same time, the team has one person in charge of each architecture.
Also, there are roles in the team responsible for coordinating quality assurance  efforts, building a package set and for having an updated set of documents.
When referring to the release engineer, a representative for the release engineering team is meant.

When a release is coming, the FreeBSD project changes shape somewhat.
A release schedule is made containing feature- and code-freezes, release of interim releases and the final release.
A feature-freeze means no new features are allowed to be committed to the branch without the release engineers' explicit consent.
Code-freeze means no changes to the code (like bugs-fixes) are allowed to be committed without the release engineers' explicit consent.
This feature- and code-freeze is known as stabilising.
During the release process, the release engineer has the full authority to revert to older versions of code and thus "back out" changes should they find that the changes are not suitable to be included in the release.

There are three different kinds of releases:

. .0 releases are the first release of a major version. These are branched of the -CURRENT branch and have a significantly longer release engineering cycle due to the unstable nature of the -CURRENT branch
. .X releases are releases of the -STABLE branch. They are scheduled to come out every 4 months.
. .X.Y releases are security releases that follow the .X branch. These come out only when sufficient security fixes have been merged since the last release on that branch. New features are rarely included, and the security team is far more involved in these than in regular releases.

For releases of the -STABLE-branch, the release process starts 45 days before the anticipated release date.
During the first phase, the first 15 days, the developers merge what changes they have had in -CURRENT that they want to have in the release to the release branch.

Updates are less likely to be allowed during this period, except for important bug fixes and security updates.
In this final period, all releases are considered release candidates.
At the end of the release process, a release is created with the new version number, including binary distributions on web sites and the creation of CD-ROM images.
However, the release is not considered "really released" until a <<tool-pgp>>-signed message stating exactly that, is sent to the mailing list freebsd-announce; anything labelled as a "release" before that may well be in-process and subject to change before the PGP-signed message is sent. footnote:[Many commercial vendors use these images to create CD-ROMs that are sold in retail outlets.].

The releases of the -CURRENT-branch (that is, all releases that end with ".0") are very similar, but with twice as long timeframe.
It starts 8 weeks prior to the release with announcement of the release time line.
Two weeks into the release process, the feature freeze is initiated and performance tweaks should be kept to a minimum.
Four weeks prior to the release, an official beta version is made available.
Two weeks prior to release, the code is officially branched into a new version.
This version is given release candidate status, and as with the release engineering of -STABLE, the code freeze of the release candidate is hardened.
However, development on the main development branch can continue.
Other than these differences, the release engineering processes are alike.

*.0 releases go into their own branch and are aimed mainly at early adopters. The branch then goes through a period of stabilisation, and it is not until the <<role-releng, Release Engineering Team>> decides the demands to stability have been satisfied that the branch becomes -STABLE and -CURRENT targets the next major version. While this for the majority has been with *.1 versions, this is not a demand.

Most releases are made when a given date that has been deemed a long enough time since the previous release comes.
A target is set for having major releases every 18 months and minor releases every 4 months.
The user community has made it very clear that security and stability cannot be sacrificed by self-imposed deadlines and target release dates.
For slips of time not to become too long with regards to security and stability issues, extra discipline is required when committing changes to -STABLE.

. Make release schedule
. Feature freeze
. Code freeze
. Make branch
. Release candidate
. Stabilize release (loop back to previous step as many times as necessary; when release is considered stable, proceed with next step)
. Build packages
. Warn mirrors
. Publish release

[<<freebsd-releng, FreeBSD, 2002E>>]

[[tools]]
== Tools

The major support tools for supporting the development process are Bugzilla, Mailman, and OpenSSH.
These are externally developed tools and are commonly used in the open source world.

[[tool-svn]]
=== Subversion (SVN)

Subversion ("SVN") is a system to handle multiple versions of text files and tracking who committed what changes and why.
A project lives within a "repository" and different versions are considered different "branches".

[[tool-bugzilla]]
=== Bugzilla

Bugzilla is a maintenance database consisting of a set of tools to track bugs at a central site.
It supports the bug tracking process for sending and handling bugs as well as querying and updating the database and editing bug reports.
The project uses its web interface to send "Problem Reports" to the project's central Bugzilla server.
The committers also have web and command-line clients.

[[model-mailman]]
=== Mailman

The FreeBSD Project uses it to run 16 general lists, 60 technical lists, 4 limited lists and 5 lists with SVN commit logs.
It is also used for many mailing lists set up and used by other people and projects in the FreeBSD community.
General lists are lists for the general public, technical lists are mainly for the development of specific areas of interest, and closed lists are for internal communication not intended for the general public.
The majority of all the communication in the project goes through these 85 lists  [<<ref-bsd-handbook, FreeBSD, 2003A>>, Appendix C].

[[tool-pgp]]
=== Pretty Good Privacy

Pretty Good Privacy, better known as PGP, is a cryptosystem using a public key architecture to allow people to digitally sign and/or encrypt information in order to ensure secure communication between two parties.
A signature is used when sending information out to many recipients, enabling them to verify that the information has not been tampered with before they received it.
In the FreeBSD Project this is the primary means of ensuring that information has been written by the person who claims to have written it, and not altered in transit.

[[tool-ssh2]]
=== Secure Shell

It allows other connections, called tunnels, to be established and protected between the two involved systems.
This standard exists in two primary versions, and only version two is used for the FreeBSD Project.
The most common implementation of the standard is OpenSSH that is a part of the project's main distribution.
Since its source is updated more often than FreeBSD releases, the latest version is also available in the ports tree.

[[sub-projects]]
== Sub-projects

=== The Ports Subproject

A "port" is a set of meta-data and patches that are needed to fetch, compile and install correctly an external piece of software on a FreeBSD system.
The amount of ports has grown at a tremendous rate, as shown by the following figure.

.Number of ports added between 1996 and 2008 [[fig-ports]]
image::portsstatus.png[Refer to tables below for a screen-reader friendly version.]

<<fig-ports>> shows the number of ports available to FreeBSD in the period 1995 to 2008.
It looks like the curve has first grown exponentially, and then from the middle of 2001 to the middle of 2007 grown linearly at a rate of about 2000 ports/year, before its growth rate gets lower.

Approximate dates each multiple of 1000 ports is reached

[.informaltable]
[cols="1,1", options="header"]

|2nd quarter 2007
|===

Approximate number of ports at the start of each year

[.informaltable]
[cols="1,1", options="header"]

|===

As the external software described by the port often is under continued development, the amount of work required to maintain the ports is already large, and increasing.
This has led to the ports part of the FreeBSD project gaining a more empowered structure, and is more and more becoming a sub-project of the FreeBSD project.

Ports has its own core team with the <<role-ports-manager>> as its leader, and this team can appoint committers without FreeBSD Core's approval.
Unlike in the FreeBSD Project, where a lot of maintenance frequently is rewarded with a commit bit, the ports sub-project contains many active maintainers that are not committers.

Unlike the main project, the ports tree is not branched.
Every release of FreeBSD follows the current ports collection and has thus available updated information on where to find programs and how to build them.
This, however, means that a port that makes dependencies on the system may need to have variations depending on what version of FreeBSD it runs on.

With an unbranched ports repository it is not possible to guarantee that any port will run on anything other than -CURRENT and -STABLE, in particular older, minor releases.
There is neither the infrastructure nor volunteer time needed to guarantee this.

For efficiency of communication, teams depending on Ports, such as the release engineering team, have their own ports liaisons.

[[sub-project-documentation]]
=== The FreeBSD Documentation Project

The FreeBSD Documentation project was started January 1995.
From the initial group of a project leader, four team leaders and 16 members, they are now a total of 44 committers.
The documentation mailing list has just under 300 members, indicating that there is quite a large community around it.

The goal of the Documentation project is to provide good and useful documentation of the FreeBSD project, thus making it easier for new users to get familiar with the system and detailing advanced features for the users.

The main tasks in the Documentation project are to work on current projects in the "FreeBSD Documentation Set", and translate the documentation to other languages.

Like the FreeBSD Project, documentation is split in the same branches.
This is done so that there is always an updated version of the documentation for each version.
Only documentation errors are corrected in the security branches.

Like the ports sub-project, the Documentation project can appoint documentation committers without FreeBSD Core's approval. [<<freebsd-doceng-charter, FreeBSD, 2003B>>].

The Documentation project has extref:{fdp-primer}[a primer].
This is used both to introduce new project members to the standard tools and syntaxes and to act as a reference when working on the project.

:sectnums!:


Return to bug 264771