Page MenuHomeFreeBSD
Differential D29252 Diff 85746 documentation/content/en/books/handbook/mirrors/_index.adoc

Changeset View

Standalone View

View Options

documentation/content/en/books/handbook/mirrors/_index.adoc

Show First 20 Lines • Show All 380 Lines • ▼ Show 20 Lines
* {mirrors-us-ftp6} (ftp) * {mirrors-us-ftp6} (ftp)
* {mirrors-us-ftp8} (ftp) * {mirrors-us-ftp8} (ftp)
* {mirrors-us-ftp10} (ftp) * {mirrors-us-ftp10} (ftp)
* {mirrors-us-ftp11} (ftp) * {mirrors-us-ftp11} (ftp)
* {mirrors-us-ftp13} (ftp / {mirrors-us-ftp13-http} / rsync) * {mirrors-us-ftp13} (ftp / {mirrors-us-ftp13-http} / rsync)
* {mirrors-us-ftp14} (ftp / {mirrors-us-ftp14-http}) * {mirrors-us-ftp14} (ftp / {mirrors-us-ftp14-http})
* {mirrors-us-ftp15} (ftp) * {mirrors-us-ftp15} (ftp)
[[git]]
== Using git
[[git-intro]]
=== Introduction
As of December 2020, FreeBSD uses git as the primary version control system for storing all of FreeBSD's base source code and documentation.
emasteUnsubmitted
Done
Inline Actions

Maybe "base system source code"?

emaste: Maybe "base system source code"?
[NOTE]
====
Git is generally a developer tool. Users may prefer to use `freebsd-update` (crossref:cutting-edge[updating-upgrading-freebsdupdate,“FreeBSD Update”]) to update the FreeBSD base system, and `portsnap` (crossref:ports[ports-using,“Using the Ports Collection”]) to update the FreeBSD Ports Collection.
vigole.com_outlook.comUnsubmitted
Done
Inline Actions

Maybe: portsnap --> net/gitup

vigole.com_outlook.com: Maybe: portsnap --> net/gitup
debdrupUnsubmitted
Done
Inline Actions

Ports hasn't yet transistioned to git, so this is a bit premature.

debdrup: Ports hasn't yet transistioned to git, so this is a bit premature.
impAuthorUnsubmitted
Done
Inline Actions

This was a conscious choice. portsnap is still the right thing to use.

imp: This was a conscious choice. portsnap is still the right thing to use.
====
This section demonstrates how to install Git on a FreeBSD system and use it to create a local copy of a FreeBSD repository.
Additional information on the use of Git is included.
[[git-ssl-certificates]]
=== Root SSL Certificates
Older FreeBSD systems do not have proper root certificates.
emasteUnsubmitted
Done
Inline Actions

Subversion?

emaste: Subversion?
pauamma_gundo.comUnsubmitted
Done
Inline Actions
=== Root SSL Certificates
- Installing package:security/ca_root_nss[] allows Subversion to verify the identity of HTTPS repository servers. The root SSL certificates can be installed from a port:
+ Installing package:security/ca_root_nss[] allows Git to verify the identity of HTTPS repository servers. The root SSL certificates can be installed from a port:
[source,bash]
pauamma_gundo.com:
impAuthorUnsubmitted
Done
Inline Actions

Also, this is only for older systems since Kyle committed the root CAs to base.

imp: Also, this is only for older systems since Kyle committed the root CAs to base.
Installing package:security/ca_root_nss[] on these systems allows Git to verify the identity of HTTPS repository servers.
The root SSL certificates can be installed from a port:
[source,shell]
....
# cd /usr/ports/security/ca_root_nss
# make install clean
....
or as a package:
[source,shell]
....
# pkg install ca_root_nss
....
[[git-install]]
=== Installation
Git can be installed from the Ports Collection:
[source,shell]
....
# cd /usr/ports/devel/git
# make install clean
....
Git can also be installed as a package:
[source,shell]
....
# pkg install subversion
pauamma_gundo.comUnsubmitted
Done
Inline Actions
....
- # pkg install subversion
+ # pkg install git
....
pauamma_gundo.com:
....
[[git-usage]]
=== Running Git
emasteUnsubmitted
Done
Inline Actions

do we want to expand git here with example URL / directory?

emaste: do we want to expand `git` here with example URL / directory?
pauamma_gundo.comUnsubmitted
Done
Inline Actions

Isn't "local working copy" SVN parlance? I thought with Git it was called a clone instead.

pauamma_gundo.com: Isn't "local working copy" SVN parlance? I thought with Git it was called a clone instead.
impAuthorUnsubmitted
Done
Inline Actions

working tree. And no example. not here, we do just below.

imp: working tree. And no example. not here, we do just below.
To fetch a clean copy of the sources into a local directory, use `git`.
This directory of files is called the _working tree_.
[WARNING]
====
Move or delete an existing destination directory before using `git clone` for the first time.
Cloning over an existing non-`git` directory will fail.
====
Git uses URLs to designate a repository, taking the form of _protocol://hostname/path_.
The first component of the path is the FreeBSD repository to access.
There are two different repositories, `src` for the FreeBSD systerm source code, and `doc` for documentation.
In the future a third repository, 'ports' will be available for the FreeBSD ports code.
For example, the URL `https://git.FreeBSD.org/src.git` specifies the main branch of the src repository, using the `https` protocol.
.Handy URL Table
[options="header,foooter"]
|=======================================================
|Item | Git URL
| Web-based repository browser src | `https://cgit.freebsd.org/src`
| Read-only src repo via HTTPS | `https://git.freebsd.org/src.git`
emasteUnsubmitted
Done
Inline Actions

Can we put an entry across both columns for the headings ("src", "ports"), or maybe make this something like Web-based repository browser - doc repository

emaste: Can we put an entry across both columns for the headings ("src", "ports"), or maybe make this…
impAuthorUnsubmitted
Done
Inline Actions

Too much of a hassle for this round. Spending a ton of time on this (which it needs to look good, imho), will make the rest of these changes later. I'll do that in a followup to get the rest of the things in .

imp: Too much of a hassle for this round. Spending a ton of time on this (which it needs to look…
| Read-only src repo via anon-ssh | `ssh://anongit@git.freebsd.org/src.git`
| Read/write src repo for committers | `ssh://git@gitrepo.freebsd.org/src.git` (*)
| Web-based repository browser doc | `https://cgit.freebsd.org/doc`
| Read-only doc repo via HTTPS | `https://git.freebsd.org/doc.git`
| Read-only doc repo via anon-ssh | `ssh://anongit@git.freebsd.org/doc.git`
| Read/write doc repo for committers | `ssh://git@gitrepo.freebsd.org/doc.git` (*)
|=======================================================
- (*) `git` is a special user on the repository server which will map your registered ssh key in FreeBSD.org to your identity, no need to change it.
[WARNING]
emasteUnsubmitted
Done
Inline Actions

Can we give advice on what we mean by "the future" - e.g. after svn is retired, ...?

emaste: Can we give advice on what we mean by "the future" - e.g. after svn is retired, ...?
impAuthorUnsubmitted
Done
Inline Actions

I'm unsure when lwshu plans to retire things, so I can't be super specific...
I'll change it to 'sometime after the switch to git is complete' which is better on many levels.

imp: I'm unsure when lwshu plans to retire things, so I can't be super specific... I'll change it to…
====
Sometime after the switch to git is complete, `gitrepo.freebsd.org` will change to simply `repo.freebsd.org`.
====
pauamma_gundo.comUnsubmitted
Done
Inline Actions

Again, SVN vocabulary

pauamma_gundo.com: Again, SVN vocabulary
To get started, clone a copy of the FreeBSD repository:
emasteUnsubmitted
Done
Inline Actions

Do we want to mark up [-b branch] indicating it is optional?

emaste: Do we want to mark up `[-b branch]` indicating it is optional?
pauamma_gundo.comUnsubmitted
Done
Inline Actions

Isn't there more to it than that? To get the r* numbers imported from SVN? (I'm on the fence with this. On the con side, it may be unneeded complication, on the pro side, sometimes non-contributors need to refer to r* numbers.)

pauamma_gundo.com: Isn't there more to it than that? To get the r* numbers imported from SVN? (I'm on the fence…
debdrupUnsubmitted
Done
Inline Actions

There's a section below which covers daily usage, where the notes (which store the svn revision IDs) are fetched.

I think it would be better to have a single section which describes the flow with explanations, rather than two sections where one has explanations and another has the flow, as this can lead to confusion.

debdrup: There's a section below which covers daily usage, where the notes (which store the svn revision…
impAuthorUnsubmitted
Done
Inline Actions

No. There's nothing more to it. That's all you need.

There's a different chapter that goes into all the details. This is just the basics.

imp: No. There's nothing more to it. That's all you need. There's a different chapter that goes…
[source,shell]
....
# git clone [ -b branch ] https://git.FreeBSD.org/repo.git wcdir
....
where:
* _repo_ is one of the Project repositories: `src`, `ports`, or `doc`.
* _branch_ depends on the repository used.
`ports` and `doc` are mostly updated in the `main` branch, while `src` maintains the latest version of -CURRENT under `main` and the respective latest versions of the -STABLE branches under `stable/12` (12._x_) and `stable/13` (13._x_).
* _wcdir_ is the target directory where the contents of the specified branch should be placed.
This is usually [.filename]#/usr/ports# for `ports`, [.filename]#/usr/src# for `src`, and [.filename]#/usr/doc# for `doc`.
emasteUnsubmitted
Done
Inline Actions

svn.FreeBSD.org?

emaste: svn.FreeBSD.org?
This example checks out the `main` branch of the system sources from the FreeBSD repository using the HTTPS protocol, placing the local working copy in [.filename]#/usr/src#.
If [.filename]#/usr/src# is already present but was not created by `git`, remember to rename or delete it before the checkout.
Git will refuse to do anything otherwise.
[source,shell]
....
# git clone https://git.FreeBSD.org/git.src /usr/src
pauamma_gundo.comUnsubmitted
Done
Inline Actions
....
- # git clone https://git.FreeBSD.org/git.src /usr/src
+ # git clone https://git.FreeBSD.org/src.git /usr/src
....
pauamma_gundo.com:
....
emasteUnsubmitted
Done
Inline Actions

This looks like literal lwdir, do we want to make it more clear that this is a placeholder

emaste: This looks like literal lwdir, do we want to make it more clear that this is a placeholder
impAuthorUnsubmitted
Done
Inline Actions

Yea, unsure how to do that in asciidoc.
Will do something if someone can suggest how to do things inside a literal.

imp: Yea, unsure how to do that in asciidoc. Will do something if someone can suggest how to do…
Because the initial checkout must download the full branch of the remote repository, it can take a while.
Please be patient.
After the initial checkout, the local working copy can be updated by running:
[source,shell]
....
# cd wcdir
# git pull --rebase
....
To update [.filename]#/usr/src# created in the example above, use:
[source,shell]
pauamma_gundo.comUnsubmitted
Done
Inline Actions
The update is much quicker than a checkout, only transferring files that have changed.
- There are also external mirrors maintained by project members available, please refer to "External mirrors" section.
+ There are also external mirrors maintained by project members available, please refer to the "External mirrors" section.
=== SSH related information
pauamma_gundo.com:
impAuthorUnsubmitted
Done
Inline Actions

This likely should be a link.

imp: This likely should be a link.
....
# cd /usr/src
# git pull --rebase
....
The update is much quicker than a checkout, only transferring files that have changed.
There are also external mirrors maintained by project members available, please refer to the "External mirrors" section.
=== SSH related information
- `ssh://${user}@${url}/${repo}.git` can be written as `${user}@${url}:${repo}.git`, i.e., following two URLs are both valid for passing to git:
- `ssh://anongit@git.freebsd.org/${repo}.git`
- `anongit@git.freebsd.org:${repo}.git`
As well as the read-write repo:
- `ssh://git@(git)repo.freebsd.org/${repo}.git`
- `git@(git)repo.freebsd.org:${repo}.git`
- gitrepo.FreeBSD.org host key fingerprints:
- ECDSA key fingerprint is `SHA256:seWO5D27ySURcx4bknTNKlC1mgai0whP443PAKEvvZA`
- ED25519 key fingerprint is `SHA256:lNR6i4BEOaaUhmDHBA1WJsO7H3KtvjE2r5q4sOxtIWo`
- RSA key fingerprint is `SHA256:f453CUEFXEJAXlKeEHV+ajJfeEfx9MdKQUD7lIscnQI`
- git.FreeBSD.org host key fingerprints:
- ECDSA key fingerprint is `SHA256:/UlirUAsGiitupxmtsn7f9b7zCWd0vCs4Yo/tpVWP9w`
- ED25519 key fingerprint is `SHA256:y1ljKrKMD3lDObRUG3xJ9gXwEIuqnh306tSyFd1tuZE`
- RSA key fingerprint is `SHA256:jBe6FQGoH4HjvrIVM23dcnLZk9kmpdezR/CvQzm7rJM`
These are also published as SSHFP records in DNS.
emasteUnsubmitted
Done
Inline Actions

mirrors

emaste: mirrors
pauamma_gundo.comUnsubmitted
Done
Inline Actions
=== For Users
- Using `git clone` and `git pull` from the official distributed mirros is recommended. The GeoDNS should direct you to the nearest mirror to you.
+ Using `git clone` and `git pull` from the official distributed mirrors is recommended. The GeoDNS should direct you to the nearest mirror to you.
=== For Developers
pauamma_gundo.com:
=== Web-based repository browser
The FreeBSD project currently uses cgit as the web-based repository browser: https://cgit.freebsd.org/
The URL of the indivirual repository is at: https://cgit.freebsd.org/${repo}/
pauamma_gundo.comUnsubmitted
Done
Inline Actions
=== For Developers
- This section describes the read-write access for committers to push the commits from develoeprs or contributors. For read-only access, please refer to the users section above.
+ This section describes the read-write access for committers to push the commits from developers or contributors. For read-only access, please refer to the users section above.
We only document the important URLs here, the full information of retrieving all data in the repository is available at:
pauamma_gundo.com:
=== For Users
Using `git clone` and `git pull` from the official distributed mirrors is recommended.
The GeoDNS should direct you to the nearest mirror to you.
=== For Developers
This section describes the read-write access for committers to push the commits from developers or contributors.
For read-only access, please refer to the users section above.
==== Daily use
* Clone the repository:
```
git clone -o freebsd --config remote.freebsd.fetch='+refs/notes/*:refs/notes/*' https://git.freebsd.org/${repo}.git
```
Then you should have the official mirrors as your remote:
```
$ git remote -v
freebsd https://git.freebsd.org/${repo}.git (fetch)
freebsd https://git.freebsd.org/${repo}.git (push)
```
* Config the FreeBSD committer data:
pauamma_gundo.comUnsubmitted
Done
Inline Actions
freebsd https://git.freebsd.org/${repo}.git (push)
```
- * Config the FreeBSD committer data:
+ * Configure the FreeBSD committer data:
The commit hook in repo.freebsd.org checks the "Commit" field matches the
pauamma_gundo.com:
The commit hook in repo.freebsd.org checks the "Commit" field matches the
committer's information in FreeBSD.org. The easiest way to get the suggested
pauamma_gundo.comUnsubmitted
Done
Inline Actions
git config user.name (your name in gecos)
- git config user.email (your login)FreeBSD.org
+ git config user.email (your login)@FreeBSD.org
````
pauamma_gundo.com:
config is by executing `/usr/local/bin/gen-gitconfig.sh` script on freefall:
```
$ gen-gitconfig.sh
[...]
git config user.name (your name in gecos)
git config user.email (your login)@FreeBSD.org
````
* Set the push URL:
```
$ git remote set-url --push freebsd git@gitrepo.freebsd.org:${repo}.git
```
Then you should have separated fetch and push URLs as the most efficient setup:
```
$ git remote -v
freebsd https://git.freebsd.org/${repo}.git (fetch)
freebsd git@gitrepo.freebsd.org:${repo}.git (push)
```
Again, note that `gitrepo.freebsd.org` will be canonicalized to `repo.freebsd.org` in the future.
* Install commit message template hook:
```
emasteUnsubmitted
Done
Inline Actions

For now, we can mention that these will likely change

emaste: For now, we can mention that these will likely change
fetch https://cgit.freebsd.org/src/plain/tools/tools/git/hooks/prepare-commit-msg -o .git/hooks
chmod 755 .git/hooks/prepare-commit-msg
```
==== "admin" branch
The `access` and `mentors` files are stored in an orphan branch, `internal/admin`, in each repository.
Following example is how to check out the `internal/admin` branch to a local branch named `admin`:
```
git config --add remote.freebsd.fetch '+refs/internal/*:refs/internal/*'
git fetch
git checkout -b admin internal/admin
```
Alternatively, you can add a worktree for the `admin` branch:
```
git worktree add -b admin ../${repo}-admin internal/admin
```
For browsing `internal/admin` branch on web:
https://cgit.freebsd.org/${repo}/log/?h=internal/admin
pauamma_gundo.comUnsubmitted
Done
Inline Actions
git push freebsd HEAD:refs/internal/admin
```
- Or setting `push.default` to `upstream` which will make `git push` to push the current branch back to its upstream by default, which is more suitable for our workflow:
+ Or set `push.default` to `upstream` which will make `git push` to push the current branch back to its upstream by default, which is more suitable for our workflow:
```
git config push.default upstream
pauamma_gundo.com:
For pushing, either specify the full refspec:
```
git push freebsd HEAD:refs/internal/admin
```
Or set `push.default` to `upstream` which will make `git push` to push the current branch back to its upstream by default, which is more suitable for our workflow:
```
pauamma_gundo.comUnsubmitted
Done
Inline Actions
Those mirrors are not hosted in FreeBSD.org but still maintained by the project members.
- Users and developers are welcomed to pull or browse repositories on those mirrors.
+ Users and developers are welcome to pull or browse repositories on those mirrors.
The project workflow with those mirrors are still under discussion.
pauamma_gundo.com:
git config push.default upstream
```
[WARNING]
====
These internal details may change often.
====
=== External mirrors
Those mirrors are not hosted in FreeBSD.org but still maintained by the project members.
Users and developers are welcome to pull or browse repositories on those mirrors.
The project workflow with those mirrors are still under discussion.
==== Codeberg
- doc: https://codeberg.org/FreeBSD/freebsd-doc
- src: https://codeberg.org/FreeBSD/freebsd-src
pauamma_gundo.comUnsubmitted
Done
Inline Actions
General usage and questions about git in the FreeBSD project: [freebsd-git](https://lists.freebsd.org/mailman/listinfo/freebsd-git)
- Commit messages will be sent to following mailing lists:
+ Commit messages will be sent to the following mailing lists:
- [dev-commits-doc-all](https://lists.freebsd.org/mailman/listinfo/dev-commits-doc-all): All changes to the doc repository
pauamma_gundo.com:
==== GitHub
- doc: https://github.com/freebsd/freebsd-doc
- src: https://github.com/freebsd/freebsd-src
==== GitLab
- doc: https://gitlab.com/FreeBSD/freebsd-doc
- src: https://gitlab.com/FreeBSD/freebsd-src
=== Mailing lists
General usage and questions about git in the FreeBSD project: [freebsd-git](https://lists.freebsd.org/mailman/listinfo/freebsd-git)
Commit messages will be sent to the following mailing lists:
- [dev-commits-doc-all](https://lists.freebsd.org/mailman/listinfo/dev-commits-doc-all): All changes to the doc repository
- [dev-commits-ports-all](https://lists.freebsd.org/mailman/listinfo/dev-commits-ports-all): All changes to the ports repository
- [dev-commits-ports-main](https://lists.freebsd.org/mailman/listinfo/dev-commits-ports-main): All changes to the "main" branch of the ports repository
- [dev-commits-ports-branches](https://lists.freebsd.org/mailman/listinfo/dev-commits-ports-branches): All changes to the quarterly branches of the ports repository
- [dev-commits-src-all](https://lists.freebsd.org/mailman/listinfo/dev-commits-src-all): All changes to the src repository
- [dev-commits-src-main](https://lists.freebsd.org/mailman/listinfo/dev-commits-src-main): All changes to the "main" branch of the src repository (the FreeBSD-CURRENT branch)
- [dev-commits-src-branches](https://lists.freebsd.org/mailman/listinfo/dev-commits-src-branches): All changes to all stable branches of the src repository
For more information, please refer to the "Commit message lists" section of C.2. "Mailing Lists" in handbook: https://www.freebsd.org/doc/en/books/handbook/eresources-mail.html
[[svn]] [[svn]]
== Using Subversion == Using Subversion
[[svn-intro]] [[svn-intro]]
=== Introduction === Introduction
As of July 2012, FreeBSD uses Subversion as the only version control system for storing all of FreeBSD's source code, documentation, and the Ports Collection. As of July 2012, FreeBSD uses Subversion as the only version control system for storing all of FreeBSD's Ports Collection.
As of December 2020, FreeBSD uses git as the primary version control system for storing all of FreeBSD's source code and documentation.
Changes from the git repo on the `stable/11`, `stable/12` and related releng branches are exported to the subversion repository.
This export will continue through the life of these branches.
[NOTE] [NOTE]
==== ====
Subversion is generally a developer tool. Users may prefer to use `freebsd-update` (crossref:cutting-edge[updating-upgrading-freebsdupdate,“FreeBSD Update”]) to update the FreeBSD base system, and `portsnap` (crossref:ports[ports-using,“Using the Ports Collection”]) to update the FreeBSD Ports Collection. Subversion is generally a developer tool. Users may prefer to use `freebsd-update` (crossref:cutting-edge[updating-upgrading-freebsdupdate,“FreeBSD Update”]) to update the FreeBSD base system, and `portsnap` (crossref:ports[ports-using,“Using the Ports Collection”]) to update the FreeBSD Ports Collection.
==== ====
This section demonstrates how to install Subversion on a FreeBSD system and use it to create a local copy of a FreeBSD repository. Additional information on the use of Subversion is included. This section demonstrates how to install Subversion on a FreeBSD system and use it to create a local copy of a FreeBSD repository. Additional information on the use of Subversion is included.
[[svn-ssl-certificates]] [[svn-ssl-certificates]]
=== Root SSL Certificates === Root SSL Certificates
Installing package:security/ca_root_nss[] allows Subversion to verify the identity of HTTPS repository servers. The root SSL certificates can be installed from a port: Older FreeBSD systems do not have proper root certificates.
pauamma_gundo.comUnsubmitted
Done
Inline Actions

"Older" meaning? (IIRC, it's only 11.4.)

pauamma_gundo.com: "Older" meaning? (IIRC, it's only 11.4.)
Installing package:security/ca_root_nss[] on these systems allows Subversion to verify the identity of HTTPS repository servers.
The root SSL certificates can be installed from a port:
[source,shell] [source,shell]
.... ....
# cd /usr/ports/security/ca_root_nss # cd /usr/ports/security/ca_root_nss
# make install clean # make install clean
.... ....
or as a package: or as a package:
▲ Show 20 Lines • Show All 191 Lines • Show Last 20 Lines