Index: head/tools/tools/README =================================================================== --- head/tools/tools/README (revision 307679) +++ head/tools/tools/README (revision 307680) @@ -1,70 +1,71 @@ # $FreeBSD$ This directory is for tools. A tool is something which is sometimes useful, and doesn't fit any of the other categories. Please make a subdir per program, and add a brief description to this file. ansify Convert K&R-style function definitions to ANSI style ath Tools specific to the Atheros 802.11 support cfi Common Flash Interface (CFI) tool commitsdb A tool for reconstructing commit history using md5 checksums of the commit logs. crypto Test and exercise tools related to the crypto framework cxgbetool A tool for the cxgbe(4) driver. cxgbtool A tool for the cxgb(4) driver. diffburst OBSOLETE: equivalent functionality is available via split -p. For example: "split -p ^diff < patchfile". See split(1). drm Tools specific to the DRM/KMS device drivers. editing Editor modes and the like to help editing FreeBSD code. epfe Extract printing filter examples from printing.sgml. ether_reflect An Ethernet packet reflector for low level testing. find-sb Scan a disk for possible filesystem superblocks. gdb_regofs A simple tool that prints out a register offset table for mapping gdb(1) register numbers to struct reg and struct fpreg offsets. The tool is useful on selected platforms only. genericize Turn a kernel config into something that can more easily be diffed against the appropriate GENERIC. +git Tools to simplify the use of git by committers. hcomp Compress header files by removing comments and whitespace. html-mv Rename HTML generated filenames to human readable filenames. ifinfo Uses the interface MIB to print out all the information an interface exports in an ugly form. indent_wrapper Tool for style(9) checking SVN/GIT patches. iso Tool to compare the iso3166 and iso639 files in /usr/share/misc with the data from the master sites. iwi Tools specific to the Intel PRO/Wireless 2200BG/2225BG/2915ABG support. kdrv KernelDriver; add/list/remove third-party kernel driver source to/in/from a kernel source tree. kernelcruft Shellscript to find orphaned *.c files in /sys kerninclude Shellscript to find unused #includes in the kernel. kernxref Shellscript to cross reference symbols in the LINT kernel. kttcp An in-kernel version of the ttcp network performance tool mctest A multicast test program mid Create a Message-ID database for mailing lists. mwl Tools specific to the Marvell 88W8363 support ncpus Count the number of processors netmap Test applications for netmap(4) notescheck Check for missing devices and options in NOTES files. npe Tools specific to the Intel IXP4XXX NPE device nxge A diagnostic tool for the nxge(4) driver pciid Generate src/share/misc/pci_vendors. pciroms A tool for dumping PCI ROM images. WARNING: alpha quality. pirtool A tool for dumping the $PIR table on i386 machines at runtime. portsinfo Generate list of new ports for last two weeks. recoverdisk Copy as much data as possible from a defective disk. scsi-defects Get at the primary or grown defect list of a SCSI disk. sysdoc Build a manual page with available sysctls for a specific kernel configuration. tinybsd Script to build FreeBSD embedded systems. track Track the progress of a world / kernel build vimage An interim utility for managing the virtualized network stack infrastructure. vop_table Generates a HTML document that shows all the VOP's in the kernel. vxge A diagnostic tool for the vxge(4) driver whereintheworld Summarizes "make world" output. Index: head/tools/tools/git/HOWTO =================================================================== --- head/tools/tools/git/HOWTO (nonexistent) +++ head/tools/tools/git/HOWTO (revision 307680) @@ -0,0 +1,144 @@ +# $FreeBSD$ + +This directory contains tools intended to help committers use git when +interacting with standard FreeBSD project resources like Differential or svn. + +I. arcgit + +arcgit is a wrapper script around the arc command line tool that simplifies the +automatic creation of a series of Differential reviews from a series of git +commits. The intended workflow is: + +1. Create a series of commits in git. Each commit will be a separate review, so + try to make each commit as self-contained as possible. The series of commits + should demonstrate a logical progression towards your end goal. For example, + one commit might refactor existing code to expose a new API without changing + any current functionality. A subsequent commit could then introduce your new + code that uses the new API. + + It usually will not be helpful to present your code in the order in which it + was actually written and can actually be harmful. For example, if you + introduced a bug early in your development process that you fixed in a + subsequent commit, it is a waste of your reviewer's time to have them review + old code with known bugs. Instead, it would probably be best to squash the + bug fix into the commit that introduced it, so that the bug is never + presented to your reviewers in any review. + + The commit headline and commit message will be imported verbatim into + Differential, so try to give each commit a meaningful commit message that + gives your reviewers the necessary context to understand your change. + +2. Create your reviews bu running this command in your git repo: + $ arcgit -r C1~..C2 -R reviewer -T testplan + + C1 should be the first commit that you want reviewed, and C2 should be the + last commit that you want reviewed. You may add multiple reviewers by + specifying the -R option multiple times. You can CC (AKA subscribe) people + to a review with the -C option. Note that if you subscribe a mailing list + to a review, the mailing list will be emailed for every comment or change + made to each review. Please be judicious when subscibing mailing lists to + reviews. It may be better to instead send a single email to the appropriate + list announcing all of the reviews and giving a short summary of the change + as a whole, along with a link to the individual reviews. + +3. When you need to make a change and upload it to a review, use the following + process. First, check out the branch corresponding to the review (arcgit + automatically creates this branch for every review that it creates): + + $ git checkout review_D1234 + + Next, make your change and perform whatever testing is necessary. Commit it + to your repository with this command: + + $ git commit --fixup HEAD + + You can upload the change to the code review by running this command in your + repository while (ensure that you are still on the review_D1234 branch): + + $ arc diff --update D1234 review_D1234_base + + When you run arc, it will pull up your editor and give you a chance to + change the message that will be shown in Differential for this upload. It's + recommended that you change it from the default "fixup! ...." as that does + not give your reviewers an indication of what changes were made. It's not + recommended that you give the code review fixes meaningful commit messages + directly because we will be using git's autosquash feature in the next step, + which depends on the fixup! tag being present. + + Do not merge in other branches, or rebase your review branches at this point. + Any changes that are made will show up in your reviews, and that will create + extra noise that your reviewers will have to ignore. If a reviewer requests + a change that requires your commit to be based off of a later version of + head, I would suggest deferring the change from the review and creating a + new review with the change once you hit step 5. + +4. Once the reviews have been approved, you need to prepare your patch series + to be committed. This involves squashing the fixes made in code review + back into the original commit that they applied to. This gives you a clean + series of commits that are ready to be commited back to svn. + + First, merge each of your review branches back into your main development + branch. For example: + + $ git checkout myfeature_dev + $ for branch in review_D1234 review_D1235 review_D1236; do \ + git merge $branch || git mergetool -y || break; done + + Next, do an interactive rebase to squash your code review fixes back into the + main commit: + + $ git rebase -i -k review_D1234_base + + review_D1234 should be the name of the *first* review that was created for + you in step 2. For every commit, your editor will be pulled up and you will + be given one last chance to edit your commit message. Make sure that you fill + in the "Reviewed by:" tag indicating who accepted the review. This would + be a good point to add other tags like MFC after:, Sponsored by:, etc. + + The rebase will not introduce any actual changes to your code if done + properly. You can use this command to double-check that no changes were + inadvertently made here: + + $ git diff myfeature_dev@{1} + +5. Finally, you should get up to date with the latest changes from head: + + $ git pull --rebase origin master + + It is not recommended that you combine this step with the rebase done in the + previous step. The reason for this is that if you perform an interactive + rebase that changes the commit that you branch from, it becomes much more + difficult to use the reflog to confirm that the interactive rebase did not + introduce unwanted changes. + + At this point, you are ready to commit your changes to head. The importgit + script can be used to import your commits directly into git. + +II. importgit + +importgit is a script that can take a series of commits from git and commit them +to a svn repository. The script uses the git commit messages for the svn commit +message, which allows importgit to be fully automated. This does mean that once +you start importgit, it will start commit things to svn without giving any +further chance to sanity check what it's doing. + +importgit only supports importing commits that add or modify files. It does not +support importing commits that rename or delete files, to ensure that git's +rename detection heuristics do not introduce an error in the import process. +importgit also does not support importing merge commits. Only linear history +can be imported into svn. + +importgit must be run from a clean subversion checkout. You should ensure that +the working tree is up-to-date with "svn up" before running importgit. +importgit will run svn directly, so make sure that your ssh-agent is running +and has your ssh key loaded already. Run importgit as follows: + + $ importgit -r D1~..D2 -g /path/to/git/repo + +This will import every commit between D1 and D2, including both D1 and D2. The +invocation is very similar to the invocation given to arcgit but there is an +important point to note. When you rebased your commits as you followed steps 4 +and 5, the commit hashes of all of your commits changed, including C1 and C2. +You must go back and find the new commit hashes of your commits to pass to +importgit. Passing -r C1~..C2 would import your commits as they were *before* +your code review fixes were applied. Property changes on: head/tools/tools/git/HOWTO ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: head/tools/tools/git/arcgit =================================================================== --- head/tools/tools/git/arcgit (nonexistent) +++ head/tools/tools/git/arcgit (revision 307680) @@ -0,0 +1,214 @@ +#!/bin/sh +# +# Copyright (c) 2015 Ryan Stone. All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. +# +# $FreeBSD$ + +# This script is used to submit a series of git commits to Differential. Each +# commit is submitted as a separate review. For each review, this script will +# create a branch called review_DXXXX (e.g. review_D2185 for Differential +# revision D2185). When you need to make a change to a review, checkout the +# review_D2185 branch, commit your change with "git commit --fixup HEAD". To\ +# upload the change to Differential, use the command: +# $ arc diff --update D2185 review_D2185_base +# +# When your reviews are complete, merge all of the review_DXXXX branches +# together, and then do a git rebase -ik to meld the code review fixes into the +# commit that they fixed. Now you have a clean series of patches to commit to +# svn. + +usage() +{ + echo "Usage: arcgit <-c commit | -r commit1~..commit2> [-R reviewer] " >&2 + echo " [-C subscriber] [-T testplan] [-n]" >&2 +} + +error() +{ + echo "$@" >&2 + usage + rm -f $phab_before $phab_after $arc_msg + exit 1 +} + +create_review() +{ + local commit phab_id arc_dir + unset phab_before phab_after arc_msg + commit=$1 + + phab_before=`mktemp -t arcoutput` + phab_after=`mktemp -t arcoutput` + echo "Create review for '`git show $commit -s --oneline`'" + + if [ -n "$dry_run" ] + then + return + fi + + git checkout $commit > /dev/null || error "Could not checkout $commit" + + arc_dir="$(git rev-parse --show-toplevel)/.git/arc" + arc_msg="$arc_dir/create-message" + mkdir -p $arc_dir + git show -s --format='%B' HEAD > $arc_msg + echo >> $arc_msg + echo "Test Plan:" >> $arc_msg + cat $test_plan >> $arc_msg + echo >> $arc_msg + echo "Reviewers:" >> $arc_msg + echo "$reviewers" >> $arc_msg + echo >> $arc_msg + echo "Subscribers:" >> $arc_msg + echo "$cc_list" >> $arc_msg + echo >> $arc_msg + + arc list > $phab_before + yes | env EDITOR=true arc diff --create --allow-untracked HEAD~ + arc list > $phab_after + + headline="$(git show $commit -s --format=%s)" + phab_id=`comm -13 "$phab_before" "$phab_after" | fgrep "$headline" \ + | egrep -o 'D[0-9]+:' | tr -d ':'` + + if [ -z "$phab_id" ] + then + error "Could not get review ID" + fi + + git branch review_${phab_id}_base HEAD~ + + git checkout -b review_$phab_id + cat - < /dev/null 2> /dev/null +then + error "Install devel/git first" +fi + +if ! type arc > /dev/null 2> /dev/null +then + error "Install devel/arcanist first" +fi + +git update-index -q --refresh +if ! git diff-index --quiet --cached HEAD +then + error "index is unclean" +fi + +if ! git diff-files --quiet +then + error "Working directory is unclean" +fi + +if git ls-files --other --error-unmatch . > /dev/null 2> /dev/null +then + error "Working directory contains untracked files" +fi + +# We have to do a git checkout in order to run arc, so save the original branch +# so that we can check it out again once we're done. +if ! orig_branch=$(git symbolic-ref --short -q HEAD) +then + orig_branch=$(git show -s --pretty='%H' HEAD) +fi + +git log --format=%H $range | tail -r | while read -r commit +do + create_review $commit < /dev/null +done + +# Note that due to the use of the pipeline above, the body of the while loop +# above runs in a subshell. If it exits with an error, execution resumes +# here rather than exiting the script, so we have to cache the right exit code +# and return it when we're done cleaning up. +code=$? + +git checkout $orig_branch + +exit $code + Property changes on: head/tools/tools/git/arcgit ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: head/tools/tools/git/importgit =================================================================== --- head/tools/tools/git/importgit (nonexistent) +++ head/tools/tools/git/importgit (revision 307680) @@ -0,0 +1,182 @@ +#!/bin/sh +# +# Copyright (c) 2015 Ryan Stone. All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. +# +# $FreeBSD$ + +usage() +{ + echo "Usage: importgit <-c commit | -r c1..c2> -g /path/to/git/repo [-n]" >&2 +} + +error() +{ + local print_usage + + if [ "$1" = "-u" ] + then + shift + print_usage=1 + else + print_usage= + fi + + echo "$@" >&2 + if [ -n "$print_usage" ] + then + usage + fi + exit 1 +} + +unset git range commit dry_run + +while getopts ":c:g:nr:" o +do + case "$o" in + c) + range="${OPTARG}~..${OPTARG}" + ;; + g) + git_repo=$OPTARG + ;; + n) + dry_run=1 + ;; + r) + range=$OPTARG + ;; + *) + error -u "Unrecognized argument '-$OPTARG'" + esac +done + +shift $((OPTIND - 1)) +OPTIND=1 + +if [ -n "$1" ] +then + error -u "Unrecognized argument $1" +fi + +if [ -z "$range" ] +then + error -u "-c or -r argument is mandatory" +fi + +if ! echo "$range" | egrep -qs '^[^.]+\.\.[^.]*$' +then + error -u "$range is not a range of commits. Did you mean '-c $range'?" +fi + +if [ -z "$git_repo" ] +then + error -u "-g argument is mandatory" +fi + +git="$git_repo/.git" + +if [ ! -d "$git" ] +then + error "$git_repo does not seem to be a git repo" +fi + +if ! type git > /dev/null 2> /dev/null +then + error "Install devel/git first" +fi + +if ! type svn > /dev/null 2> /dev/null +then + error "Install devel/subversion first" +fi + +if [ -n "$(svn status)" ] +then + error "Working tree is not clean" +fi + +if ! svn --non-interactive ls > /dev/null +then + error "Could not communicate with svn server. Is your ssh key loaded?" +fi + +git --git-dir=$git log --format=%H $range | tail -r | while read -r commit +do + echo "Applying `git --git-dir=$git show -s --oneline $commit`" + + if [ -n "$(git --git-dir=$git show --diff-filter=CDRTUXB $commit)" ] + then + error "Commit performed unsupported change (e.g. delete/rename)" + fi + + if [ "$(git --git-dir=$git show -s --format=%P $commit | wc -w)" -ne 1 ] + then + error "Cannot import merge commits" + fi + + git --git-dir=$git diff --diff-filter=A --name-only \ + ${commit}~..$commit | while read -r newfile + do + if [ -f "$newfile" ] + then + error "New file $newfile already exists in tree" + fi + done + + # The previous while loop ran in a subshell, so we have to check if it + # exited with an error and bail out if so. + ret=$? + if [ "$ret" -ne 0 ] + then + exit $ret + fi + + if [ -n "$dry_run" ] + then + continue + fi + + git --git-dir=$git show $commit | patch -p 1 -s || \ + error "Failed to apply patch" + + git --git-dir=$git diff --diff-filter=A --name-only \ + ${commit}~..$commit | while read -r newfile + do + svn add --parents --depth=infinity $newfile || \ + error "Failed to add new file" + done + + # The previous while loop ran in a subshell, so we have to check if it + # exited with an error and bail out if so. + ret=$? + if [ "$ret" -ne 0 ] + then + exit $ret + fi + + git --git-dir=$git show -s --format='%B' $commit | svn commit -F - || \ + error "Failed to commit" +done + Property changes on: head/tools/tools/git/importgit ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property