Subversion (svn) je nástroj pro archivaci (a v případě potřeby obnovu) starších verzí softwaru, který vyvíjíte. Pokud se vás v jednom projektu hrabe víc, Subversion vám pomůže ohlídat, abyste si vzájemně neničili práci.

Subversion vám poskytuje různé možnosti, kde založit tzv. repozitář (tedy archív verzí vašeho projektu), a různé možnosti, jak k němu přistupovat. Lokální repozitář vám stačí, používáte-li Subversion pro archivaci projektu, na kterém pracujete pouze vy, případně i další členové ÚFALu, ale všem vám stačí přistupovat k repozitáři z terminálu či skriptů běžících na linuxových počítačích na ÚFALu. Lokální repozitář je prostě podstrom adresářové struktury na síťových discích (konkrétně v /net/work/public/SVN, ale můžete si ho vytvořit i jinde). Když chcete s lokálním repozitářem pracovat, musíte mít na dané místo souborový přístup. Serverový repozitář využijete, jestliže potřebujete s repozitářem pracovat z počítače, ze kterého není přímo vidět disk, na kterém je repozitář uložen (např. z Windows, z domova) nebo pokud s vámi na projektu spolupracují lidé vně ÚFALu, kteří ani úplný přístup do naší sítě nemají. U serverového repozitáře je potřeba nakonfigurovat metody, jakými se k němu bude po síti přistupovat, a také přístupová práva uživatelů.

Pro projekty ÚFALu, na nichž chcete spolupracovat i s lidmi zvenku, (serverové repozitáře) použijte vyhrazený stroj svn.ms.mff.cuni.cz. Návod na konfiguraci najdete níže v kapitole Subversion Server. Návod na vytvoření jednoduchého lokálního repozitáře najdete níže v kapitole Lokální Subversion.

Toto je Milanovo e-mailové shrnutí z 26.9.2007. Týká se konfigurace serverových repozitářů na serveru svn.ms.mff.cuni.cz.

Mili kolegove,
po diskusi s Ondrou Bojarem a drobnem vylepseni naseho serveru svn jeste jednou sepisuji co je potreba si projit a nastavit pri konfiguraci noveho projektu:

1. nalogujete se na svn.ufal.ms.mff.cuni.cz (= svn.ms.mff.cuni.cz) (ucty byly preneseny z ufal.mff.cuni.cz)
2. pro vytvoreni projektu pouzijete skript (kteremu date jeste jmeno projektu):

   /home/howto/create_new_project.sh <jmeno_projektu>

3. pak se rozhodnete kterymi cestami budete s projektem pracovat - nejlepsi je asi nakonfigurovat vsechny (viz. nize) - tedy svn, webovy pristup do svn a Trac (take webovy). Vsude jsou prednastaveny nejake zakladni ucty, ale nejlepsi je tyto veci upravit podle vlastnich potreb. Napriklad nemusite chtit, aby ufal/maly… mel rw prava ve vasem projektu, jak tomu defaultne je!

Toto rozhraní je užíváno SVN klienty, kteří dostanou repository udanou ve tvaru https://svn.ms.mff.cuni.cz/svn/PROJEKT, tj. např.:

    svn --username ufal checkout https://svn.ms.mff.cuni.cz/svn/$PROJEKT

Co se tyce svn weboveho rozhrani, tak ten modul ma nasledujici konfiguraky (viz. /etc/httpd/conf.d/subversion.conf):

• uzivatele: AuthUserFile /home/svn/users/passwords (Ondrej tomu laicky rika .htaccess)
• jejich opravneni: AuthzSVNAccessFile /home/svn/permissions/svnauthz.conf (zde se rika, zda RO nebo RW)

O zbytek se stara webovy modul subversion - ja mu jen rikam tohle.

Konfiguraky jsou globalni, tj. pro vsechny projekty spolecne.

O přidávání uživatelů a hesel viz níže.

Toto rozhraní je užíváno SVN klienty, kteří dostanou repository udanou ve tvaru svn…, tj. např.:

  svn --username ufal checkout svn://svn.ms.mff.cuni.cz/$PROJEKT

Pristupova prava pro svn protokol se urcuji pro kazdy projekt nezavisle, v adresari conf v dane repository (jak je vytvorena mym skriptem), tj. napr.:

  vim /home/svn/repos/$PROJEKT/conf/passwd  # vytvořit uživatele
  vim /home/svn/repos/$PROJEKT/conf/authz   # nastavit jim práva

Vzory nastaveni pristupu jsou zde: /home/svn/repos/default/conf

Pozor - hesla se tu nastavuji jako textove retezce, neni to htaccess! Po siti tedy tecou nekryptovane!

Trac - pouziva take overovani vuci:

  AuthUserFile /home/svn/users/passwords

Trac ma navic vlastni system prav. Trac vas rozpozna podle loginu, se kterym jste se prihlasili pri http autentizaci, ktera je pri pristupu vynucena. Pokud v danem projektu nejste zavedeni, tak se k nemu dostanete s nejakymi default pravy.

Takto spustíte konfigurační skript pro Trac vašeho projektu:

  trac-admin /home/trac/$PROJEKT

Vlastni uzivatele pro webove pristupy pridate do centralniho souboru webovych hesel v /home/svn/users/passwords (na coz prava mate) pomoci:

  htpasswd -bs /home/svn/users/passwords <uzivatelske_jmeno> <heslo>

nebo

  htpasswd -s /home/svn/users/passwords <uzivatelske_jmeno>
  # na heslo budete dotázáni, heslo se nezobrazí ani na terminálu

(Z návodu ve wiki University of Maryland převzal a upravil Dan Zeman.)

Tento návod se týká lokálních repozitářů SVN, tedy takových, ke kterým budete přistupovat pouze z linuxových počítačů v síti ÚFALu.

You can use svn to keep track of your own code.

New svn repository can be created by

svnadmin create /net/work/public/SVN/$MY_PROJECT

where obviously $MY_PROJECT should be replaced by the desired name for your project.

Beware! Do not use older versions (e.g. 1.1.4) of Subversion to create a repository! By default, they will use the Berkeley DB backend, which is not recommended. You may be able to “successfully” finish the repository creation but things are likely to get broken later! Check your version by running

svn --version

Version 1.3.1 and above should work fine.

Let's say you want to archive the directory /home/$USER/proj and all its subdirectories. The following code puts your project in the proj folder under version control. After executing the commands, your proj folder will become a working copy of the files stored in the repository.

svnadmin create /net/work/public/SVN/proj
cd /home/$USER
mv proj proj.old
mkdir tmp
cd tmp
mkdir proj
cp -r /home/$USER/proj.old proj/trunk
mkdir proj/branches
mkdir proj/tags
svn import proj file:///net/work/public/SVN/proj --message 'Initial import'
cd ..
svn checkout -q file:///net/work/public/SVN/proj/trunk proj
rm -rf tmp
rm -rf proj.old

Výše uvedený postup je možné po úpravách použít i pro vzdálený repozitář na serveru SVN. Předpokládáme, že jste si už vytvořili repozitář $REP podle návodu uvedeného dříve, a že jste k němu nakonfigurovali přístupová práva. Řádek svnadmin create … zde tedy vynecháváme.

cd /home/$USER
mv $PROJ $PROJ.old
mkdir tmp
cd tmp
mkdir $PROJ
cp -r /home/$USER/$PROJ.old $PROJ/trunk
mkdir $PROJ/branches
mkdir $PROJ/tags
svn --username $USER import $PROJ https:///svn.ms.mff.cuni.cz/svn/$REP/$PROJ --message 'Initial import'
cd ..
svn checkout -q https:///svn.ms.mff.cuni.cz/svn/$REP/$PROJ/trunk $PROJ
rm -rf tmp
rm -rf $PROJ.old

Pokud jste si jisti, že pro tento projekt nebudete nikdy potřebovat větve ani otagované verze, můžete pravděpodobně vynechat vytváření podsložek branches a tags a celý projekt posunout o patro výš. To jsem ale nezkoušel.

Rozmyslete si, zda chcete zachovat pouze aktuální verzi vašeho projektu, nebo i některé starší. Pokud chcete zachovat pouze aktuální verzi, vybalte si ji do nějaké pracovní složky pomocí cvs checkout a postupujte podle výše uvedeného návodu pro přidání projektu pod SVN.

Pokud chcete zachovat i některé důležité starší verze, vybalte nejprve nejstarší z nich a podle výše uvedeného návodu ji zaveďte jako nový projekt pod SVN. Potom postupně vybalujte další verze, přepište jimi vaši pracovní kopii SVN projektu a uložte je do SVN pomocí svn commit. Pozor na případy, kdy mezi dvěma verzemi přibyly nové soubory nebo naopak zmizely staré! Takové soubory můžete odhalit pomocí svn status a zařadit nebo vyřadit pomocí svn add, resp. svn remove.

# Get version of August 16, 2004, from CVS.
cvs checkout -D 2004/8/16 $USER/parser
cd $USER/parser
cvs status

# Import this version to SVN.
rm -rf CVS
cd ..
rm -rf CVS
mv parser trunk
mkdir branches
mkdir tags
cd ..
mv $USER parser
svnadmin create /net/work/public/SVN/parser
svn import parser file:///net/work/public/SVN/parser --message 'Initial import'
rm -rf parser
svn checkout -q file:///net/work/public/SVN/parser/trunk parser

# Tag this version of the project.
svn copy -q file:///net/work/public/SVN/parser/trunk \
  file:///net/work/public/SVN/parser/tags/disertace \
  -m "Version from the dissertation"

# Before checking out a new CVS version, remove all files but not the ''.svn'' folder.
# (Note that better approach is needed if we have subfolders.)
cd parser
rm -f *
rm -rf CVS
cd ..

# Check out the current CVS version of the project.
ln -s . $USER
cvs checkout $USER/parser
rm -f $USER
rm -rf CVS
cd parser
rm -rf CVS
svn status

# Add new files (status ?), remove obsolete files (status !)
# Thereafter, added files have status A, removed D.
svn rm obsolete-file # ! -> D
svn add new-file     # ? -> A
svn status

# Commit the changes, update working copy.
svn commit -m 'Current version. Moving from CVS completed.'
svn update

Imagine that your makefile creates a tmp file or directory. You do not want to version this file, because it can be any time recreated by calling make. And you do not want to see the file listed in svn status output. Then you need to edit the svn:ignore property of the folder containing tmp.

1. Go to the folder containing tmp.
2. Call svn propedit svn:ignore ..
3. Enter files to be ignored, one per line. But you can use wildcards, as in *.d. Note that some files are ignored by default (e.g. *.o).
4. Call svn status. Now you should not see the files you said should be ignored, but for now you will see the folder containing them. That's because this folder's property (svn:ignore) has changed.
5. You need to call svn commit to make the property stored in the repository and available to other users.

Note that you just set property of one folder, not of a whole subtree of folders. That is, tmp files in subfolders will not be ignored until you set the same property in the subfolders. In general, properties can be set recursively but it is not trivial with multi-line property values, which is the case of svn:ignore.

The basic idea behind subversion is that it keeps multiple versions of entire directory trees (Note that this differs from cvs, which deals in per-file revisions). You can store an entire snapshot of the directory tree at any time by committing it to the repository. If you'd like to store a snapshot of something you're working on, but don't want to put it into the main development line just yet, you can store it as a branch, and when you've finished working on your branch, you can merge it into the main development line (trunk).

You should 'never' work directly on the live code that lives in /fs/clip-mteval/Programs. Rather, you will always work on a local copy, called the working copy of the code, which is a replica of some snapshot of the repository, with any local changes you have made. To propagate information from a working directory, you must commit changes to the repository, and then export those changes to the live code directory.

To create a working copy of the repository, first make sure you are 'not' logged in to csubmit01 or csubmit02. Then cd to some directory where you would like to work (preferably somewhere in your home directory path), and run the following commands:

svn checkout -q file:///fs/clip-mteval/svn/Programs/trunk Programs
cd Programs
make

(Ignore hundreds of compiler warnings.)

This will create a pristine copy of the latest repository version in your local directory. You can then proceed to make any changes you would like. (Note that the checkout command does 'not' imply locking.)

If you already have a working copy, you can use the following command to bring it up to date with changes made to the repository since you last checked the files out:

svn update

If you want to access to svn repository from a host outside the UMIACS network, you can do so via ssh from the remote machine:

svn checkout -q svn+ssh://csubmit01.umiacs.umd.edu/fs/clip-mteval/svn/Programs/trunk Programs

Note: for many operations, you may need to enter your UMIACS password multiple times. For example, checking out a client requires at least 3 password entries. This is due to the svn client establishing multiple ssh connections to the remote host.

Once you have a working copy that's up-to-date, you can proceed to make and test any changes that you are working on. It is important to remember that in subversion, a version of a project is a snapshot of an entire directory. Therefore, if your change involved moving, copying, or renaming files, you need to make subversion aware of these changes. Rather than use the commands rm, cp, and mv, you should use the subversion equivalents:

svn delete <file>
svn copy <file-from> <file-to>
svn move <file-from> <file-to>

Additionally, whenever you create a file in the directory, you'll want to use the command

svn add <file>

Note that these commands do not update the repository with the local copies of these files, if they've been modified since you checked them out; it only signals to the repository that you've made changes to the directory structure. These will become permanent only once you've committed the files.

Changes that do not change the directory structure can be made in the working copy of the file directly using whatever method is appropriate.

Before you commit your changes to the repository, you may need to examine them to see precisely what they are (in fact, in the case of conflicting changes committed by different users, subversion will 'require' you to do this). To do this, run the command.

svn status

This will give you a listing of all the files in the working directory and their status with respect to the original repository version. If a file has changed and you want to see the changes, you can use

svn diff <file>

If you discover that you've made an error, you can undo it using

svn revert <file>

Note that reverting also works for add, move, copy, and delete operations as well!

Most importantly, svn status will tell you if the changes that you've made in a working copy of a file are in conflict with the changes that another user has committed since you last checked out the file. If this happens, it will report:

C filename

You will not be able to commit changes to this file until you've fixed the conflict manually. This is described next.

'You should not even attempt to commit files to the main branch until you can run svn status without seeing conflicts'. The group policy on regression and commits is as follows:

* When you begin a modification, checkout a fresh copy of the latest code
* Run a test on whatever portion of the code you plan to update. Store the results somewhere where you can look at them later.
* Make your modifications.
* Run a test of your modifications. Check your results against the ones you stored previously and make sure that the differences are the ones you intended.
* Run svn status and resolve any conflicts.
* Run svn update to retrieve modifications made by others.
* 'Run your test again' to make sure that the results you intend from your modification have 'not' been affected by other changes to the code.
* Repeat the last four steps as necessary until there are no more conflicts.
* Commit your changes.

Before you check in files, you must first run svn update to receive any non-conflicting updates from the repository. This will also, like svn status, tell you if there are any conflicts with your intended changes. If this happens, subversion will inform you of the filename that is in conflict, and place two unversioned copies of it in your working directory: file.r<OLD> and file.r<NEW>, which are, respectively, the version of the file when you last checked it out, and the current version in the repository (<OLD> and <NEW> are the version numbers of these revisions). Coupled with your working version, you should determine the necessary changes, make them in your working copy, and then run:

svn resolved <file>

This will tell subversion that your working copy has successfully integrated the two versions. Now you can check them in using:

svn commit

Subversion will prompt you for a log message describing your changes.

Alternatively, you can specify a log message on the command line, using the -m flag.

svn commit -m 'Replaced all perl scripts with python'

You can review log messages from previous changes using

svn log

The version of the code that lives in /fs/clip-mteval/Programs is 'not' a working copy. The eventual expectation is that this directory will go away. Instead of running scripts from the Programs directory, you will run code from a local copy that you have checked out or exported from the repository. When running experiments, you should make note of the code revision or tag that you used.

Note that once you checkout or export the codebase from the directory, you will need to run make in the toplevel directory. This builds certain executables that are required. For the time being, this only seems to work correctly on the wine cluster.

Because you will always run code from a local directory, your programs and scripts should never refer to the directory /fs/clip-mteval/Programs. This is sure way to cause major debugging problems later. Instead, they should refer to the local directory where your script is run from. Since the directory differs for each user, your program will need to determine it at runtime. Let's suppose that you have a script foo, from which you would like to call the script bar. Here's an easy way to do this in perl:

#!/usr/bin/perl
open(ROOT,"dirname $0|");
while (<ROOT>){
  chomp;
  $root_dir = $_;
}
$cmd = "$root_dir/bar.pl";
system($cmd);

In sh or bash:

#!/bin/sh
ROOTDIR=`dirname $0`
CMD=$ROOTDIR/bar.sh
$CMD

In python:

#!/usr/bin/python
import sys, os.path
progname = sys.argv[0]
root_dir = os.path.dirname(progname)
cmd = os.path.join(root_dir, "bar.py") 
os.system(cmd)

Note that you only have to do this with pathnames that refer to things in the repository. It is perfectly fine to use absolute pathnames for other dedicated resources, such as things that reside in /fs/clip-mteval/Corpora.

Branching in subversion is merely a matter of copying a directory version to a new name. For instance, if I want to work on a complicated branch, and still keep it under version control, I might create a new branch under my user name:

svn copy -q file:///fs/clip-mteval/svn/Programs/trunk \
  file:///fs/clip-mteval/svn/Programs/branches/alopez \
  -m "Personal development branch for alopez"

This creates a new branch called alopez. I can then checkout and work with that branch:

svn checkout -q file:///fs/clip-mteval/svn/Programs/branches/alopez alopez
cd alopez
make

The changes I create will not affect the main branch of code (the ones in trunk) until I explicitly merge them into the code. But, how do I do this? I cannot simply copy all of my modified files back into the main trunk, becuase I may clobber changes that have been made there since my branch. And I can't run svn update to receive the changes from the main trunk into my own working copies, because svn status only works on working copies that are in the same branch, not independent development branches. What I need to do is merge the changes that I've made in my branch into the current revision of trunk. This is made possible with the svn merge command. Suppose that I started my branch of the code at revision 250 of the trunk. I continued working on this up until revision 271. The code in the trunk was last modified at revision 274. To merge my changes into the trunk, I must first checkout the code from the trunk into a new working copy:

svn checkout -q file:///fs/clip-mteval/svn/Programs/trunk Programs

Now, the changes that I made occurred in the alopez branch between revisions 250 and 271 (I must note the revisions manually). What svn merge does is determine the differences between two objects in the repository, and then apply them to the working copy. So here's how I run merge:

svn merge file:///fs/clip-mteval/svn/Programs/tags/alopez@250 file:///fs/clip-mteval/svn/Programs/tags/alopez@271 Programs''

This command takes the differences between the alopez branch from rev. 250 to 271, and applies them to the working copy. Note that this may create conflicts! (To preview these, I can use the –dry-run flag to svn merge). Once I've successfully resolved any conflicts, merged the files, and run regression tests as described above, I can commit the files to the development trunk.

Tags are subversion's mechanism for creating special named snapshots of particular directories. Essentially, a tag is just a copy of the directory from some particular revision (or combination of revisions). Tags are useful for creating releases. In general, you should not check out tagged versions of the project, because modifications should only be based on the trunk. Instead, you should export them if you need a snapshot of the code at some particular point in time.

To tag the current version, use:

svn copy -q file:///net/work/public/SVN/project/trunk \
  file:///net/work/public/SVN/project/tags/$TAGNAME \
  -m "Version from the dissertation"

To export a tagged version, use:

svn export -q file:///net/work/public/SVN/project/tags/$TAGNAME \
  ~/projects/project

What do you do if something is broken that wasn't broken before? You can simply check out a copy of the file (or the entire project) from a point before the problem arose. In order to see the revisions in which a particular file changed, you can run

svn log $FILE

This will show you the change log. Hopefully, this will be sufficiently informative as to when particular changes occurred, but if you aren't sure what happened in a particular revision, you can always run

svn diff $FILE -r $REVISION

To see the changes that have been made. Once you figure out which version you need, you can restore your working copy to that revision using

svn update $FILE -r $REVISION

Warning: If you update the whole project (svn update . in the root of the working copy) this way, svn will behave as if the revisions after the one you rolled back to did not exist. For instance, it will not show you the renewed files as changed when you call svn status. If you call svn log, it will show only revisions up to the one you rolled back to. If you call svn commit without making additional changes, nothing will happen. However, svn diff -r HEAD:BASE will show the differences between your current (rolled-back) version and the most recent one in the repository. So how do we force svn to store the restored old revision as the newest one in the repository?

The following chapters from the SVN book have some answers:

• http://svnbook.red-bean.com/nightly/en/svn.tour.history.html#svn.tour.history.snapshots
• http://svnbook.red-bean.com/nightly/en/svn.branchmerge.basicmerging.html#svn.branchmerge.basicmerging.resurrect
• http://svnbook.red-bean.com/nightly/en/svn.branchmerge.basicmerging.html#svn.branchmerge.basicmerging.undo

The following code should undo the changes between the revisions 302 and 303 (note the minus sign before 303, meaning negative change) while keeping the other changes between 303 and HEAD.

svn merge -c -303 https://svn.ms.mff.cuni.cz/svn/$REPOSITORY/$PROJECT/trunk
# alternatively: svn merge -r 303:302 https://svn.ms.mff.cuni.cz/svn/$REPOSITORY/$PROJECT/trunk
svn status
svn diff
svn commit -m "Undoing change committed in r303."

You can use svn to keep track of your own code. Let's say you want to archive the directory /nfshomes/$USER/proj and all its subdirectories. You can do so with the following commands.

'Beware!' /usr/bin/svn on the C cluster is version 1.1.4 (run svn –version to confirm). 'Do not use this version' to create a repository! By default, it will use the Berkeley DB backend, which is not recommended. You may be able to “successfully” finish the repository creation but things are likely to get broken later! /usr/local/bin/svn on the wine cluster (e.g., the vouvray machine) is version 1.3.1 and it should work fine, although 1.4 is already available.

cd /nfshomes/$USER
mv proj proj.old
svnadmin create svn
mkdir tmp
cd tmp
mkdir proj
cp -r /nfshomes/$USER/proj.old proj/trunk
mkdir proj/branches
mkdir proj/tags
svn import . file:///nfshomes/$USER/svn --message 'Initial import'
cd ..
svn checkout -q file:///nfshomes/$USER/svn/proj/trunk proj
rm -rf tmp
rm -rf proj.old

See VersionControlSetup for the series of commands used to set up the MTEval archive.

You can accomplish virtually anything you can imagine with the repository, although it may not be immediately obvious how. For instance:

• Get the latest revision number of your current branch: svn info -r HEAD. You can also try svn log | head -2
• View the logs of all changes to your current branch since you last updated: svn log -r HEAD:BASE
• View all the differences between your working copy and the latest revision of your current branch: svn diff -r HEAD .
• Merge into your current working copy the bug fixes checked in to r475 of the bittorrent branch (even if that's not your working branch): svn merge -r 474:475 file:///fs/clip-mteval/svn/Programs/branches/bittorrent .
• Change your mind when you realize those bug fixes did more harm than good: svn merge -r 475:474 file:///fs/clip-mteval/svn/Programs/branches/bittorrent .

If you get the message:

svn: Working copy path 'path/file' does not exist in repository

it means svn has screwed up. The only way I know to fix it is as follows:

* chmod path/.svn/entries u+w
* edit path/.svn/entries
* locate the entry for file
* locate the setting revision=“0”
* change the setting to something non-0 – the current version of this checkout should work
* save path/.svn/entries
* chmod path/.svn/entries u-w
* pray

All of the information in this tutorial comes directly from the online Subversion book. It is not necessary to read the entire book to get a good idea of how subversion works. In general, if you only have a limited period of time in which to get acquainted, you will get the most mileage out of the following parts of the book:

• Nečíslovaný seznam Chapter 2: Basic Concepts. This is an intro to version control concepts; if you are familiar with the basic ideas, you can probably skip this chapter. Covers concepts such as repositories, working copies, and copy-modify-merge version control.
• Section 3.5: Basic Work Cycle. Covers 90% of what you will need to know about using subversion, in about fifteen minutes. If you don't read anything else, read this section, and you'll be able to get started very quickly.
• Chapter 3: Guided Tour. Includes the above (Section 3.5) as well as some additional details. This is 95% of what you need to know.
• Chapter 4: Branching and Merging. This is the other 5%. If you are working on merging changes into the main branch of code, look here.
• Chapter 9: Subversion Complete Reference. Compact documentation on all of the command-line tools and their options. This isn't something you need to read front-to-back, but rather, where you should look if you can't remember a particular command-line option.

Most likely, this is all you will need to look at to do most of your work. Most of this material comes from Section 3.5.