Gear/Introduction

From ALT Linux Wiki
< Gear
Revision as of 17:44, 8 March 2009 by Ab (talk | contribs)

This is introduction to gear for the people who already have the RPM packaging experience and know how to use git.

Purpose of gear

Gear is a set of tools for maitenance and release management of RPM packages based on git repository. A gear-enabled git repository of a software package combines

  • software development history
  • modifications neccessary to produce an RPM package
  • rules to bind software releases to specific RPM-based distribution releases

Developers using Gear need to understand

  • format of rules file describing transformation procedures that convert a given git changeset and branches to number of source files for RPM package
  • utilities for building packages and working with source code (importing upstream sources and managing patches)

RPM-specific stuff

RPM source packages are built using the following files:

  • .spec file, which is roughly debian/* without debian/patches/*
  • a number source files/archives
  • optionally patches containing local fixes to the upstream source code

Gear gives a way to automate generation of patches and source files off the git repostiory.

Working with upstream source code

Source code of a project is generally tracked in the separate branch that could be synchronized with corresponding upstream repository by traditional git tools. When upstream of the project does not use a VCS that allows direct import into git, source code still can be managed with archives and source RPM packages.

SRPMs

$ gear-srpmimport some-0.1.src.rpm
$ gear-srpmimport some-0.2.src.rpm
...

will import the given SRPMs to the branch ("srpms" by default) in the repository (like git-import-dsc).

Importing SRPM creates git layout similar to the "Like SRPM" scenario described below.

Tarballs

$ gear-update foo-0.2.tar.gz foo
$ gear-update foo-2.0.tar.gz foo
...

will update the subdirectory foo in git repository with contents of tarball. This operation is similar to the git-import-orig from dpkg-buildpackage. As SRPMs may contain several sources, the source code usually stored in a subdirectory of a repository.

Upstream VCS

This should be obvious: just fetch/pull if upstream uses git, or use conversion tools (e.g. svn2git, cvs2git) if it does not yet migrated to git.

Packaging and patching

There are three major scenarios for keeping changes in the repository:

  • Work with changes much like in SRPM-era
  • Keep patches in one branch
  • Keep patches in specialized topic branches

Scenario 1. "Like SRPM"

This simple scenario where git is used just for keeping history, and the usual maintainer's workflow is not changed.

Branches in repository: 

master
upstream

Tree layout: 

foo/                      (in upstream, master)
.gear/rules               (in master)
foo.spec                  (in master)
foo-something-fixed.patch (in master)
foo-another-fixed.patch   (in master)

git repository is generated by importing SRPMs or by creating from scratch.

Importing tarball to the upstream branch

Use master branch to make it a bit easier 

$ mkdir foo
$ cd foo
$ git init foo
$ gear-update ../foo-1.0.tar.gz foo
$ git checkout -b upstream

Add spec and patches

$ vi foo.spec
$ vi foo-something-fixed.patch
$ vi foo-another-fixed.patch

Add gear/rules

The .gear/rules will have the following content: 

copy: foo-something-fixed.patch
copy: foo-another-fixed.patch
tar.gz: foo

This sample rules file describes a source tarball from a foo directory and two patches.

Specfile is picked up automatically if there is only .spec in the root directory fo git.

Commiting the stuff

Gear is relying on whole history being recorded by git. This means that any changes you'd like to see in RPM, must be committed to the repository. 

$ git add .   # Commit entire workdir
$ gear-commit # this is wrapper around git-commit which uses the last
              # changelog entry as a commit message.

Scenario 2. "Small fixes"

This is scenario is useful for packages which deviate from upstream with a small number of non-overlapping fixes here and there. This is also easiest scenario to use.

Branches in repository: 

master
upstream

Tree layout: 

foo/        (in upstream, patched in master)
.gear/rules (in master)
foo.spec    (in master)

Working on package

$ git checkout master
$ vim ...     # hack-hack
$ git add ... # add new files if needed (optional)
$ git commit

All the changes are applied just on top of the upstream source code.

.gear/rules

The .gear/rules file will be of the following form: 

tar: v@version@:foo
diff: diff: v@version@:foo foo

This will generate foo.tar, containing upstream source code, taken from the tag v${version}, where version is parsed from the foo.spec, and the diff is containing the difference between the directory foo in v${version} tag and current content of the directory foo.

Gear does not use real git tags, but instead it uses the tags stored in .gear/tags. The reason for that is to keep packages reproduceable over time as tags may move but SHA1 labels recorded at the time of run of gear-update-tag will be able to address original changesets properly.

The gear-update-tag utility writes down SHA1 lables of all tags referenced in package process to .gear/tags. 

$ gear-update-tag --all

Don't forget to commit the changes after updating: 

$ git-commit .gear/tags -m 'tags updated'

Rebasing to new upstream version

$ git merge upstream
... Fix conflicts, update changelog, remove unnecesary patches etc ...
$ gear-commit

That's all! As patches are stored in git branch, it's easy to update to new upstream version: merge will detect most incosistences. As an additional bonus, if upstream uses git as well, patches forwarded upstream and accepted there will not generate conflicts on merge.

New package revision

$ vim ... # hack-hack, update changelog
$ gear-commit

Also simple.

Scenario 3. "Full-blown development"

This is scenario for the packages which need the lot of work downstream (e.g. kernel).

Branches: 

upstream
topic-A
topic-B
...
master

Tree layout: 

foo/        (in upstream, patched in topic-*, master)
.gear/rules (in master)
foo.spec    (in master)

Working on package

All the work is done in topic branches, each is dedicated for some topic: 

$ git checkout topic-A
$ vim ... # hack-hack-hack
$ git commit
$ git checkout topic-B
...

As patches may overlap, some conflict resolution need to be involved. The naive approach would be using the following branching scheme: 

* upstream
|
\--> topic-A
\--> topic-B
\--> master

And then generating patch-per-topic by using .gear/rules. This will not work, as the generated patches will conflict.

Instead, the following scheme is used: 

* upstream
\-> topic-A
    \-> topic-B
        \-> master

And gear-merge(1) utility merges the branches as described in .gear/merge file: 

merge: upstream topic-A
merge: topic-A topic-B
merge: topic-B master

After using this utility (and resolving all conflicts occupied), master branch gets a cumulative change.

.gear/rules

The .gear/rules file will be exactly the same as for previous scenario, so the gear-update-tag(1) is also required: 

tar: v@version@:foo
diff: diff: v@version@:foo foo

Rebasing to new upstream version

$ ... # obtain the new upstream code in upstream branch
$ gear-merge
... Fix conflicts, update changelog, remove unnecessary patches etc...
$ gear-commit

Due to gear-merge, it's only necessary to fix the conflicts once.

New package revision

$ vim ... # hack-hack, commit
$ gear-merge

Builds

$ gear-hsh

Here the .gear/rules is consulted and the following is performed:

  • pkg.tar package is constructed, according to the .gear/rules file
  • hasher is run with the given source package.

There are also another build-commands: gear-rpm (uses rpmbuild instead of hasher) and gear-remote-hsh/gear-remote-rpm, which use remote host for actual building (communicating over SSH).