mariatuplayground/ Morph/ story

Baserock development: a bedside story

Assumptions:

  • A native development system is installed: Baserock core and development strata, and whatever else is considered essential for development. (Until Baserock is bootstrapped, we can just pretend whatever Linux system is native.)
  • Chunk morphologies are kept with the upstream projects they are for. Initially, the upstream project is branched and the morphology is added, and this branch is rebased with any new upstream developments. Eventually, we hope that at least some upstreams will include the morphologies in their upstream code.
  • All other morphologies (strata, systems) are kept in the same dedicated git repo. Having them all in different repos would be too hard to deal with for users.
  • We want to only ever clone the git repositories we actively work on, not everything. For example, if working on busybox, there should be no need to clone (never mind build) the gcc repo.
  • The morph command can work on all cloned repositories at once, but only when they're under the same parent directory.

Configure morph (intended to be similar to what git does):

$ morph config --global cache=file://$HOME/.cache/morph/binaries
... where built binaries (chunks, strata, system images) are stored
... some day this will be a suitable server
$ morph config --global git-base-url=git://git.gitorious/baserock/
... where git repositories are looked for by default
... git-base-url is for resolving relative URLs
... absolute URLs may always be specified

Checkout a Baserock base system and rebuild it:

$ cd $HOME/baserock
... this is the directory where we'll put all repos we work on
$ morph clone morphs
... this clones git://git.gitorous.org/baserock/morphs
$ cd morphs
$ morph build base-system.morph

The system image gets put into the cache. FIXME: There needs to be convenient ways of getting stuff out of a cache.

Test built system in an emulator:

$ morph qemu base-system.morph
... this will use the system image built for base.morph
... the image could also be tested on an actual device, by
... getting it out from the cache and installing onto the device
... (in whatever manner is suitable for that specific device)

Next, fix a bug in an upstream project.

Making changes is complicated by the fact that many projects need to be touched: the actual upstream project and the various morphologies, which may be distributed in several git repos. In order to make a change in, say, busybox, you need to create a branch for that change, and then change the stratum morph to refer to that branch instead of the default one, and then percolate the change up to the system image. Thus, we must "fork" all the relevant parts of the stack.

$ cd $HOME/baserock/morphs
$ git checkout master
... base the changes on the current master branch
$ morph fork --fork-name=baserock/fix-bug-12765 busybox
... this git clones any named repos, if necessary, to the parent directory
... of the current repository (i.e., busybox goes into ../busybox)
... it also creates and checkouts a new branch in each repo
... it tries to be safe: no uncommitted changes, etc
... also changes the all morphs in the morphs repository so that 
... they build the forked busybox
... if necessary, the branch may be manually adjusted (e.g., rebased)

Now make some actual changes.

$ cd ../busybox
$ $EDITOR miscutils/time.c && make && make check
... user makes the necessary changes and tests them locally
... e.g., they might run an upstream test suite
$ git commit -m "Fix bug 12765 by doing foo to bar."
... commit the changes so that they can be built by morph
$ morph build ../morphs/base.morph
... build the new system image
$ morph qemu ../morphs/base.morph
... run the new system image in an emulator
... manually verify that changes actually fix the problem
$ morph system-test ../morphs/base.morph --test-system=qemu
... run automatic system integration tests using an emulator
... with suitable options, this should be able to run the tests against
... a real device, after the user has manually installed the new system
... image on the device
$ morph install ../morphs/base.morph
... install the new stratum onto the current system
... manually test the bug fix
... might be done on an actual device instead of the development system
$ morph rollback base
... revert any changes to the stratum on the current system

$ cd ../morphs
$ morph patch --old-branch=master | less
... this generates diffs from all the upstream repos affected by the change
... i.e., the busybox one
... repos can also be specified explicitly
... this does not include changes to the morphologies outside morphs repo
$ morph patch --old-branch=master --send=patches@busybox.org
... send patch upstream
$ morph push
... push changes to the git server
... affects both the morphs repo and all other affected repos

FIXME: Figure out what patch submission formats are needed. For example, patch series?

However, sometimes it takes a while for upstreams to incorporate patches. Meanwhile, we'll want to continue development as if they already had. For this, we'll want to have some branch management.

See repositories for a discussion on branch management.

In order to use the changes done in the branches created by morph fork, we need to change the master branch of morphs to use them.

$ git checkout master
$ $EDITOR base.morph
... add the busybox baserock/bug-12765 feature branch to the 
... branches to use for integration

Make a release candidate of Baserock:

$ cd $HOME/baserock/morphs
$ morph release-candidate --release-name=1.23.4.7.1
... create a baserock/petrifying/1.23.4.7.1 branch
... create a similar branch in all constituent projects
... update morphologies to refer to those branches

Make an actual release:

$ morph release --release-name=1.23.4.7.1
... create branch baserock/petrified/1.23.4.7.1
... convert all branch references in morphologies to absolute SHA-1 ids