PerforceToSubversionReplication

From Eigenpedia

Jump to: navigation, search

Contents

Overview

This page describes some experimental support for replicating the depot from perforce.eigenbase.org to Subversion repositories.

Goals:

  • Make code and history stored in the Eigenbase depot accessible for read-only access via Subversion clients
  • Provide some fault tolerance for cases where there is Eigenbase server downtime (not currently achieved since the Subversion server is running on the same machine as Perforce)

Non-goals:

  • Support for bidirectional replication is not planned. Perforce will remain the only read/write depot, with Subversion instances as read-only replicas.

Current State

Public-access replication has been set up on perforce.eigenbase.org. You can access it via svn command line as:

# Check out Mondrian
svn co svn://perforce.eigenbase.org/trunk/open/mondrian 
# Check out Eigenbase platform development tree
svn co svn://perforce.eigenbase.org/trunk/open/dev

You can also browse it, e.g. browse Eigenbase platform development tree.

Incremental replication runs via crontab every 5 minutes, so that is the maximum lag you should see between when a change gets checked into Perforce and when it appears in Subversion.

Tools

Replication is based on the p42svn Perl script:

http://p42svn.tigris.org/source/browse/*checkout*/p42svn/tags/0.21/p42svn.pl

If you want to set up your own private replication, you first have to

  • download the Perforce C++ and Perl API's from Perforce ftp
  • build and install (as root) p4perl; see the instructions in the release notes
  • install any necessary Perl module dependencies (such as Date::Format and File::MMagic) if you don't have them already; CPAN is your friend

Script

This is the replication script we use:

#!/bin/bash
repo=/home/p42svn/repos
repodir=trunk/open
p4branch=//open

set -e

cd /home/p42svn

if [ ! -e $repo ] ; then
    svnadmin create $repo
    svn_chng=0
else
    svn_chng=`svn propget -r HEAD --revprop p4_eigenchange file://$repo`
fi


while true; do
    start=`/usr/local/bin/p4 changes //open/... | awk '{print $2}' |sort -n |grep -A 1 "^$svn_chng\$" |tail -1`
    end=`/usr/local/bin/p4 changes //open/... | awk '{print $2}' |sort -n |grep -A 50 "^$svn_chng\$" |tail -1`
    if [ "$start" -eq "$svn_chng" ] ;then break;fi
    svnlook tree --full-paths $repo >filelist
    ./p42svn_mache.pl --branch $p4branch=$repodir --changes $start-$end --existing-files filelist --svn-change-prop p4_eigenchange > dump 2> dump.$end.log
    svnadmin load $repo < dump > load.$end.log 2>&1
    svn_chng=`svn propget -r HEAD --revprop p4_eigenchange file://$repo`
    if [ $svn_chng -ne $end ] ; then 
        echo "Import failed for changes above $svn_chng, check the load.log";
        exit 1
    fi
done

Note that the p42svn_mache.pl script includes patches by Nicholae Mihalache, available here.

Personal tools