$Id: //guest/jeffery_g_smith/perforce/utils/sttop4/main/README.txt#9 $ StarTeam to Perforce Converter Release Notes Copyright 1998 Perforce Software. All rights reserved. Based on VSStoP4: Written by James Strickland, April 1998 Maintained by Robert Cowham, since 2000 Updated to support StarTeam conversions: Jeffery G. Smith, MedPlus, Inc. 2004-2005 Latest status - J Smith 2004/12/17: This is a first pass at this. Version 0.1 Summary ======= - Obtain and install a version of Perl for your machine. This script assumes you are running on Windows (as it reads registry settings for date and time). You can download a precompiled binary from http://www.activestate.com/. Note that this script has been mostly tested with Perl 5.8. - Install P4Perl from http://public.perforce.com//guest/tony_smith/perforce/API/Perl/index.html. This is the Perl interface to Perforce and is required because it is significantly faster than executing p4.exe at the command line. The latest versions of P4Perl may work fine, but the following are known to work: http://public.perforce.com/guest/tony_smith/perforce/API/Perl/released/p4perl56-setup-3.4804.exe or http://public.perforce.com/guest/tony_smith/perforce/API/Perl/released/p4perl58-setup-3.4804.exe Just double-click on Windows installer and run it (no real need to download and save). - Install the StarTeam SDK for the version of StarTeam you are using. - Untar or Unzip the converter on a disk large enough to hold all files stored in StarTeam, plus the space required for the "metadata". The directory you put the converter in must not be under the Perforce server root! Also advise not having spaces in the path. - For improved performance, see the tips below (which require extra modules to be installed). - If you are not a superuser in your depot then you will get error messages since the script tries to update date/time and username in changelists and labels - limited to superusers. - Change directory to the directory the converter was installed into and edit the file "config.ini" to setup for your environment. - Recommended practice is to import into an area of your repository such as: //depot/import/vss/projectA/... and then branch afterwards into your new structure for use in Perforce, e.g. //depot/main/projectA/... This gives you all the history (if you follow back branching history) and allows you to restructure things and get a better structure than perhaps you were forced to use in VSS (you can branch different projects to different levels). - Make sure that you do not have a client workspace the same as whatever you specified as p4client in config.ini since it will be overwritten by the script. - Then run "perl convert.pl" (note Performance tips below before doing this). Troubleshooting =============== If you get an error message: ***** Failed to run command: perl mkdepot.pl Then: Look in logfile.log for the last SS.exe command executed. Try and run the same command from the command line. Note that the flag "-vN" is the version that is trying to be extracted. It is also worth trying to get this version out using the VSS GUI. Also make sure that analyze.exe runs without errors. In the worst case you may need to ignore that particular file (or version). You can contact the curator if the above doesn't help, but be warned that some VSS repositories are corrupted in a way that certain files are not retrievable... Performance =========== The script can take a long time (many hours) to run if you have a large StarTeam repository. The script needs to parse the history of StarTeam and then get individual versions of each file out and check them in to Perforce. This by its nature takes time. - For larger installations with large labels, try installing the Perl module DB_File. Running the following command: perl -e "use DB_File;" will give an error message if it is not installed. To install, use PPM (Perl Package Manager from ActiveState - installed with Perl): c:\ppm PPM interactive shell (2.1.5) - type 'help' for available commands. PPM> install DB_File It is highly recommended, that you read the following before attempting to do the conversion. Differences between StarTeam and Perforce ========================================= StarTeam and Perforce both use a client/server architecture where the server maintains the centralized depot. Local files may be mapped from depot files to any desired location in the local filesystem. Perforce uses RCS format files to store revisions, but all metadata is stored in a single, central database. Perforce stores useful information that StarTeam does not. Perforce uses the notion of an atomic change which may consist of many add, edit, and/or delete operations. It is guaranteed that either all of a change is visible, or none of it is. StarTeam does not provide atomic change transactions; when checking in a number of files one is given the option of using the same change description for each, but the fact that the revisions were all logically part of one change is lost. StarTeam branches are generated by creating "views" under the original project. Each file version has a branching version number associated with it similar to RCS versions which can be used to determine the branching point (e.g. 1.2.1.0 is a branch from 1.2, 1.4.2.3.1.2 is a branch from 1.4.2.3, which itself is the second branch from 1.4). Perforce handles branches differently. In Perforce when you wish to create a "branch", you simply copy the files, typically to another directory. The recommended convention, and the convention used by the converter, is to make the second component of the depot pathname the branchname. Thus, when it is time for the main line of development to be released as v7, you would typically do a "p4 integrate //depot/main/... //depot/v7/..." which (since there aren't any files in //depot/v7 currently) would create copies of all files in //depot/main in //depot/v7. To refer to the mainline version of foo.c use //depot/main/foo.c; for the v7 branched copy, use //depot/v7/foo.c This works well because Perforce implements a lazy copy. Archive files are never duplicated; only the metadata changes. Merging operations are recorded in Perforce - if, for example, patches were made to v7 code these patches could be merged back into the main line of development with "p4 integrate //depot/v7/... //depot/main/..." Only files which have changes that have not been previously integrated into main will be integrated, and the integration will be recorded so that next time those changes will not have to be revisited. In StarTeam there is no way of knowing that a revision was the result of a merge aside from noting that in the change description. Labels are commonly used in StarTeam; they are available in Perforce but not usually needed. Perforce uses atomic change transactions, and thus the state of the depot or any part of it is easy to specify. Perforce keeps track of deleted revisions (obviating the need for a special "DELETED" label of some sort) and branches are simply differently named files (labelling branches is optional in StarTeam). Possible Missing/Incorrect Information in StarTeam metadata - no identification of checkins which are logically all part of the same change (identify by similar time + same author and change description) - no identification of branches which are logically all part of the same branch (hopefully identify by label, "floating" label is a branch label) - no identification of merged branches - no consistent method for identifying deleted files (hopefully identify by a label such as "OBSOLETE" or "DELETED") - incorrect file type information ("binary" files with GENERATEDELTA or EXPANDKEYWORDS) Restrictions and limitations ============================ Filenames may not contain ASCII unprintable characters, at signs (@) or hashes (#). Note that the converter changes things like German umlauts in filenames. Userids and labels may not contain spaces or ASCII unprintable characters. Again, these characters will be mapped to underscores. The date format used by StarTeam must be of the format: MM/DD/YY hh:mm:ss AM|PM where each number can be one or two digits. The converter is not regularly tested against old server versions, and thus it is not guaranteed to always work against all older versions. Unless you have a good reason to use an old server version, you should run the converter against the latest version of Perforce. How does the converter work? ============================ The converter works in four phases: Phase I: extract StarTeam metadata Phase II: improve the metadata (compact changes using heuristics) Phase III: execute appropriate StarTeam and p4 commands to construct Perforce database Phase IV: niceties after the fact The StarTeam repository is in a proprietary binary format, however the metadata is available using the StarTeam SDK, and individual revisions can be extracted also using this. The converter marks files as deleted immediately after the checkin of a revision which is marked in StarTeam with a label indicating it should be deleted. When the converter runs it reads configuration values from the file "config". Several values are required; the others have defaults. See the comments in "config" for a description of each option. How do I check the end result, and what does it all mean? ========================================================= If you want to be certain that the revisions stored in Perforce are correct, answer yes to the question "Do you wish to verify the result?" (If you say no and then change your mind, just run "perl verify.pl"). The verification step isn't done by default because it takes a long time, and because there is no portable way to determine whether earlier phases of the converter succeeded. NOTE: the verification step does *not* verify that all files were submitted with the correct file type. On Windows/NT submitting a "binary" file as "text" will cause either a verification failure (due to CRLF -> LF translation) or *no error* despite a truncated file. The latter error can occur because files opened as "text" files on NT consider ^Z (ASCII 26) to be an end of file marker. Thus, when the file is submitted, only bytes up to the first occurence of ^Z will be submitted. The solution for this is to force the file to be submitted as type binary by adding the filename extension to the "type_binary:" line in the config file. You can verify the file type the converter chose by examining the file "metadata/files". You can verify that all changes were successfully submitted by running "p4 changes -s pending"; it should produce no output. Look in the directory "metadata" for files containing a mapping of all StarTeam files and revision numbers to the associated Perforce filename and change number. The sorted, more human readable version is in the file "mapping"; the tabular version is in "mapping.ns" (ns for "not sorted"). Change numbers should not be confused with version numbers. To see the files and versions affected by a particular change, type p4 describe -s (Without the -s all differences between files involved in the change are displayed as well). To see the list of all changes (you may want to pipe this through more): p4 changes To see the list of labels: p4 labels To see the list of changes affecting a particular file: p4 filelog filename To list revisions associated with the label "labelname": p4 files @labelname A note about re-running the converter ===================================== If you want to re-run the converter it would be best to erase the directory tree "data", so that there cannot be any conflicts between runs. If you don't want to keep the results of the previous conversion, delete all files in the server root and directories below it. originally by James Strickland (james@perforce.com) Modified by Robert Cowham (robert@vaccaperna.co.uk) Modified by Peter Steiner (peter.steiner@hugwi.ch) Modified by Jeffery G. Smith (jsmith@medplus.com)