- <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"><html><head>
- <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
- <meta name="GENERATOR" content="Mozilla/4.05 [en] (X11; U; OSF1 V4.0 alpha) [Netscape]">
- <meta name="Author" content="Jeff A. Bowles"><title>FAQ for integrating Perforce with builds</title></head>
- <body><x-html>
- <h1>
- FAQ for integrating Perforce with your build scripts</h1>
- <p>
- Contributed by Jeff Bowles
- </p><h3>
- Note - the intended audience for this FAQ is a shell/perl hacker who writes
- or maintains local build scripts and wants to integrate Perforce into them.</h3>
- <ol>
- <h5>
- <b>Labels</b></h5>
- <li>
- <a href="#createlabel">How do I create
- a label? What should I include in a label?</a></li>
- <li>
- <a href="#overnightlabels">If I want
- to create a new label for each overnight build, how would I do it?</a></li>
- <li>
- <a href="#labels_vs_changenums">Should I
- use labels or change numbers to track the contents of overnight builds?</a></li>
- <li>
- <a href="#makepatch">So the release
- needs to be remade three months after it went out. What do I do?</a></li>
- <li>
- <a href="#makeofficial">I like this
- overnight build and want to officially designate it as a "release" build.
- What should I do?</a></li>
- <li>
- <a href="#releasebuild">I need to create a
- release build - what's different in that process?</a></li>
- <h5>
- <b>Clients</b></h5>
- <li>
- <a href="#populate_build_area">How do I populate
- the build area from a nightly build script? What are the problems that
- will come up?</a></li>
- <h5>
- Updating depot files from a script</h5>
- <li>
- <a href="#checkin_logs">How do I
- update the depot to include a new log, or to include a new line in a logfile?</a></li>
- <li>
- <a href="#should_I_checkin_logs">Should l check in my log files?
- What files might need to be modified and checked in as part of a build?</a></li>
- <h5>
- Putting a release out</h5>
- <li>
- <a href="#backup_for_release">What should I back
- up when I put a release out?</a></li>
- <li>
- <a href="#escrow">How do I make
- an "escrow" copy of the files (that built the release) for the off-site
- vaults?</a></li>
- </ol>
- <hr>
- <ol>
- <li>
- <a name="createlabel"></a><b>How
- do I create a build label? What should I include in a label?</b></li>
- <ul>
- <li>
- Pick a regular format for the label name, e.g. "<i>bld_xxx_1</i>" where
- '<i>xxx</i>' is the name of the codeline you're building and '<i>1</i>'
- is just a counter, i.e. the first such label. Remember this format - you'll
- need it in the next question!</li>
- <li>
- Make the label itself:</li>
- <p><tt><font color="#990000"> p4 label bld_xxx_1</font></tt>
- </p>
- <p>It'll dump you into an editor, and you can change the default "Views:"
- lines from
- </p>
- <p><tt><font color="#990000"> //depot/...</font></tt>
- </p>
- <p>to look solely at the codeline you want, e.g.
- </p>
- <p><tt><font color="#990000"> //depot/xxx/...</font></tt>
- </p>
- <p>Save/Quit the editor.
- </p>
- <li>
- Populate the label with the file/revision list you want:</li>
- <p><tt><font color="#990000"> p4 labelsync -l bld_xxx_1
- //depot/xxx/...</font></tt>
- </p>
- <p><font color="#000000">or</font>
- </p>
- <p><tt><font color="#990000"> p4 labelsync -l bld_xxx_1
- //depot/...</font></tt>
- </p>
- <p>In this second example, it's okay to include all files in the depot
- - it'll really only include files described in the label spec you edited
- when you ran "<tt><font color="#990000">p4 label</font></tt>".
- <br>
- </p>
- <li>
- <b>Always include all files [that are under source management] that are
- needed to recreate the build</b> . If your build scripts aren't in codeline
- "xxx" you'll want to have the label specification include the directory
- where they are, and "<tt><font color="#990000">p4 labelsync</font></tt>"
- needs to include them.</li>
- </ul>
- <hr>
- <li>
- <a name="overnightlabels"></a><b>If
- I want to create a new label for each overnight build, how would I do it?</b></li>
- <br>
- <ol>
- <li>
- Make sure you're using a regular name for the label, e.g. "bld_xxx_yyy"
- where "xxx" is the codeline name and "yyy" is a counter.</li>
- <li>
- Create the first label in the sequence, which would be "bld_xxx_1". See
- the previous question for that.</li>
- <li>
- In the Unix shell, you would do the following (cut and paste):</li>
- <p><tt><font color="#990000"> CodeLineName=xxx</font></tt>
- <br><tt><font color="#990000"> LabelSpec=tst_${CodeLineName}_</font></tt>
- <br><tt><font color="#990000"> toplabelnumber=`p4 labels |
- grep $LabelSpec | sed "s/^Label $LabelSpec//" | \</font></tt>
- <br><tt><font color="#990000">
- sed "s/ .*//" | sort -r -n | head -1`</font></tt>
- <br><i><tt><font color="#990000"> # at this point, we know
- the most recent label in this codeline and just need</font></tt></i>
- <br><i><tt><font color="#990000"> # to create a new label and
- populate it.</font></tt></i>
- <br><tt><font color="#990000"> TopLabel=$LabelSpec$toplabelnumber</font></tt>
- <br><tt><font color="#990000"> echo Current Label is $LabelSpec$toplabelnumber</font></tt>
- <br><tt><font color="#990000"> if [ "$toplabelnumber" != ""
- ]</font></tt>
- <br><tt><font color="#990000"> then</font></tt>
- <br><tt><font color="#990000">
- newlabelnumber=`expr $toplabelnumber + 1`</font></tt>
- <br><tt><font color="#990000">
- NewLabel=$LabelSpec$newlabelnumber</font></tt>
- <br><tt><font color="#990000">
- echo "New Label is $NewLabel"</font></tt>
- <br>
- <br><tt> <i><font color="#990000">#
- the following two commands are from the previous FAQ item.</font></i></tt>
- <br><tt><font color="#990000">
- <u>p4 label -o -t $TopLabel $NewLabel | p4 label -i</u></font></tt>
- <br><tt><font color="#990000">
- p4 labelsync -l $NewLabel //depot/...</font></tt>
- <br><tt><font color="#990000">
- <i># It's best to obsolete labels that are kinda of old, so we</i></font></tt>
- <br><i><tt><font color="#990000">
- # do it here.</font></tt></i>
- <br><tt><font color="#990000">
- if [ "$toplabelnumber" -gt 10 ]</font></tt>
- <br><tt><font color="#990000">
- then</font></tt>
- <br><tt><font color="#990000">
- obsoletelabelnumber=`expr $toplabelnumber - 10`</font></tt>
- <br><tt><font color="#990000">
- ObsoleteLabel=$LabelSpec$obsoletelabelnumber</font></tt>
- <br><tt><font color="#990000">
- p4 label -d $ObsoleteLabel</font></tt>
- <br><tt><font color="#990000">
- fi</font></tt>
- <br><tt><font color="#990000"> else</font></tt>
- <br><tt><font color="#990000">
- echo "There was no label to match!" 1>&2</font></tt>
- <br><tt><font color="#990000">
- exit 1</font></tt>
- <br><tt><font color="#990000"> fi</font></tt>
- <br>
- </p>
- <li>
- Perl hackers will look at the previous "Bourne shell" example and use it
- as an example of why you don't want to do string processing in "Bourne
- shell". One perl treatment of this follows:
- </li>
- <tt><font color="#990000"> $CodeLineName = "xxx";</font></tt><br>
- <tt><font color="#990000"> $LabelSpec="tst_" . $CodeLineName
- . "_";</font></tt>
- <p><tt><font color="#990000"> @AllLabels = `p4 labels`;</font></tt>
- <br><tt><font color="#990000"> @ThisCodeLineLabels = sort(sortrevlabel
- grep(/$LabelSpec/, @AllLabels));</font></tt>
- <br><tt><font color="#990000"> $TopLabelLine = $ThisCodeLineLabels[0];</font></tt>
- <br><tt><font color="#990000"> $TopLabel = $1 if ($TopLabelLine
- =~ /^Label\s(\S+)\s/);</font></tt>
- <br><tt><font color="#990000"> $TopLabelNumber = $1 if ($TopLabel
- =~ /^\S+_(\d+)$/);</font></tt>
- </p>
- <p><tt><font color="#990000"> print "The last label in this
- codeline is $TopLabel\n";</font></tt>
- <br><tt><font color="#990000"> $NewLabelNumber = $TopLabelNumber
- + 1;</font></tt>
- <br><tt><font color="#990000"> $NewLabel = "$LabelSpec$NewLabelNumber";</font></tt>
- <br><tt><font color="#990000"> print "New Label is $NewLabel\n";</font></tt>
- <br><tt><font color="#990000"> system("<u>p4 label -o -t $TopLabel
- $NewLabel | p4 label -i</u>");</font></tt> <i># in real life, return code would be checked!</i>
- <br><tt><font color="#990000"> system("p4 labelsync -l $NewLabel
- //depot/...");</font></tt>
- <br><tt><font color="#990000"> if ($TopLabelNumber > 10 )</font></tt>
- <br><tt><font color="#990000"> {</font></tt>
- <br><tt><font color="#990000">
- $ObsoleteLabelNumber = $TopLabelNumber - 10;</font></tt>
- <br><tt><font color="#990000">
- $ObsoleteLabel = "$LabelSpec$ObsoleteLabelNumber";</font></tt>
- <br><tt><font color="#990000">
- system("p4 label -d $ObsoleteLabel");</font></tt>
- <br><tt><font color="#990000"> }</font></tt>
- </p>
- <p><tt><font color="#990000"> sub sortrevlabel {</font></tt>
- <br><tt><font color="#990000">
- my($labelnum_a, $labelnum_b) = ();</font></tt>
- <br><tt><font color="#990000">
- $labelnum_a = $1 if ($a =~ /.*_(\d+)/);</font></tt>
- <br><tt><font color="#990000">
- $labelnum_b = $1 if ($b =~ /.*_(\d+)/);</font></tt>
- <br><tt><font color="#990000">
- return ($labelnum_b <=> $labelnum_a);</font></tt>
- <br><tt><font color="#990000">
- }</font></tt>
- </p>
- <li>
- <font color="#000000">The underlined part in the above fragments probably
- should have a "sed" command between the two 'p4 label' commands, to change
- the description for each label.</font></li>
- </ol>
- <hr>
- <li>
- <a name="labels_vs_changenums"></a><b>Should
- I use labels or change numbers to track the contents of overnight builds?</b></li>
- <br>
- <ul>
- <li>
- There's no right answer. The command</li>
- <p><tt><font color="#990000"> p4 changes -m1 //depot/xxx/...</font></tt>
- </p>
- <p>will tell a build script what the top change number for codeline 'xxx'
- (assuming it's mapped there), so it can parse this output and use that
- in build logs and build e-mail. The developer would type:
- </p>
- <p><tt><font color="#990000"> p4 sync @12345</font></tt>
- </p>
- <p>to pull over all files current as of change #12345, but that has a problem:
- it'll pull over all files [that his/her client maps in] as of that revision,
- not just the files for codeline 'xxx'. You can tell them to type:
- </p>
- <p><tt><font color="#990000"> p4 sync //depot/xxx/...@12345</font></tt>
- </p>
- <p>but, by then, you've lost the simplicity that using the change numbers
- might've given you.
- <br>
- </p>
- <li>
- You should probably always use labels for release builds - when you create
- a release you'll often need to include an additional change to one particular
- file that you need to track, and you want the label to include <u>exactly</u>
- the files used to build the release on the build client.</li>
- </ul>
- <hr>
- <li>
- <a name="makepatch"></a><b>So the
- release needs to be remade three months after it went out. What do I do?</b></li>
- <dl>You'll need a slightly-hacked version of your "get the source and do
- a build" script. The hacks are:
- <ul>
- <li>
- It shouldn't use or touch any labels except for the release label you've
- remembered to create when you made the release. Delete the lines in the
- script that create labels or run "p4 labelsync" on them.</li>
- <li>
- Change the line in the script that does the "<tt><font color="#990000">p4
- sync -f ...#have</font></tt>" so that it looks like:</li>
- <p><tt><font color="#990000"> p4 sync -f //depot/xxx/...@labelname</font></tt>
- </p>
- <p>where "xxx" is the name of the codeline, and "labelname" is the release
- label.
- </p>
- <li>
- Bug your system administrators about whether compiler versions, OS releases,
- etc, have changed since the release was created. Listen carefully to their
- answers.</li>
- <li>
- Run the script. It should create the original release faithfully, although
- most compilers embed timestamps within the .exe and .obj files, so you
- might not have byte-for-byte identical files.</li>
- <li>
- Rerun your regressions to gain confidence in the newly-built product.</li>
- </ul>
- </dl>
- <hr>
- <li>
- <a name="makeofficial"></a><b>I like
- this overnight build and want to officially designate it as a "release"
- build. What should I do?</b></li>
- <dl>See the previous step - you just need to run "p4 label" (to create
- a label) and then "p4 labelsync" to the overnight label or change number
- that created the build:
- <p><tt><font color="#990000"> p4 labelsync -l release_label_name
- //depot/xxx/...@bld_xxx_23</font></tt>
- </p><p>or
- </p><p><tt><font color="#990000"> p4 labelsync -l release_label_name
- //depot/xxx/...@12345</font></tt>
- </p><p>The first case is for "overnight build #23 for codeline xxx", if you're
- using that convention for overnight builds; the second case is if you use
- change numbers to track the overnight build contents.
- <br> </p></dl>
- <hr>
- <li>
- <a name="releasebuild"></a><b>I need to create
- a release build - what's different in that process?</b></li>
- <br>
- <dl>This depends on what packages the release for deployment. If you use
- the same build mechanisms for overnight and release builds, you should
- only need to do a few steps:
- <ul>
- <li>
- Run the overnight build script to create the build.</li>
- <li>
- Label the build area that created the release, using "p4 label" and then
- "p4 labelsync". <i>Pick the 'view' for the label carefully, so that it
- doesn't include parts of the depot not included in this release.</i></li>
- <li>
- Run "p4 label" a second time on the new label, and change the status of
- the label to "locked" so that "p4 labelsync" cannot be run against it accidently.</li>
- </ul>
- </dl>
- <hr>
- <li>
- <a name="populate_build_area"></a><b>How do
- I populate the build area for an overnight build? What are the problems
- that will come up?</b></li>
- <br>
- <ul>
- <li>
- If the Perforce client root is C:/p4root, and the codeline is C:/p4root/xxx,
- then you'll need to do 2-3 things to populate the area prior to building:</li>
- <ul>
- <li>
- Make a copy of the scripts that'll do the populating, if there's a chance
- that you'll delete them in the next two steps! (Leave a place for your
- scripts to stand.)</li>
- <li>
- Run the OS command to delete the codeline directory, e.g.</li>
- <p><tt><font color="#990000"> del/s c:\p4root\xxx</font></tt>
- </p>
- <p>it's important to do this if you want a clean build - don't rely on
- incremental compile for overnight compiles unless the compilation process
- takes many hours.
- </p>
- <li>
- Repopulate the source:</li>
- <p><tt><font color="#990000"> p4 sync -f C:/p4root/xxx/...#have</font></tt>
- </p>
- <li>
- Update the source to the most recent revisions:</li>
- <p><tt><font color="#990000"> p4 sync C:/p4root/xxx/...</font></tt>
- </p>
- <li>
- <font color="#000000">Update your build labels (or remember the current
- change numbers) to reflect this new content.</font></li>
- </ul>
- Deleting the entire codeline build tree (and any "obj" or "lib" directories
- if they're not directly under C:\p4root\xxx) is important because old object
- files often can bite you.
- <br>
-
- <li>
- Most problems that come up in this area have to do with obsolete object/library
- files. <u>Entirely removing the source area and repopulating saves you
- from this headache completely.</u></li>
- </ul>
- <hr>
- <li>
- <a name="checkin_logs"></a><b>How
- do I update the depot to include a new log, or to include a new line in
- a logfile?</b></li>
- <br>
- <ul>
- <li>
- To include a new log or edit a file, you should:</li>
- <ul>
- <li>
- Open the file for either "add" or "edit";</li>
- <li>
- Modify the file in whatever ways make sense;</li>
- <li>
- Use "p4 change -o" and "p4 submit -i" to create the submitted change.</li>
- </ul>
- An example of this, in a Unix shell script, follows:
- <p><tt><font color="#990000"> operation=add</font></tt>
- <br><tt><font color="#990000"> filename=newfile.txt</font></tt>
- <br><tt><font color="#990000"> <u>filetype=ltext</u></font></tt>
- <br><tt><font color="#990000"> [ -f $filename ] &&
- operation=edit</font></tt>
- <br><tt><font color="#990000"> description="log of build for
- `date`"</font></tt>
- </p>
- <p><tt><font color="#990000"> p4 $operation -t $filetype $filename</font></tt>
- </p>
- <p><tt><font color="#990000"> echo New information as of `date`
- >> $filename</font></tt>
- <br><tt><font color="#990000"> <i># this point, we need to
- construct the change submission with</i></font></tt>
- <br><i><tt><font color="#990000"> # an accurate description
- in the change text.</font></tt></i>
- <br><tt><font color="#990000"> p4 change -o > xx.tmp</font></tt>
- <br><tt><font color="#990000"> (</font></tt>
- <br><tt><font color="#990000">
- echo "/^Files:/+1,\$v/$filename/d"</font></tt>
- <br><tt><font color="#990000">
- echo "/<enter description here>/s/.*/ $description/"</font></tt>
- <br><tt><font color="#990000">
- echo w</font></tt>
- <br><tt><font color="#990000">
- echo q</font></tt>
- <br><tt><font color="#990000"> ) | ed - xx.tmp</font></tt>
- <br><tt><font color="#990000"> p4 submit -i < xx.tmp</font></tt>
- <br><tt><font color="#990000"> rm xxx.tmp</font></tt>
- </p>
- <p><font color="#000000"><i>Perl</i> hackers won't see substantial gains
- by doing this in <i>perl</i>, and the 'ed' is actually easier from the
- shell script.</font>
- <br>
- </p>
- <li>
- <font color="#000000">The underlined part above is important, if the file
- you're working with is a giant log file, because giant text files should
- include this <u>hint</u> so that the files will be stored in more efficient
- ways.</font></li>
- </ul>
- <hr>
- <li>
- <a name="should_I_checkin_logs"></a><b>Should l check in my
- log files? What files might need to be modified and checked in as part
- of a build?</b></li>
- <br>
- <ul>
- <li>
- There's no good answer for this - log files are usually huge, and can start
- eating space quickly. It's probably best to keep them in a disk area that's
- backed up regularly, but not to check them in if you can avoid it.</li>
- <li>
- If, however, you track overnight builds based on change numbers instead
- of labels, you might want to have a file somewhere in the <i>depot</i>
- that contains one-line entries of the form:</li>
- <br>
- <tt><font color="#990000"> change 12345 - overnight build
- for 07/22/98</font></tt><br>
- <tt><font color="#990000"> change 12366 - overnight build
- for 07/23/98</font></tt><br>
- and so on. That should be modified (appended to) by the build scripts
- using code similar to the example above. <u>It gets really hard to track
- which change corresponded to which nightly build, otherwise.</u><br>
-
- </ul>
- <hr>
- <li>
- <a name="backup_for_release"></a><b>What should
- I back up when I put a release out?</b></li>
- <br>
- <br>
- <ul>
- <li>As a minimum, you'll need to back up the source files and build scripts
- in Perforce that created the product. This makes several assumptions:
- Usually, however, <u>you'll probably just want to back up the entire Perforce
- depot so that you have a complete snapshot of everything</u>, and then
- if you restore from this backup you can refer to current labels such as
- the <i>product release label</i> you made when you released the product.
- (You can also refer to intermediate build labels if you choose, since they'll
- be included in the backup.)
- <i><font color="#000000">The second and third steps are slight overkill
- for many situations, but if you wanna be incredibly paranoid, that's the
- approach to take.</font></i>
- </li>
- <li>
- That the build environment (machine, OS, compilers, etc) are easily recreateable
- from media you have in-hand;</li>
- <li>
- That, if you only back up the top revision of those files, you won't need
- file revision history and the like.</li>
- <li>To make the most comprehensive backup, you'll need to do the following:
- </li>
- <li>
- Checkpoint the Perforce database and save the checkpoint and the subdirectory
- $P4ROOT/depot with all its contents:</li>
- <p><tt><font color="#990000"> cd $P4ROOT</font></tt>
- <br><tt><font color="#990000"> p4d -r
- . -jc</font></tt>
- </p>
- <p><font color="#000000">Note that this is documented in the "FAQ for System
- Administrators" and also on-line in the Perforce manuals.</font>
- </p>
- <li>
- <font color="#000000">Save a copy of the Perforce executables you're using,
- server and client versions.</font></li>
- <li>
- <font color="#000000">Do a full system backup of the build machine on which
- the product was created. This isn't so much to get the source onto media
- as to get compilers, OS, etc.</font></li>
- <li>
- <font color="#000000">Put this all onto a medium you trust, that'll be
- around for a while.</font></li>
- <li>
- <font color="#000000">Write the <i>product release label,</i> the one that
- you created when you built the product, on the media case. Include with
- the case a hard-copy printout of the exact commands you typed to create
- the media, e.g. <i>created using "tar rvf".</i></font></li>
- </ul>
-
- <hr>
- <li>
- <a name="escrow"></a><b>How do
- I make an "escrow" copy of the files (that built the release) for the off-site
- vaults?</b></li>
- <br>
- <br>You can use a label to create an initial fileset that you'll archive.
- It's pretty easy:
- <ul>
- <li>
- Create a label using <tt><font color="#990000">p4 label, <i>escrowlabelname1.0,
- </i></font></tt>that maps in only the directories you want. (You'll edit
- the label specification to either include as many directories as you choose,
- or perhaps the exact file list itself.)</li>
- <li>
- Populate the label, using:</li>
- <p><tt><font color="#990000">
- p4 labelsync -l <i>escrowlabelname1.0</i> //buildclientname/...#have</font></tt>
- </p>
- <p>or,
- </p>
- <p><tt><font color="#990000">
- p4 labelsync -l <i>escrowlabelname1.0 </i> @release_source_label</font></tt>
- </p>
- <p>The first example relies on the build client having the exact files,
- at this moment, that it did when it created the build; the second example
- relies on having a label of all the source files that were built to create
- the release.
- </p>
- <li>
- Create a new Perforce client that maps its client area to an empty part
- of the disk, and on that new client run the command:</li>
- <br>
- <tt><font color="#990000">
- p4 sync @<i>escrowlabelname1.0</i></font></tt>
- <li>
- Back up this new area.</li>
- <li>
- Run <tt><font color="#990000">p4 label <i>escrowlabel1.0</i> </font></tt>
- and mark the label "locked". This will prevent accidently updates of that
- label.</li>
- <li>
- When release 2.0 goes out and you need to do this again, follow the same
- steps. When you create the label, however, use the original (release 1.0)
- escrowlabel as a template, using<tt><font color="#990000"> p4
- label -t <i>escrowlabel1</i>.0 </font></tt> escrowlabel2.0 .</li>
- </ul>
- </ol>
- </x-html><x-html>
- <i>Last modified on 7 November 2003.</i>
- </x-html></body></html>
# | Change | User | Description | Committed | |
---|---|---|---|---|---|
#1 | 5415 | michael | Move user FAQ to //guest branch and provide pointers. Remove old branching FAQ. Update p...age to reference Perforce Software documentation. « |
19 years ago | |
//public/perforce/faq/build.html | |||||
#7 | 3870 | Jeff Bowles | Cleaning up formatting. | 21 years ago | |
#6 | 3868 | Jeff Bowles | cleaning up a lot of bullet lists. | 21 years ago | |
#5 | 1959 | Jeff Bowles | Minor hacks to clean up links. | 23 years ago | |
#4 | 112 | Laura Wingerd |
Publish "Branching FAQ" (pull in changes from //guest/laura_wingerd/perforce/faq/... ) |
26 years ago | |
#3 | 27 | Jeff Bowles | Adding important comment. | 26 years ago | |
#2 | 20 | Jeff Bowles | Cleaning up numbered lists. | 26 years ago | |
#1 | 1 | laura |
Add FAQs. (Still needs index page.) |
26 years ago |