<?xml version="1.0" encoding="ISO-8859-1" ?>
<?xml-stylesheet type="text/xsl" href="../xsl/road-faq.xsl"?>
<content>
<topic title="Perforce Admin FAQ" file="p4-faq-admin.xml" fileid="$Id: //main/2005/road/web/xml/p4-faq-admin.xml#2 $$Change: 540 $" fileChange="$DateTime: 2005/10/22 20:18:53 $$Author: bbarber $">
Mostly written 2002-2003. Please report errors to <a href="mailto:bradb@shore.net">bradb@shore.net</a>
</topic>
<section id="p4-setup" title="Perforce Setup" order="sorted">
<item id="codeline" title="Codelines for Perforce">
<p>The codeline is the foundation for Perforce. </p>
<ul><li>
A <i>codeline</i> is a set of files treated as a unit, for example, the source files for Release 6.2 of a product.
</li><li>
A <i>codeline diagram</i> is a diagram of the branch history with important events identified. Typically, the mainline is straight, version branches are
above the mainline while development branches are below the mainline. A sync merge is a dashed line (all revisions merged), while partial merges are
dotted lines. A codeline diagram helps keep developers organized, especially when working on multiple codelines simultaneously.
</li><li>
A <i>codeline policy</i> defines who can change a codeline and other conventions about writing and maintaining the files.
</li><li>
A <i>branch</i> is a variant of a codeline. For example, a <i>version branch</i> is the released version of a codeline. Create a branch when development needs to
occur in parallel, usually with different codeline policies.
</li><li>
A <i>file revision</i> is a revision of a file (e.g., README.txt#2). Perforce keeps file revisions forever. Most Perforce commands are linear in the number of file revisions selected.
</li></ul>
<p>In Perforce, codelines are simply naming conventions. Once set up, these conventions are difficult or impossible to change. So it is worth some thought ahead of time.
The goal is to categorize all source files into codelines, while keeping the number of file revisions low. From trial-and-error, the following organization should work well.</p>
<ul><li>
//codeline-type/branch-name/depot-name/project-name/... -- The <i>codeline-type</i> identifies the type of a codeline (see following for examples). The <i>branch-name</i> identifies the codeline branch. The <i>depot-name</i> identifies files owned by an organization, for example, 'QA' or ' product'. The <i>project-name</i> identifies a specific project.
</li><li>
In small organizations, consider dropping the depot-name.
ATG placed the depot-name first (e.g., //QA/Domino/...). While this convention organizes the files for an individual group, it can spread files for the same codeline throughout the hierarchy. Performance suffered as the number of branches and depots increased.
</li><li>
//main/2005/... -- The <i>mainline</i> holds the ongoing path of development. It is rebranched each year. This allows reorganization while limiting the number of file revisions.
If you do not branch main, the total number of file revisions steadily increases. Performance suffers when there's more than half a million file revisions.
Branching main helps identify out-of-date clients. It forces developers to rethink their clients once a year. It allows out-of-date projects to quietly go away.
</li><li>
//version/Domino_1.2/... -- Version codelines should be clearly idenfied by a unique name and version number. A version codeline should be branched prior to deployment. Avoid branching too early. As a rule of thumb do not branch if most developers are still working on the release. Use version codelines for ongoing patches.
</li><li>
//dev/eisbock/... -- A <i>development codeline</i> is a temporary codeline for individual developers or small groups. New projects typically start as a development codeline.
</li><li>
//final/Domino_1.2/... -- A final codeline contains a snapshot of the files as of a release. It is equivalent to a label, but easier to explore and more visible.
</li><li>
//archive/main/2005/... -- The <i>archive</i> consists of codeline fragments that are no longer needed. To keep these fragments accessible, branch them to //archive. Whole projects are archived by branching //main for the following year.
</li><li>
//user/bbarber/... -- Each user has a codeline to organize as desired.
</li><li>
.../_etc/specs/... -- Keep non-source files in an <i>_etc</i> directory. It appears first in the directory hierarchy, while distinguising source from non-source files.
</li></ul>
</item>
<item id="newdepot" title="How to create a depot">
<p>To create the <tt>xyz</tt> depot</p>
<ol>
<li>You need superuser permission.</li>
<li><tt>p4 depot xyz</tt></li>
<li>Enter a description -- the first line goes to <tt>p4
depots</tt></li>
<li>Change the mapping -- prefix "depots/" (e.g., "Map:
depots/road/..."). This places all files into the subdirectory, Perforce/depots/.</li>
<li><tt>p4 protect</tt> -- add entries for //xyz/...</li>
<li>Let the owner know that a file must be added before //xyz will
appear.</li>
</ol>
<p>If you get an error like "Mapping '//customer/...' is not under
'//road-disk/...'", Try creating the default client first and then
edit it.</p>
<p>Perforce stores text files as RCS ",v" files in
<tt> /work/perforce/depots</tt></p>
<p>Perforce stores binary files as compressed files (".tgz") in ",d"
directories. All files are owed by 'perforce' with group
'perforce'. Under Windows, all files and directories are lower-case.
</p>
</item>
<item id="newproject" title="How to create a new project or branch">
<p>Projects and branches should be controlled by 'p4 protect'. Otherwise, a misspelling can permanently create a new project or branch. On Windows, incorrect capitalization can have the same effect.</p>
<p>To create the <tt>xyz</tt> project or branch</p>
<ul><li>
You need superuser permission
</li><li>
If appropriate, create a protection group named 'xyz'.
Set MaxResults to 5000 -- this can be increased if needed.
</li><li>
<tt>p4 protect</tt> -- add entries for xyz/...
</li><li>
Let the owner know that a file must be added before xyz/ will
appear.
</li></ul>
</item>
<item id="newp4t" title="How to create a remote Perforce server">
<p>See</p>
<ul>
<li><a href="#hluser">How to create</a> a user on harry-lime</li>
<li>Consider using p4proxy instead. It avoids the headaches of a remote server.</li>
</ul>
<ol>
<li>Create a clean copy of the Perforce databases. Redo if the
changelist number changes.
<pre>
p4 changes -m 1; cp $p4p4/db.user $p4p4/db.group $p4p4/db.protect $p4p4/p4d.* $p4p4/README . ; p4 changes -m 1
</pre></li>
<li>Copy the files over to Perforce.
<p>scp * bug-ex:/work/perforce</p>
</li>
<li>Check in the Perforce license
</li>
<li>Install the latest server from Perforce</li>
</ol>
</item>
<item id="license" title="How to install a new Perforce license">
<p>To install the license file on each
server, do the following:</p>
<ul>
<li>Find the license file for each server; the license file should
have the server's IP address within the "IPAddress" field.</li>
<li>Log into the server, and cd into the /work/perforce
directory.</li>
<li>Su to the perforce user.</li>
<li>Move the old license file to license.date:<br/>
<tt>mv license license.9.10.01</tt></li>
<li>Copy the new license file to the file
/work/perforce/license.</li>
<li>Exit out of the perforce user shell.</li>
<li>Restart the server to enable the new license:<br/>
<tt>sudo /etc/init.d/p4d stop<br/>
sudo /etc/init.d/p4d start</tt></li>
<li>Run p4 info to make sure that the new license file is being
read.</li>
</ul>
</item>
<item id="clientup" title="How to upgrade Perforce clients and documentation">
<p>Perforce upgrades its server and clients twice a year.
New clients can run with old servers. To upgrade the clients and
user documentation</p>
<ol>
<li>Download the new installers and clients and ask IT to add them
to cmb-files/installers. It is best to place copies in your Unix
home directory. Consider creating a custom installer with preset
options [see perforce.zip on the Perforce FTP site]. It includes an
example configuration file that is optimized for performance.</li>
<li>Download the user, admin, and cmdref guides. Do <i>not</i>
download the plugsin or p4web guides.</li>
<li>Open the corresponding web directories for edit
[perforce/cmdref,p4guide,p4sag,p4win].</li>
<li>For each directory, cd into the directory and 'unzip -a -o
~/cmdref.zip'.</li>
<li>Diff the file contents for cmdref. Place any new files in
'//road/web/main/perforce/p4_cmd.jhtml'. It will be sync'd to
cmdref/index.jhtml. Check the link for Keywords to "Keyword
Expansion" in o.ftypes.html.</li>
<li>Revert cmdref/index.html. It is a symlink to
cmdref/index.jhtml</li>
<li>Revert unchanged files (p4 revert -a ...) and add new
ones.</li>
<li>Update the 021 version numbers in the perforce.jhtml doc links
(e.g., http://www.perforce.com/perforce/doc.021/manuals/)</li>
</ol>
</item>
<item id="svrup" title="How to upgrade a Perforce server">
<p>The Perforce documentation has <a href=
"http://www.perforce.com/perforce/doc.021/manuals/p4sag/01_install.html#1051322">
detailed instructions</a> for upgrading a server. Here are the steps we
use at ATG [rjohnson, Michael Shields, bbarber]:</p>
<ol>
<li>Complete your acceptance testing.</li>
<li>Modify scripts and procedures for changes in functionality as
described in the release notes.</li>
<li>Delete abandoned clients and review <a href=
"#periodic">Periodic Maintenance</a>
<p>Periodic deleting of the abandoned clients is certainly
something that should be suggested for a site such as yours. In
general, deleting abandoned clients will reduce the size of your
db.have table, which can make for more efficient access.</p>
</li>
<li>Download new the p4d from <a href=
"http://www.perforce.com/perforce/loadprog.html">the Perforce
download page</a>.</li>
<li><a href=
"http://www.perforce.com/perforce/doc.021/manuals/p4sag/03_superuser.html#1046957">
Verify</a> the depots before upgrading (run as perforce user). This
step checks that the recorded MD5 for each revision matches the
archived files. It may be interrupted and restarted later. [cbb
5/02]:<br/>
<p>super user bbarber * //...<br/>
p4 verify -qu //... >& verify.log<br/>
<br/>
Note: the verify run will take 4-5 hours on geezer //... [5/02]
harry-lime verify</p>
</li>
<li>Shutdown Perforce (run as root user):<br/>
<tt>/etc/init.d/p4d stop</tt></li>
<li>Run a checkpoint on the server (use the standard cron script to
do this since it has everything already set up):<br/>
<tt>su perforce<br/>
/work/perforce/p4d.crondaily</tt></li>
<li>Make a backup of the Perforce dir, just in case:<br/>
<tt>cd /work; /road/local/bin/tar -zcf perforce_bak.tgz
perforce</tt></li>
<li>Copy new p4d into /work/perforce, and set execute
permissions:<br/>
<tt>mv /work/perforce/p4d /work/perforce/p4d.old<br/>
cp /home/rjohnson/newperforce/p4d /work/perforce<br/>
chmod 755 /work/perforce/p4d</tt></li>
<li>Run upgrade:<br/>
<tt>cd /work/perforce<br/>
./p4d -xu</tt></li>
<li>Restart p4d:<br/>
<tt>su root<br/>
/etc/init.d/p4d start</tt></li>
<li>Run post-upgrade verification:<br/>
<tt>p4 verify -q //...</tt><br/>
Note: this verify run will also take 4-5 hours on geezer //...
<p>Remove your super user access.</p>
</li>
<li>Update the Solaris i486 client on atlas and the solaris clients
in /road/local/bin</li>
</ol>
<p>NOTE: the upgrade will require about three times as much disk
space as all of the current db.* files added together. Once the
upgrade is finished, the db.* files should be a bit smaller. The
total upgrade time for geezer is around 12 hours.</p>
</item>
</section>
<section id="Admin" title="Perforce Admin">
<item id="deluser" title="How to delete a user or clientspec">
<p>To delete a user, first delete the user's clients.</p>
<p>To reinstate a user, recreate the user with the same userid.
Perforce uses the userid to indicate the user (e.g., change
lists).</p>
<p>To delete a user:</p>
<ol>
<li>Use 'p4 clients | grep userid' to list the clients named for
that user. This will not list the clients owned by a user but named
something else.</li>
<li>Use 'p4 opened -a | grep userid' to list opened files.
<p>These files may contain valuable edits on the user's
workstation.</p>
<p>If the opened files are assigned a changelist, use 'p4 change -o
nnn' to print out the changelist. It may indicate the purpose of
the change.</p>
<p>Notify the manager and ask them to submit the files if needed.
Do not delete the client until the manager is done.</p>
<p>If the manager does not want the change and the edits are
assigned to a changelist, use 'p4 change -f -d nnn' to delete the
changelist. You need 'open' or 'edit' permission on these
files.</p>
</li>
<li>Use 'p4 client -d client_name' to delete clients without opened
files. Use 'p4 client -d -f client_name' to delete a client with
opened files.</li>
<li>Use 'p4 opened -a | grep userid' and check for opened
changelists. Delete with 'p4 change -d -f ...'</li>
<li>Use 'p4 groups userid' to list the group memberships. Remove
the name for the corresponding protections/group/ file.</li>
<li>Use 'p4 user -d -f ...' to delete the user. If any clients
remain, it will report an error.</li>
</ol>
</item>
<item id="reorgdepot" title="How to reorganize a depot">
<p>You should use 'p4 rename' if possible. An alternative
is to directly edit the checkpoint file. The main problem is that
it is easy to change too much or too little and end up with
garbage.</p>
<p>See also <iref file="p4-faq.xml" item="reorg-codeline" title="How to reorganize a
codeline."/></p>
<pre>
Paul Goffin PGoffin@baltimore.com
Mon, 16 Aug 1999 16:36:42 +0100
http://maillist.perforce.com/pipermail/perforce-user/1999-August/002864.html
> 1. Stop Perforce.
> 2. Take a checkpoint.
> 3. Run sed on the checkpoint file (it's a plain-text file, one
> database record per line).
> 4. Rebuild the database files from the checkpoint.
> 5. Move your depot directory so it has the correct name.
> 6. Restart Perforce.
>
> The hard part is step 3: performing the correct transformation on the
> checkpoint file. I don't know whether the web site or FTP site has a
> collection of suitable sed/awk/perl scripts (for applying common
> transformations on a repository) but it's something they could
> consider.
I'd use vi (or ed):
1,$s/\/\/DEPOT/\/\/AUDIO/g
:wq
Or, these days, I often use MS dev studio for this sort of
thing. Then I just do a find/replace of //DEPOT for //AUDIO
There is one more thing of concern. If the "DEPOT" is
actually the default depot "depot", then it may not
have been "created". So you'll need to add the following
line to the checkpoint:
@pv@ 0 @db.depot@ @AUDIO@ 0 @subdir@ @AUDIO/...@
(use your editor to search for "db.depot" if its not there, you'll
need the above. If it is there, it will have been
missed by my search & replace as the db.depot entry is the
only one that doesn't use the "//" prefix, so it'll
need to be changed anyway)
Paul
Paul D Goffin, Technical Leader, Process & Certification, Baltimore
Technologies Ltd.
Fredric Fredricson fredric.fredriksson@mydata.se
Wed, 23 Sep 1998 22:37:04 +0200
http://maillist.perforce.com/pipermail/perforce-user/1998-September/001400.html
There is a small problem with the solution suggested
in the technote:
The technote suggests a simple sed script to modify
the path in the depot. This means that if a change
description contains a file name in depot format
this will also be changed.
This *may* cause p4d to fail to read the modified
checkpoint file.
Paul Goffin PGoffin@baltimore.com
Thu, 10 Feb 2000 09:57:42 -0000
http://maillist.perforce.com/pipermail/perforce-user/2000-February/003701.html
If it's a real big rearangement, you
might want to consider the UNSUPPORTED
mechanism of directly editing the
metadata and moving the archive files around.
I do this sort of thing quite often - and
we don't have any problems from doing it.
But then, I'm VERY careful (I write
scripts to do the editing and then print
them out and go over them "with
a fine tooth comb" before actually
commiting the changes.)
Anyway, from the now defunct TechNote 20 (I expect because
users screwed this up too often):
"That way you move your history too.
To preserve all history but avoid an integration record
By editing a checkpoint and recovering from the checkpoint, you can preserve
your entire depot, but you won't have an integration record and you won't
need to save the old tree.
You'd probably use this method if you are about to go production with your
Perforce system, already have lots of changes you want to save, but realize
you'd like to change the structure.
Assume /src1/perforce is where you've installed p4d. (P4ROOT).
Take a checkpoint of your current depot.
p4d -r /src1/perforce -jc
Edit the checkpoint file and change all the places where the file says
//depot/file.c to //depot/main/file.c
You could write a sed script to do this if you wanted:
sed 's?//depot/core?//depot/main?g' new.checkpoint
Move the rcs files from under the /src1/perforce/depot/core to
/src1/perforce/depot/main.
Recover from your checkpoint:
rm /src1/perforce/db.*
p4d -jr new.checkpoint
</pre>
</item>
<item id="qcheck" title="How to checkpoint one table">
<p>Perforce maintains separate tables for users, have
lists, changes, etc. There is a corresponding "db.*" file for each
table. A checkpoint lists each row of the table with @pv@ [Put
Value] and the table's revision number.</p>
<p>To produce a checkpoint for selected tables, follow Wally
Dutchess's suggestion:</p>
<blockquote>Copy the db.* files to a temp root directory, remove
all db. files except the ones you want, then perform a checkpoint
in the temp root -- bingo, a lowfat checkpoint.</blockquote>
</item>
<item id="maxresults" title="How to increase MaxResults and MaxScanRows">
<p>MaxResults and MaxScanRows should be large enough to
check out source files without sycning individual directories.</p>
<p>To increase the default value of MaxResults and MaxScanRows</p>
<ol>
<li>Edit the default value in p4d-watch-admin.pl
<p>cd geezer:/road/perforce/triggers<br/>
p4 edit p4d-watch-admin.pl<br/>
...modify $::maxResults or $::maxScanRows<br/>
p4 submit</p>
</li>
<li>Restart the daemon
<p>cd /etc/init.d<br/>
sudo p4d-watch.d stop<br/>
sudo p4d-watch.d start</p>
</li>
<li>Submit all of the groups
<p>p4 edit //road/Perforce/main/protections/group/...<br/>
p4 submit</p>
</li>
<li>Check the result
<p>p4 group -o it</p>
</li>
</ol>
</item>
<item id="triggers" title="How to write triggers">
<p>For introduction see Perforce's
<a href="http://www.perforce.com/perforce/technotes/note028.html">Tech Note 28</a>.
and <a href="p4sag/06_scripting.html">System Admin: Triggers</a>.</p>
</item>
<item id="periodic" title="Periodic maintenance">
<p>The Perforce databases need cleanup from time to time.
Here are some items:</p>
<ul>
<li><b>p4 typemap</b> -- The file
//road/Perforce/main/docs/typemap.txt records the output from 'p4
typemap'. It is the association map between file extensions and
Perforce types. It identifies binary files and ktext files.</li>
<li><b>p4 labels | grep \'Build</b> -- Build labels are not deleted
and occupy a lot of disk space. Be careful of deleting more than
150 labels at a time; the journal file may fill up disk on /var.
Allow time between deletions, otherwise Perforce becomes
unavailable. Even a sleep of 5 seconds makes a big difference.
<p>df<br/>
sleep 30;p4 label -d -f eisbock-3338</p>
</li>
<li><b>p4 changes -s pending</b> -- particularly slow if someone
has many files open [cf. bug 41890]. Need a script to send e-mail
to old changelist owners.</li>
<li>disk cleanup -- delete unneeded files from
/work/perforce/depots/... Delete old checkpoints and
journals from /work/perforce. Once
checkpoints are one tape, they can be safely deleted. Old binary
files may be deleted from the depot.
<p>Delete all 300-day or older files: find . -mtime +300 -atime
+300 -type f -exec sudo rm {} \;</p>
</li>
<li><b>p4 opened -a | sort +7</b> -- non-existent users may have
opened files. A Perforce superuser should delete these clients with
<b>p4 client -d</b>. Also delete the
corresponding changelists with <b>p4 change -d -f nnn</b>.
<b>Note:</b> double-check your commands. There is no recovery.
<p>p4 opened -a | sed 's/\(.*\) by \(.*\)/\2 \1/' | sort >
opened.txt</p>
</li>
<li><b>obsolete clients</b> -- Send e-mail to everyone with old
clients. If necessary, delete old clients that are inactive and do
not have opened files. Let users know that they should remove the
Host field before deleting clients on other hosts.
<p>Be careful. Use 'p4 -ztag' instead of 'p4 clients'. The later
reports client updates while the former reports syncs. Look at
clients before deleting them and check for opened files. The client
may be active even though it has opened files on a retired
codeline.</p>
<p>p4 -ztag clients | egrep '\.\.\. (client|Access|Owner)' >
clients.txt<br/>
TextPad Search: ... client \([-.0-9a-zA-Z_]*\)\n... Access
\([0-9]+\)\n... Owner \(-_[a-zA-Z]+\)<br/>
TextPad Replace: \3 \2 \1<br/>
sort +2 -n clients.txt</p>
<p>There should be a client report that gets e-mailed to each user.
It can show clients sorted by date with instructions on how to
delete clients on hosts that no longer exist.</p>
</li>
<li><b>unaligned clients</b> -- Send e-mail to everyone with
unaligned clients. The compute phase for 'p4 sync' is slower if the
client is unaligned. It will block other users.
<p>From Michael Shields of Perforce, <a href=
"p4-docs/mapstate.htm">FAQ on mapstate</a></p>
<pre>
> > 21. Run the mapstate0.sh script. You'll find the mapstate0.sh script in the
> public depot at //guest/michael_shields/scripts/mapstate0.sh.
./mapstate0.sh | sed 's/.* [0-9]* //' | sort >mapstate.txt
Attempt to set mapstate for each of these clients
p4 -c road-nibbler-web fstat //user/xyz
...
Rerun mapstate0.sh
> > 22. Ask users with clients in the mapstate0.sh output to:
> > a. p4 sync #have (beware of opened files)
> > b. p4 fstat //depot/foo (can be //depot/foo, even if it doesn't exist)
The above steps identify which clients are misaligned with respect to the
client's view and it's have list. Users with clients in the resulting list
should (and would want to) align their client's view and it's have list. Step 22
a. will align the client's view and it's have list (provided that none of the
misaligned files are open), and step 22 b. will set the MapState flag now that
the client is aligned.
The mapstate0.sh script may need some customization for your environment: this
script is written to use an awk that understands strftime(), such as gawk. If
your awk does not understand strftime(), you'll need to remove the final pipe to
awk.
</pre>
<p>How many files? Only worry about</p>
<pre>
^\([^ ]+\) \([^ ]*\) ... MapState 0
echo \1 \2 `p4 -c \2 files //\2/...#have | wc -l`
</pre>
<p>Suggested e-mail:</p>
<pre>
Please realign your misaligned clients. See the list below. Jeanne, please
check the qa-script clients. Mary, please check the techsupport clients.
To realign client 'clientname'
Go to the client's host machine
p4 -c clientname sync #have
p4 -c clientname fstat //user/xyz
You can drop the "-c clientname" if it is your active client.
Why?
Whenever you edit a clientspec, Perforce's 'have' list of your files may not
match your clientspec. Perforce must scan your 'have' list when you use
depot syntax (e.g., //user/xyz). With 30,000 files, this is a slow operation that
blocks everyone's use of Perforce.
'p4 sync' or 'p4 sync #have' realigns your files with your clientspec.
'p4 fstat //user/xyz' (or any other use of depot syntax) scans your 'have' list
and verifies alignment. For details, see Michael Shield's FAQ on mapstate.
</pre></li>
<li><b>users w/o passwords</b> -- copy db.users to a new directory.
<p>p4d -jd users.txt<br/>
egrep '@@ *$' users.txt</p>
</li>
<li><b>perforce-users@list</b> -- update the e-mail list.
<p>p4 -Ztag users | grep "... Email" | sed 's/... Email //' |
sort<br/>
All lists, show users<br/>
uniq -23 (to delete)<br/>
uniq -13 (to add, remove lionbridge, releng, perforce)</p>
</li>
<li><b>Hostless clients</b>
<p>p4 -ztag clients | egrep '\.\.\. (client|Host|Owner)' >
clients.txt<br/>
TextPad Search:... client \([-.0-9a-zA-Z_]*\)\n... Owner
\([-a-zA-Z0-9]+\)\n... Host \([-.a-zA-X _]*$\)<br/>
TextPad Replace: \2 \1 @\3@<br/>
sort +2 -n clients.txt</p>
</li>
<li><b>Clients with duplicate roots</b> -- some users make good use
of duplicate roots. They require "sync -f". For example cpatti has
one client per codeline. He can quickly switch between codelines
w/o updating all of the data.</li>
<li><b>Unused branches</b> -- You can obliterate entire branches.
This takes a while since it scans all of the database files.
Perforce is unusable in the meantime.
<p>Make a client of everything that you want obliterated.
Double-check it! Then run 'p4 obliterate -y ...' The files and all
record of the branch will go away.</p>
<p>I've requested an option to delete unrev'd files from
branches.</p>
</li>
</ul>
</item>
<item id="monitor" title="How to monitor the Perforce server">
<p>Richard Baum of Perforce (reb@perforce.com) made the
following recomendations in his 2001 User Conference paper
[Solaris]:</p>
<ul>
<li>Use 'ps -ef' or 'ps -axl' to track the processes</li>
<li>Use 'swap -s' to report memory usage.</li>
<li>Use 'vmstat 3' to monitor swap space usage</li>
<li>Use 'dmesg' or 'syslogd' to report log messages</li>
<li>Use 'netstat -i -I 1e0 3' to monitor network usage</li>
</ul>
<p>From perforce/technotes/note040.html, compare the response time
of (1) 'p4 info' and (2) p4win's "Show Connection Info". If both
are slow, you have network problems. If (1) is slow but (2) is
fast, you have DNS problems.</p>
<p>Review the event log, <tt>/home/perforce/logs/p4d</tt>. Notes
from Michael Shields of Perforce (CALL#306631):</p>
<ul>
<li>Does SubmitChange need any db locks?
<p>Yes, it takes locks on a number of db.* tables, but releases the
locks before transferring files from the client to the server. The
general rule is that we avoid network I/O while we have locks on
the db.* files.</p>
<p>It's interesting to note that it is during dm-SubmitChange that
the transferring of files from the client to the server occurs.
Perhaps you were transferring large files or there was a network
anomaly?</p>
</li>
<li>Does Perforce always lock the entire database if a lock is
needed?
<p>There is no such thing as a "database lock". When a command
starts it's compute phase, it locks only the tables that are
needed, and only in the mode (read vs. write) that are needed. Most
commands take read locks on either db.rev or db.have, so multiple
readers can coexist with no problem. The challenge begins when a
writer is introduced. We try to ensure that write locks are taken
and released as quickly as possible. If a write lock is taken on
either db.rev or db.have, the entire metadata is effectively
locked.</p>
</li>
<li>Does your log analyzer give stats for each command, and list
the outliers?
<p>Yes. See
public.perforce.com:1666//guest/michael_shields/src/p4loga. It is a
C++ program compiled with Jam.</p>
</li>
</ul>
<p>See <iref file="p4-faq.xml" item="slow" title="Why is Perforce Slow?"/></p>
</item>
<item id="retire" title="How to retire a codeline">
<ol>
<li>Notify the developers and ask them to check in their
changes.</li>
<li>From a good build, sync the mainline into the branch.</li>
<li>Notify developers about conflicts. If the sync looks worrisome,
set up a meeting to resolve conflicts and create a good build.</li>
<li>After a good build is achieved on the branch, merge the branch
into the mainline.</li>
<li>Check the source for clients with opened files. There should be
no opened files if you are retiring the branch.
<p>p4 opened -a ... | sed -e 's/.* by //' -e 's/@.*//' | sort
-u</p>
</li>
<li>Remove 'write' permission to the codeline.</li>
<li>If users have opened files, give them 'write' permission to
check in their changes, or 'open' permission to revert their
changes.</li>
<li>If the branch should disappear, delete all the files.</li>
</ol>
</item>
<item id="p4scripts" title="How to script Perforce">
<p>Perforce scripting can be tricky since the client does not
consistently handle errors.</p>
<p>The Bugzilla-Perforce interface uses a variety of techniques
including the Perl API. See
//road/Bugzilla/main/bugzilla/p4_diffs.cgi, perforce.pl, perldoc -t
P4, perldoc P4::Client, and perldoc P4::UI.</p>
<ul>
<li>How to hide the Perforce password
<ul>
<li>Set P4USER and P4PASSWD in the .profile file of the user.
<pre>
export P4PORT=perforce.atg.com:1666
export P4CONFIG=p4config.txt
export P4USER=$USER
export P4CLIENT=$USER-$HOST-all
if [ -f .p4passwd ]; then
chmod 600 .p4passwd
export P4PASSWD=`cat .p4passwd`
fi
</pre></li>
<li>Store the userid and passwd in a P4CONFIG file that is read
protected. Then as long as you are in a subdirectory, Perforce
commands will execute with this userid/passwd.
<p>In the Perl API, see NewPerforceConnection() in
//road/Bugzilla/main/bugzilla/perforce.pl. It sets the current
working directory to the P4CONFIG file.</p>
</li>
<li>Store the passwd in a read protected password file, and pass it
to p4 via '-P'.
<p>my $p4passwd = `sed -n '/^P4PASSWD=/s/P4PASSWD=//p'
$config_file` || die "Could not read P4PASSWD= from
$config_file";<br/>
$p4passwd =~ s/\s+//g;<br/>
`p4 -u releng -P $p4passwd ...`</p>
</li>
</ul>
</li>
<li>How to change the directory
<p>The p4 client does not recognize 'cd' commands in a script. It
uses the value of 'PWD' to set the current directory.</p>
<p>cd ${HOME}<br/>
PWD=`pwd`<br/>
export PWD</p>
</li>
<li>How to avoid connection errors [from cpatti:]
<p>In order to lock all the labels in Perforce, I wrote a little
'one-liner' shell script in zsh:</p>
<pre>
for x in `p4 labels | awk '{print $2}'`
do
p4 label -o $x | sed 's/unlocked/locked/' | p4 label -i
done
</pre>
This worked great, with the exception that every 10th or 15th label
would get a "Client Error: Can't connect".
<p>A little investigation revealed that the Perforce server had
about 9,000 TIME_WAIT sockets waiting to close, and so it couldn't
allocate any more connections fast enough to satisfy the request
which times out.</p>
<p>*SO* the solution was to add a minimal delay into the loop:</p>
<pre>
for x in `p4 labels | awk '{print $2}'`
do
p4 label -o $x | sed 's/unlocked/locked/' | (sleep 1;p4 label -i)
done
</pre>
<p>That little sleep 1 makes all the difference. Now every request
goes through.</p>
</li>
</ul>
</item>
<item id="trace" title="How to trace Perforce">
<ul>
<li>To trace p4, use 'p4 -vrpc=1' or 'p4 -vrpc=5'. The first just
shows the call flow, the second includes all the data too [Tony
Smith].
<p>Use 'p4 -Ztag' to show individual fields of the response.</p>
</li>
<li>To trace scc.dll: Using utility regedt32/regedit, create a
registry key named:
<p>HKEY_CURRENT_USER\Software\Perforce\p4scc\TraceFlags</p>
<p>It must be a string value (REG_SZ) containing '*' to "log
everything". The logging information is written to a file named
"PerforcePlugin.log" in the "temp" directory. [Wally Dutchess,
Perforce]</p>
</li>
<li>To trace the p4d server, turn on logging (e.g., p4d -v3). </li>
<li>Use <a href="http://public.perforce.com:8080/guest/ralf_martin/P4GraphicalLogfileAnalyzer/?ac=83">P4GLA</a> for a graphical view of the p4d server logs. </li>
</ul>
</item>
<item id="multiwork" title="How to work with many users using only one client workspace">
<p>In most cases, each user and each host should have
their own client spec. Perforce uses the client spec to distinguish
who has what files.</p>
<p>The following note is from Jeff Bowles:</p>
<p>When I'm teaching the Perforce course, I say very specific
things about sharing workspaces. What follows is mostly a technical
set of guidelines, but at the end is a suggestion about when
sharing a workspace might make sense and saying that "usually, it's
not necessary."</p>
<p>The rule-of-thumb that I recommend is this:</p>
<blockquote>If it's the same physical area on disk with the same
pathname then you can consider it the same workspace. It doesn't
matter if it's accessed by multiple machines or multiple users, so
long as it's the same disk: serial number 129-123023--12A or
whatever. Note that this disk should be mounted so that you can use
the same pathname for the workspace root, e.g. M:\homer or
/usr/marge/workspace.</blockquote>
<p>Then I give examples:</p>
<blockquote>If you have a removable ZIP drive in your E:\ drive at
work, and you have a removable ZIP drive in your E:\ drive at home,
and you wanna carry the cartridge back and forth, it can be the
same workspace and therefore, the same workspace name: lisa.work or
whatever. Why? Because it's the same physical disk, on the same
platform type (oops, should've mentioned that), in the same
pathname.
<p>If you have a removable ZIP drive in your E:\ drive at work, and
you have a removable ZIP drive in your F:\ drive at home, and you
wanna carry the cartridge back and forth, you cannot do it without
making at least one change: the pathname to the workspace needs to
be the same on both machines. So you might want to use the OS
mechanism to refer to the ZIP drive as Z:\ on both machines: on
Windows/NT, that would use the command ``subst Z: E:\ '' on one
machine and ``subst Z: F:\ '' on the other.</p>
<p>If you have a fixed disk, e.g. your C: drive, and you wanna
share it between two computers, you'll probably have to refer to
your workspace using some shared-drive-letter like Z: or M: using
that ``subst'' trick, to make it a shared workspace.</p>
<p>If the workspace will be shared between Unix and NT users, and
you want to use 'samba' or NFS to do it, stop. My advice is don't
do it. (Otherwise, if the Unix user does a 'p4 sync' they'll pull
down files with Unix end-of-line, and the NT users using that
shared disk will see the wrong end-of-line and ... well ... it can
be done, but it's just borrowing trouble. Use separate
workspaces.</p>
</blockquote>
<p>I hasten to point out that recent Perforce releases include a
"Host:" field in the client specification for a workspace that
defaults to the machine you were on when you ran "p4 client". (I
think this was done because it was too easy for me to masquerade as
your client and run 'p4 sync' to pull things onto my own C: drive,
but updating your workspaces "p4 have" data.) So if you're sharing
a workspace between computers, you might need to remove this
field.</p>
<p>The rule: if it's the same platform, pathname, and physical
disk, then you can share the workspace.</p>
<p>Note that this rule is a technical one, not a recommendation of
a procedure. For shared development, probably separate workspaces
makes more sense. But for the once-every-blue-moon patches that
release 1.2 requires (and you're developing release 7.0 now) it
might make sense to share the 1.2 patch workspace. Your choice.</p>
<p><i>Jeff Bowles</i>, Jan 2001.</p>
</item>
<item id="p4search" title="How to search some files in the depots">
<p>In the Perforce root directory</p>
<ul>
<li>Use <tt>find . -mindepth 2 -type d -iname '*.gz'</tt> for all
compressed binary revisions</li>
<li>Use <tt>find . -mindepth 2 -type f -iname '*,v'</tt> for text
files</li>
<li>Use <tt>find . -mindepth 2 -type f -iname '*,v' -not -name
"*'*" | xargs grep '\$ +\$'</tt> for all text files containing "$
$". This does not work for files with spaces or single quotes in
them.</li>
</ul>
</item>
<item id="p4stop" title="How to shut down Perforce">
<p>To shut down Perforce, use the following commands</p>
<pre>
su perforce
/work/perforce/p4d.crondaily
exit
p4 admin stop
</pre>
</item>
</section>
</content>