<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta charset="utf-8" />
<meta name="generator" content="pandoc 3.10" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
<title>p4seedreplica</title>
<style>
/* Default styles provided by pandoc.
** See https://pandoc.org/MANUAL.html#variables-for-html for config info.
*/
html {
color: #1a1a1a;
background-color: #fdfdfd;
}
body {
margin: 0 auto;
max-width: 36em;
padding-left: 50px;
padding-right: 50px;
padding-top: 50px;
padding-bottom: 50px;
hyphens: auto;
overflow-wrap: break-word;
text-rendering: optimizeLegibility;
font-kerning: normal;
}
@media (max-width: 600px) {
body {
font-size: 0.9em;
padding: 12px;
}
h1 {
font-size: 1.8em;
}
}
@media print {
html {
background-color: white;
}
body {
background-color: transparent;
color: black;
font-size: 12pt;
}
p, h2, h3 {
orphans: 3;
widows: 3;
}
h2, h3, h4 {
page-break-after: avoid;
}
}
p {
margin: 1em 0;
}
a {
color: #1a1a1a;
}
a:visited {
color: #1a1a1a;
}
img {
max-width: 100%;
}
svg {
height: auto;
max-width: 100%;
}
h1, h2, h3, h4, h5, h6 {
margin-top: 1.4em;
}
h5, h6 {
font-size: 1em;
font-style: italic;
}
h6 {
font-weight: normal;
}
ol, ul {
padding-left: 1.7em;
margin-top: 1em;
}
li > ol, li > ul {
margin-top: 0;
}
blockquote {
margin: 1em 0 1em 1.7em;
padding-left: 1em;
border-left: 2px solid #e6e6e6;
color: #606060;
}
code {
white-space: pre-wrap;
font-family: Menlo, Monaco, Consolas, 'Lucida Console', monospace;
font-size: 85%;
margin: 0;
hyphens: manual;
}
pre {
margin: 1em 0;
overflow: auto;
}
pre code {
padding: 0;
overflow: visible;
overflow-wrap: normal;
}
.sourceCode {
background-color: transparent;
overflow: visible;
}
hr {
border: none;
border-top: 1px solid #1a1a1a;
height: 1px;
margin: 1em 0;
}
table {
margin: 1em 0;
border-collapse: collapse;
width: 100%;
overflow-x: auto;
display: block;
font-variant-numeric: lining-nums tabular-nums;
}
table caption {
margin-bottom: 0.75em;
}
tbody {
margin-top: 0.5em;
border-top: 1px solid #1a1a1a;
border-bottom: 1px solid #1a1a1a;
}
th {
border-top: 1px solid #1a1a1a;
padding: 0.25em 0.5em 0.25em 0.5em;
}
td {
padding: 0.125em 0.5em 0.25em 0.5em;
}
header {
margin-bottom: 4em;
text-align: center;
}
#TOC li {
list-style: none;
}
#TOC ul {
padding-left: 1.3em;
}
#TOC > ul {
padding-left: 0;
}
#TOC a:not(:hover) {
text-decoration: none;
}
span.smallcaps{font-variant: small-caps;}
div.columns{display: flex; gap: 1.5em;}
div.column{flex: auto;}
@media screen {
div.columns{gap: min(4vw, 1.5em);}
div.column{overflow-x: auto;}
}
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
/* The extra [class] is a hack that increases specificity enough to
override a similar rule in reveal.js */
ul.task-list[class]{list-style: none;}
ul.task-list li input[type="checkbox"] {
font-size: inherit;
width: 0.8em;
margin: 0 0.8em 0.2em -1.6em;
vertical-align: middle;
}
.display.math{display: block; text-align: center; margin: 0.5rem auto;}
/* CSS for syntax highlighting */
html { -webkit-text-size-adjust: 100%; }
pre > code.sourceCode { white-space: pre; position: relative; }
pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
pre > code.sourceCode > span:empty { height: 1.2em; }
.sourceCode { overflow: visible; }
code.sourceCode > span { color: inherit; text-decoration: inherit; }
div.sourceCode { margin: 1em 0; }
pre.sourceCode { margin: 0; }
@media screen {
div.sourceCode { overflow: auto; }
}
@media print {
pre > code.sourceCode { white-space: pre-wrap; }
pre > code.sourceCode > span { text-indent: -5em; padding-left: 5em; }
}
pre.numberSource code
{ counter-reset: source-line 0; }
pre.numberSource code > span
{ position: relative; left: -4em; counter-increment: source-line; }
pre.numberSource code > span > a:first-child::before
{ content: counter(source-line);
position: relative; left: -1em; text-align: right; vertical-align: baseline;
border: none; display: inline-block;
-webkit-touch-callout: none; -webkit-user-select: none;
-khtml-user-select: none; -moz-user-select: none;
-ms-user-select: none; user-select: none;
padding: 0 4px; width: 4em;
color: #aaaaaa;
}
pre.numberSource { margin-left: 3em; border-left: 1px solid #aaaaaa; padding-left: 4px; }
div.sourceCode
{ }
@media screen {
pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; }
}
code span.al { color: #ff0000; font-weight: bold; } /* Alert */
code span.an { color: #60a0b0; font-weight: bold; font-style: italic; } /* Annotation */
code span.at { color: #7d9029; } /* Attribute */
code span.bn { color: #40a070; } /* BaseN */
code span.bu { color: #008000; } /* BuiltIn */
code span.cf { color: #007020; font-weight: bold; } /* ControlFlow */
code span.ch { color: #4070a0; } /* Char */
code span.cn { color: #880000; } /* Constant */
code span.co { color: #60a0b0; font-style: italic; } /* Comment */
code span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } /* CommentVar */
code span.do { color: #ba2121; font-style: italic; } /* Documentation */
code span.dt { color: #902000; } /* DataType */
code span.dv { color: #40a070; } /* DecVal */
code span.er { color: #ff0000; font-weight: bold; } /* Error */
code span.ex { } /* Extension */
code span.fl { color: #40a070; } /* Float */
code span.fu { color: #06287e; } /* Function */
code span.im { color: #008000; font-weight: bold; } /* Import */
code span.in { color: #60a0b0; font-weight: bold; font-style: italic; } /* Information */
code span.kw { color: #007020; font-weight: bold; } /* Keyword */
code span.op { color: #666666; } /* Operator */
code span.ot { color: #007020; } /* Other */
code span.pp { color: #bc7a00; } /* Preprocessor */
code span.sc { color: #4070a0; } /* SpecialChar */
code span.ss { color: #bb6688; } /* SpecialString */
code span.st { color: #4070a0; } /* String */
code span.va { color: #19177c; } /* Variable */
code span.vs { color: #4070a0; } /* VerbatimString */
code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warning */
</style>
<style>
:root {
color: #1f2933;
background: #eef3f6;
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
line-height: 1.6;
}
body {
box-sizing: border-box;
max-width: 58rem;
margin: 2.5rem auto;
padding: 3rem clamp(1.5rem, 5vw, 4.5rem);
background: #ffffff;
border-top: 5px solid #4b778d;
box-shadow: 0 0.75rem 2.5rem rgb(31 41 51 / 12%);
}
h1, h2, h3 {
color: #173f52;
line-height: 1.2;
}
h1 {
margin-top: 0;
padding-bottom: 0.7rem;
border-bottom: 1px solid #d6e1e6;
font-size: 2.4rem;
}
h2 {
margin-top: 2.4rem;
padding-bottom: 0.35rem;
border-bottom: 1px solid #d6e1e6;
font-size: 1.55rem;
}
h3 {
margin-top: 1.8rem;
font-size: 1.2rem;
}
a, a:visited { color: #0d6b85; }
a:hover { color: #a74725; }
#TOC {
margin: 2rem 0;
padding: 1.25rem 1.5rem;
background: #f3f7f8;
border-left: 4px solid #4b778d;
}
#TOC > ul { margin-bottom: 0; }
#TOC li { margin: 0.3rem 0; }
code {
padding: 0.1em 0.3em;
background: #edf2f4;
border-radius: 3px;
color: #173f52;
}
pre {
padding: 1rem 1.2rem;
background: #edf2f4;
border-left: 4px solid #4b778d;
border-radius: 4px;
box-shadow: inset 0 0 0 1px #d6e1e6;
}
pre code {
padding: 0;
background: transparent;
color: #1f2933;
}
@media (max-width: 600px) {
body {
margin: 0;
padding: 1.5rem;
box-shadow: none;
}
h1 { font-size: 2rem; }
}
@media print {
:root { background: #ffffff; }
body {
max-width: none;
margin: 0;
padding: 0;
border: 0;
box-shadow: none;
}
}
</style>
</head>
<body>
<header id="title-block-header">
<h1 class="title">p4seedreplica</h1>
</header>
<nav id="TOC" role="doc-toc">
<ul>
<li><a href="#p4seedreplica" id="toc-p4seedreplica">p4seedreplica</a>
<ul>
<li><a href="#requirements" id="toc-requirements">Requirements</a></li>
<li><a href="#downloads" id="toc-downloads">Downloads</a></li>
<li><a href="#discover-depot-paths"
id="toc-discover-depot-paths">Discover Depot Paths</a></li>
<li><a href="#verify-missing-revisions"
id="toc-verify-missing-revisions">Verify Missing Revisions</a></li>
<li><a href="#seed-a-replica" id="toc-seed-a-replica">Seed a
Replica</a></li>
<li><a href="#command-options" id="toc-command-options">Command
Options</a></li>
<li><a href="#html-documentation" id="toc-html-documentation">HTML
Documentation</a></li>
</ul></li>
</ul>
</nav>
<h1 id="p4seedreplica">p4seedreplica</h1>
<p><code>p4seedreplica</code> feeds missing archive revisions to a
Perforce replica without overwhelming its archive pull queue. Its
workflow has three phases:</p>
<ol type="1">
<li><code>discover</code> generates bounded depot path
specifications.</li>
<li><code>verify</code> finds missing archive revisions in those
paths.</li>
<li><code>seed</code> submits the missing revisions in controlled
batches and can resume from a checkpoint.</li>
</ol>
<h2 id="requirements">Requirements</h2>
<p>Run the tool as a Perforce user that can run <code>p4 sizes</code>
and <code>p4 verify</code> against the source server and
<code>p4 pull -lsh</code> against the replica. The <code>p4</code> CLI
must be available on <code>PATH</code>, unless its location is provided
with <code>--p4bin</code>.</p>
<p>Set <code>P4PORT</code> and <code>P4USER</code> in the environment,
or pass <code>-p</code>/<code>--p4port</code> and
<code>-u</code>/<code>--p4user</code> to each command. Point
<code>discover</code> and <code>verify</code> at the source server, and
point <code>seed</code> at the replica.</p>
<p>Every P4 command includes <code>-Ztrack</code>. With
<code>--debug</code>, the tool logs each command’s elapsed time,
user/system CPU time, and database lock totals. At the end of each
phase, it writes command and database-table totals, plus archive library
opens, closes, checkins, and exists counts, to standard output.</p>
<h2 id="downloads">Downloads</h2>
<p>Build release binaries with:</p>
<div class="sourceCode" id="cb1"><pre
class="sourceCode bash"><code class="sourceCode bash"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="fu">make</span> dist</span></code></pre></div>
<p>The resulting files are written to <code>bin/</code>:</p>
<ul>
<li><code>p4seedreplica.darwin-arm64.gz</code></li>
<li><code>p4seedreplica.darwin-amd64.gz</code></li>
<li><code>p4seedreplica.linux-arm64.gz</code></li>
<li><code>p4seedreplica.linux-amd64.gz</code></li>
<li><code>p4seedreplica.windows-amd64.exe</code></li>
</ul>
<p>To build a binary for the current platform instead, run
<code>make</code> or <code>make build</code>.</p>
<h2 id="discover-depot-paths">Discover Depot Paths</h2>
<p>Generate a bounded set of depot paths from the source server:</p>
<div class="sourceCode" id="cb2"><pre
class="sourceCode bash"><code class="sourceCode bash"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a><span class="ex">p4seedreplica</span> discover <span class="at">-o</span> depot-paths.txt <span class="at">-p</span> source:1666</span></code></pre></div>
<p>The generator considers only top-level local and stream depots. For
local depots, it runs <code>p4 sizes -sh</code> with a five minute
timeout. When a path exceeds that timeout, the command is stopped, its
immediate directories are obtained with <code>p4 dirs -D</code>, and
each directory is checked in the same way.</p>
<p>For stream depots, the generator reads the depot’s
<code>StreamDepth</code>, then lists stream roots with one direct
<code>p4 dirs</code> request at that depth. It randomly samples stream
roots with <code>p4 sizes -sh</code> to estimate file density, grouping
them by their shared path prefixes. Groups estimated to exceed
<code>--max-stream-files</code> are split into the next path level. This
avoids issuing a size command for every stream, so the file limit is an
estimate rather than a strict cap. A stream depot with no
<code>StreamDepth</code> metadata or no stream roots is ignored.
Independent depots are processed concurrently; use
<code>--path-workers</code> to change the default of <code>10</code>
workers.</p>
<p>Use <code>--path-timeout</code> to change the per-path timeout,
<code>--max-stream-files</code> to set the estimated stream-group limit
(default: <code>1000000</code>), and <code>--stream-size-samples</code>
to set the number of random samples (default: <code>10</code>).
Generation refuses to overwrite an existing output file. Add
<code>--debug</code> to log every <code>p4</code> command to standard
error:</p>
<div class="sourceCode" id="cb3"><pre
class="sourceCode bash"><code class="sourceCode bash"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="ex">p4seedreplica</span> discover <span class="at">-o</span> depot-paths.txt <span class="at">--debug</span></span></code></pre></div>
<h2 id="verify-missing-revisions">Verify Missing Revisions</h2>
<p>Verify the generated depot paths against the source server and write
missing revisions to an input file for seeding:</p>
<div class="sourceCode" id="cb4"><pre
class="sourceCode bash"><code class="sourceCode bash"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a><span class="ex">p4seedreplica</span> verify <span class="dt">\</span></span>
<span id="cb4-2"><a href="#cb4-2" aria-hidden="true" tabindex="-1"></a> <span class="at">--paths</span> depot-paths.txt <span class="dt">\</span></span>
<span id="cb4-3"><a href="#cb4-3" aria-hidden="true" tabindex="-1"></a> <span class="at">-o</span> missing-revisions.txt <span class="dt">\</span></span>
<span id="cb4-4"><a href="#cb4-4" aria-hidden="true" tabindex="-1"></a> <span class="at">-p</span> source:1666</span></code></pre></div>
<p><code>verify</code> runs <code>p4 verify -q --only MISSING</code>
concurrently, once for each line in the depot path file. A line can
contain multiple space-separated filespecs, which are sent in one
<code>p4 verify</code> command. Use <code>--workers</code> to change the
default of <code>10</code> workers. It will not replace an existing
output file unless <code>--replace</code> is specified.</p>
<h2 id="seed-a-replica">Seed a Replica</h2>
<p>The input file contains one revision per line, in the form
<code>//depot/path/file#revision</code>. Run the seeding step with an
appropriate batch size and pull-queue limit:</p>
<div class="sourceCode" id="cb5"><pre
class="sourceCode bash"><code class="sourceCode bash"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a><span class="ex">p4seedreplica</span> seed <span class="dt">\</span></span>
<span id="cb5-2"><a href="#cb5-2" aria-hidden="true" tabindex="-1"></a> <span class="at">--input</span> missing-revisions.txt <span class="dt">\</span></span>
<span id="cb5-3"><a href="#cb5-3" aria-hidden="true" tabindex="-1"></a> <span class="at">--batch-size</span> 10000 <span class="dt">\</span></span>
<span id="cb5-4"><a href="#cb5-4" aria-hidden="true" tabindex="-1"></a> <span class="at">--max-pull-queue</span> 100000 <span class="dt">\</span></span>
<span id="cb5-5"><a href="#cb5-5" aria-hidden="true" tabindex="-1"></a> <span class="at">-p</span> replica:1666</span></code></pre></div>
<p>For each batch, the tool submits
<code>p4 -x <batch-file> verify -qt --only MISSING</code>, then
waits until <code>p4 pull -lsh</code> reports no more than
<code>--max-pull-queue</code> transfers. It stops if the pull queue
output cannot be parsed, rather than submitting unbounded work.</p>
<p>Each completed batch is recorded in
<code>missing-revisions.txt.seedreplica.json</code>. Resume an
interrupted run with:</p>
<div class="sourceCode" id="cb6"><pre
class="sourceCode bash"><code class="sourceCode bash"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a><span class="ex">p4seedreplica</span> seed <span class="at">--input</span> missing-revisions.txt <span class="at">--resume</span> <span class="at">-p</span> replica:1666</span></code></pre></div>
<p>Use <code>--follow</code> to keep waiting for appended revisions
after the end of the file, or <code>--max-batches</code> to stop after a
controlled number of batches.</p>
<h2 id="command-options">Command Options</h2>
<pre class="text"><code>discover
-o, --output FILE Generated depot path file; required
-p, --p4port P4PORT Source server override
-u, --p4user P4USER Perforce user override
--path-timeout DURATION Per-path p4 sizes timeout (default: 5m)
--max-stream-files N Estimated maximum files per stream path (default: 1000000)
--stream-size-samples N Random stream paths sampled for file estimates (default: 10)
--path-workers N Parallel depot path workers (default: 10)
verify
--paths FILE Depot path specifications; required
-o, --output FILE Missing revision output file; required
-p, --p4port P4PORT Source server override
-u, --p4user P4USER Perforce user override
--workers N Parallel verification workers (default: 10)
--replace Replace an existing output file
seed
--input FILE Missing revision input file; required
-p, --p4port P4PORT Replica server override
-u, --p4user P4USER Perforce user override
--batch-size N Revisions per verify command (default: 10000)
--max-pull-queue N Queue limit before waiting (default: 100000)
--poll-interval DURATION Queue polling interval (default: 30s)
--max-batches N Stop after this many batches; zero is unlimited
--checkpoint FILE Checkpoint file (default: input.seedreplica.json)
--resume Resume from the checkpoint
--follow Wait for additional input after reaching EOF
--dry-run Validate and checkpoint batches without running p4
All commands accept --p4bin PATH, --debug, -V/--version, and -h/--help.</code></pre>
<h2 id="html-documentation">HTML Documentation</h2>
<p>Generate a standalone HTML version of this document with:</p>
<div class="sourceCode" id="cb8"><pre
class="sourceCode bash"><code class="sourceCode bash"><span id="cb8-1"><a href="#cb8-1" aria-hidden="true" tabindex="-1"></a><span class="fu">make</span> html</span></code></pre></div>
</body>
</html>