# p4seedreplica

`p4seedreplica` feeds missing archive revisions to a Perforce replica without
overwhelming its archive pull queue. Its workflow has three phases:

1. `discover` generates bounded depot path specifications.
2. `verify` finds missing archive revisions in those paths.
3. `seed` submits the missing revisions in controlled batches and can resume
  from a checkpoint.

## Requirements

Run the tool as a Perforce user that can run `p4 sizes` and `p4 verify` against
the source server and `p4 pull -lsh` against the replica. The `p4` CLI must be
available on `PATH`, unless its location is provided with `--p4bin`.

Set `P4PORT` and `P4USER` in the environment, or pass `-p`/`--p4port` and
`-u`/`--p4user` to each command. Point `discover` and `verify` at the source
server, and point `seed` at the replica.

Every P4 command includes `-Ztrack`. With `--debug`, the tool logs each
command's elapsed time, user/system CPU time, and database lock totals. At the
end of each phase, it writes command and database-table totals, plus archive
library opens, closes, checkins, and exists counts, to standard output.

## Downloads

Build release binaries with:

```bash
make dist
```

The resulting files are written to `bin/`:

- `p4seedreplica.darwin-arm64.gz`
- `p4seedreplica.darwin-amd64.gz`
- `p4seedreplica.linux-arm64.gz`
- `p4seedreplica.linux-amd64.gz`
- `p4seedreplica.windows-amd64.exe`

To build a binary for the current platform instead, run `make` or
`make build`.

## Discover Depot Paths

Generate a bounded set of depot paths from the source server:

```bash
p4seedreplica discover -o depot-paths.txt -p source:1666
```

The generator considers only top-level local and stream depots. For local
depots, it runs `p4 sizes -sh` with a five minute timeout. When a path exceeds
that timeout, the command is stopped, its immediate directories are obtained
with `p4 dirs -D`, and each directory is checked in the same way.

For stream depots, the generator reads the depot's `StreamDepth`, then lists
stream roots with one direct `p4 dirs` request at that depth. It randomly
samples stream roots with `p4 sizes -sh` to estimate file density, grouping
them by their shared path prefixes. Groups estimated to exceed
`--max-stream-files` are split into the next path level. This avoids issuing a
size command for every stream, so the file limit is an estimate rather than a
strict cap. A stream depot with no `StreamDepth` metadata or no stream roots
is ignored. Independent depots are processed concurrently; use
`--path-workers` to change the default of `10` workers.

Use `--path-timeout` to change the per-path timeout,
`--max-stream-files` to set the estimated stream-group limit (default:
`1000000`), and `--stream-size-samples` to set the number of random samples
(default: `10`). Generation refuses to overwrite an existing output file. Add
`--debug` to log every `p4` command to standard error:

```bash
p4seedreplica discover -o depot-paths.txt --debug
```

## Verify Missing Revisions

Verify the generated depot paths against the source server and write missing
revisions to an input file for seeding:

```bash
p4seedreplica verify \
  --paths depot-paths.txt \
  -o missing-revisions.txt \
  -p source:1666
```

`verify` runs `p4 verify -q --only MISSING` concurrently, once for each line
in the depot path file. A line can contain multiple space-separated filespecs,
which are sent in one `p4 verify` command. Use `--workers` to change the
default of `10` workers. It will not replace an existing output file unless
`--replace` is specified.

## Seed a Replica

The input file contains one revision per line, in the form
`//depot/path/file#revision`. Run the seeding step with an appropriate batch
size and pull-queue limit:

```bash
p4seedreplica seed \
  --input missing-revisions.txt \
  --batch-size 10000 \
  --max-pull-queue 100000 \
  -p replica:1666
```

For each batch, the tool submits `p4 -x <batch-file> verify -qt --only
MISSING`, then waits until `p4 pull -lsh` reports no more than
`--max-pull-queue` transfers. It stops if the pull queue output cannot be
parsed, rather than submitting unbounded work.

Each completed batch is recorded in
`missing-revisions.txt.seedreplica.json`. Resume an interrupted run with:

```bash
p4seedreplica seed --input missing-revisions.txt --resume -p replica:1666
```

Use `--follow` to keep waiting for appended revisions after the end of the
file, or `--max-batches` to stop after a controlled number of batches.

## Command Options

```text
discover
  -o, --output FILE           Generated depot path file; required
  -p, --p4port P4PORT         Source server override
  -u, --p4user P4USER         Perforce user override
    --path-timeout DURATION Per-path p4 sizes timeout (default: 5m)
    --max-stream-files N    Estimated maximum files per stream path (default: 1000000)
    --stream-size-samples N Random stream paths sampled for file estimates (default: 10)
    --path-workers N        Parallel depot path workers (default: 10)

verify
    --paths FILE            Depot path specifications; required
  -o, --output FILE           Missing revision output file; required
  -p, --p4port P4PORT         Source server override
  -u, --p4user P4USER         Perforce user override
    --workers N             Parallel verification workers (default: 10)
    --replace               Replace an existing output file

seed
    --input FILE            Missing revision input file; required
  -p, --p4port P4PORT         Replica server override
  -u, --p4user P4USER         Perforce user override
    --batch-size N          Revisions per verify command (default: 10000)
    --max-pull-queue N      Queue limit before waiting (default: 100000)
    --poll-interval DURATION Queue polling interval (default: 30s)
    --max-batches N         Stop after this many batches; zero is unlimited
    --checkpoint FILE       Checkpoint file (default: input.seedreplica.json)
    --resume                Resume from the checkpoint
    --follow                Wait for additional input after reaching EOF
    --dry-run               Validate and checkpoint batches without running p4

All commands accept --p4bin PATH, --debug, -V/--version, and -h/--help.
```

## HTML Documentation

Generate a standalone HTML version of this document with:

```bash
make html
```