# Session Handoff: sync_jira_to_p4jobs.sh

**Date:** 2026-07-03
**Prepared for:** Claude Code (or next session)
**Author:** Claude (claude.ai chat session)

---

## Context

Tom Tyler is a maintainer of the **Perforce Server Deployment Package (SDP)**,
an open source project managing the Perforce Helix Core server engine.  He
has admin control over the **Public Depot** — a Perforce server at
`workshop.perforce.com` hosting open source Perforce-adjacent projects.

The SDP project historically used native P4 Jobs for issue tracking, with a
`JobIncrement` trigger that named jobs after JIRA issue keys (e.g. `SDP-1234`).
Jobs were publicly visible at URLs like
`https://swarm.workshop.perforce.com/jobs/SDP-563`.

The SDP project was then migrated to JIRA (`perforce.atlassian.net`) as the
source of truth.  JIRA is Perforce-internal and not publicly accessible.
The P4 Jobs in the Public Depot stopped being updated.

The goal of this session was to restore public visibility of SDP issues by
writing a script that polls JIRA and mirrors issue data back into the Public
Depot P4 job database, one-way (JIRA → P4, never the reverse).

---

## Deliverables produced

Two files were written and are the primary output of this session:

### 1. `sync_jira_to_p4jobs.sh`

The main script.  Key characteristics:

- **Language:** Bash, following the SDP Bash Coding Standard (draft at
  `https://swarm.workshop.perforce.com/view/p4-sdp/dev_rebrand/doc/SDP_CodingStandard_bash.adoc`)
- **Not part of SDP:** Deployed under `/p4/common/site/bin/` per SDP
  convention for site-local (non-SDP) additions.  Uses `$P4CSBIN` env var.
- **Versioning deferred:** Per-standard versioning requires a Classic→Streams
  depot migration that is still in progress; version is pinned at `1.0.0`.

### 2. `sync_jira_to_p4jobs_README.md`

Deployment and operations guide.  Covers installation, credential setup,
scheduling, field mapping, resilience design, and the SDP site directory
convention.

---

## Design decisions (record of rationale)

### Architecture
- **Incremental polling** via a persistent watermark file (`last_sync_time`).
  JQL query: `project in (...) AND updated >= '<watermark>' ORDER BY updated ASC`.
  Watermark is recorded at *run start* (not end) to avoid gaps.
- **Watermark only advances on a fully-clean run.**  If any JIRA page or P4
  job upsert fails, the watermark stays put and the next run retries.
- **flock advisory lock** prevents overlapping cron invocations.
- **`p4 job -i -f` is idempotent** — safe to run repeatedly.
- **P4 preflight:** `p4 info` is called before any JIRA fetching; if P4 is
  unreachable the script bails without touching the watermark.

### JIRA API
- Uses `/rest/api/3/search/jql` with `nextPageToken` pagination.
  (The old `startAt`-based `/rest/api/3/search` was removed from Atlassian
  Cloud in 2025 and must not be used.)
- HTTP Basic auth: `JiraUser:JiraToken` (email + API token, never a password).
- Per-call `JiraPollDelay` (default 2 s) for politeness to Atlassian.
- Retry with back-off on HTTP 429 and 5xx; non-retryable on 4xx.
- JSON parsed with `python3` inline heredocs to avoid `jq` dependency.

### P4 job form / jobspec
- The actual Public Depot jobspec was obtained and examined.  Field names
  and valid values differ from SDP defaults.  Key fields written:

  | P4 field      | Source                                  |
  |---------------|-----------------------------------------|
  | `Job`         | JIRA issue key (e.g. `SDP-1234`)        |
  | `Status`      | Mapped from JIRA status (see below)     |
  | `Project`     | `JiraProjectMap[]` config mapping       |
  | `Severity`    | Fixed: `C` (no JIRA equivalent)         |
  | `Type`        | JIRA issue type → `Bug/Doc/Feature/Problem` |
  | `ReportedBy`  | Reporter email or displayName           |
  | `ReportedDate`| `fields.created` (YYYY/MM/DD)           |
  | `ModifiedBy`  | Reporter email (JIRA updater not in API)|
  | `ModifiedDate`| `fields.updated` (YYYY/MM/DD)           |
  | `Description` | Summary line + blank + body (see below) |

  Fields **not written** (preserved on update): `OwnedBy`, `DevNotes`,
  `Component`, `Release`.

- **Description convention:** First line = JIRA summary (issue title), then
  a blank line, then the description body.  This is a standard Public Depot
  P4 Jobs convention — tools that show one line of a job show the title.
  There is no separate `Summary` field in this jobspec.

- **`p4 job -f` requirement:** The `ReportedDate` (`once`) and `ModifiedBy`/
  `ModifiedDate` (`always`) fields are read-only without `-f`.  The service
  account therefore needs `admin` access in `p4 protect`.

### Status mapping
The Public Depot jobspec has ten Status values.  Mapping uses both the coarse
JIRA status *category* and the specific JIRA status *name*, checking the name
first for finer resolution:

| JIRA status name (pattern)             | P4 Status    |
|----------------------------------------|--------------|
| `*block*`                              | `blocked`    |
| `*review*`                             | `review`     |
| `*duplicate*`                          | `duplicate`  |
| `*obsolete*`, `*wontfix*`              | `obsolete`   |
| `*punt*`                               | `punted`     |
| `*done*`, `*resolved*`, `*complete*`   | `closed`     |
| category = To Do / New                 | `open`       |
| category = In Progress                 | `inprogress` |
| category = Done (fallback)             | `closed`     |
| (anything else)                        | `open`       |

### Project mapping
The jobspec `Project` field is a large enum of workshop project names
(e.g. `perforce-software-sdp`).  JIRA project keys (e.g. `SDP`) do not
match these values directly.  An associative array `JiraProjectMap` in the
config file provides the mapping:

```ini
JiraProjectMap[SDP]=perforce-software-sdp
```

If a JIRA key has no mapping entry, a warning is emitted and the lowercased
key is used as a fallback (which may fail P4 validation — treat as a
configuration error to be fixed).

### SDP site directory convention
This script is not part of SDP and must not go in `$P4CBIN` (`/p4/common/bin`).
All paths use `$P4CSBIN` (`/p4/common/site/bin`) for the script and config,
and `/p4/common/site/etc/` for credential files.

### SDP Coding Standard compliance
Key standard requirements applied:
- 3-space indentation throughout
- Global script variables: `UpperCamelCase` (e.g. `JiraUrl`, `StateDir`)
- Perforce/shell env variables: `ALL_UPPERCASE` (e.g. `P4PORT`, `P4USER`)
- Function-local variables: `lowerCamelCase` with `local` (not `declare`)
- Timestamped log files: `$LOGS/<ScriptName>.<YYYY-MM-DD-HHMMSS>.log`
  with a stable `<ScriptName>.log` symlink; `set -C` noclobber for safety
- `terminate()` function exits with `$ErrorCount`
- `-si` silent mode (for cron), `-V`/`--version`, `-man` full docs
- Function header comments with `Input:` / `Output:` / `Returns:`
- `case` arms use `(pattern)` form (parens on both sides)

### ShellCheck findings addressed
- **SC2324** (`missing+=1`): Fixed — changed to `local -i missing=0` and
  `(( missing += 1 ))`.
- **SC2034** (`rc` unused in `sync_project`): Fixed — removed the variable.
- **SC2001** (sed for tab insertion in Description): Kept `sed`; added
  `# shellcheck disable=SC2001` with explanation.  The bash parameter
  expansion alternative (`${var//^/tab}`) cannot insert a literal tab
  portably.
- **SC2329** (`dbg` never invoked): Intentionally ignored — standard
  template function, retained for future use.

### JobIncrement trigger safety
The `JobIncrement.pl` trigger fires only when `Job:` is `new`.  Since this
script always supplies an explicit JIRA key as the job name, the trigger is
a no-op for every job this script touches.  Other projects using
`JobIncrement` with their own prefixes are completely unaffected.

---

## Files and deployment paths

```
/p4/common/site/bin/sync_jira_to_p4jobs.sh      # script
/p4/common/site/bin/sync_jira_to_p4jobs.cfg     # config (generated by --init)
/p4/common/site/etc/.jira_api_token             # JIRA API token (chmod 600)
/p4/common/site/etc/.p4_jira_sync_ticket        # P4 ticket (chmod 600)
/p4/<instance>/tmp/jira_sync/last_sync_time     # watermark
/p4/<instance>/tmp/jira_sync/last_sync.lock     # flock file
/p4/<instance>/tmp/jira_sync/jira_page_cache/   # raw JIRA JSON (2-day TTL)
/p4/<instance>/logs/sync_jira_to_p4jobs.*.log   # timestamped logs
/p4/<instance>/logs/sync_jira_to_p4jobs.log     # symlink → latest log
```

---

## Known gaps / next steps for Claude Code

The following items are known open questions or likely next tasks:

1. **Test against live JIRA and P4.**  The script has not been run against
   the real `perforce.atlassian.net` or `workshop.perforce.com`.  First run
   should use `-n -v 5` (NoOp + debug) and carefully review the NoOp output
   before removing `-n`.

2. **`ModifiedBy` accuracy.**  JIRA's `/rest/api/3/search/jql` does not
   return the last-update user in the fields list (it returns `reporter`).
   The current code sets `ModifiedBy` to the reporter, which is wrong for
   issues updated by someone other than the reporter.  Options:
   - Accept the inaccuracy (low impact for a mirror field).
   - Fetch `/rest/api/3/issue/<key>` per-issue for the `updated` user
     (expensive — one extra API call per issue).
   - Use the `changelog` API endpoint (`/rest/api/3/issue/<key>/changelog`)
     to find the most recent changelog entry's `author`.

3. **Severity mapping.**  Currently hardcoded to `C` for all issues.  JIRA
   has a `priority` field (`Highest`/`High`/`Medium`/`Low`/`Lowest`) that
   could map to `A`/`B`/`C`.  A `jira_priority_to_p4_severity` function
   would follow the same pattern as the existing mapping functions.

4. **`Component` and `OwnedBy` mapping.**  These are jobspec fields left
   unset by the sync.  JIRA has `components[]` and `assignee` fields that
   could populate them.  For `Component`, JIRA components are free-form and
   may not match the SDP components list in `components.txt`.

5. **Initial full sync sizing.**  `InitialLookbackSeconds=0` (all issues)
   on a project with thousands of issues will work correctly but will be
   slow due to per-issue Python JSON processing and `JiraPollDelay` pacing.
   May want to run the first full sync with `JiraPollDelay=0` and
   `JiraPageSize=100` and then restore defaults.

6. **`p4_vars` sourcing.**  On the Public Depot server, the SDP environment
   (`p4_vars`) must be sourced before running the script to set `$P4CSBIN`,
   `$LOGS`, `$P4PORT`, `$P4USER`, etc.  The cron entry or systemd unit
   should source `/p4/common/bin/p4_vars <instance>` first, or the config
   file must provide explicit fallback values for all variables.

7. **ShellCheck full pass.**  The session used manual grep-based checks;
   a proper `shellcheck -S warning sync_jira_to_p4jobs.sh` pass on a host
   where shellcheck is installed would be a good final gate before production
   deployment.

8. **Swarm display.**  The goal is public visibility via Swarm Workshop URLs
   (`https://swarm.workshop.perforce.com/jobs/SDP-563`).  Once the sync is
   running, verify that Swarm picks up the updated jobs correctly and that
   the Description line rendering looks right in the Swarm job view.

---

## Reference links

- SDP Bash Coding Standard (draft):
  `https://swarm.workshop.perforce.com/view/p4-sdp/dev_rebrand/doc/SDP_CodingStandard_bash.adoc`
- JobIncrement trigger docs:
  `https://swarm.workshop.perforce.com/view/guest/perforce_software/sdp/dev/Unsupported/doc/JobIncrement.html`
- JobIncrement trigger source:
  `https://swarm.workshop.perforce.com/view/guest/perforce_software/sdp/dev/Unsupported/Samples/triggers/JobIncrement.pl`
- JIRA Cloud REST API (search/jql):
  `https://developer.atlassian.com/cloud/jira/platform/rest/v3/api-group-issue-search/`
- Public Depot Swarm:
  `https://swarm.workshop.perforce.com`