# P4LF Session Log — 2026-06-25

## Session Goal

First design session for the P4 Log Feeder (p4lf) project. No coding — design and architecture only.

## Prompts / Key Interactions

1. **Kickoff**: "Let's get started." — Agent reviewed README.md and AGENTS.md to understand the project scope.

2. **Architecture question** (paraphrased from AGENTS.md): Should p4lf be implemented in Bash using `p4 logtail`, or in Go using the `go-libtail` library at https://github.com/rcowham/go-libtail?

3. **Research**: Agent researched the go-libtail library in depth (all source files, README, open issues).

## Key Findings from go-libtail Research

- go-libtail is a pure-Go, channel-based, line-streaming file tailer. It handles log rotation well via inode-following (Linux inotify) and truncation detection.
- **Critical gap**: Issue #2 (open since Oct 2019, still unresolved): No API to start from an arbitrary byte offset, and no way to read or persist the current byte offset. On restart, the only options are start-from-BOF (`readall=true`) or start-from-EOF (`readall=false`). This makes checkpoint/resume impossible without forking the library.
- Used in production in p4prometheus and go-libp4dlog by the same author (rcowham), but for streaming-from-end use cases, not chunked delivery with restarts.
- Requires logrus as a mandatory parameter (no stdlib log support).
- Lines channel is unbuffered; slow consumers block the reader goroutine; BufferedTailer mitigates this but drops lines if buffer overflows.
- Uses Go 1.25.

## Key Decision: Go + Custom File Reader

**Neither `go-libtail` nor `p4 logtail` (Bash) were chosen.**

### Rationale

- **go-libtail rejected** because: No offset API means no reliable checkpoint/resume. On restart with `readall=true`, all prior data is re-sent to Splunk (duplicate data). With `readall=false`, data during the outage window is silently dropped. The rotation detection capability — go-libtail's main value — can be replicated with ~100 lines of idiomatic Go using `os.File` + `os.SameFile()` + `fsnotify`.

- **Bash + `p4 logtail` rejected** because: `p4 logtail` is a network call to the Perforce server (performance risk), requires the `p4` binary at runtime, and Bash is harder to maintain and test. The P4LOG is a local file; reading it directly is simpler and safer.

- **Go + custom reader chosen** because: single deployable binary, direct local file reads (no p4d performance impact), full control over offset tracking and rotation detection, idiomatic Go file I/O with `fsnotify` for event-driven wake-ups, and easy cross-compilation.

## Design Summary

### Architecture

```
P4LOG (local file)
      │ fsnotify (inotify on Linux) + polling fallback
      ▼
  p4lf binary (Go)
  ├── State file (inode + byte offset), written atomically after each chunk
  ├── Rotation detector (inode comparison on each open/event)
  └── Chunk assembler (accumulate between LogTailDelay intervals)
        │ gzip + rename
        ▼
  LogChunksDir/ → log.<offset1>.<offset2>.gz
        │
        ▼ (external automation)
  Splunk / other consumer
```

### Config Settings (revised vs. README)

| Setting | Status | Notes |
|---|---|---|
| `LogTailDelay` | Keep | Chunk assembly interval |
| `LogChunksDir` | Keep | Output directory for chunks |
| `MaxLogChunks` | Keep | Back-pressure: stop writing if too many un-consumed files |
| `MinLogSpace` | Keep | Disk space guard |
| `MaxLogSize` | Keep | p4lf's own log rotation |
| `Debug` | Keep | Verbosity (0=off, 1=on, 2=pedantic) |
| `StateFile` | **New** | Path to persist inode+offset for checkpoint/resume; default `$LOGS/p4lf.state` |
| `ReadFromStart` | **New** | On first run (no state file): read from BOF (true) or EOF (false); default true |
| `MaxLogOffsetDelta` | **Dropped** | No longer needed; we read the file directly, not via offset-based API |
| `MaxLogStartOffset` | **Renamed `MaxRotationRecoverySize`** | If new P4LOG post-rotation exceeds this size, start from EOF and log a warning rather than re-reading everything |

### Log Rotation Handling

1. Normal rotation: fsnotify fires `RENAME`/`CREATE`; new file's inode differs from state → reset offset to 0, emit rotation event to p4lf.log.
2. SDP rotation (backup_functions.sh): handled same as above.
3. Gap at rotation: data written to P4LOG after last chunk and before rotation is unrecoverable. p4lf logs a warning with the approximate byte range missed.
4. `MaxRotationRecoverySize`: If the freshly rotated-in log is already large (service was down), only read from BOF if size ≤ threshold; otherwise start from EOF and warn.

### Deployment (SDP structure)

| Artifact | Path |
|---|---|
| Binary | `/p4/common/site/log_feeder/p4lf` |
| Config | `/p4/common/site/log_feeder/p4lf.cfg` |
| State file | `$LOGS/p4lf.state` (default, configurable) |
| p4lf log | `$LOGS/p4lf.log` |
| systemd unit | `/etc/systemd/system/p4lf.service` (User=perforce) |

### Signals

- `SIGTERM` / `SIGINT`: flush current chunk, write state, clean exit
- `SIGHUP`: reload config + rotate p4lf's own log

## Status at End of Session

- Design complete; no code written (as instructed).
- Session log written to `ai/session_log_2026-06-25.md`.
- AGENTS.md not updated (no new standing instructions needed yet).
- Next session should begin implementation: Go module setup, config parsing, file reader with state file, chunk assembly, systemd unit.