-
Status
- Punted
-
Project
- perforce-software-sdp
-
Severity
- B
-
Reported By
- Adam Morriss
-
Reported Date
- 3 years ago
-
Modified By
- C. Thomas Tyler
-
Modified Date
- 3 years ago
-
Owned By
- amo
-
Dev Notes
- 2021-01-30 ttyler: Changed status from open to punted.
See comments.
-
Type
- Bug
Hello @amo!
Thanks for the job contributions as always! Just a heads up, though, after some debate, we do not plan to implement this one. There was a debate on this a while ago.
For many settings, using db.config with 'p4 configure set' commands is preferred. However, when it comes to P4JOURNAL, this is one of a few settings that are absolutely mission-critical to get right, and data loss is possible if it is ever changed. SDP's job is to set these required-at-startup and mission-critical settings like P4ROOT, P4JOURNAL, P4LOG, and others.
For SDP, the best practice is and will always be never set P4JOURNAL in db.config. The P4JOURNAL setting is one of several managed as a shell environment variable by the SDP, and passed in as command line flags to p4d on server startup to guarantee they cannot be changed. The value of P4JOURNAL is critical to the SDP's job of maintaining the offline_db.
There is no benefit to allowing P4JOURNAL to be defined twice in different ways (and thus potentially to different values). Further, this is actual harm to the data set, as it becomes brittle, unable to be restored on other machines with different structures (e.g. for analysis) until P4JOURNAL (and P4PORT) are removed from db.config with 'p4d -cunset' commands.
For many settings, using 'p4 configure' commands and db.config are the way to go, just not P4JOURNAL, P4ROOT, P4PORT, or P4LOG.
I am aware of other discussions on this topic, and there is a distinct difference between SDP best practice and non-SDP best practice.
Setting P4JOURNAL (and P4LOG, and the server spec containing P4PORT) in the database means you can't forget it, you can only override it. Overriding it incorrectly, just like forgetting to set the override, has caused problems, and potentially data loss.
This is a only partly a case of a best practice that is specific to SDP. It is partly specific to SDP, as SDP already has a way of defining certain key values, and allowing them to be set in any other way breaks the SDP horribly.
But even if not SDP, the p4d server itself has a class of of things you can't set in db.config, like P4ROOT, because they must be defined at server start. P4PORT and P4JOURNAL can be defined in db.config, but they're ignored when they conflict with command line flags anyway, but they have greater precedence than P4PORT and P4JOURNAL shell environment variables, leading to tremendous confusion. Allowing db.config to be used at all for these key settings is a bad practice, because allowing that requires use of 'p4d -cset', and that causes writes to the wrong P4JOURNAL which then don't make it to standby replicas. Maybe p4d should disallow setting of P4PORT and P4JOURNAL in db.config just as it does for P4ROOT? We don't need yet another way to define these key settings, as that contributes to more complex order-of-precdence rules one must understand in order to start the server. I'll contemplate a p4d job for that.
In any case, for SDP, the shell environment variables always match the 'p4d' command line startup flags for '-r', '-p' and '-J', which were crafted so that 'ps -ef' shows those values for each p4d process (especially important when there are multiple p4d instances). That's the only way those settings should be touched.
Digression: We also need be wary of of 'sidetrack servers', where a second p4d points to the same P4ROOT and P4JOURNAL, but listens on a different P4PORT and writes to a separate P4LOG. Those have their use cases (mainly temporarily gathering performance data). But they cause confusion due to 'p4 monitor show' having different results on different ports, and also a 'p4 failover' assumes a single 'p4d'.
In any case, non-SDP environments are fading away, with SDP now being the standard and by far the most tested install method and also with the most at-scale operational usage. Next year a helix-sdp package will further that trend so that all benefit from the same best practices. Dunno if you're a Mandalorian fan, but for SDP, "This is the way."
A server needs only to know P4ROOT to find the database, including all the remaining configuration. Override with command-line flags only if needed (such as with a sidetrack server) but the default shouldn't require you to override. The 'ps' output is cosmetic at best.