Helix SwarmHelix Swarm
Loading...

anon

anon
  • 1
    Owner
  • 6
    Members
Owners
amo (Adam Morriss)
Members
hws_workshop (hws_workshop)jobxfr (Background User for Joh Replication (jobxfr.py))p4admin (p4admin)perforce (Perforce maintenance)perforce_software (perforce_software)
swarm-user (swarm-user)
  • Activity
  • Reviews

  • All Activity
  • Followed Activity
  • Reviews
  • Commits
  • Comments
  • Jobs
2 years agoswarm-user created workflow (Global Workflow) for
2 years agoswarm-user modified SDP-778 for
NFS sharing /p4/common causes upgrade.sh to classify major upgrade as patch.
Add a comment
2 years agoswarm-user modified SDP-919 for
p4verify.sh: Added -ignores <regex_ignores_file>' option.
Add a comment
2 years agoswarm-user modified SDP-918 for
Add db.storage to list of tables to checkpoint on an edge.
Add a comment
2 years agoswarm-user modified SDP-787 for
Define client.sendq.dir as part of template config variables

Important for parallel syncs and reducing contention
Add a comment
2 years agoswarm-user modified SDP-244 for
Define client.readonly.dir configurable and create dir on hxlogs volume.

See also: SDP-243.
Add a comment
2 years agoswarm-user modified SDP-243 for
Define client.readonly.dir configurable and create dir on /hxlogs volume.

See also: SDP-244.
Add a comment
2 years agoswarm-user modified SDP-917 for
Upgrade on Ubuntu does not attempt 'sudo setcap'.
Add a comment
2 years agoswarm-user modified SDP-916 for
Enhance granularity and extractability of timing info for p4verify.sh.
Add a comment
2 years agoswarm-user modified SDP-915 for
Add -paths option to p4verify.sh to verify specified paths.
Add a comment
2 years agoswarm-user modified SDP-912 for
Added P4D r23.1 OOM defense in SDP upgrade.sh.

Per release notes:
#2397436 (Job #112553) **
    The server on Linux can now protect itself... from the OOM killer
    during database writes.  This happens automatically when the
    necessary capabilities have been granted to the server executable,
    e.g. "sudo setcap 'CAP_SYS_RESOURCE=+ep' p4d".  A new server
    startup log entry is now written with details about the detected
    level of support for features like this. «
Add a comment
2 years agoswarm-user modified SDP-868 for
Fix call to p4login.bat in configure_new_server.bat (missing instance arg).
Add a comment
2 years agoswarm-user modified SDP-859 for
Windows template_configure_new_server.bat set net.autotune=0

This seems to be causing quite frequent hangs, especially over
slow links and with lar...ge submits or syncs (e.g. UE5 size of 100k+ files
100GB+). «
Add a comment
2 years agoswarm-user modified SDP-900 for
Add support for r23.1 hyper-parallel multi-thread per db table checkpoint capability.
Add a comment
2 years agoswarm-user modified SDP-909 for
p4verify.sh: Add safety feature to avoid verifying cache mode replica by default.
Add a comment
2 years agoswarm-user modified SDP-903 for
Broken / changed behavior in p4verify log name rotation.

In https://swarm.workshop.perforce.com/files/guest/perforce_software/sdp/dev/Server/Unix/p4/...common/bin/p4verify.sh?v=43 at line 589 the OldLogTimestamp is set to "$(ls -l --time-style +'%Y%m%d-%H%M%S' "$Log" | awk '{print $6}')". However, this does not work on (at least) Red Hat and Rocky 8.7 as the "print $6" output is the file size. Instead, $7 is needed.

In the same file at line 590 the OldLog is set to "$LOGS/p4verify.${OldLogTimestamp}.log" but prior to this change the old log file was "${LogToRotate}.${Datestamp}". As an example (after locally addressing the issue above), the new format naming results in p4verify.2023-05-09_09-40-05.log whereas the old name was p4verify.log.2023-05-09_09-28-17 so the files no longer sort the same. I *think* it was an intentional change to retain the .log extension on files but don't see it called out on https://swarm.workshop.perforce.com/projects/perforce-software-sdp/jobs/SDP-829 unless it fell under "general log file handling"?

Additionally, the previous time format was '%Y-%m-%d_%H-%M-%S' but is now '%Y%m%d-%H%M%S' so there aren't as many spacers for readability. «
Add a comment
2 years agoswarm-user modified SDP-907 for
Added '-Q <MaxTotalPullQueueSize>' option to p4verify.sh.
Add a comment
2 years agoswarm-user modified SDP-904 for
p4verify.sh: Add '-w <Wait>' and '-q <MaxActivePullQueueSize>' options.
Add a comment
2 years agoswarm-user committed change 29537 into
Added: //.swarm/attachments/0000000109-Screen-Shot-2023-04-25-at-7.35.42-AM.png
Add a comment
2 years agoswarm-user modified SDP-797 for
sync_replica.sh to maintain offline_db on a 'ham' replica.

The sdp_sync.sh script maintains the offline_db on a regular
an HA that uses the first fo...rm of the journalPrefix (per
the journalPrefix standard).  That form is: /p4/N/checkpoints/p4_N

Minor adjustment is needed for this script to maintain the
offline_db on a 'ham' replica, a metadata-only HA relpica
with lbr.replication=shared that shares /hxdepots over NFS
with its P4TARGET server. replica that uses the second form of
the journaPrefix.  That form is:

/p4/N/checkpoints.<ShortServerID>/p4_N.<ShortServerID>

Adjustments needed:
* On a ham replica, we don't need the rsync, due to the NFS
sharing. So if lbr.replication=1, skip the rsync.
* Load the latest checkpoint taken on the master (and in the
first form of the checkpoints direcotry) into the local
offline_db of the standby.

See also: SDP-676 «
Add a comment
2 years agoswarm-user modified SDP-888 for
mkrep.sh gets generated ServerID wrong if '-f' used when not daisy chaining.
Add a comment
2 years agoswarm-user modified SDP-813 for
mkrep.sh: Add preflight check for running on master and P4MASTER_ID OK.
Add a comment
2 years agoswarm-user modified SDP-880 for
p4login expands only first comma in SDP_AUTOMATION_USERS var.
Add a comment
2 years agoswarm-user modified SDP-872 for
For parallel checkpoint replay, need to add '-N'.

We need to add the '-N' parameter with '-jrp' when recovering the offline,
database since there is...n't a database in place for the command to read the
setting from. «
Add a comment
2 years agoswarm-user modified SDP-878 for
p4login fails to get P4TARGET if replica ServerID is subset of another.

The p4login script has a bad regex that needs to be tightened.

If the Serv...erID of current replica is a subset of the name of
another server spec, both values are returned and the p4login script
fails. «
Add a comment
2 years agoswarm-user modified SDP-876 for
Enhanced docs re: MAILTO/MAILFROM settings in Instance Vars file.
Add a comment
2 years agoswarm-user modified SDP-860 for
Directories that differ only by case aren't being caught by script.
Add a comment
2 years agoswarm-user modified SDP-868 for
Fix call to p4login.bat in configure_new_server.bat (missing instance arg).
Add a comment
2 years agoswarm-user modified SDP-867 for
Change server.extensions.dir to /hxlogs.
Add a comment
2 years agoswarm-user modified SDP-866 for
Add configurables for Helix Authentication Service (HAS).
Add a comment
2 years agoswarm-user modified SDP-840 for
Set net.paralllel.{max,threads} to reasonable defaults.

Also set net.parallel.sync.svrthreads.
Add a comment
2 years agoswarm-user modified SDP-859 for
Windows template_configure_new_server.bat set net.autotune=0

This seems to be causing quite frequent hangs, especially over
slow links and with lar...ge submits or syncs (e.g. UE5 size of 100k+ files
100GB+). «
Add a comment
2 years agoswarm-user modified SDP-865 for
Enhance robustness of systemctl management of p4d/p4p/p4broker.
Add a comment
2 years agoswarm-user modified SDP-864 for
Added '-delay <delay>' option to load_checkpoint.sh to delay call to p4verify.sh.

This is a workaround for p4d server job079842 (possibly to b...e fixed in P4D 2023.1). «
Add a comment
2 years agoswarm-user modified SDP-858 for
mkrep.sh: Add '-os' option to overwrite an existing ServerID.
Add a comment
2 years agoswarm-user modified SDP-857 for
mkrep.sh: Add preflight check to avoid overwriting existing ServerID.
Add a comment
2 years agoswarm-user modified SDP-777 for
mkrep.sh: Support daisy chaining of forwarding replicas.
Add a comment
2 years agoswarm-user modified SDP-624 for
Add mkrep.sh support for '<N>' per standard: <HelixServerTag>_<ReplicaTypeTag>[<N>]_<SiteTag>'

The Server Spec naming... standard allows for multiple edges/replicas to
be defined for the same site tag.  For example, you might have:

* p4d_edge_tok - An edge server in Tokyo
* p4d_edge2_tok - A second edge server in Tokyo
* p4d_edge3_tok - A third edge server in Tokyo

mkrep v2.7.x supports only the first example that doesn't use the
optional numeric integer. The second and third examples can't
be created with mkrep.sh presently.

Ref: Server Spec Naming Standard: https://swarm.workshop.perforce.com/projects/perforce-software-sdp/view/main/doc/SDP_Guide.Unix.html#_server_spec_naming_standard «
Add a comment
2 years agoswarm-user modified SDP-862 for
Adjust p4pcm.pl so it does not require a local perl build.
Add a comment
2 years agoswarm-user modified SDP-861 for
mkdirs.sh now creates crontab for proxy-only installs.
Add a comment
2 years agoswarm-user modified SDP-776 for
Add support for parallel checkpoint ops with -jcp/-jdp/-jrp.

When parallel checkpoints are added to p4d, the SDP will need
to adapt to expose parall...el checkpoint feature (with the usual
$P4D_VERSION check to only attempt to use it if the p4d version
is new enough to support it).

To define the number of threads, supply the -N option and/or
interact with the db.checkpoint.threads configurable. «
Add a comment