| 10 years agoLester Cheung created job000279 forDefine exclusion list to avoid checking certain files into ClearCase.
The "checkin everything outstanding" makes script creation much easier.
But we... also need an "I know that's checked out but I don't/never want to
check it in".
Perhaps a definable exclusion list for the checkin everything? We need
this to support one-sided move and no-parent-directory checkin scenarios.
Perhaps define a .ccignore file syntax, in the proud tradition of
(.cvs|.git.p4)ignore files. « | |
|
|
| 10 years agoLester Cheung created job000280 for Support running on top of an existing VOB for incremental testing. | |
|
|
| 10 years agoLester Cheung created job000278 forAdd syntax to define interval between actions.
It would be useful to have a deterministic time interval between certain combinations of actions. Acce...ss to a sleep command? « | |
|
|
| 10 years agoLester Cheung created job000277 forDefine how Perforce Swarm is to be included in the SDP.
We have examples of this being done at customer sites, and
need to bake it in to the stock S...DP.
The essential idea is that we'll maintain an SDP-like
structure (e.g. /p4/1/swarm), and make it clear how to
associated a Swarm instance with each p4d instance.
UPDATE: This job has been suspended, pending details
coming about Swarm 2.0. « | |
|
|
| 10 years agoLester Cheung created job000276 forDefine how Git Fusion is to be included in the SDP.
This should be minimal effort, esseintally
providing SDP-like paths for the location of Git
tri...ggers.
We'll assume an Linux install for this. « | |
|
|
| 10 years agoLester Cheung created job000275 forNormalize and enhance SDP email subject line and logs.
I think the emailed logs (backups, verify, etc) are a bit 'messy',
...
and don't provide an immediate indication that the jobs finished
successfully. I'd like to tidy them up so they display more
clearly In email (at moment in zimbra in the list if inbox emails
the subject line merges with the start of the content so is not
clear) , to be easier to read the content, and more importantly to
have a positive success (or failure) indicator in both the subject
line + body, making it easier to read & also to use email rules to
process/filter accordingly. « | |
|
|
| 10 years agoLester Cheung created job000274 forSDP triggers break with Streams workspaces in P4V.
The 'SetWsOptionsAndView.py' triggers in the SDP does not
interact wel...l P4V when using Streams workspaces.
This needs to be tested from the P4V perspective with Streams
workspaces to make sure they don't induce any improper
behavior. Presently, they do and must be disabled when
Streams workspaces are used. « | |
|
|
| 10 years agoLester Cheung created job000273 forAdd logic to cleanup logs and journal on replicas.
Add logic to the SDP processing on replica machines to ensure
that excess logs, journal and check...point files are cleaned
up on replica machines.
This might be a candidate for putting into sync_replica.sh
and weekly_sync_replica.sh. The idea is to honor
the KEEP_LOGS and KEEP_CKPS settings in p4_vars intuitively
on replica machines, not just on the master server. « | |
|
|
| 10 years agoLester Cheung created job000272 forMake p4_vars safe to run multiple times, or else add logic to make
sure it only gets run once.
If run multiple times, we'll need smarts to ensure t...he PATH is set
exactly as expected, and doesn't (for example) just continue to
append each time p4_vars is sourced.
UPDATE: Suspended this job for now. No plans to implement. « | |
|
|
| 10 years agoLester Cheung created job000271 for Update the maintenance scripts to use a common config file to simplify
configuration at customer sites. | |
|
|
| 10 years agoLester Cheung created job000270 for Exit code as detected by runCmd() is wrong (always zero) when
the specified command contains redirect operators or if
output capturing is disabled. | |
|
|
| 10 years agoLester Cheung created job000270 for Exit code as detected by runCmd() is wrong (always zero) when
the specified command contains redirect operators or if
output capturing is disabled. | |
|
|
| 10 years agoLester Cheung created job000269 for Change the MAIL environment variable to something more SDP specific to
avoid collisions with customer environment variables. | |
|
|
| 10 years agoLester Cheung created job000268 forWorking with a customer today...
he thought (and I agree) that the p4verify.sh script could use some work. Specifically, the second loop that greps fo...r unload depots:
- there can only be one unload depot
- would fail if some local depot happened to have the string 'unload' in its name (e.g. imagine a depot called "sunload" :)
The thought is that the script would be better if it just looped over the depots and checked their types, and if a depot was of type unload then the -U would be added to the verify. « | |
|
|
| 10 years agoLester Cheung created job000267 for SDPEnv.py for Windows SDP setup requires P4Python which can be a pain to install sometimes. | |
|
|
| 10 years agoLester Cheung created job000267 for SDPEnv.py for Windows SDP setup requires P4Python which can be a pain to install sometimes. | |
|
|
| 10 years agoLester Cheung created job000266 for Windows SDP Server Upgrade scripts are out of date and won't work. | |
|
|
| 10 years agoLester Cheung created job000265 for Fixing a collection of typos I spotted in the documentation | |
|
|
| 10 years agoLester Cheung created job000263 forAdded support for systemd init mechanism.
Red Hat 7 has replaced SysV init scripts with systemd. Seems like systemd
is controversial in the Li...nux community on several levels, but regardless, it's
now baked into a popular Linux distro. We should considering supporting it, as
the Perforce-related services can't start on RHEL7 unless we do figure out
how to convert the init scripts to use systemd.
We don't necessarily need to continue supporting automatic restart of the
Perforce service on reboot. It is a conveneicne, but some of the most risk-averse
sites have explicilty avoided having p4d start automatically on reboot. That
is because doing so could potentially make a bad corruption situation worse. For
example, if a kernel panic, sudden power loss, or p4d bug results in database
corruption, it is be best not to start p4d right away anyway, and instead
recover from offline databases and the active journal before restarting.
The SDP p4d_N_init script template implements a safety feature to address
the specific situation of “sudden death” scenarios. If the fast ‘p4d -xvU’
check fails, p4d_N_init refuses to restart p4d. That check tests whether any
databases were left open, which you would expect after a suddent death scenario,
but not after a normal 'p4 admin stop'. « | |
|
|
| 10 years agoLester Cheung created job000264 forThe SDP replica status scripts no longer indicate which
instance they are related to. Earlier versions showed
the 'p4 pull -lj' command used,... which had the collateral
benefit of indicating which instance they were associated
with. « | |
|
|
| 10 years agoLester Cheung created job000262 for The sdp_sync.sh script does not properly clean up old
sdp_sync.*.log files. | |
|
|
| 10 years agoLester Cheung created job000261 forAs noted at the bottom of the notes, the easiest way to get this info.
is with:
/p4/common/bin/p4master_run <instance> p4 set
No other spe...cial command/tool needed.
Create simple info SDP script to detail current settings,
file locations, and basic usage. (sdp_info.py ?)
Such a file will help customers and support personnel alike
by giving quick insight into a particular configuration without
having to refer to external doc or remember the particular
names and locations of SDP files.
For example:
> sdp_info.py -h
usage: sdp_info.py [-hsu]
Show usage and setting information for the collection of scripts
collectively known as the Server Deployment Package (SDP).
-h print this message
-s summary of server and connection info
-u summary of usage infromation
====== ( -s summary option)
master P4PORT: myhost:1666
case handling: sensitive
broker: yes
broker PORT: myhost:1888
proxy: no
log file location:
db file location:
archive file location:
See the instance_vars file for specific instance details: <here>
====== ( -u usage option)
Common commands:
# setup
<path to SDP scripts>/bin/master_run.py $INSTANCE_VAR/init.py
# restart
<path to SDP scripts>/bin/master_run.py $INSTANCE_VAR/
# daily checkpoint
<path to SDP scripts>/bin/master_run.py $INSTANCE_VAR/
# weekly checkpoint and db rotation
<path to SDP scripts>/bin/master_run.py $INSTANCE_VAR/
Note: Checkpoint operations on the master server instance can
entail significant downtime. Recommend timing during low activity
period and setting downtime expectation with end users.
Warning: The weekly checkpoint and db rotation will...
Administrators are advised to ensure that the backup db ifiles are
up to date and uncorrupted by reviewing the logs in <here> and
verifying that checkpoints are completing and being moved without error.
UPDATES:
[ 2014-10-08 ttyler ]
A requirement of this new 'sdp info' mechanism is that it should
behave well with all versions of the SDP dating back to the
first version submitted to p4prod (2007/10/08). That way, Support
can distributed it to customers and ask them to run it to at least
get some sense of what a customer has.
It will be able to display more info for newer versions (e.g. those
with a 'Id:' ktext marker in mkdirs.sh, and even newer ones with a
'Version' file to display). But should at least function with older
versions, even if it's just wrapping 'p4master_run <instance> p4
set' for older versions (with instance defaulting to 1).
[ 2014-09-18 amorriss ]
The inclusion of a post-info trigger by default in all SDP installs
could be used to flag the existence of the SDP; whether simply
"echo SDP installed here" at least, better with some sanity check
(the existence of a certain flag or file?) and retrieving
'p4 configure show' information and similar would be most useful.
[ 2014-09-18 amorriss ]
Once identified as an 'SDP' installation, then the info-gathering
tools could be used. Alternatively, a 'p4 info' command could be
reworked such that it does some self-identification and provides
the output required.
Note that some or more of the information requested is displayed
by the following command:
/p4/common/bin/p4master_run <instance> /p4/common/bin/p4 set « | |
|
|
| 10 years agoLester Cheung created job000262 for The sdp_sync.sh script does not properly clean up old
sdp_sync.*.log files. | |
|
|
| 10 years agoLester Cheung created job000260 for Add a README file to the SDP to discuss what it is,
how to contribute, etc. | |
|
|
| 10 years agoLester Cheung created job000260 for Add a README file to the SDP to discuss what it is,
how to contribute, etc. | |
|
|
| 10 years agoLester Cheung created job000259 for Publish the Perforce Server Deployment Package (SDP) to
The Workshop. | |
|
|
| 10 years agoLester Cheung created job000259 for Publish the Perforce Server Deployment Package (SDP) to
The Workshop. | |
|
|
| 10 years agoLester Cheung created job000258 for Discuss strategies for archiving branches. | |
|
|
| 10 years agoLester Cheung created job000259 for Publish the Perforce Server Deployment Package (SDP) to
The Workshop. | |
|
|
| 10 years agoLester Cheung created job000257 for bbi.sh chokes on large data sets. | |
|
|
| 10 years agoLester Cheung created job000257 for bbi.sh chokes on large data sets. | |
|
|
| 10 years agoLester Cheung created job000257 for bbi.sh chokes on large data sets. | |
|
|
| 10 years agoLester Cheung created job000256 forEnhance messages from Python reporting 'None' for Perforce errors.
Error messages should be more helpful, and should use '-s' to help
identify probl...ems. The output of failed Perforce commands should be
displayed. « | |
|
|
| 10 years agoLester Cheung created job000255 forRetool to use 'p4 reconcile'.
Prior to 'p4 reconcile', the BBI used a series of commands to
figure out what changed in a workspace. Commands s...earched for
newly added files, newly added symlinks, deleted files, and
modified files. Now 'p4 reconcile' does that more efficiently,
and all in one step.
As a bonus, this will add automatic detection to P4BBI, as
'p4 reconcile' detects renames. (It uses a heuristic which
amounts to a guess, rather than being truly deterministic,
but it's proven to be good enough to be helpful -- so far
perfect in my experience).
This will also fix a problem encountered when files with '@' in
the filename are deleted between baseline updates. The older method
of using 'p4 diff -sd ... | p4 -x - delete' failed with such files,
but 'p4 reconcile' handles that case just fine. « | |
|
|
| 10 years agoLester Cheung created job000255 forRetool to use 'p4 reconcile'.
Prior to 'p4 reconcile', the BBI used a series of commands to
figure out what changed in a workspace. Commands s...earched for
newly added files, newly added symlinks, deleted files, and
modified files. Now 'p4 reconcile' does that more efficiently,
and all in one step.
As a bonus, this will add automatic detection to P4BBI, as
'p4 reconcile' detects renames. (It uses a heuristic which
amounts to a guess, rather than being truly deterministic,
but it's proven to be good enough to be helpful -- so far
perfect in my experience).
This will also fix a problem encountered when files with '@' in
the filename are deleted between baseline updates. The older method
of using 'p4 diff -sd ... | p4 -x - delete' failed with such files,
but 'p4 reconcile' handles that case just fine. « | |
|
|
| 10 years agoLester Cheung created job000254 forAdd support for Streams.
The BBI config file will need to support a STREAM entry, with a format
something like this:
STREAM|<Stream>|<Type...>|<Owner>|<Parent>|<Description>|[|<StreamOptions>]
Like:
STREAM|//fgs/main|mainline|p4import|none|FGS mainline
STREAM|//fgs/R2.1|release|p4import|//fgs/main|FGS R2.1|ownersubmit unlocked toparent nofromparent
This can be used to create a new stream spec, or reparent an existing one.
In addition to providing stream spec manipulation, the stream workspace will
need to be adjusted between each basline import, and the worspace reset.
The BBI config file will need to support a POPULATE entry, with a format
something like this:
POPULATE|<Stream>|<OptionFlags> « | |
|
|
| 10 years agoLester Cheung created job000254 forAdd support for Streams.
The BBI config file will need to support a STREAM entry, with a format
something like this:
STREAM|<Stream>|<Type...>|<Owner>|<Parent>|<Description>|[|<StreamOptions>]
Like:
STREAM|//fgs/main|mainline|p4import|none|FGS mainline
STREAM|//fgs/R2.1|release|p4import|//fgs/main|FGS R2.1|ownersubmit unlocked toparent nofromparent
This can be used to create a new stream spec, or reparent an existing one.
In addition to providing stream spec manipulation, the stream workspace will
need to be adjusted between each basline import, and the worspace reset.
The BBI config file will need to support a POPULATE entry, with a format
something like this:
POPULATE|<Stream>|<OptionFlags> « | |
|
|
| 10 years agoLester Cheung created job000253 for Subtask: Implement CBD REWRITE logic for Classic in Cbd.py. | |
|
|
| 10 years agoLester Cheung created job000252 forSubtask: Define Key Structure for Classic keys.
Streams uses cbd_stream_* and a defined format. Classic
needs a similar defined format for the... REWRITE logic to
work. « | |
|
|
| 10 years agoLester Cheung created job000251 forSubtask: Develop WSTemplateUpdate trigger.
Develop WSTemplateUpdate trigger to CBD keys (similar to SSTemplateUpdate), and
also update user workspac...es based on the updated template (unique to the Classic
implementation, as this "comes for free" with Streams). « | |
|
|
| 10 years agoLester Cheung created job000250 for Complete implementation of CBD for Classic. | |
|
|
| 10 years agoLester Cheung created job000249 forAdd support for Task streams.
At issue: In SSTemplateUpdate.py, the 'p4 describe' output for task streams
doesn't list depot files (unless run from... the Stream workspace associated
with that task stream). This breaks upon creation of a new Task Stream.
It doesn't prevent creation of the Task Stream, but fails to update CBD
keys for that task stream. « | |
|
|
| 10 years agoLester Cheung created job000248 forKeys not cleaned up when import or shore is removed from *.cbdsst.
Removing an import or share in the *.cbdsst leaves the keys hanging around.
"Blob...bing" the CBD keys (See CBD-16) should solve this issue, but I'm
filing this bug to remember to verify that. « | |
|
|
| 10 years agoLester Cheung created job000247 forExtra whitespace in cbdsst isn't handled perfectly.
(Pretend = is space)
If you have 'share==...'
It will screw up the stream sp...ec, it needs to be 'share=...'.
With python we could probably make that a bit more 'friendly' and strip extra whitespace problems. « | |
|
|
| 10 years agoLester Cheung created job000246 for Support depots with multiple modules. | |
|
|
| 10 years agoLester Cheung created job000245 forAdd Windows support for server-side CBD back end.
While the broker & trigger back end implementation works with any and all client platforms, it... only works when the back end is Linux. Add support for Windows only environments (and of course do it in a way that doesn't break existing Unix/Linux platform support). « | |
|
|
| 10 years agoLester Cheung created job000244 forVersion 'Remapped:' and 'Ignored:' fields.
The initial CBD implementation updated only the 'Paths:' in
the live Stream Spec when the versioned form,... the *.cbdsst
file, was updated, but ignored Remapped and Ignored fields.
That functions as initially designed, based on the "miminally
intrusive" philosphy to avoid unnecessary impact of the custom
CBD logic. However, later it was deeemd better for all aspects
of the stream spec to be dictated by the versioned from of the
file if we're going to have one at all, so they can go through
the same merge down/copy up flow as Paths updates. « | |
|
|
| 10 years agoLester Cheung created job000243 forUse a single-key lookup for CBD keys on 'p4 sync.'
Replace the two-key text-string lookup with a single-key dictionary
object lookup.
Currently C...BD uses cbd_stream_path<N> and cbd_stream_vspec<N>
keys to define the revision specifiers for each path. An
attendees at the Perforce Merge 2014 conference suggested using
a single key using a dicutionary object (like Swarm). Brilliant
idea! « | |
|
|
| 10 years agoLester Cheung created job000242 forTask: Confirm that any arbitrary revision specifiers work.
Task: Confirm that any arbitrary revision specifiers work when
used in the *....cbdsst/*.cbdwst files.
This should 'just work' with CBD, since it rewrites the 'p4 sync'
commands, and the 'p4 sync' can handle any valid revision specifier.
This is known to work with changelist and labels, but needs to be
tested for the various forms of date and date/time stamp entries,
which might foil the Python regex stuff. « | |
|
|
| 10 years agoLester Cheung created job000241 forImplement Component Groups and Heirarchy.
This will be needed to support CBD in large enterprise environments.
This implies supporting an 'include...' syntax in the versioned *.cbdsst
(Streams) and *.cbdwst (Classic) files.
The include will specify another versioned file, or alternately a
component group name, which references multiple include tags. « | |
|
|
