P4XFER-3 #2

  • //
  • spec/
  • job/
  • P4XFER-3
  • View
  • Commits
  • Open Download .zip Download (5 KB)
# The form data below was edited by robert_cowham
# Perforce Workshop Jobs
#
#  Job:           The job name. 'new' generates a sequenced job number.
#
#  Status:        Job status; required field.  There is no enforced or
#                 promoted workflow for transition of jobs from one
#                 status to another, just a set of job status values
#                 for users to apply as they see fit.  Possible values:
#
#                 open - Issue is available to be worked on.
#
#                 inprogress - Active development is in progress.
#
#                 blocked - Issue cannot be implemented for some reason.
#
#                 fixed - Fixed, optional status to use before closed.
#                 
#                 closed - Issue has been dealt with definitively.
#
#                 punted - Decision made not to address the issue,
#                    possibly not ever.
#
#                 suspended - Decision made not to address the issue
#                    in the immediate future, but noting that it may
#                    have some merit and may be revisited later.
#
#                 duplicate - Duplicate of another issue that.
#
#                 obsolete - The need behind the request has become
#                    overcome by events.
#
#  Project:       The project this job is for. Required.
#
#  Severity:      [A/B/C] (A is highest)  Required.
#
#  ReportedBy     The user who created the job. Can be changed.
#
#  ReportedDate:  The date the job was created.  Automatic.
#
#  ModifiedBy:    The user who last modified this job. Automatic.
#
#  ModifiedDate:  The date this job was last modified. Automatic.
#
#  OwnedBy:       The owner, responsible for doing the job. Optional.
#
#  Description:   Description of the job.  Required.
#
#  DevNotes:      Developer's comments.  Optional.  Can be used to
#                 explain a status, e.g. for blocked, punted,
#                 obsolete or duplicate jobs.  May also provide
#                 additional information such as the earliest release
#                 in which a bug is known to exist.
#
# Component:      Projects may use this optional field to indicate
#                 which component of the project a givenjob is associated
#                 with.
#
#                 For the SDP, the list of components is defined in:
#                 //guest/perforce_software/sdp/tools/components.txt
#
#  Type:          Type of job [Bug/Feature].  Required.
#
#  Release:       Release in which job is intended to be fixed.

Job:	P4XFER-3

Status:	closed

Project:	perforce-software-p4transfer

Severity:	A

ReportedBy:	ronprestenback

ReportedDate:	2019/04/18 16:29:56

ModifiedBy:	robert_cowham

ModifiedDate:	2021/04/06 04:43:36

OwnedBy:	ronprestenback

Description:
	If transfer is stopped while in some states, it cannot be resumed without manual intervention and cleanup.
	
	Case: If all files in a changelist were opened for edit/add/integrate/whatever on the target server, but not yet submitted (e.g. script was submitting when it was killed manually or threw an exception), the transfer cannot be resumed until the files are reverted.
	Reason: After resuming the transfer, when the script goes to checkout the files, it will get an error from the p4 server (don't recall the exact errors, but they're different depending on why the file was checked out: edit/integrate/add, etc.)  In cases where all files were checked out in the previous run, the submit will fail because the new changelist has nothing in it.  In cases where the script was still checking out files when it was killed, the new changelist will not contain all of the files that were in the source changelist, causing validation to fail.
	
	Case: If the script gets killed/excepted after submit completed, but before the counter was updated, the script will attempt to process that changelist again.
	Reason: There is only one counter that is set at the end of the changelist replication process.  We probably need another counter that is set when we get to a point in the changelist replication process where recovery steps are now necessary (basically everything after replication process has started talking to the destination server).  Recovery code could use that counter to determine whether recovery actions are necessary (if p4transfer_started < p4transfer_completed).  The two most common issues I ran into was files left checked out, and changelists not updated with the origin changelist's user and time.
	
	
	My solution was to create a batch file that performed three steps before I could restart the script, if it was killed or threw an exception while process a changelist:
	1. revert everything on the target server
	2. sync target workspace to #0
	3. sync source workspace to #0
	
	The downside to this (and the only reason I think it might be worth attempting to add support for recovery) is if the changelist in question requires a long sync, it really slows down the process.  I was willing to live with this drawback, because adding recovery steps can be really complicated (maybe impossible) to implement reliably.  You could probably get away with not syncing source workspace to #0 (which would mitigate the sync time drawback), as long as the files have not been modified such that they're now different from the source server's version.

DevNotes:
	This has been implemented in github version

Type:	Bug
# Change User Description Committed
#2 default
#1 default