P4XFER-3
Closed
If transfer is stopped while in some states, it cannot be resumed without manual intervention and cleanup.
Case: If all files in a changelist were opened for edit/add/integrate/whatever on the target server, but not yet submitted (e.g. script was submitting when it was killed manually or threw an exception), the transfer cannot be resumed until the files are reverted.
Reason: After resuming the transfer, when the script goes to checkout the files, it will get an error from the p4 server (don't recall the exact errors, but they're different depending on why the file was checked out: edit/integrate/add, etc.) In cases where all files were checked out in the previous run, the submit will fail because the new changelist has nothing in it. In cases where the script was still checking out files when it was killed, the new changelist will not contain all of the files that were in the source changelist, causing validation to fail.
Case: If the script gets killed/excepted after submit completed, but before the counter was updated, the script will attempt to process that changelist again.
Reason: There is only one counter that is set at the end of the changelist replication process. We probably need another counter that is set when we get to a point in the changelist replication process where recovery steps are now necessary (basically everything after replication process has started talking to the destination server). Recovery code could use that counter to determine whether recovery actions are necessary (if p4transfer_started < p4transfer_completed). The two most common issues I ran into was files left checked out, and changelists not updated with the origin changelist's user and time.
My solution was to create a batch file that performed three steps before I could restart the script, if it was killed or threw an exception while process a changelist:
1. revert everything on the target server
2. sync target workspace to #0
3. sync source workspace to #0
The downside to this (and the only reason I think it might be worth attempting to add support for recovery) is if the changelist in question requires a long sync, it really slows down the process. I was willing to live with this drawback, because adding recovery steps can be really complicated (maybe impossible) to implement reliably. You could probably get away with not syncing source workspace to #0 (which would mitigate the sync time drawback), as long as the files have not been modified such that they're now different from the source server's version.
Case: If all files in a changelist were opened for edit/add/integrate/whatever on the target server, but not yet submitted (e.g. script was submitting when it was killed manually or threw an exception), the transfer cannot be resumed until the files are reverted.
Reason: After resuming the transfer, when the script goes to checkout the files, it will get an error from the p4 server (don't recall the exact errors, but they're different depending on why the file was checked out: edit/integrate/add, etc.) In cases where all files were checked out in the previous run, the submit will fail because the new changelist has nothing in it. In cases where the script was still checking out files when it was killed, the new changelist will not contain all of the files that were in the source changelist, causing validation to fail.
Case: If the script gets killed/excepted after submit completed, but before the counter was updated, the script will attempt to process that changelist again.
Reason: There is only one counter that is set at the end of the changelist replication process. We probably need another counter that is set when we get to a point in the changelist replication process where recovery steps are now necessary (basically everything after replication process has started talking to the destination server). Recovery code could use that counter to determine whether recovery actions are necessary (if p4transfer_started < p4transfer_completed). The two most common issues I ran into was files left checked out, and changelists not updated with the origin changelist's user and time.
My solution was to create a batch file that performed three steps before I could restart the script, if it was killed or threw an exception while process a changelist:
1. revert everything on the target server
2. sync target workspace to #0
3. sync source workspace to #0
The downside to this (and the only reason I think it might be worth attempting to add support for recovery) is if the changelist in question requires a long sync, it really slows down the process. I was willing to live with this drawback, because adding recovery steps can be really complicated (maybe impossible) to implement reliably. You could probably get away with not syncing source workspace to #0 (which would mitigate the sync time drawback), as long as the files have not been modified such that they're now different from the source server's version.
- Status
- Closed
- Project
- perforce-software-p4transfer
- Severity
- A
- Reported By
- ronprestenback
- Reported Date
- 6 years ago
- Modified By
- Robert Cowham
- Modified Date
- 4 years ago
- Owned By
- ronprestenback
- Dev Notes
- This has been implemented in github version
- Type
- Bug
Hi Ron
The downside you mention was a killer for some situations where multi GB changelists were being synced over a slow link. Thus it was left as a wrapper for user to tidy...
Will review.
Robert