# The form data below was edited by tom_tyler
# Perforce Workshop Jobs
#
# Job: The job name. 'new' generates a sequenced job number.
#
# Status: Job status; required field. There is no enforced or
# promoted workflow for transition of jobs from one
# status to another, just a set of job status values
# for users to apply as they see fit. Possible values:
#
# open - Issue is available to be worked on.
#
# inprogress - Active development is in progress.
#
# blocked - Issue cannot be implemented for some reason.
#
# fixed - Fixed, optional status to use before closed.
#
# closed - Issue has been dealt with definitively.
#
# punted - Decision made not to address the issue,
# possibly not ever.
#
# suspended - Decision made not to address the issue
# in the immediate future, but noting that it may
# have some merit and may be revisited later.
#
# duplicate - Duplicate of another issue that.
#
# obsolete - The need behind the request has become
# overcome by events.
#
# Project: The project this job is for. Required.
#
# Severity: [A/B/C] (A is highest) Required.
#
# ReportedBy The user who created the job. Can be changed.
#
# ReportedDate: The date the job was created. Automatic.
#
# ModifiedBy: The user who last modified this job. Automatic.
#
# ModifiedDate: The date this job was last modified. Automatic.
#
# OwnedBy: The owner, responsible for doing the job. Optional.
#
# Description: Description of the job. Required.
#
# DevNotes: Developer's comments. Optional. Can be used to
# explain a status, e.g. for blocked, punted,
# obsolete or duplicate jobs. May also provide
# additional information such as the earliest release
# in which a bug is known to exist.
#
# Component: Projects may use this optional field to indicate
# which component of the project a givenjob is associated
# with.
#
# For the SDP, the list of components is defined in:
# //guest/perforce_software/sdp/tools/components.txt
#
# Type: Type of job [Bug/Feature/Problem]. Required.
# Feature and Bug are common terms.
# A Problem is suspected bug, or one without a clear
# understanding of exactly what is broken.
#
# Release: Release in which job is intended to be fixed.
Job: SDP-741
Status: open
Project: perforce-software-sdp
Severity: C
ReportedBy: tom_tyler
ReportedDate: 2022/02/06 13:11:16
ModifiedBy: tom_tyler
ModifiedDate: 2022/02/06 13:11:16
OwnedBy: tom_tyler
Description:
Add support for optional db3 to maintain fast recovery capability.
There are 2 primary benefits of an offline_db:
1. No downtime is needed for routine (e.g. daily) checkpoint operations.
2. For large data sets, recovery using the offline_db is much faster
than recovery using the traditional "checkpoint and journal" approach.
Benefit #1 is always available.
Benefit #2, however, is only available if the need to recover occurs
when the offline_db is not currently involved in the routine daily
checkpoint processing. If a recovery is required when the offline_db
is unavailable, the slower-at-scale traditional "checkpoint and
journal" recovery approach must be used. (For purposes of this job,
we are only considering recovery only on the master server machine.
Standby replicas provide an entirely different and complementary
option for recovery, though that is more complex to maintain).
If a 3rd copy of the databases is used (a db3), the capability to
recover quickly using the offline_db can be preserved continuously.
This would require another full copy of the metadata, and so would
need to be an optional feature so customers can decide whether to
prioritize for disk space or continuous fast recovery capability.
It must also be easy to swap back and forth between modes in such a
way that only once cycle of daily_checkpoint.sh is required to switch
modes.
Currently, the UNIX/Linux SDP has db1 and db2 folders in /hxmetadata
(optionally /hxmetadata1 and /hxmetadata2 on separate volumes). At
any given time, P4ROOT points (via symlink) to one of these folders,
and 'offline_db' to the other. We could add a 'db3' or use the
existing 'save' directory under the P4ROOT. We may also want to
support the option (in the SDP and Helix Installer) of adding a
separate /hxmetadata3 storage volume.
DevNotes:
2022-02-06 tom_tyler: Early versions of SDP used a 3-database model
using P4ROOT, P4ROOT/save, and offline_db, where the 'save' folder
was used to store the extra copy. Long ago, the 3-database model
was changed to a 2-database model because customers at the time
generally preferred that model, and some couldn't afford a full
extra copy of the databases due to disk space constraints. It's
time to bring back the 3 database model as that would be preferred
by most nowadays, as disk space is generally more abundant.
We just need it to be optional for those that don't have extra
storage.
Component: core-unix
Type: Feature