Release Notes for P4, the Helix Command Line, P4D, the Helix Versioning Engine P4P, the Helix Proxy and P4Broker, the Helix Broker Version 2019.2 Introduction This document lists all user-visible changes to: * Helix Versioning Engine (P4D) * Command line client (P4) * Helix Proxy (P4P) * Helix Broker (P4Broker) in 2019.2 release. Release notes for other Perforce Helix products are available separately on the Perforce Documentation web page. Developer notes for Perforce API programming, the Helix Broker and also a separate document which programmers should read in addition to this release note document. Perforce numbers releases YYYY.R/CCCCC, e.g. 2002.1/30547. YYYY is the year; R is the release of that year; CCCCC is the bugfix change level. Each bugfix in these release notes is marked by its change number. Any build includes (1) all bugfixes of all previous releases and (2) all bugfixes of the current release up to the bugfix change level. Both 'p4' and 'p4d' will report their version information by passing the '-V' flag. Additionally, 'p4' can report the server's information with the 'p4 info' command. Version information includes a release version and a build change number. The build change number can be compared with change numbers and patch information mentioned in the matching version release notes file to determine if a particular build of a version includes the change mentioned. Supported Platforms for p4d, p4, p4p and p4broker Linux kernel 2.6+ (glibc 2.6+) for Intel(x86, x86_64) Windows 8.1 for Intel(x86, x64) Windows 10 for Intel(x86, x64) Windows 2012 for Intel(x64) Windows 2016 for Intel(x64) Windows 2019 for Intel(x64) Apple OS X 10.12+ for Intel(x86_64) -------------------------------------------------------------------------- Important Platform End-of-Life Notice This notice identifies the platforms that will be placed in End-of-Life (EOL) status on October 2020. This applies to p4, p4d, p4p, p4ftp, and p4broker components of the server release. The following platforms have entered the Obsolescent phase in this release: Mac OS X 10.5, 10.6, 10.7, 10.8, 10.9(x86, x86_64), 10.10(x86_64) 10.11(x86_64) Linux kernels (glibc < 2.6) for Intel(x86, x86_64) Windows 2008 for Intel(x86_64), Windows 8 for Intel(x86, x64) OBSOLESCENT Phase: Starting with this release,these platforms will no longer be supported. Customers will receive full support and patches for the deprecated platforms during this phase for older releases. The platforms will remain in this phase until October 2020 after which they will transition into the discontinued phase. DISCONTINUED (EOL) Phase: Patches will no longer be issued for these platforms. We will no longer provide technical support to customers with servers on these platforms -------------------------------------------------------------------------- Important limitation for Windows server Support for Helix Core Extensions was disabled for Windows servers due to unrelated performance concerns on that platform requiring a conflicting toolchain change. Extensions remain fully supported on Linux and Mac. -------------------------------------------------------------------------- Important security note This release links OpenSSL version 1.1.1a. -------------------------------------------------------------------------- Important export note This product is subject to U.S. export control laws and regulations including, but not limited to, the U.S. Export Administration Regulations, the International Traffic in Arms Regulation requirements, and all applicable end-use, end-user and destination restrictions. Licensee shall not permit, directly or indirectly, use of any Perforce technology in or by any U.S. embargoed country or otherwise in violation of any U.S. export control laws and regulations. -------------------------------------------------------------------------- Upgrading the Server ** IMPORTANT UPGRADE NOTES ** CHANGES TO DISTRIBUTED UPGRADE RECOMMENDATIONS For 2019.1 or later of the Helix Versioning Engine (P4D), significant schema upgrades performed by 'p4d -xu' will only be performed on servers where those upgrades are needed. This means that metadata changes made by upgrades must be replicated and so must be journaled. To help prevent mistakes, upgrades will not run if journaling is disabled. If offline databases are in use, upgrades should not be performed on them, as the upgrade performed on the live database will be journaled and applied during normal journal replay. When upgrading from 2019.1 or later of the Helix Versioning Engine (P4D), replicas may be upgraded safely prior to the server they are replicating from. This upgrade pattern, referred to as "outside-in", allows a multi-server environment be upgraded server-by-server starting with the outermost replica, without requiring the entire set of servers to be taken offline at once. When upgrading from a version prior to 2019.1 to 2019.1 or later, we strongly recommend that all servers are upgraded in a single upgrade session. More information can be found in the Helix Core Server Administrator Guides: https://www.perforce.com/manuals/p4sag/Content/P4SAG/chapter.upgrade.html https://www.perforce.com/manuals/p4dist/Content/P4Dist/intro.maintaining.html ------------------------------------------------------------------- CHANGES TO UNLICENSED SERVER For v16.1 onwards of the Helix Versioning Engine (P4D) the number of free users allowed with an unlicensed server has changed to 5, the number of clients remains at 20. Previous versions of the server will continue to support the 20/20 usage allowance. ------------------------------------------------------------------- BEFORE UPGRADING To upgrade your Helix Versioning Engine (P4D), your Perforce license file must be current. Expired licenses do not work with upgraded servers. In addition to your usual checkpointing scheme, always checkpoint your server immediately before undertaking an upgrade. For any Helix4Git instances that are currently polling repos, we recommend: 1) first upgrade your Helix4Git instance, 2) and then upgrade your Helix Versioning Engine as soon as possible thereafter. Otherwise, there could be issues with repo caching. Read-only archive directories can interfere with the upgrade process. This may be the case for archive paths migrated from a different version control system, such as CVS, or in cases where 3rd party imported content has been made read only by a system admin. Make sure all such archive paths are writeable before attempting the upgrade. DOWNGRADING A 2019.2 SERVER IS NOT POSSIBLE UPGRADING PROCEDURE IS DIFFERENT DEPENDING ON WHAT YOUR CURRENT PERFORCE VERSION IS: UPDATE DATABASE SCHEMA vs UPDATE DATABASE FORMAT ------------------------------------------------------------------- ** UPDATE DATABASE SCHEMA If your current server version is 2013.3 or above, then this upgrade is a UPDATE DATABASE SCHEMA upgrade: Unix Platforms 1. Issue the 'p4d -r -J -xu' command. This command performs the significant schema upgrades, then exits. 2. Restart your server with your site's usual parameters. Windows Platforms 1. Issue the 'p4d -r -J -xu' command from a command prompt window. This command performs the significant schema upgrades, then exits. 2. Restart your Helix Versioning Engine (P4D) service. Note: Personal DVCS servers will automatically perform any database schema upgrades; whilst the upgrade steps described above are unnecessary for these servers, they are safe to run. ------------------------------------------------------------------- ** UPDATE DATABASE FORMAT If your current server version is 2013.2 or earlier, then this upgrade is an UPDATE DATABASE FORMAT upgrade: Due to the btree format introduced in 2013.3 this is not an UPDATE DATABASE SCHEMA upgrade. A checkpoint using your old server and then a restore using the 2019.2 server is required before the internal upgrades 'p4d -xu' (if required) can be performed. Note: If you have made use of the (undocumented) '+T' modifier (storing file content in the tiny.db database) this data is not checkpointed (and therefore not restored). See 'p4 help undoc' for backup/restore procedures for this table. Once the database has been restored with the 2019.2 version, update your database schema: Unix Platforms 1. Issue the 'p4d -r -J -xu' command. This command performs the schema upgrades, then exits. 2. Restart your server with your site's usual parameters. Windows Platforms 1. Issue the 'p4d -r -J -xu' command from a command prompt window. This command performs the schema upgrades, then exits. 2. Restart your Helix Versioning Engine (P4D) service. -------------------------------------------------------------------------- Interoperating With Previous Releases 1. Unless stated otherwise you can intermix any release of Helix Command Line (P4) with any release of the Helix Versioning Engine (P4D), but any functionality new to 2019.2 requires you to upgrade the client and/or P4D. See marks in the notes below. * -- requires new p4 client program including all client applications and derived APIs ** -- requires new p4d server program *** -- requires new p4p proxy program **** -- requires new p4broker broker program Any replica servers must be at least the same release level as the master server and any functionality that requires an upgrade for the master requires an upgrade for the replica. A replica can be running a later release than the master if the master is running at least 2019.1. However in this case some new functionality might not be available on the upgraded replica until the master has been upgraded. 2. Beginning with 99.2, remote depots will interoperate between UNIX and NT. Beginning with 98.2, remote depots will interoperate across (98.2 and higher) releases. In 98.1 and before, remote depots will only operate with another server of the same release. 3. As of 2005.1, remote depot support for 98.2 and 99.1 servers has been dropped. Attempts by 98.2 and 99.1 servers to contact 2005.1 servers still works, but the depots will appear devoid of files. -------------------------------------------------------------------------- Technology Preview features: Technology Preview features are currently unsupported, might not be functionally complete, and are not suitable for deployment in production. These features are provided to the customer to solicit interest and feedback, with the goal of full support in future releases. Customers are encouraged to provide feedback and functionality suggestions for Technology Preview features before they become fully supported. -------------------------------------------------------------------------- Important note: Please refer to: http://www.perforce.com/perforce/r19.2/user/relnotes.txt to get up to date GA and post-GA information about this release. -------------------------------------------------------------------------- Major new functionality in 2019.2 #1859368 (Bug #99850) ** Minimal downtime upgrades. Support for outside-in upgrades has been reestablished. Please see the upgrade notice at the beginning of this file. #1856324 (Bug #94869) ** Configurable history is now recorded. Configurables changes on a 2019.2 or later server with either 'p4 configure set/unset' or 'p4d -cset/-cunset' are now recorded. This history is made visible with 'p4 configure history', which behaves similarly to 'p4 configure show'. #1828890 (Bug #95112, #90346, #67029) ** Structured logging improvements. Structured logs now have a versioned schema that allow new versions of existing events to be added. Updated versions of events are represented by having the server version included after a period in the event type field. The event types with new versions this release are: 2.49 - CommandEnd 6.49 - Audit 8.49 - NetworkPerformance 9.49 - DatabasePerformance The 'serverlog.version.N' configurable can be used to pin a structured log file to a specific server version's format. To retain the prior structured log events format, set the 'serverlog.version.N' configurable to 48. #1873195 (Bug #99688, #98735, #97689, #23625) ** Orphaned depot files detection and archive cleanup. The '-l' and '-d' flags have been added to 'p4 storage'. These flags can be used to detect and clean up any orphaned depot files left over from a failed submit or shelve operation. See 'p4 help storage' for more detail. Minor new functionality in 2019.2 #1872848 (Bug #98957, #96750) ** Add the ability to create custom fields in stream specs. When adding custom fields to specs use the 'NNN' placeholder value for the field code value to automatically assign the next available code value. #1862346 (Bug #98937) ** A new command 'p4 streamspec' has been added. The new command allows you to add custom fields to stream specs. #1859414 (Bug #99722) ** For the scenario where a stream spec change in a pending changelist is shelved, the change is reverted and then the client workspace is switched to a different stream: it is now possible for a file to be added to the pending changelist and shelved with the stream spec from the previous stream. #1856483 (Bug #99721) ** A stream spec can now be deleted from a shelved change from the context of any stream. Also, the stream spec does not have to be open for edit to be deleted from a shelved change. #1853371 (Bug #98048) ** 'p4 verify' now supports using -S and -v together to recalculate the digests of shelved files. #1847657 (Bug #92631) ** A new field 'gconn-serverId' has been added in the repo spec. Data for this field comes from the Helix4Git instance which is mirroring that repo. #1843953 (Bug #95377) ** A 'failed-over' trigger has been added. It is fired when a standby server becomes the new master and first starts up during a successful 'p4 failover' command. #1842551 (Bug #98058) ** A new flag '-Z' has been added to the 'p4 verify' command. This is equivalent to a 'p4 storage -v' request, causing the verify command to perform a checksum compare using the db.storage table instead of the db.rev table. This avoids duplicate checks of lazy-copied files. #1840946 (Bug #99415) ** We are now recording the serverSize field in the matching storage record with the file size of the depot file in the server. Additionally, if the type of submitted revision is compressed and p4 configurable 'submit.storagefields' is set, then the digest of depot file is computed and stored in the compCksum field in the storage record. #1833159 (Bug #99047, #96805) ** The new 'p4 extension --run' option lets a user interact with a Server-side Extension directly. The Extension can send plain or tagged messages and progress updates to the client program. One use for this is to implement custom commands, e.g. 'p4 extension --run fancyBranchMaker args'. #1826797 (Bug #94456) ** 'p4 obliterate supports' a new -p purge option. This removes the need of running a two step process of 'p4 archive' followed by 'p4 archive -p'. This marks the revision as purged and leaves the integration history intact while still removing the archive file. #1821614 (Bug #98956) ** Added a new server spec field, 'updateCachedRepos' for use only by servers of type 'connector'. Prior to his change 'gconn --poll-repos' queried its Helix 'connector' server spec for a list of git repos contained in the 'externalAdress' field. With this release and upgrade, this list for repos has moved from 'externalAddress' to the new 'updateCachedRepos'. Note: Helix4Git should be upgraded before Helix Core. If not, gconn --poll-repos' will find an empty list of repos in 'externalAdress'. #1808665 (Bug #96987) ** A new configurable, db.monitor.term.allow, when set to 1 allows users to terminate their own processes. If set to 2, users can also pause and resume their own processes. A minimum of 'read' permission is required for these users to be able to use 'p4 monitor terminate'. Bugs fixed in 2019.2 Patch 1 #1896318 (Bug #100568) ** A 'p4 storage -l pause //depot/...' request now waits until the orphan scanner has entered the paused state before returning. #1895480 (Bug #100608) ** Avoid error on conflicting clients for commands that are read-only. For 'p4 opened -c ' or -u, the current client of the user is irrelevant and no error should be displayed if that client is tied to another server. The same should be the case for 'p4 client' when the client name is provided as an argument or for 'p4 client -i' which should validate the client name listed in the form rather than the client tied to the user initiating the command. #1891322 (Bug #100615, #100616) ** Log a message if computing the digest for a keyed file fails during the storage upgrade process. Avoid holding the write lock on the storage table during digest calculation during the storage upgrade process. #1889618 (Bug #100613) ** Creating file locks (.lck) could take a long time to fail. Check the error returned by the OS and fail quickly if it is a permanent error. #1888789 (Bug #100682) ** The orphan scanner did not correctly follow relative symlinks on the Windows platform. This has been fixed. #1887331 (Bug #83623) * ** The default value of the 'filesys.lockdelay' has been decreased from 300 to 90 to ensure abandoned .lck files are correctly deleted. #1887330 (Bug #100634) ** 'p4 storage -l start //depot/...' can fail if an orphan scanner with a name that is a substring of 'depot' exists. This has been fixed. #1875057 (Bug #100028) ** 'p4 clean' and 'p4 reconcile' now handle the '*' wildcard correctly on Windows. Bugs fixed in 2019.2 #1882284 (Bug #100524) ** Changing the file type from a tempobj type (+S) to a non tempobj type incorrectly removes all previous revisions. This has been fixed. #1877678 (Bug #100400) ** A call to FileSys::Stat() issued on a symlink would return inconsistent results between operating systems. This has been fixed. #1869587 (Bug #99913) ** A journal rotation beyond the LEOF is no longer sent to a replica that should be constrained by the LEOF. #1867765 (Bug #99911) ** A metadata 'pull' or 'journalcopy' request for a journal fragment from the current journal no longer fails if the current journal is rotated just prior to determining the size of the current journal. #1866020 (Bug #100158) ** UTF8 type files sync'ed from cache mode replica with a pre-2016.2 client could report a depot file error. This has been fixed. #1865912 (Bug #100130) ** A crash when replicating a file into a directory with no write permissions is now prevented. #1861550 (Bug #98869) ** The 'pull -L' thread no longer applies transactions beyond the LEOF to the metadata of a 'standby' or 'forwarding-standby' server, nor prematurely advances to the next journal if the LEOF is in a prior journal. #1858397 (Bug #100012) ** If a stream spec is in a change but not shelved, and file is shelved but not opened, then 'p4 shelve -d -c N' would incorrectly report "Shelved change N partially deleted, still contains stream ...". The stream spec was never in the shelf, the message was not correct. Fixed to report "Shelved change N deleted." #1858257 (Bug #99978) ** The upgrade storage walker would incorrectly take the walker lock in exclusive mode, blocking 'safe' command during the 2nd phase of the upgrade. This has been fixed. #1855841 (Bug #99869) If a shelved change had an open stream spec change for one stream and a file change from that stream, and then the stream spec and file were reverted, and the client was set to a different stream, 'p4 describe -S clno' would give an error "RpcTransport: partner is not a Perforce client/server". This has been fixed. #1854429 (Bug #99609) ** If 'p4 submit -b' was run from an edge server on a change with missing archives, a pull thread on the commit server could crash and cause more than just the missing archive to fail to transfer. This has been fixed. #1854349 (Bug #99762) ** 'p4 graph tag' now allows partial SHA1 arguments. #1852875 (Bug #99206) ** The environment P4EXTENSIONS ending with trailing slash caused should be silently disregarded but caused a runtime error. The runtime issue has been fixed, the environment still requires a pattern to match in extension filenames. #1850983 (Bug #99608) ** Improve argument validation on 'p4 extension' to prevent certain combinations from entering a bad state. #1850862 (Bug #99597) ** A metadata 'pull' or 'journalcopy' request for a journal fragment from the current journal no longer fails if the current journal is rotated just prior to opening the current journal. #1850807 (Bug #99749) ** Trailing slashes at end of directory name when installing an extension was failing. This is fixed. #1848299 (Bug #31585) ** 'p4 fstat' could scan db.rev excessively when the -e flag was combined with file spec using a revision specifier. This has been fixed. #1847529 (Bug #96725) ** Attempts to restart the former master server after successful failover will now fail if its serverid has not been modified via 'p4d -xD' to a different serverid than the new master. #1842553 (Bug #28576) ** A 'p4 obliterate' command called without the '-y' confirm flag now only takes a READ lock. #1842404 (Bug #99542) ** 'journalcopy -i 0' no longer incorrectly detects an incorrectly terminated journal when rotated immediately after an unrelated journal growth. #1840225 (Bug #98278) ** If an edge server is restarted and must replay the journal that contains a record of an install of an extension and subsequent delete of that same extension, the edge journal replay would be blocked at that position attempting to download an extension archive that is no longer needed. This has been fixed. #1839667 (Bug #75382) ** The maximum size of shared memory for monitor table use was too small (2048 8K pages). Now it's 32768 pages for all platforms except Windows 32bit where memory address space is severely limited. #1839424 (Bug #98597) ** Messages enumerating the mandatory standby servers that are not alive are now also reported in the server log of the replica servers to which replication is stalled. #1836925 (Bug #97741) ** P4 extension ClientApi.new() can now process a port supplied via p4 -p . #1836209 ** Adjustments to allocation routines lead to less memory usage in the Windows server. #1830413 (Bug #96675) ** Extension package command no longer accepts "." for current working directory. The directory name must be supplied either as an absolute or relative path. #1829901 (Bug #99300) ** Journal header notes included with database table unloads created by 'p4 journaldbchecksums -u' now contain the correct case/unicode flags and the server root of the server they were unloaded from. #1826907 (Bug #99208) ** 'rdb.lbr' is now locked along with the db.* files during server shutdown. This prevents incomplete btree updates of 'rdb.lbr' by other threads as the server is shut down. #1826716 (Bug #94998) ** Protections, permissions and groups for a user are now cached and reused by additional commands run on the same connection. #1823272 (Bug #98566, #99136) ** The server no longer allows force deletion of a submitted change that resides in an unloaded task stream, and when a task stream change isn't unloaded, it now reports the correct number of files in the change preventing the delete. #1821164 (Bug #90948) ** Archive pull threads initiated by 'p4 pull -u --batch' didn't resume active archive transfers after connectivity issues with the commit server. This has been fixed. #1814097 (Bug #98909) ** 'p4 change -df' of submitted changes containing stream edits now displays the correct messages about the stream and files in the change that prevent the delete. #1814096 (Bug #98925, #98910) ** Submitted changes containing stream only edits now have the correct change root field set in db.change. #1812373 (Bug #97309) * A 'p4 reconcile' that would add a huge number of files could cause an Rpc buffer too big error. This has been fixed. #1812099 (Bug #98861) ** 'p4 renameuser' would leave orphaned locks when the user being renamed has opened exclusive checkout files (+l). This has been fixed. #1812005 (Bug #97309) ** The service types ClusterServer and Zookeeper stored in the server spec are no longer supported; assigning these service types in a server spec will raise a warning. #1808584 (Bug #98852) ** The server now consistently enforces a 1024 argument limit to the arguments supplied to trigger programs. #1808412 (Bug #98651) ** The server can enter an infinite loop trying to join the rev and have table during a sync request using the #have revision specifier and a changed client view. This has been fixed. #1804485 (Bug #90995) ** Changelist checksums for changes containing task stream files could erroneously report as 'differ' in the integrity.csv structured log on replica servers. This has been fixed. #1803518 (Bug #95206) ** 'p4 unshelve' is now allowed for 'partitioned' type clients.