svn.pm #7

# Change User Description Committed
#7 28901 marc_tooley ""
#6 5849 marc_tooley VCP: final (?) fix for tracking down copyfrom-rev entries.

After additional experimentation, it turns out SVN copyfrom-rev
is the rev the svn workspace is currently updated to when the
user does an svn rename, or an svn copy. Therefore, the source
revision is the last *active* revision from that source file.

In the event of an svn rename(), however, svn will report the
deleted revision first, prior to reporting the newly-added
revision. This confuses VCP's LAST_REV_BY_FILEBRANCH struct
and foces us to search back prior to the delete via some other
means.

Well, now we have that other means. Hopefully.

This should be the last fix to get things working with SVN
sources. I hope.
#5 5841 marc_tooley VCP bugfixes: works with external-to-current-path branch sources now.

VCP now converts branches that are from paths outside of the current
scanning range into pure "adds" instead.
#4 5840 marc Spelling fix..
#3 5839 marc_tooley Source::svn problem fix.

For those times that svn doesn't tell us the truth about the
copyfrom-rev, we have to try to retrieve the real one. Sometimes we
haven't seen the copyfrom yet, though, so we have to check to see whether
it's available. Partial fix committed to get it working in special-case
customer scenario.
#2 5836 marc_tooley Fix our svn source of a branch recognition mechanism

Unless there's some sneaky way to call svn that I'm not aware of
(certainly possible!) when you retrieve a file's copyfrom-rev in
a:

svn log -v --xml URL

... it sometimes gives you the copyfrom-rev which doesn't actually
match the true copyfrom-rev. So, this adds in logic to detect that
*and* correct our idea of the true nature of the source.
#1 5835 marc_tooley VCP branch to prep for incoming bugfixes for the Source/svn.pm module.
//guest/perforce_software/revml/lib/VCP/Source/svn.pm
#2 5403 Barrie Slaymaker - Misc logging, maintainability & debugging improvements
#1 5343 Barrie Slaymaker - cvs branched to svn (non functional)
//guest/perforce_software/revml/lib/VCP/Source/cvs.pm
#130 4517 Barrie Slaymaker - VCP::Source::cvs uses earlier_ids to prevent deletes from
  occuring before clones or branches
#129 4507 Barrie Slaymaker - RevML:
    - added <action>, removed <delete>, <placeholder> and <move>
    - added <from_id> for clones (and eventually merge actions)
    - Simplified DTD (can't branch DTD based on which action
      any more)
- VCP::Source::cvs, VCP::Filter::changesets and VCP::Dest::p4
  support from_id in <action>clone</action> records
- VCP::Dest::perl_data added
- VCP::Rev::action() "branch" added, no more undefined action
  strings
- "placeholder" action removed
#128 4487 Barrie Slaymaker - dead code removal (thanks to clkao's coverage report)
#127 4476 Barrie Slaymaker - misc bugfixes
#126 4220 Barrie Slaymaker - VCP::Source::cvs no longer passes a leading '/' on module name for checkout
#125 4039 Barrie Slaymaker - VCP::Source::scan_metadata() API now in place,
- VCP::Source::copy_revs() is fully deprecated.
#124 4035 Barrie Slaymaker - VCP::Source::cvs no longer specifies a user_id or time for
  concocted delete revs
#123 4034 Barrie Slaymaker - VCP::Source::cvs does not set the user_id on branch revs
#122 4032 Barrie Slaymaker - VCP::Dest::p4 now estimates missing metadata
#121 4021 Barrie Slaymaker - Remove all phashes and all base & fields pragmas
- Work around SWASHGET error
#120 4012 Barrie Slaymaker - Remove dependance on pseudohashes (deprecated Perl feature)
#119 4005 Barrie Slaymaker - VCP::Source::cvs: minor, abandoned code removed
#118 3999 Barrie Slaymaker - VCP::Source::cvs parses branch numbers more correcly
#117 3995 Barrie Slaymaker - VCP::Source::cvs parses large RCS text sections with lots of
  "@@" escapes.
#116 3994 Barrie Slaymaker - VCP::Source::cvs splits dead revs in to edit followed by a
         delete rev if need be (not always, but only when needed).
#115 3993 Barrie Slaymaker - Fold in changes from clkao's SVN work
#114 3982 Barrie Slaymaker - VCP::Source no longer leaks memory by delete()ing from a phash
- VCP::Source::cvs now flushes to disk more often to conserve RAM
#113 3979 Barrie Slaymaker - VCP::Source::cvs branch number regex fixed
- VCP::Dest::null --dont-get-revs option added
#112 3970 Barrie Slaymaker - VCP::Source handles rev queing, uses disk to reduce RAM
- Lots of other fixes
#111 3930 Barrie Slaymaker - VCP::Source::cvs and VCP::Dest::p4 handle cloning deletes
- "placeholder" actions and is_placeholder_rev() deprecated in
  favor of is_branch_rev() and is_clone_rev().
- Misc cleanups and minor bugfixes
#110 3923 Barrie Slaymaker - VCP::Source::cvs now uses source_...() where possible to
  avoid using modified fields (for instance fields touched by Map:)
#109 3916 Barrie Slaymaker - Reduce memory consumption
#108 3907 Barrie Slaymaker - Debugging cleanups
#107 3904 Barrie Slaymaker - VCP::Source::cvs copes with consecutive dead revisions by
  concocting edit revisions with the same rev_id but different
  ids.  This is experimental.
#106 3894 Barrie Slaymaker - VCP::Source::cvs RCS scan RE no longer explodes
#105 3855 Barrie Slaymaker - vcp scan, filter, transfer basically functional
    - Need more work in re: storage format, etc, but functional
#104 3850 Barrie Slaymaker - No longer stores all revs in memory
#103 3836 Barrie Slaymaker - Sources no longer cache all revs in RAM before sending
#102 3819 Barrie Slaymaker - Factor send & queueing of revs up in to VCP::Source
#101 3818 Barrie Slaymaker - VCP::Source::{cvs,p4,vsS} use less memory
#100 3813 Barrie Slaymaker - VCP::Rev::previous() is no more
#99 3811 Barrie Slaymaker - fetch_*() and get_rev() renamed get_source_file()
#98 3804 Barrie Slaymaker - Refactored to prepare way for reducing memory footprint
#97 3800 Barrie Slaymaker - <branches> removed from all code
#96 3763 Barrie Slaymaker - VCP::Source::cvs now explains what was ignored and what was
  scanned when it finds both foo,v and Attic/foo,v.
#95 3747 Barrie Slaymaker - VCP::Source::cvs branches vendor tags properly
#94 3746 Barrie Slaymaker - VCP::Source::cvs parses vendor tags when no revisions are present
  on the vendor branch (as per Marc Tooley's patch)
    - add test for said parsing
#93 3744 Barrie Slaymaker - VCP::Source::cvs understands a source filespec of "..." for local
  respositories (unless --use-cvs)
#92 3681 Barrie Slaymaker - VCP now scans much more of real_vss_1 and converts it to revml
#91 3677 Barrie Slaymaker - rev_root sanity check is now case insensitive on Win32
- Parens in source filespecs are now treated as regular
  characters, not capture groups
- ** is not treated as '...'
#90 3568 Barrie Slaymaker - Use xchdir() instead of chdir()
#89 3532 John Fetkovich changed File::Spec->rel2abs( blah, start_dir )
to      start_dir_rel2abs blah
everywhere.

which
   does the same thing
   and is defined in VCP::Utils
#88 3523 John Fetkovich more ui defaults and checks added
#87 3489 Barrie Slaymaker - Document options emitted to .vcp files.
#86 3477 Barrie Slaymaker - Make --rev-root only available in VCP::Source::p4
#85 3462 Barrie Slaymaker - Make sure bootstrap regexps get compiled
#84 3460 Barrie Slaymaker - Revamp Plugin/Source/Dest hierarchy to allow for
  reguritating options in to .vcp files
#83 3418 Barrie Slaymaker - Better progress reporting.
- VCP::Source::cvs now actually passes the -k option through to cvs
#82 3384 John Fetkovich moved setting of default repo_id
#81 3285 John Fetkovich In 'sub new' constructor, Only call parse_cvs_repo_spec if a $spec is
       provided.  parse_cvs_repo_spec also now sets repo_id.
#80 3274 John Fetkovich split part of 'sub new' into 'sub init'
#79 3206 John Fetkovich documentation changes
#78 3205 John Fetkovich pod improvements
#77 3199 John Fetkovich Improved documentation of --bootstrap switch.
#76 3167 Barrie Slaymaker Add profiling report that details various chunks of time
       taken.
#75 3166 Barrie Slaymaker Remove stale code, update to _run3 calling conventions
#74 3155 Barrie Slaymaker Convert to logging using VCP::Logger to reduce stdout/err spew.
       Simplify & speed up debugging quite a bit.
       Provide more verbose information in logs.
       Print to STDERR progress reports to keep users from wondering
       what's going on.
       Breaks test; halfway through upgrading run3() to an inline
       function for speed and for VCP specific features.
#73 3133 Barrie Slaymaker Make destinations call back to sources to check out files to
       simplify the architecture (is_metadata_only() no longer needed)
       and make it more optimizable (checkouts can be batched).
#72 3131 Barrie Slaymaker Double the speed of the RCS file parser.
       Deprecate VCP::Revs::shift() in favor of remove_all().
#71 3129 Barrie Slaymaker Stop calling the slow Cwd::cwd so much, use start_dir
       instead.
#70 3120 Barrie Slaymaker Move changeset aggregation in to its own filter.
#69 3106 Barrie Slaymaker Remove an unused field (state) from VCP::Rev
       optimize and bugfix labelmap
#68 3096 Barrie Slaymaker Tuning
#67 3086 Barrie Slaymaker Optimize change aggregation from something like O(N^2)
       down to something more reasonable.  Noticable only
       on large transfers.
#66 3081 Barrie Slaymaker Get cvs->p4 propogation branches with multiple tags working
       to spec.
#65 3075 Barrie Slaymaker Make all empty branches be timestamped at $last_rev_time_in_cvsroot + 1
       second.
#64 3068 Barrie Slaymaker Note a cleanup to be done someday
#63 3067 Barrie Slaymaker Improve revision linking logic for direct-read case
#62 3061 Barrie Slaymaker Make VCP use the first label for a branch instead of the last
#61 3038 Barrie Slaymaker Get proper identification of founding revisions implemented.
#60 3035 Barrie Slaymaker code format tweak.
#59 3032 Barrie Slaymaker Fix rev_id parsing RE
#58 3013 Barrie Slaymaker Clean up minor undefined var warning discovered in testing
#57 3010 Barrie Slaymaker Log the number of tag applications (the xzfree86 repo has a
       lot of tags applied to each file rev, I need numbers).
#56 3007 Barrie Slaymaker Read CVS vendor branche tags
#55 2982 Barrie Slaymaker Treat 1.0, 2.0, 3.0 as first revs
#54 2979 Barrie Slaymaker Put all 1.x, 2.x, 3.x, etc. on the main dev trunk
       (1.1.1.x, 1.1.2.x, etc. are still separate branches)
#53 2973 Barrie Slaymaker Fix handling of branched but unchanged files
#52 2972 Barrie Slaymaker Interim checkin
#51 2938 John Fetkovich added empty() calls
#50 2925 Barrie Slaymaker Source cleanup; no significant changes
#49 2900 Barrie Slaymaker Handle case where the first rev in a branch is deleted.
#48 2824 John Fetkovich removed CVS_CONTINUE field from Source/cvs.pm, and added
       CONTINUE field and continue accessor to Source.pm.  Moved parsing
       of the --continue option also.
#47 2809 Barrie Slaymaker Implement --repo-id in Plugin.pm, refactor source & dest
       options parsing starting in VCP::Source::cvs (need to
       roll out to other sources and dests), get t/91cvs2revml.t
       passing again (first time in months! branching and
       --continue support works in cvs->foo!).
#46 2802 John Fetkovich Added a source_repo_id to each revision, and repo_id to each
Source and Dest.  The repo_ids include repository type
(cvs,p4,revml,vss,...) and the repo_server fields.  Changed the
$self->...->set() and $self->...->get() lines in VCP::Dest::* to
pass in a conglomerated key value, by passing in the key as an
ARRAY ref.  Also various restructuring in VCP::DB.pm,
VCP::DB_file.pm and VCP::DB_file::sdbm.pm related to this
change.
#45 2800 Barrie Slaymaker Get --continue working in cvs->foo transfers.
#44 2743 John Fetkovich Add fields to vcp:
         source_name,
         source_filebranch_id,
         source_branch_id,
         source_rev_id,
         source_change_id

        1. Alter revml.dtd to include the fields
        2. Alter bin/gentrevml to emit legal RevML
        3. Extend VCP::Rev to have the fields
        4. Extend VCP::{Source,Dest}::revml to read/write the fields
           (VCP::Dest::revml should die() if VCP tries to emit illegal
           RevML)
        5. Extend VCP::{Source,Dest}::{cvs,p4} to read the fields
        7. Get all tests through t/91*.t to pass
           except those that rely on ch_4 labels
#43 2667 Barrie Slaymaker Convert more to IPC::Run3
#42 2389 John Fetkovich removed calls to methods:
         command_stderr_filter
         command_ok_result_codes
         command_chdir
       and replaced with named Plugin::run_safely method parameters
         stderr_filter
         ok_result_codes
         in_dir
       respectively, where possible.
#41 2337 Barrie Slaymaker Correct the parser, reduce memory usage
#40 2331 Barrie Slaymaker tune memory usage of VCP::Source::cvs
#39 2322 Barrie Slaymaker Fix jack-in-the-bug options parsing exposed by .vcp files
#38 2321 Barrie Slaymaker Fix a jack-in-the-bug triggered by changing gentrevml's time outputs.
#37 2293 Barrie Slaymaker Update CHANGES, TODO, improve .vcp files, add --init-cvs
#36 2267 Barrie Slaymaker factor out cvs2revml, test both --use-cvs and direct modes, with times
#35 2266 Barrie Slaymaker clean up --use-cvs doc
#34 2245 Barrie Slaymaker cvs -r (re)implemented for direct reads, passes all cvs-only tests
#33 2241 Barrie Slaymaker RCS file scanning improvements, implement some of -r
#32 2240 Barrie Slaymaker Start on cvs -r option support.
#31 2236 Barrie Slaymaker Debug, speed up cvs file parsing
#30 2235 Barrie Slaymaker Debugging cvs speed reader.
#29 2228 Barrie Slaymaker working checkin
#28 2199 Barrie Slaymaker um, comment out the cache I was using to debug.
#27 2153 Barrie Slaymaker checkin
#26 2151 Barrie Slaymaker checkin
#25 2042 Barrie Slaymaker Basic source::p4 branching support
#24 2026 Barrie Slaymaker VCP::8::cvs now supoprt branching
#23 2009 Barrie Slaymaker lots of fixes, improve core support for branches and VCP::Source::cvs
       now supports branches.
#22 2006 Barrie Slaymaker more preparations for branching support,
       handling of cvs :foo:... CVSROOT specs,
       misc fixes, improvements
#21 1998 Barrie Slaymaker Initial, revml and core VCP support for branches
#20 1855 Barrie Slaymaker Major VSS checkin.
 Works on Win32
#19 1728 Barrie Slaymaker CVS on win32, minor bugfixes
#18 1367 Barrie Slaymaker lots of docco updates
#17 1358 Barrie Slaymaker Win32 changes
#16 1330 Barrie Slaymaker Ignore cvs lock mgmt warnings in VCP::Source::cvs.
#15 723 Barrie Slaymaker VCP::Dest::cvs tuning and cvs and p4 bugfixes
#14 705 Barrie Slaymaker Release 0.22.
#13 692 Barrie Slaymaker Add VCP::Utils::p4 and use it to get VCP::Dest::p4 to create it's
own client view as needed.
#12 689 Barrie Slaymaker reinstate -f behavior as the default for VCP::Source::cvs, clean
up -D --> -d doco.
#11 687 Barrie Slaymaker remove -f, tweak deduce_rev_root
#10 630 Barrie Slaymaker Fix bug in CVS log file parsing that made it think it was always
seeing the same file, different revisions over and over again.
Reported by Matthew Attaway.
#9 628 Barrie Slaymaker Cleaned up POD in bin/vcp, added BSD-style license.
#8 627 Barrie Slaymaker Beef up CVS log file parsing.
#7 626 Barrie Slaymaker Removed POD that was older than the current feature set.
#6 625 Barrie Slaymaker Add NOTE about required " " in cvs options.
#5 624 Barrie Slaymaker Add a space to bin/vcp SYNOPSIS after the cvs -r option.
#4 480 Barrie Slaymaker 0.06 Wed Dec 20 23:19:15 EST 2000
   - bin/vcp: Added --versions, which loads all modules and checks them
     for a $VERSION and print the results out.  This should help with
     diagnosing out-of-sync modules.
   - Added $VERSION vars to a few modules :-).  Forgot to increment any
     $VERSION strings.
   - VCP::Dest::cvs: The directory "deeply" was not being `cvs add`ed on
     paths like "a/deeply/nested/file", assuming "deeply" had no files
     in it.
   - VCP::Dest::revml: fixed a bug that was causing files with a lot of
     linefeeds to be emitted in base64 instead of deltaed.  This means
     most text files.
   - Various minor cleanups of diagnostics and error messages, including
     exposing "Can't locate Foo.pm" when a VCP::Source or VCP::Dest
     module depends on a module that's not installed, as reported by
     Jeff Anton.
#3 478 Barrie Slaymaker 0.05 Mon Dec 18 07:27:53 EST 2000
   - Use `p4 labels //...@label` command as per Rober Cowham's suggestion, with
     the '-s' flag recommended by Christopher Siewald and
     Amaury.FORGEOTDARC@atsm.fr.  Though it's actually something like

       vcp: running /usr/bin/p4 -u safari -c safari -p localhost:5666 -s files
       //.../NtLkly //...@compiler_a3 //.../NtLkly //...@compiler_may3

     and so //on //for 50 parameters to get the speed up.  I use the
     //.../NtLkly "file" as //a separator between the lists of files in various
     //revisions.  Hope nobody has any files named that :-).  What I should do
     is choose a random label that doesn't occur in the labels list, I guess.
   - VCP::Source::revml and VCP::Dest::revml are now binary, control code, and
     "hibit ASCII" (I know, that's an oxymoron) clean.  The <comment>, <delta>,
     and <content> elements now escape anything other than tab, line feed,
     space, or printable chars (32 <= c <= ASCII 126) using a tag like '<char
     code="0x09">'.  The test suite tests all this.  Filenames should also
     be escaped this way, but I didn't get to that.
   - The decision whether to do deltas or encode the content in base64 is now
     based on how many characters would need to be escaped.
   - We now depend on the users' diff program to have a "-a" option to force it
     to diff even if the files look binary to it.  I need to use Diff.pm and
     adapt it for use on binary data.
   - VCP::Dest::cvs now makes sure that no two consecutive revisions of the
     same file have the same mod_time.  VCP::Source::p4 got so fast at pulling
     revisions from the repositories the test suite sets up that CVS was not
     noticing that files had changed.
   - VCP::Plugin now allows you to set a list of acceptable result codes, since
     we now use p4 in ways that make it return non-zero result codes.
   - VCP::Revs now croaks if you try to add two entries of the same VCP::Rev
     (ie matching filename and rev_id).
   - The <type> tag is now limited to "text" or "binary", and is meant to
     pass that level of info between foreign repositories.
   - The <p4_info> on each file now carries the one line p4 description of
     the file so that p4->p4 transferes can pick out the more detailed
     info.  VCP::Source::p4, VCP::Dest::p4 do this.
   - VCP::{Source,Dest}::{p4,cvs} now set binaryness on added files properly,
     I think.  For p4->p4, the native p4 type is preserved.  For CVS sources,
     seeing the keyword substitution flag 'o' or 'b' implies binaryness, for
     p4, seeing a filetype like qr/u?x?binary/ or qr/x?tempobj/ or "resource"
     implies binaryness (to non-p4 destinations).  NOTE: Seeing a 'o' or 'b'
     in a CVS source only ends up setting the 'b' option on the destination.
     That should be ok for most uses, but we can make it smarter for cvs->cvs
     transfers if need be.
#2 468 Barrie Slaymaker - VCP::Dest::p4 now does change number aggregation based on the
  comment field changing or whenever a new revision of a file with
  unsubmitted changes shows up on the input stream.  Since revisions of
  files are normally sorted in time order, this should work in a number
  of cases.  I'm sure we'll need to generalize it, perhaps with a time
  thresholding function.
- t/90cvs.t now tests cvs->p4 replication.
- VCP::Dest::p4 now doesn't try to `p4 submit` when no changes are
  pending.
- VCP::Rev now prevents the same label from being applied twice to
  a revision.  This was occuring because the "r_1"-style label that
  gets added to a target revision by VCP::Dest::p4 could duplicate
  a label "r_1" that happened to already be on a revision.
- Added t/00rev.t, the beginnings of a test suite for VCP::Rev.
- Tweaked bin/gentrevml to comment revisions with their change number
  instead of using a unique comment for every revision for non-p4
  t/test-*-in-0.revml files.  This was necessary to test cvs->p4
  functionality.
#1 467 Barrie Slaymaker Version 0.01, initial checkin in perforce public depot.