Helix SwarmHelix Swarm

p4ftpsync-manual.html

  • //
  • guest/
  • richard_gruet/
  • p4ftpsync/
  • p4ftpsync-manual.html #2
  • View
  • Commits
  • Open Download .zip Download (32 KB) 
  1. <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
  2. <html xmlns="http://www.w3.org/1999/xhtml">
  3. <head>
  4. <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
  5. <meta name="description" content="Manual for p4ftpsync, a Python script to synchronize a remote site with a P4 depot via FTP." />
  6. <meta name="keywords" content="Perforce,P4,FTP,sync,synchronization,remote,site,web,python" />
  7. <meta name="author" content="Richard Gruet" />
  8. <title>p4ftpsync Manual</title>
  9.  
  10. <style type="text/css">
  11.     h1,h2,h3,h4,h5 {
  12.         font-family:Arial, Helvetica, sans-serif;
  13.         margin-left: 5px;
  14.     }
  15.     
  16.     h1 {
  17.         margin: 25px 10% 40px 10%;
  18.         margin-bottom: 40px;
  19.         border: gray dotted 2px;
  20.         background-color:#FFCC99;
  21.     }
  22.     
  23.     h2 {
  24.         border-bottom: black solid 1px;
  25.     }
  26.     
  27.     h4 {
  28.         margin-bottom: 5px;
  29.     }
  30.     div {
  31.         margin-bottom: 10px;
  32.         margin-left: 15px;
  33.         font-family: Georgia, "Times New Roman", Times, serif;
  34.         /* font-family: Geneva, Arial, Helvetica, sans-serif; */
  35.         font-size:12px;
  36.         }
  37.  
  38.     div.optionCategory {
  39.         margin-top: 20px;
  40.         margin-right: 5%;
  41.         font-weight: bold;
  42.         font-style: italic;
  43.         border-bottom: gray solid 1px;
  44.     }
  45.     
  46.     div.option {
  47.         font-weight: bold;
  48.         margin-left: 40px;
  49.         margin-bottom: 2px;
  50.     }
  51.     
  52.     div.optionDesc {
  53.         margin-left: 80px;
  54.     }
  55.  
  56.     pre {
  57.         border: black solid 1px;
  58.         border-width: 1px 0 1px 0;
  59.         margin: 7px 5% 7px 3%;
  60.         padding: 5px 5px 5px 5px;
  61.         background-color: #EFEFEF /* #FFCC99;*/
  62.     }
  63. </style>
  64. </head>
  65.  
  66. <body>
  67. <div style="font-size:smaller">
  68.   <a href="javascript:location='mailto:\u0070\u0034\u0066\u0074\u0070\u0073\u0079\u006e\u0063\u0040\u0072\u0067\u0072\u0075\u0065\u0074\u002e\u006e\u0065\u0074';void 0">Richard Gruet</a> - Created September 15, 2005 - Last revised 
  69.   <!-- #BeginDate format:Am1 -->September 20, 2005<!-- #EndDate --></div>
  70. <h1 align="center">p4ftpsync <span style="font-size:smaller">- Synchronizes a remote site with a P4 depot via FTP</span></h1>
  71.  
  72. <h2>Table Of Contents</h2>
  73. <div class="toc">
  74.   <ul>
  75.     <li><a href="#overview">Overview</a></li>
  76.     <li><ul>
  77.       <li><a href="#Whatsthat">What's that ?</a></li>
  78.       <li><a href="#WhatsFor">What's for ?</a></li>
  79.     </ul></li>
  80.     <li><a href="#requirements">Requirements</a></li>
  81.     <li><a href="#installation">Installation</a></li>
  82.     <li><a href="#startup">Start-up</a></li>
  83.     <li><a href="#commandReference">Command Reference</a></li>
  84.       <li><ul>
  85.         <li><a href="#reverseSync">Reverse Synchronization</a></li>
  86.         <li><a href="#validOptions">Valid Options</a></li>
  87.         <li><a href="#defaultValues">Default Values</a></li>
  88.         <li><a href="#cache">Cache</a></li>
  89.         <li><a href="#generatedScript">Generated FTP script</a></li>
  90.         <li><a href="#log">Log</a></li>
  91.       </ul></li>
  92.     <li><a href="#limitations">Limitation, Bugs, Todo</a></li>
  93.     <li><a href="#license">License</a></li>
  94.   </ul>
  95. </div>
  96.  
  97. <!-- OVERVIEW -->
  98. <a name="overview" id="overview"></a><h2>Overview</h2>
  99.  
  100. <a name="Whatsthat" id="Whatsthat"></a>
  101. <h3>What's that ?</h3>
  102. <div><code>p4ftpsync.py</code> is a Python script which synchronizes a remote site with a P4 repository via Ftp, and vice versa. It is specially useful when you have not enough control over the remote host to be able to install a Perforce client there, which would make synchronization easy (at least the P4 to remote site sync).</div>
  103.  
  104. <a name="WhatsFor" id="WhatsFor"></a><h3>What for ?</h3>
  105. <div>I am a member of a team which designs web sites. We want to keep the pages and code for our sites under P4 control,
  106.      and <strong>synchronize</strong> the remote live sites from time to time to reflect the changes. Occasionally we
  107.      have also to modify the pages <strong>directly</strong> on the live sites (in emergency situations!), and some
  108.      of our clients can even edit the pages directly using Macromedia <a href="http://www.macromedia.com/software/contribute/"
  109.      title="Macromedia Contribute">Contribute</a>: In these cases we want to have the P4 repository <strong>synced back</strong>.</div>
  110.  
  111. <div>If you have enough control over the web host where your live site resides, i.e. if you can install a P4 client, the first sync
  112.     (P4 to the live site) is easily accomplished via a <code>P4 sync</code> command, but not the second since P4 doesn't detect
  113.      automatically changes in workspaces, you still have to manually open the files for add or delete.
  114.      And if you have not enough privileges to be able to install a P4 client, which is the case of a lot of small to medium
  115.      sites using cheap Web Hosting, you just can't use the P4 client install/P4 sync strategy [this strategy and others are developped
  116.      in an interesting article on the P4 web site: <a href="http://www.perforce.com/perforce/wcm.html"> Web Content management in
  117.      Perforce</a>].</div>
  118.  
  119. <div>I searched for alternate solutions but could not find any - I could have missed something though, please let me know! -
  120.      So I eventually wrote a program to perform the P4 to remote site synchronization, which I extended later to perform
  121.      the reverse sync as well.</div>
  122.  
  123. <div> The problem is to emulate a P4 remote client accessible only via FTP. The basic idea is to use a dedicated local
  124.       client/workspace that reflects (mirrors) the state of the remote site. If we P4 sync this workspace to a certain
  125.       revision, we get the exact list of changes that occurred, which we can then propagate (<em>push</em>) to the remote
  126.       site using FTP operations. Conversely, with the restriction that the remote site be synced to the head revision,
  127.       we can detect changes made directly to the site by comparing the remote site and the local workspace (synced to
  128.       the head revision beforehand).</div>
  129.  
  130. <div>Basically that's what <code>p4ftpsync</code> does. It's plain <a href="http://python.org/"
  131.     title="Python language official site" target="_blank">Python</a>, one single loooooong file (>3000 lines!),
  132.      uses no external library (not even the Perforce Python API) and requires only the basic P4 client install + the
  133.      Python language (not required with the (almost) standalone executable version). It works with Perforce
  134.      server version 2005.1 but should work with older versions as well (it doesn't use any fancy features but relies
  135.      on P4 output parsing so it could be sensitive to change in format). The code uses some tricks to optimize speed
  136.      (file caching, multi-threaded FTP operations) and so far it seems to be rather reliable, at least for common simple
  137.      operations (eg sync to head revision).</div>
  138.  
  139. <!-- REQUIREMENTS -->
  140. <a name="requirements" id="requirements"></a><h2>Requirements</h2>
  141. <ul>
  142.   <li>Python 2.2+ (not required if the Windows standalone executable version is used). </li>
  143.   <li>Basic (text mode) Perforce client installed locally. No Python API required.</li>
  144.   <li>Your remote site accessible via FTP. The more possible simultaneous connections the faster transfers will be. </li>
  145. </ul>
  146.  
  147. <!-- INSTALLATION -->
  148. <a name="installation" id="installation"></a><h2>Installation</h2>
  149. <div>You have the choice between 2 distributions : </div>
  150. <ul>
  151.   <li>A Python script <a href="p4ftpsync.py" title="Python script">p4ftpsync.py</a>
  152.      (you will need <a href="http://www.python.org/download/">Python</a> 2.2+ to execute it).</li>
  153.   <li>A Win32 standalone executable <a href="p4ftpsync.exe" title="Standalone executable">p4ftpsync.exe</a>.
  154.       Actually you may also need to download <a href="MSVCR71.dll" title="MicroSoft C Runtime library">MSVCR71.dll</a>
  155.       if not already present on your system (this is because Python 2.4 is compiled with MicroSoft VC7, which creates
  156.       executables that depend on that dll). Put it in the same directory as the .exe.</li>
  157. </ul>
  158.  
  159. <div>In both cases you must have installed at least the <a href="http://perforce.com/perforce/loadprog.html"
  160.      title="Perforce downloads" target="_blank">Perforce client</a>, and added the P4 install directory to
  161.      your <code>PATH</code> environment variable before you can run <code>p4ftpsync</code>.
  162.      Check it by opening a terminal and type: </div>
  163. <pre>p4</pre>
  164. <div>You should get a P4 help message. If not, check your P4 install and <code>PATH</code> envt var.</div>
  165.  
  166. <div><code>p4ftpsync</code> also requires that you <strong>create a dedicated P4 <em>client</em></strong> (workspace).
  167.     This workspace will be used to <em>mirror</em> the remote site in order to perform the synchronization.
  168.     Use the Perforce command <code>p4 client</code> to create a client. Define the client <em>map spec</em>
  169.     so to include the part of the depot that you want to synchronize.</div>
  170.  
  171. <!-- START-UP -->
  172. <a name="startup" id="startup"></a><h2>Start-up </h2>
  173. <div>We assume that you have installed everything as described above and created a dedicated P4 client
  174.     (called here <em>syncClient</em>). The examples described use the python script <code>p4ftpsync.py</code>,
  175.     but apply to the executable version as well.</div>
  176.  
  177. <div>To get help, type :</div>
  178. <pre>[python] p4ftpsync[.py] -h  (or --help)
  179. </pre>
  180. <div>You can skip the "python" if <code>p4ftpsync.py</code> is in your <code>PATH</code> and files with extension
  181.     <code>.py</code> are associated to the Python interpreter (which is generally the case). On windows, you can even
  182.     omit the extension <code>.py</code>, provided you have added .py to the list of extensions in the <code>PATHEXT</code>
  183.     environment variable.</div>
  184.  
  185. <div>Let's try a normal (P4 to remote site) synchronization. Say you have your site on ftp server <em>ftpServerAddress</em>
  186.     accessible by user <em>userName</em>, password <em>passwd</em>, rooted at <em>/mySite/www/</em>. Say the corresponding
  187.     location in the P4 depot is at <em>//depot/mySite/www/...</em>. You want to synchronize the remote site with the
  188.     head revision of the depot. Type:
  189. </div>
  190. <pre>
  191. p4ftpsync.py -v -t --p4Passwd <em>password</em> --p4Client <em>syncClient</em> --ftpHost <em>ftpServerAddress</em>
  192.  --ftpUser <em>userName</em> --ftpPasswd <em>passwd</em> --ftpRoot <em>/mySite/www/</em> <em>//depot/mySite/www/...</em>
  193. </pre>
  194. <div>
  195.   <ul>
  196.     <li>If you omit a mandatory option, p4ftpsync will ask you to enter it interactively. Some P4 options are
  197. 	    <em>guessed</em> from the environment, and p4ftpsync saves certain option values from one session to the
  198. 		other and uses them as defaults (more details about this later).</li>
  199.     <li>Option <strong>-v</strong> (or --verbose) displays more detailed information messages on the screen (there
  200.     is also an -always verbose- log, see file <code><em>sameDirAsP4ftpsync</em>/p4ftpsync.log</code>.</li>
  201.     <li>Option <strong>-t</strong> (or --test)  performs a <em>dry run</em>: synchronization actions are listed but not
  202.     actually executed. The 2 options can be grouped together as <code>-vt</code> since they have no parameters.</li>
  203.   </ul>
  204. </div>
  205.  
  206. <div>If you perform this sync for the <strong>first time</strong>, <code>p4ftpsync</code> will try to upload
  207.     <strong>all</strong> the files in <em>//depot/mySite/www/...</em> to the remote site. Actually, the actions
  208. 	are the same as a <code>p4 sync //depot/mySite/www/...</code> would do, except that they are converted to ftp
  209. 	actions to update the remote site (viewed as a remote P4 workspace). Next runs will only transfer changes since
  210. 	the last run, as <code>P4 sync</code> would do (however option <strong>-f</strong> allows to <strong>force</strong>
  211.     the complete refresh of the workspace and therefore will upload all the files again).
  212. </div>
  213. <div>As you can see, there are a <strong>lot of parameters</strong> to provide to <code>p4ftpsync</code> !
  214.      Fortunately the program tries to <strong>help you</strong> :
  215.   <ul>
  216.     <li>by defaulting missing P4 parameters p4Host, p4User, p4Passwd to their current values in the environment
  217.     (P4PORT, P4USER, P4PASSWORD - actually the program gets this info by parsing the output of a <code>p4 info</code> command).</li>
  218.     <li> The values of several options are also <strong>saved</strong> on disk during a session and used as defaults in
  219.          the next session (passwords are <strong>never</strong> stored).</li>
  220.     <li> Finally, the user is prompted to dynamically enter the missing <strong>required</strong> parameters. </li>
  221.   </ul>
  222.   In spite of these facilities, if you have to routinely perform a sync task, it is more convenient  to define
  223.   a small <strong>script</strong> for each site to synchronize, which will call <code>pftpsync</code> with the
  224.   appropriate arguments. I find personally easier to define <strong>two</strong> scripts for the normal and reverse
  225.   sync respectively (since they have different options), that I call (on Windows) <code>sync<em>MySite</em>live.bat</code>
  226.   and <code><strong>r</strong>sync<em>MySite</em>live.bat</code>. Here is an example of such scripts :
  227. </div>
  228. <pre>
  229. @echo off
  230. REM <strong>sync<em>MySite</em>live.bat</strong>:
  231. REM Synchronizes (depot head revision -> live site) the <em>MySite</em> live site via FTP.
  232. REM
  233. REM For reverse synchronization, see rsync<em>MySite</em>live.bat.
  234. REM Uses P4 workspace ("client") <em>syncClient</em> as a mirror.
  235. REM You can pass additional args like :
  236. REM   -v, --verbose  to get a more detailed trace,
  237. REM   -f, --force to force P4 to resync to the given revision
  238. REM   ... and many more! use option -h for details.
  239. REM 
  240. p4ftpsync.py %* --p4Passwd <em>password</em> --p4Client <em>syncClient</em> --ftpHost <em>ftpServerAddress</em>
  241.  --ftpUser <em>userName</em> --ftpPasswd <em>passwd</em> /mySite/www/ --exclude @<em>p4ftpSyncDir</em>\sync<em>MySite</em>live.excludes.txt
  242.  //depot/mySite/www/...
  243. </pre>
  244. <div>(Replace <code>p4ftpsync.py</code> with <code>p4ftpsync.exe</code> if needed)</div>
  245.  
  246. <pre>
  247. @echo off
  248. REM <strong>rsync<em>MySite</em>live.bat</strong>:
  249. REM <strong>Reverse</strong> synchronization (live site -> P4) of live site <em>MySite</em> via FTP.
  250. REM Creates a P4 changelist for the changes and submits it.
  251. REM
  252. REM For normal synchronization (p4 -> live site), see sync<em>MySite</em>live.bat.
  253. REM Uses P4 workspace ("client") <em>syncClient</em> as a mirror.
  254. REM You can pass additional args like :
  255. REM   -v, --verbose  to get a more detailed trace,
  256. REM   -f, --force to disable the file desc cache and force re-reading of remote files
  257. REM   ... and many more! use option -h for details.
  258. REM 
  259. p4ftpsync.py %* <strong>--reverse --submit</strong> --p4Passwd <em>password</em> --p4Client <em>syncClient</em> --ftpHost <em>ftpServerAddress</em>
  260.  --ftpUser <em>userName</em> --ftpPasswd <em>passwd</em> /mySite/www/ <strong>--comment "GG3:rsync: Integrated changes made on the
  261.  <em>mySite</em> live site." --mailto "john@doe.org,jane@jungle.com"</strong> --exclude @<em>p4ftpSyncDir</em>\sync<em>MySite</em>live.excludes.txt
  262.  <strong>--smtpServer smtp.myIsp.com</strong> //depot/mySite/www/...
  263. </pre>
  264. <div>(options specific to <strong>reverse</strong> sync are in bold).
  265.   <ul>
  266.     <li>option <strong>--reverse</strong> tells <code>p4ftpsync</code> to perform a <em>reverse</em> synchronisation, instead
  267.        of the default "normal" one.</li>
  268.     <li>option <strong>--submit</strong> tells <code>p4ftpsync</code> to automatically submit the P4 changelist created
  269.        for the changes detected (if any). By default the changelist is not submitted to let you have a look at the
  270.        changes and possibly revert some of them (useful during your first trials).</li>
  271.     <li>option <strong>--comment</strong> overrides the default auto-generated changelist comment with a customized one.</li>
  272.     <li>option <strong>--mailto</strong> tells <code>p4ftpsync</code> to send a report e-mail to each address listed.
  273. 	    The	<strong>--smtpServer</strong> option specifies the address of the SMTP server to use (the default is
  274. 	  <em>localhost:25</em>. Use options --smtpUser and --smtpPasswd if  the server requires authentication).
  275. 	  Default is to <strong>not</strong> send an email report.</li>
  276.   </ul>
  277. </div>
  278.  
  279. <div>Both scripts share the same <strong>list of exclusions</strong> (which makes sense). This is why the list is
  280.      contained in a <strong>external file</strong> (<em>p4ftpSyncDir</em>\syncMySitelive.excludes.txt), rather than
  281.      directly passed as a command line argument (this is indicated by the use of <code><strong>@</strong></code>
  282.      in option --exclude). Files to exclude from the sync are specified as <em>patterns</em>, one per line.
  283.      Patterns are actually Python <em>regular expressions</em>, implicitely terminated by <code>$</code>
  284.      and preceded by <code>^.*</code>, which in practice means that patterns will be matched against the
  285. 	 <strong>end</strong> of the files to check. For examples of patterns, see option <strong>--exclude</strong>
  286. 	 in the <em><a href="#commandReference">Command Reference</a></em> section below.
  287. </div>
  288. <div>Once you have perfectly tuned the parameters in your 2 scripts, you may consider <strong>scheduling</strong> their
  289.      execution, for example on a daily basis, using Control Panel/Scheduled Tasks on Windows, cron on Unix, etc...<br />
  290.      Synchronizing and reverse synchronizing is like having two different p4 clients accessing and modifying the
  291.      same files. If a file can be modified <strong>directly</strong> on the live site, then a situation of
  292.      <strong>conflict</strong> (e.g. simultaneous edits of the same file) is possible, and must be <em>resolved</em>.
  293.      By scheduling the <strong>reverse</strong> sync <strong>first</strong>, then the "normal" sync, rather
  294.      than the contrary, one guarantees that any conflict will be detected by Perforce and scheduled for resolve.
  295.      This is not true if the scripts are scheduled the other way (normal, then reverse): in this case any change 
  296.      on the live site will be silently overwritten by a change (done elsewhere) already submitted in P4.
  297. </div>
  298. <div>The strategy above is somewhat "blind" in that it synchronizes the live site at fixed intervals without any
  299.      consideration for actual changes made. You don't always want to propagate changes submitted in P4 to
  300.      the live site ASAP. Maybe you want to accumulate changes somewhere and once you are ready, transfer
  301.      them to the live site. A possible strategy to achieve this is to create a <strong>branch for the sync</strong>:
  302.      you make the changes into the main/development branch, and when you feel ready you <em>integrate</em>
  303.      them into the sync branch, which of course will be the one specified for the <em>whatToSync</em> parameter
  304.      to <code>p4ftpsync</code>.
  305. </div>
  306.  
  307. <!-- COMMAND REFERENCE -->
  308. <a name="commandReference" id="commandReference"></a>
  309. <h2>Command Reference</h2>
  310. <div style="font-size:smaller">(this is basically a formatted copy of the ouput of <code>p4ftpsync --help</code>)</div>
  311.  
  312. <pre>[python] p4ftpsync[.py] [options] <em>whatToSync</em></pre>
  313. <div> <em>whatToSync</em> specifies what to sync, and must be a valid P4 file spec such as the ones used in the
  314.       <code>P4 sync</code> command (e.g. <code>//depot/MyProject/MyDir/...</code>,
  315.       <code>//depot/Proj2/main.c#2</code>, <code>@label</code>, etc...). For the <em>reverse</em> sync, the revision
  316.       range info is N.S., since the comparison is always done with the HEAD revision.</div>
  317.  
  318. <a name="reverseSync" id="reverseSync"></a><h4>Reverse sync</h4>
  319. <div><code>p4ftpsync</code> may also be run in <em>reverse sync mode</em> (option -r). In this mode, the files
  320.  on the remote site are compared with the latest revision files in P4 and any change on the remote site is reported
  321.  into a new P4 <em>changelist</em> (optionally submitted at the end of the process). This mode is handy if some changes are
  322.  done directly on the remote site and you want to easily keep your P4 repository up to date. <em>Symbolic links</em>
  323.  are supported on the remote site but <strong>mapped to real files</strong> locally/in P4 named like the link.</div>
  324.  
  325. <a name="validOptions" id="validOptions"></a><h4>Valid options</h4>
  326.  
  327. <div class="optionCategory">Perforce options</div>
  328.  
  329. <div class="option">--p4Port <span style="font-weight:normal"><em>host</em>[:<em>port</em>]</span></div>
  330. <div class="optionDesc">P4 host:port to use, default port 1666 [default: current config, see
  331.      <a href="#defaultValues" target="_self">Default values</a>]</div>
  332.  
  333. <div class="option">--p4User <span style="font-weight:normal"><em>userName</em></span></div>
  334. <div class="optionDesc">P4 user name to use [default: see <a href="#defaultValues" target="_self">Default values</a>].</div>
  335.  
  336. <div class="option">--p4Passwd <span style="font-weight:normal"><em>password</em></span></div>
  337. <div class="optionDesc">P4 user password [default: see <a href="#defaultValues" target="_self">Default values</a>].</div>
  338.  
  339. <div class="option">--p4Client <span style="font-weight:normal"><em>client</em></span></div>
  340. <div class="optionDesc">P4 client to use as the mirror of the remote site [default: see
  341.      <a href="#defaultValues" target="_self">Default values</a>]. </div>
  342.  
  343. <div class="optionCategory">FTP options (for the remote site)</div>
  344.  
  345. <div class="option">--ftpHost <span style="font-weight:normal"><em>host</em>[:<em>port</em>]</span></div>
  346. <div class="optionDesc">FTP host IP address, default port 21 [mandatory]</div>
  347.  
  348. <div class="option">--ftpUser <span style="font-weight:normal"><em>userName</em></span></div>
  349. <div class="optionDesc">FTP user name [mandatory]</div>
  350.  
  351. <div class="option">--ftpPasswd <span style="font-weight:normal"><em>password</em></span></div>
  352. <div class="optionDesc">FTP user password [default: will be prompted]</div>
  353.  
  354. <div class="option">--ftpRoot <span style="font-weight:normal"><em>rootRelativePath</em></span></div>
  355. <div class="optionDesc">Path of the root folder of the site [mandatory]</div>
  356.  
  357. <div class="optionCategory">Normal sync (p4 to remote site) options</div>
  358. <div class="option">-o, --scriptDir <span style="font-weight:normal"><em>localPath</em></span></div>
  359. <div class="optionDesc">Directory in which to generate the Python update script in normal sync
  360.                        [default: <code><em>thisProgDir</em>/p4FtpSyncScripts</code>]</div>
  361.   
  362. <div class="optionCategory">Reverse sync (remote site to P4) options</div>
  363.  
  364. <div class="option">-r, --reverse</div>
  365. <div class="optionDesc">Reverse synchronization: Changes occurred on the remote site are detected, and
  366.          a P4 changelist is created but not submitted, unless option -s is specified.</div>
  367.  
  368. <div class="option">-s, --submit</div>
  369. <div class="optionDesc">Submit the P4 changelist created for the changes detected. The default is to <strong>not</strong>
  370.          submit, so the changes can be reviewed before (manual) submit.</div>
  371.  
  372. <div class="option">-c, --comment <span style="font-weight:normal"><em>"COMMENT"</em></span></div>
  373. <div class="optionDesc">Optional description for the changelist. A default will be generated if none is specified.</div>
  374.  
  375. <div class="option">-m, --mailto <span style="font-weight:normal"><em>addr1,addr2,...</em></span></div>
  376. <div class="optionDesc">A list of email addresses to send a "file change report" to [default: don't send an email].
  377.                     For now the mail server settings are globals in <code>p4ftpsync.py</code> !</div>
  378.  
  379. <div class="option">--smtpServer <span style="font-weight:normal"><em>host</em>[:<em>port</em>]</span></div>
  380. <div class="optionDesc">SMTP server IP address to use for sending the above mail [default localhost:25]</div>
  381.  
  382. <div class="option">--smtpUser <span style="font-weight:normal"><em>userName</em></span></div>
  383. <div class="optionDesc">SMTP user name if authentication is required [default: None]</div>
  384.  
  385. <div class="option">--smtpPasswd <span style="font-weight:normal"><em>password</em></span></div>
  386. <div class="optionDesc">SMTP user password [default: None]</div>
  387.  
  388. <div class="option">--fromAddr <span style="font-weight:normal"><em>addr</em></span></div>
  389. <div class="optionDesc">Email address to put in the 'From' field of the report emails [default: p4ftpsync@p4ftpsync.net -dummy!]</div>
  390.  
  391. <div class="optionCategory">Common sync. options</div>
  392.  
  393. <div class="option">-f, --force</div>
  394. <div class="optionDesc">For <em>normal</em> sync: If specified, the target will be resynced even if supposedly up
  395.      to date (same as P4 sync option -f).<br />
  396.      For <em>reverse</em> sync: If specified, the file description <a href="#cache" title="File description Cache">cache</a>
  397.      will <strong>not</strong> be used and the file download &amp; comparison will <strong>always</strong> be done.</div>
  398.   
  399. <div class="option">-t, --test</div>
  400. <div class="optionDesc">Test/preview mode: For <em>normal</em> sync: do not really P4 sync the mirror client workspace,
  401.     generate the script but do not actually FTP.<br />
  402.     For <em>reverse</em> sync: do not create P4 changelist or copy files to client space.</div>
  403.  
  404. <div class="option">-x, --exclude <span style="font-weight:normal"><em>FILE1,FILE2,... | <strong>@</strong>FILE</em></span></div>
  405. <div class="optionDesc">Exclude files from the list of updates to do (handy if some files must remain different locally
  406.     from remotely, e.g. config files, system files). FILEs are specified as a comma separated list (no spaces) of
  407.     <em>patterns</em>, or alternatively listed in a file (@FILE), one per line. <em>Patterns</em> are Python compatible
  408. 	<em>regular expressions</em>. They are implicitely completed with a <code>^.*</code> on the left and a <code>$</code>
  409. 	on the end (unless they already include them), meaning that if the <strong>end</strong> of the path of a file to sync
  410. 	matches one of the patterns, then it will be excluded from the sync. The <code>(?i)</code> flag (IGNORECASE) can be added
  411. 	to a pattern to make the match case-independent.<br /><br />
  412.     <strong>Examples of patterns</strong><br />
  413.     <code>.htaccess</code> (exclude the .htaccess file wherever it appears)<br />
  414. 	<code>/A/B/f.txt</code>  (exclude file f.txt located in folder /A/B/)<br />
  415.     <code>/log/.*</code>  (exclude all files in directory log/)<br />
  416.     <code>/~.*</code> (exclude all files whose name starts with ~)<br />
  417.     <code>.pdf</code> (exclude all pdf files)<br />
  418.     <code>.pdf<strong>(?i)</strong></code> (ditto, but (?i) specifies to <strong>ignore case</strong>,
  419. 	      so .PDF matches too)<br /><br />
  420. </div>
  421.  
  422. <div class="optionCategory">Misc options</div>
  423.  
  424. <div class="option">-v, --verbose</div>
  425. <div class="optionDesc">Verbose: More trace/error messages on stdout.</div>
  426.  
  427. <div class="option">-V, --version</div>
  428. <div class="optionDesc">Print p4ftpsync version on stdout and exit.</div>
  429.  
  430. <div class="option">-h, --help</div>
  431. <div class="optionDesc">Print help message on stdout and exit.</div>
  432.  
  433. <a name="defaultValues" id="defaultValues"></a><h4>Default values</h4>
  434. <div>p4ftpsync stores (in file <code>p4ftpsync.opt</code>) the last values used for options p4Client,
  435.      ftpHost, ftpUser and ftpRoot and uses them if no value is provided on the command line.
  436.      Options are stored twice: as associated to the specific target sync spec, and as generic
  437.      "last session defaults" (ftpRoot is not stored in the latter case).<br />
  438.      When trying to reload the values for a new session, p4ftpsync first attempts to reload the
  439.      target specific values, then the non specific ones.</div>
  440.  
  441. <a name="cache" id="cache"></a><h4>File description cache (reverse sync only)</h4>
  442. <div> Remote file descriptions are saved on disk after a successful synchronization and used on
  443.   subsequent syncs to determine if a file has changed (different file descs), without having to
  444.   download the file, which can save a tremendous amount of time. There is a different cache for every
  445.   ftp host+user (file <code><em>user</em>@<em>ftpHost</em>.fdc</code>)<br /> 
  446.   
  447.   On the <strong>first</strong> run of <code>p4ftpsync</code> for a given site, <strong>all</strong>
  448.   files from the remote site will be downloaded and compared to the files in the depot, quite a lengthy
  449.   and bandwith consuming operation ! However next iterations should use the cache and be considerably
  450.   faster (the only relatively long operation that can't be shortened is the determination of the current
  451.   file structure on the remote site). When the cache is missing or corrupted, the download of the entire
  452.   site (minus the exclusions) occurs again.<br />
  453.   
  454.   The cache can be <strong>disabled</strong> with option -f, --force (it is then not used for the current
  455.   iteration but will still be saved for future use).
  456. </div>
  457.  
  458. <a name="generatedScript" id="generatedScript"></a><h4>Generated FTP Script</h4>
  459. <div>In normal sync, a <strong>Python script</strong> is created in the directory indicated by option -o
  460.      for the specified target (<em>whatToSync</em>). The script is named <em>what</em>_<em>date</em>.py
  461.      (with / and spaces replaced by _). It contains the set of FTP actions required to synchronize
  462.      the remote client (site) as requested, therefore its execution will perform the actual update,
  463.      which can prove very useful, should the FTP session fail before completion: executing the script will
  464.       <em>ftp</em> the changes again.
  465. </div>
  466.  
  467. <a name="log" id="log"></a><h4>Log</h4>
  468. <div> A file <code>p4ftpsync.log</code> is generated in <code>p4ftpsync</code>'s folder (always
  469.     <em>verbose</em>, unlike the on-screen trace which depends on option -v). Logs are rotated to
  470.     <code>p4ftpsync.log.<em>n</em>.zip</code> when they exceed a certain size (4MB by default).
  471. </div>
  472.  
  473. <!-- LIMITATIONS -->
  474. <a name="limitations" id="limitations"></a><h2>Limitations, Bugs, ToDo</h2>
  475. <div>The most obvious limitations and todos I can think of right now are:
  476.   <ul>
  477.     <li>A limited support for <em>symbolic links</em>. Currently links are recognized on the remote site
  478.         but translated into normal files in P4 during reverse sync. That is, we tolerate that links
  479.         exist on the remote site but not locally and in P4. Since P4 supports symbolic links, it
  480.         could potentially be changed, though. (<em>Due to the comparison strategy used during reverse
  481.         sync, I'm not sure that the case where 2 links withing the site point to the same file is
  482.         correctly handled: it will create as many real files in p4</em>). </li>
  483.     <li>An uncertain support of <strong>older versions of the P4 server</strong> (&lt;2005.1). This program probably works
  484.         with not-too-old versions of the server, but I wouldn't bet on it.</li>
  485.     <li>Support of <strong>all ftp servers</strong> is not guaranteed either. I tried on at least
  486.         3 different servers, but they are sometimes surprising in their use of error codes. </li>
  487.     <li>... and probably many other ones I don't think of ...</li>
  488.   </ul>
  489. </div>
  490.          
  491. <div>Please report bugs or suggestions to <a href="javascript:location='mailto:\u0070\u0034\u0066\u0074\u0070\u0073\u0079\u006e\u0063\u0040\u0072\u0067\u0072\u0075\u0065\u0074\u002e\u006e\u0065\u0074';void 0">Richard Gruet</a>.  I'd be happy to hear from you. I'll try to fix bugs
  492. and implement suggestions if I think they can improve p4ftpsync (and are not to complex !). Don't hold your breath, though.
  493. I'm quite busy.
  494. </div>
  495.  
  496. <!-- LICENSE -->
  497. <a name="license" id="license"></a><h2>License</h2>
  498. <div>Copyright &copy; 2004-2005 <a href="javascript:location='mailto:\u0070\u0034\u0066\u0074\u0070\u0073\u0079\u006e\u0063\u0040\u0072\u0067\u0072\u0075\u0065\u0074\u002e\u006e\u0065\u0074';void 0">Richard Gruet</a></div>
  499. <div>Permission to use, copy, modify, and distribute this software and its
  500.     documentation for any purpose and without fee or royalty is hereby
  501.     granted, provided that the above copyright notice appear in all copies
  502.     and that both that copyright notice and this permission notice appear
  503.     in supporting documentation or portions thereof, including modifications,
  504.     that you make.</div>
  505.  
  506. <div>THE AUTHOR RICHARD GRUET DISCLAIMS ALL WARRANTIES WITH REGARD TO
  507.     THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
  508.     FITNESS, IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL,
  509.     INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
  510.     FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
  511.     NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
  512.     WITH THE USE OR PERFORMANCE OF THIS SOFTWARE !</div>
  513.  
  514. <div>In short: I publish this program in hope it can be useful to others. Use it as you wish but keep the copyright
  515.    intact, and don't hold me responsible for any problem you run into ;)</div>
  516.  
  517. </body>
  518. </html>
# Change User Description Committed
#2 5125 Richard Gruet minor (compatible) changes; improved manual. 20 years ago
#1 5122 Richard Gruet Changed cache validation method; logs now rotated when >4MB; created user manual; other... minor improvements (comments, etc..). « 20 years ago