build.xml #3

Commits

#	Change	User	Description
#3	26953	Paul Allen	Move //guest/perforce_software/p4convert to //guest/perforce_software/p4convert/main
#2	14806	Paul Allen	Update docs and add +w.
#1	13919	Paul Allen	part 1 of copy
//guest/paul_allen/p4convert-maven/docs/perforce/build.xml
#1	13895	Paul Allen	Copying using p4convert-docbook
//guest/perforce_software/doc_build/main/perforce/build.xml
#25	12836	eedwards	Arrange for the doc build infrastructure to pass the output type through to AsciiDoctor processing, so that output-format specific image handling can work.
#24	12830	eedwards	#review-1031983 Incorporate AsciiDoc into the doc build infrastructure. This builds on Matt's initial generate.rb, but uses its own version that is more generic and adds some rudimentary validation. Requirements to build AsciiDoc guides: - in the build.properties (or build.xml) for a guide, the property "input-xml" must point to the master AsciiDoc guide. - additionally, a property "input-type" must be set to "adoc". I.e.: "input-type=adoc". - Ruby must be installed and available in your path. - AsciiDoctor must be installed and available in your path. - The master AsciiDoc file must have a metadata line providing the guide's subtitle (which, by convention, is the release string, e.g. "2015.1"), like so: :subtitle: 2015.1 The standard invocations: - "ant pdf" to create a PDF, - "ant publicsite" to create HTML destined for perforce.com, - "ant htmlhelp" to provide the files for CHM production, - "ant product" (if implemented for the guide) to produce HTML destined for product inclusion should all work just as if the guide originated in DocBook XML. As the DVCS guide is the only one authored in AsciiDoc at this point, the necessary modifications to its configuration/source are included in this review, as well as a first cut of common AsciiDoc files for license statement inclusion. Note: I had to rename aa_license to aa_license.adoc because AsciiDoctor's include mechanism requires a known filename extension for nested includes. In the future, all AsciiDoc files should have the ".adoc" extension or frustration will prevail.
#23	12826	eedwards	Allow guide-specific assets to overwrite any generic assets.
#22	12823	eedwards	Add a new 'htmlhelp' doc build target that generates the HTML, index, TOC files that can be compiled into a CHM file on a Windows machine using HTML Help Workshop. The build target can be used wherever the doc infrastructure runs, but the final CHM compilation currently needs to be performed on a Windows box. This means full CHM production automation is currently unavailable. Further investigation into non-Microsoft CHM compilers will be required for full automation. Note that any guide that requires CHM production will need to add its own 'htmlhelp' asset tree. The P4VS guide will (shortly) become a decent reference, if required. For any XSLT tinkerers, note that only a few of the 'htmlhelp' DocBook parameters are currently passed through to the transformation phase. If you need to pass any additional parameters through, add them to build.xml as appropriate.
#21	12812	eedwards	During transformation to PDF/HTML, the source XML is filtered/copied to the target-specific output directory, and then transformation operates on the filtered XML. There was an assumption that input-xml was a relative path, but oXygen transformation scenarios provide absolute paths. This change fixes that assumption so that absolute paths get mapped to their relative locations within the output directory.
#20	12811	eedwards	Add support for guide transformation targets to specify the use of assets from a different target, and to then overlay their own assets on top. This makes it easy to have a single set of images, say from the publicsite target, that the product target can use, and then apply its own variations of those images (or additional images). Guides that wish to do this should update their target-specific build.properties file to specify assets-basedir-extra to be the base path for the overlay assets, and assets-dirs-extra to the asset inclusion criteria required. These two properties function identically to assets-basedir and assets-dirs, respectively.
#19	12809	eedwards	#review-954548 Add the infrastructure to present a "PDF" link in the header bar in the HTML presentation. Not all guides provide PDF versions, so this is configurable on a guide-by-guide basis. The default is to omit the link. When a guide provides a PDF version, set the pdf.available attribute to the name of the PDF file that the link should target. For example, in P4Guide's publicsite build.properties: pdf.available=p4guide.pdf A subsequent changelist will apply this attribute to all of the server guides.
#18	12807	eedwards	Add support for a 'product' build target. The doc infrastructure already supports 'pdf' and 'publicsite' targets; the latter is used to generate the HTML hosted on perforce.com. Newer products, such as P4Insights and Swarm 2.0, will need to also produce HTML that will be hosted by the product, and which should be styled (and possibly interact) like the product itself. This change includes a mechanism to cause the build to end without failure if a guide does not provide 'product' assets, specifically it looks for: <guide>/assets/product/build.properties If that file exists, the 'product' build continues as expected, otherwise it will report going through the motions without actually running any tasks. Currently, the 'product' HTML is identical to the 'publicsite' HTML. Guides that require product-specific HTML are required to customize both XSLT and CSS as necessary, but this gives them the standard presentation and functionality to start with.
#17	12806	eedwards	Retire the mechanism for selecting the stylesheet for 'profiling', since it only worked by accident and it is unnecessary to do this differently for the various output formats. The path to profile.xsl is now specified in global.properties. This should have no effect on production of HTML or PDF for any guide.
#16	12799	eedwards	#review-919318 Updates to the CmdRef to replace links to our www.perforce.com hosted documentation to be tied to a specific release. The thinking is that this should make the links more durable within products that include HTML docs. A future changelist will apply these changes to the rest of the server docs.
#15	12789	eedwards	Upgrade the doc build process so that guide XML is pre-processed, filtering include paths to point to the correct common XML directory, and to be aware of where guides are when files such as appendices are shared between guides. This facilitates building of translated guides, which have a more nested directory structure that broke assumptions about common assets. All DocBook guides have been updated to use the filtered path identifiers. Also includes minor cleanup of commented out legal info, tidy of the root node attributes, inclusion of a Vim modeline, etc. This change also retires the original build infrastructure assets, which should be no problem since all of the guides that continued to use these have been updated to use the filtered paths that point to the current infrastructre asset path(s).
#14	12788	eedwards	Improve PDF doc generation by using a dynamic path for font embedding within the FOP configuration. This should help with docs that live outside of //depot/main/p4-doc/manuals (such as P4Convert)
#13	12776	eedwards	#review-851775 Updates to the HTML presentation: - remove the double DOCTYPE declarations. - revise markup so that the content is no longer in an immovable container. This should allow the scroll position to be maintained by the browser and work as expected after users navigate within a page and then use the browser's back button. - rename the generated search index file to 'index.js' to avoid a mimetype issue. Also, made the search code tolerant of a missing index.js. - fix scrolling to targeted locations in a guide by applying an offset to the scroll to move the target out from under the header, and to animate the scroll.
#12	12757	eedwards	Pass the "ulink.show" parameter through to XSL processing, so guides can control whether external links emit the URL.
#11	12756	eedwards	Re-introduce original image path filtering for HTML output.
#10	12747	eedwards	Add a mechanism to automatically label the top-level items in the sidebar TOC. This is disabled by default, and can be enabled by setting webhelp.autolabel=1 in guides that want it.
#9	12738	eedwards	Infrastructure updates to support new Swarm DocBook documentation: - Strip out XSLTHL; it does a poor job of highlighting the kinds of syntax included in our documentation, and it does an awful job when code to be highlighted contains other markup, such as xref, link, or replaceable. - Add in google-code-prettify (from Swarm) so that generated HTML has fairly decent syntax highlighting for many of the code examples we use. Further work on syntax highlighting, particularly for PDF output, is required. - Disable TOC generation in HTML; the HTML template already includes the full TOC as a sidebar on every page. - Update the xmlns attribute and image URL filtering. - Fix a bug in HTML page title XSL that often prevented the current section title from being included. This was more obvious in the Swarm documentation which chunks into pages by section, rather than by chapter. - Update the HTML indexer to process multiple HTML files in one pass for performance, and overall simplify the code structure. - Adjust sidebar TOC activation logic to allow Bootstrap's scrollSpy code to work with section-based chunking. - Re-introduce the "brand" markup in the HTML doc heading so that Swarm's documentation can include its logo. Other guides can do so if required, but none have been updated to do so yet. - Add facility for guide-specific custom javacsript inclusion. - Add facility to apply custom CSS classes to images in documentation. This is used in Swarm documentation to provide thumbnail, popup, and "framed" images. - Adjust xref and link text to just report the chapter or section name, without the preamble text, for example: "the section called". - Add <replaceable role="bold"> to provide a second kind of emphasis to programlistings. - Added a subtle color to <filename> content, to help distinguish from other literal text. - Removed the separator rule from the first section heading while chunking by section. - Added Swarm's orderedlist styling so that each numbered item has a circle around the number and a vertical bar indicating the extend of the item. - Added a visual indicator when a link points to an external document. - Rewrote the javascript search code to search for exact and fuzzy terms matches, as well as phrases. Relevance scoring is included, but likely needs some fine tuning to suit our documentation.
#8	12737	eedwards	Add a step to the asset copying task that makes a copy of the guide-specific XSL file, replacing @p4-assets-dir@ with the runtime property value. This facilitates building documentation outside of the p4-doc depot hierarchy. As the copied XSL lives in the same directory as the original, there should be no effect for any existing XSL files. A future changelist will update guide-specific XSL files to include the property marker for consistency/flexibility.
#7	12734	eedwards	Replace the Java-based HTML indexer with a Python-based one. The Java-based HTML indexer was based on stemming, which can reduce the size of the index notably. However, stemming producespoor search behaviour, and is language specific. For example, the word 'documentation' has a stem of 'document'. When a user searches for 'docu', 'document' is not found. Also, all punctuation was stripped, making searches for IP addresses or depot paths impossible. The Python-based indexer collects all whitespace-separated tokens. The resulting index is around 10% larger than the stemmed index, but permits much more reasonable results, particularly while the user is still typing in search terms. The Python-based indexer should be cross-platform, but has to date only been tested on Mac OSX. It should work as-is on Linux, but further work may be required on Windows or other platforms. Also, there is room for optimization, particularly when a large number of HTML documents are to be indexed. Searching multiple terms is possible, and each HTML page must match all entered terms. There is nothing fancy about handling the search terms, no conjunctions, phrase searching, etc. Depending on user feedback, we may need to add more sophistication in the future. Included in this change: - the Python indexer - removed use of Java-based indexer from common build.xml and applied the Python indexer - added additional image src filtering - better build commentary in certain targets - rewrote the client-side searching - fixed the content of the HTML <title> tags to be '<chapter> // <book>', or just '<book>' when necessary. Search results strip off the ' // <book>' to be concise. - identify results as 'pages' rather than results, as each page may contain multiple matches. - results now indicate how many matches to expect on each page. - re-enabled indexing/search for CmdRef, P4API, P4Dist, and P4Guide.
#6	12733	eedwards	Add more xmlns attribute cleanup, and adjust how included images get processed.
#5	12728	eedwards	Upgrade ANT doc build infrastructure to assemble PDFs: - remove non-namespaced DocBook source and add namespaced DocBook source. - add Apache FOP 1.1 - copy fonts, images, XSL into _build, establishing new asset structure. The original structure remains until all guides using it can be upgraded, and several other issues can be resolved. - updated build.xml to allow for per-target build properties. - upgraded the P4SAG to use the new infrastructure. - tweaked admonition presentation in PDFs to remove admonition graphics, and resemble closely the presentation used in the new HTML layout, including the same colors. With these changes, building PDFs involves using a shell, navigating into the guide's directory (just P4SAG for now), and executing "ant pdf". Issues still to be resolved: - PDF generation encounters several warnings about missing fonts (bold versions of Symbol and ZapfDingbats), and a couple of locations where the page content exceeds the defined content area. - Due to issues within Apache FOP, PDF generation emits a substantial amount of output that is not easily suppressed without losing important warning information. - Apache FOP's interface to ANT does not expose a way to set the font base directory. The current configuration does work under Mac OSX, but further testing on Windows will need to be done to determine if the relative paths defined continue to work. The workaround is for Windows users to customize the fop-config.xml to provide absolute system paths to the required fonts. - HTML generation needs further browser testing, and exhibits broken navigation on iOS browsers within the TOC sidebar. - A number of PDF and HTML presentation tweaks still need to be made, for example: sidebars, gui* DocBook tags, whitespace, section separation, etc.
#4	12720	eedwards	Apply better organization for DocBook transformation assets. In the future, other output target assets will appear as peers of webhelp, within the assets directory.
#3	12718	eedwards	WebHelp transformation improvements: - consolidate the Java indexer's output files into a single file to reduce HTTP requests, and tidy up the search assets the indexer created - move the search interface and results into their own pane on the right side of the display - make search incremental as users type - replace existing navigation links with a TOC sidebar that is displayed when the browser is wide enough, or use a footer with prev/next links when the browser is too narrow. In the narrow presentation, the TOC sidebar can be toggled open. - Various improvements to colours, formatting, at various browser sizes.
#2	12717	eedwards	Initial prototype of new HTML generation intended to exist on the Perforce public web site. The updates features a more responsive design, removal of frames, styling improvements. More work needs to be done to ensure robust presentation across various devices, and further design tweaks may be required once various stakeholders get a chance to have a look. Future infrastructure work will need to be done: - incorporate PDF generation - some organizational tweaks to make additional output targets easy to add - HTML validation assets need to be added/incorporated
#1	12716	eedwards	Initial checkin of infrastructure to process DocBook XML into WebHelp format. The purpose of this change is to lay the groundwork that will allow all published Perforce documentation to be converted to HTML (and eventually PDF and other formats) in a consistent manner, independent of tools, and eventually allow for automated documentation builds. The latest versions of DocBook (1.78.1), Saxon (6.5.5), Xalan-J (2.7.1), and XSLTHL (2.1.0) are included, as well as generic customization capability and per-guide customizations. More work needs to be done to the customization layers to fully support their use from an XML authoring tool, such as oXygen, which will appear in a future change. Note: This change only includes configuration to build WebHelp for the P4SAG. Further work is required before the infrastructure can be applied to other guides. WARNING: For anyone using oXygen to produce WebHelp, there is a known incompatibility: <imagedata> tags in oXygen are verified for file existence, and the HTML is subsequently patched to resolve to the output directory. The non-oXygen WebHelp generation does not include the patch operation, so images are broken. The <imagedata> can be updated to indicate the relative path in the output directory, but then the oXygen transformation will fail. This will be fixed in a future change.