Loading...
Helix SwarmHelix Swarm
Loading...

jam.paper.html

  • //
  • guest/
  • dick_dunbar/
  • jam/
  • src/
  • jam.paper.html #1
  • View
  • Commits
  • Open Download .zip Download (33 KB) 
  1. <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
  2. <HTML>
  3.  
  4. <HEAD>
  5. 	<META HTTP-EQUIV="Content-Type" CONTENT="text/html;CHARSET=iso-8859-1">
  6. <META NAME="VPSiteProject" CONTENT="file:///D|/rld/Docs/Project.vpp">
  7.  
  8. 	<META NAME="GENERATOR" Content="Visual Page 2.0 for Windows">
  9. 	<TITLE>&lt;Jam&gt;</TITLE>
  10. </HEAD>
  11.  
  12. <BODY TEXT="black" BGCOLOR="white" LINK="red" ALINK="#CC00CC" VLINK="#0066FF">
  13.  
  14. <H1><B>Jam - Make(1) Redux Paper</B></H1>
  15. <P>Christopher Seiwald<BR>
  16. <I>INGRES Corporation</I><BR>
  17. Seiwald@Ingres.Com Now Seiwald@perforce.com<BR>
  18. March 11, 1994 -- Modified May 20, 2004 to match current Jam implementation</P>
  19. <P><BR>
  20. Abstract</P>
  21.  
  22. <BLOCKQUOTE>
  23. 	<P>Despite the progress of UNIX, the basic mechanism by which developers build their programs - <I>make</I>(1)
  24. 	- has remained at its core unimproved since its inception. Most notably, the <I>make</I> language has seen few
  25. 	improvements. Jam is a <I>make</I> replacement that uses an extensible, expressive language for describing ways
  26. 	in which files relate. This new language simplifies the description of systems, both small and large, and renders
  27. 	extending Jam's functionality not only possible but easy.
  28. </BLOCKQUOTE>
  29.  
  30.  
  31. <BLOCKQUOTE>
  32. 	<P>Jam exists now and runs on many UNIX platforms, VMS, and NT. It is freely available in the comp.sources.unix
  33. 	archives. As proof of concept, it has been used to build a very large commercial product, generating in a single
  34. 	invocation 1,000 deliverable files from 12,000 source files.
  35. </BLOCKQUOTE>
  36.  
  37. <H2>Introduction</H2>
  38. <P>The UNIX <I>make</I>(1) program [Feldman, 1986], which automates the building of targets from their source files,
  39. is widely used. Together with its compatible successors (<I>dmake</I> [Vadura, 1990], GNU <I>make</I> [Stallman,
  40. 1991], NET2 <I>make</I> [BSD, 1991], SunOS <I>make</I> [Sun, 1989]) and mutations (<I>cake</I> [Somogyi, 1987],
  41. <I>cook</I> [Miller, 1993], <I>nmake</I> [Fowler, 1985], Plan 9 <I>mk</I> [Flandrena], <I>mms</I> on VMS, etc.),
  42. <I>make</I> enjoys world domination in its capacity.</P>
  43. <P>As <I>make</I>'s author, Stuart Feldman, noted, <I>make</I> itself is not suited for describing huge programs.
  44. This is arguably because the <I>make</I> language has one useful statement: the expression of a direct dependency
  45. among files. This makes for a clumsy, bottom-up description of how to build a system - describing large systems
  46. this way is unmanageable. <I>make</I>'s successors try to overcome this difficulty with sundry tricks: dependencies
  47. with wild-carded names matched against directory contents; parsing command output as Makefile syntax; macro expansions
  48. with and without the help of the C preprocessor, etc.</P>
  49. <P>Jam is an attempt to replace <I>make</I>'s rule system, with its bottom-up language and wayward implicit rules,
  50. with an expressive language that makes it possible to describe explicitly and cogently the compilation of programs.
  51. The current practice of building source simply because it matched wildcards is unreliable in the face of debris
  52. left around by a careless programmer. <BR>
  53. A robust product should be built explicitly, and to make this palatable, Jam makes it easy to be explicit.</P>
  54. <P>A typical sample of Jam's language as seen by end-users will serve to anchor its description:</P>
  55. <PRE>    Main    prog : prog.c ;
  56.     Libs    prog : libaux.a ;
  57.     Archive libaux.a : compile.c gram.y scan.c ;
  58. </PRE>
  59. <P>This example invokes three rules that instruct Jam to build an archive from three source files, to compile a
  60. fourth source file, and to link it against the archive. All three rules, as well as all other rules given as examples
  61. in this paper, are stock ones that come with Jam (see <I>Jambase</I>(5) [Seiwald, 1993]).</P>
  62. <P>
  63. <H2>The Jam Language</H2>
  64. <P>As with <I>make</I>, the most important statement in the Jam language is the expression of a relationship among
  65. files. With <I>make</I>, the relationship is a direct dependency; with Jam, the relationship is user-defined. The
  66. expression of such a relationship is:</P>
  67. <PRE>    &lt;Rule&gt; &lt;targets&gt; [ : &lt;sources&gt; ] ;
  68. </PRE>
  69. <P>This statement is referred to as a rule invocation, with the name of the rule leading the statement. Except
  70. for a handful of built-in rules, the definition of a rule is user-defined. The &lt;sources&gt; are optional. Each
  71. of the three lines in the example above are rule invocations.</P>
  72. <P>Because rules invoke each other, the expression of a user-defined relationship can result in other user-defined
  73. relationships being made among the same or different files. In the case of the example given, the MAIN rule will
  74. invoke the rules:</P>
  75. <PRE>    Cc   prog.o : prog.c ;
  76.     Link prog   : prog.o ;
  77. </PRE>
  78. <P>These rules (presumably) handle the cases of compiling an object module from a C source file and linking that
  79. object module into an executable.</P>
  80. <P>
  81. <H3>Rule Definition</H3>
  82. <P>A rule is defined in two parts: the Jam statements to execute when the rule is invoked (essentially a procedure
  83. call); and the actions (<I>sh</I>(1) commands) to execute in order to update the targets of the rule. A rule may
  84. have a procedure definition, actions, or both.</P>
  85. <P>A rule's procedure definition is given with:</P>
  86. <PRE>    rule &lt;Rule&gt; { &lt;statements&gt; }
  87. </PRE>
  88. <P>This statement causes &lt;statements&gt; to be interpreted by Jam whenever &lt;Rule&gt; is invoked. The &lt;targets&gt;
  89. and &lt;sources&gt; given at rule invocation are available as the special variables $(&lt;) and $(&gt;) in the
  90. &lt;statements&gt; defining the rule. &lt;statements&gt; may be any of the Jam statements listed in this document.
  91. Carrying on the Cc example, a definition for the Cc rule might be:</P>
  92. <PRE>    rule Cc { Depends $(&lt;) : $(&gt;) ; }
  93. </PRE>
  94. <P>This particular rule definition simply arranges for the targets to depend on the sources, using the built-in
  95. rule Depends (described below).</P>
  96. <P>A rule's updating actions are given with:</P>
  97. <PRE>    actions &lt;Rule&gt; { &lt;string&gt; }
  98. </PRE>
  99. <P>This causes the <I>sh</I> script &lt;string&gt; to be associated with the &lt;targets&gt; in any invocation
  100. of &lt;Rule&gt;. Later, if Jam determines that the &lt;targets&gt; are out-of-date, it will pass &lt;string&gt;
  101. to <I>sh</I> for execution. Jam expands $(&lt;) and $(&gt;) in &lt;string&gt;, but $(&lt;) and $(&gt;) in this
  102. case refer to the &lt;targets&gt; and &lt;sources&gt; after they have been bound to real file path names (see Binding,
  103. below). Finishing out the Cc example, a definition of Cc actions would be:</P>
  104. <PRE>    actions Cc {  $(CC) -c $(CCFLAGS) -I$(HDRS) $(&gt;) }
  105. </PRE>
  106. <P>
  107. <H3>Rule Effects</H3>
  108. <P>Rule invocations have no outputs or return values and, instead, do their job through three distinct types of
  109. side-effects. The first is when a rule's procedure invokes built-in rules to modify the target dependency graph.
  110. These built-ins will be discussed shortly. The second is when a rule's procedure sets variables. The third is the
  111. association of the updating actions with the targets, which occurs whenever a rule with updating actions is invoked.</P>
  112. <P>
  113. <H3>Built-in Rules</H3>
  114. <P>There are six built-in rules, five of which modify the target dependency graph. None of these rules have updating
  115. actions. The built-in rules are:</P>
  116. <PRE>    Depends, Includes, Echo, Exit, Glob, Match, Temporary, NotFile, NoCare
  117. </PRE>
  118. <P><I>Depends</I> and <I>Includes</I> take <I>&lt;targets&gt;</I> and <I>&lt;sources&gt;</I>. <I>Echo</I> takes
  119. only <I>&lt;targets&gt;</I>. <I>Temporary, NotFile,</I> and <I>NoCare</I> take only <I>&lt;targets&gt; </I>and
  120. mark them with attributes to indicate special handling when descending the dependency graph.</P>
  121.  
  122. <P><BR>
  123. <I>Depends</I> The basic builder of the dependency graph: it makes <I>&lt;sources&gt;</I> dependencies of <I>&lt;targets&gt;</I>,
  124. just like the simple <I>make</I> <CITE>:</CITE> dependency. If <I>&lt;sources&gt;</I> are newer than <I>&lt;targets&gt;</I>
  125. (using file update times for comparison), or if<I> &lt;sources&gt;</I> are being updated, then the updating actions
  126. of <I>&lt;targets&gt;</I> will be executed.</P>
  127.  
  128. <P><I>Includes</I> A variation on <I>Depends</I>: it makes<I> &lt;sources&gt;</I> dependencies of any targets of
  129. which <I>&lt;targets&gt;</I> are dependencies. This example makes both foo.c and foo.h dependencies of foo.o:</P>
  130. <PRE>    Depends  foo.o : foo.c ;
  131.     Includes foo.c : foo.h ;
  132. </PRE>
  133. <P><BR>
  134. <I>Echo</I> Just echoes its targets to the standard output, as a means for communicating with the user. Jam knows
  135. no fatal error, so the message emitted by <I>Echo</I> can only be advisory.</P>
  136.  
  137. <P><BR>
  138. <I>Temporary</I> Allows for intermediate targets to be missing and not updated if the final target is up-to-date.
  139. If a target marked <I>Temporary</I> is not present, then it simply inherits its parent's time-stamp. <I>Temporary</I>
  140. can be used for any <I>Temporary</I> target, such as the short-lived object module that is to be part of a library
  141. archive.</P>
  142.  
  143. <P><BR>
  144. <I>NotFile</I> Indicates that the target is not really a file and therefore doesn't have a time-stamp. Any updating
  145. actions are only executed if the target's dependencies were updated, rather than on the basis of time-stamp comparisons.
  146. <I>NotFile</I> is used for pseudo targets such as <CITE>all</CITE> or <CITE>install</CITE>, which have dependencies
  147. but don't actually get built themselves.</P>
  148.  
  149. <P><BR>
  150. <I>NoCare</I> Indicates that the target may both be non-existent and not have any updating actions. This loophole
  151. is used to make up for the sloppiness of the header-file scanning.</P>
  152.  
  153. <P>
  154. <H3>Jam Variables</H3>
  155. <P>Part of Jam's programmability lies in its treatment of variables. As with <I>make</I> and <I>sh</I>, Jam variables
  156. are lists of strings, with zero or more elements. But unique to Jam, the result of variable expansion is the product
  157. of the variable values and literal constants in the token being expanded. An example helps here:</P>
  158. <PRE>    $(X) -&gt; a b c 
  159.     t$(X) -&gt; ta tb tc 
  160.     $(X)$(X) -&gt; aa ab ac ba bb bc ca cb cc
  161. </PRE>
  162. <P>This approach makes quick work of many normal variable manipulations: prepending a path name to a list of file
  163. names, prepending <CITE>-l</CITE> to a list of library names, appending a <CITE>,v</CITE> to RCS file names, etc.</P>
  164. <P>Jam has a modicum of variable editing options to replace components of a path name and to subselect members
  165. of a list. These options are discussed in usable detail in Jam's manual page [Seiwald, 1993].</P>
  166. <P>Unlike <I>make</I>, Jam does not defer expansion of variables. When a variable is referenced, even to assign
  167. a new variable, the value is expanded at that time.</P>
  168. <P>Jam variables have two scopes: global and target-specific. Global variables behave much as one might expect,
  169. holding their value until reassigned. Target-specific variables take precedence over global variables when the
  170. specified target is being bound (see below) or updated. The distinction between global and target-specific variables
  171. is made when the variables are assigned. The syntax for setting the two types is, respectively:</P>
  172. <PRE>    &lt;var&gt; = &lt;value&gt; ;
  173.     &lt;var&gt; on &lt;targets&gt; = &lt;value&gt; ;
  174. </PRE>
  175. <P>Target-specific variables have several uses. A simple one is to permit different compiler flag settings for
  176. different source files. In this way, the actions of the Cc rule may be used to compile any C source file, with
  177. various flags (HDRS, CCFLAGS) being adjusted per-target. Other uses of target-specific variables will be discussed
  178. shortly.</P>
  179.  
  180. <P>
  181. <H3>Flow-of-Control</H3>
  182. <P>In addition to statements for defining and invoking rules and setting variables, the Jam language contains statements
  183. for flow-of-control and file inclusion. The statements are:</P>
  184. <PRE>    if &lt;cond&gt; { &lt;statements&gt; } [ else { &lt;statements&gt; } ]
  185.     for &lt;var&gt; in &lt;list&gt; { &lt;statements&gt; } 
  186.     while &lt;cond&gt; { &lt;statements&gt; } ;
  187.     switch &lt;value&gt; { case &lt;pattern1&gt; : &lt;statements&gt; ; ... }
  188.     break ;
  189.     continue ;
  190.     return &lt;values&gt; ;
  191.     include &lt;file&gt; ;
  192. </PRE>
  193. <P>The <CITE>if</CITE> statement does the obvious; the &lt;condition&gt; is the usual mix of comparison and logical
  194. operators applied to variables.</P>
  195. <P>The <CITE>for</CITE> statement iterates over the elements of &lt;value&gt;, assigning the (global) variable
  196. &lt;var&gt; to each element and executing the statement block.</P>
  197. <P>The <CITE>switch</CITE> statement executes the statement block whose case &lt;value&gt; matches the switch's
  198. &lt;value&gt;.</P>
  199. <P>The <CITE>include</CITE> statement sources another file containing Jam statements.</P>
  200. <P>Jam neither needs nor desires a macro preprocessor. Making rule definitions and file inclusions normal statements
  201. obviates a macro preprocessor for conditional compilation, as these statements may appear within Jam conditionals.
  202. Further, preprocessing would require the Jam language to play <CITE>dodge-em</CITE> with the preprocessor semantics.</P>
  203. <H2>Binding Files</H2>
  204. <P>Jam can find source and target files in distant directories, much like the functionality of VPATH in GNU <I>make</I>
  205. and <I>dmake</I>.</P>
  206. <P>By default, a target is located at the actual path of the target, relative to the directory of Jam's invocation.
  207. If the special variable $(LOCATE) is set to a directory name, Jam locates the target in that directory (correctly
  208. concatenating the value of $(LOCATE) and the target's path name). If $(LOCATE) is unset but the special variable
  209. $(SEARCH) is set to a directory list, Jam searches along the directory list for the target file (again, correctly
  210. concatenating the path names).</P>
  211. <P>Jam makes available the bound target names by using them when expanding $(&lt;) and $(&gt;) for updating actions.
  212. Thus, a target can be referred to by a short, unrooted name when invoking a rule to define a relationship, but
  213. any shell commands manipulating the target see a path name usable from the current directory.</P>
  214. <P>$(SEARCH) provides VPATH-like functionality, allowing Jam to be invoked in directories other than where the
  215. source lives, while $(LOCATE) liberates Jam from the directory tree altogether. With it, Jam can run anywhere.</P>
  216. <P>By setting $(SEARCH) and $(LOCATE) properly, Jam can handle a variety of build environments. For example, read-only
  217. source trees can be handled by pointing $(SEARCH) at a read-only source code directory while pointing $(LOCATE)
  218. to a working directory. As another example, <CITE>sparse</CITE> source trees can be handled by having $(SEARCH)
  219. contain two directories: first the developer's own directory, which contains only the files he is editing, and
  220. then his group's directory, which contains the master copy of all source. Most importantly, much of any build environment
  221. can be encoded in the settings of $(SEARCH) and $(LOCATE), which leaves the file names used in rule invocations
  222. free from environment.</P>
  223. <P>The power of $(SEARCH) and $(LOCATE) is realized when these variables are set per-target rather than just globally.
  224. Each individual target file can potentially be found along different search paths. In practice, related files will
  225. have the same search path, but Jam can efficiently accommodate the degenerate case of having these variables set
  226. per-target. In this way, Jam can build whole source trees, with source files scattered across directories.</P>
  227. <P>
  228. <H2>Header-File Inclusion</H2>
  229. <P>Jam handles the incidental dependencies caused when source files include other source files. To find such dependencies,
  230. Jam scans source files for header-file inclusions, using a regular expression pattern match [Spencer, 1986]. The
  231. regular expression is given in the variable $(HDRSCAN). The result of the scan is not interpreted directly by Jam;
  232. to arrange the necessary relationship, Jam calls a user-defined rule named in the variable $(HDRRULE), with the
  233. scanned file as &lt;targets&gt; and the found header-files as &lt;sources&gt;. Usually, the definition of $(HDRRULE)
  234. Includes a call to the built-in rule Includes, which updates the dependency graph appropriately. An example HDRSCAN
  235. that works for C preprocessor Includes is:</P>
  236. <PRE>    HDRSCAN = &quot;^[ \t]*#[ \t]*include[ \t]*[&lt;\\&quot;]([^\\&quot;&gt;]*)[\\&quot;&gt;].*$&quot; ; 
  237. </PRE>
  238. <P>The combination of $(HDRSCAN) and $(HDRRULE), when set per-target, enables Jam to handle just about any include-file
  239. syntax or semantics. Unfortunately, this mechanism doesn't understand conditional Includes (#include within #ifdef),
  240. and can produce bogus dependencies that must be crudely pasted over with the application of the built-in NoCare
  241. rule.</P>
  242.  
  243. <P>
  244. <H2>Time-Stamps</H2>
  245. <P>Like <I>make</I> et. al., Jam uses time-stamps to determine when targets are out-of-date. Another possible design,
  246. a more forward-looking one, would have Jam taking file update cues from an integrated source management system.
  247. This was deferred for two reasons: first, it would require picking a source management system with which to work
  248. (or attempting to engineer a generic interface to source management systems); second, it would preclude using Jam
  249. as a drop-in replacement for existing uses of <I>make</I>.</P>
  250. <P>The code in Jam that checks dependencies is isolated enough to be altered to work with a source management system.
  251. Internally, Jam already distinguishes between updates due to newer dependents and updates due to updated dependents.</P>
  252. <P>
  253. <H2>The Base Rule Set</H2>
  254. <P>A collection of rules providing <I>make</I>-like functionality is supplied with Jam. Called Jambase, the file
  255. provides a dozen-odd rules for compiling and linking C source code. Different versions of Jambase exist for UNIX,
  256. VMS, and NT, all providing the same rule set.</P>
  257.  
  258. <P>Figure 1 lists the rules defined in the current Jambase (described comprehensively in <I>Jambase</I>(5)).
  259. <PRE>
  260. <HR ALIGN="CENTER">
  261. Main image : source ;        link executable from compiled sources
  262.     Libs image : libraries ;     link libraries onto a MAIN
  263.     Undefines images : symbols ;  save undefs for linking
  264.     Setuid image ;               mark an executable SETUID
  265.     Archive lib : source ;       archive library from compiled sources
  266.     Object objname : source ;    compile object from source
  267.     HdrRule source : headers ;   handle #Includes
  268.     Cc obj.o : source.c ;        .c -&gt; .o
  269.     Lex source.c : source.l ;    .l -&gt; .c
  270.     Yacc source.c : source.y ;   .y -&gt; .c
  271.     Bulk dir : files ;           populate directory with many files
  272.     File dest : source ;         copy file
  273.     Shell exe : source ;         install a shell executable
  274.     RmTemps target : sources ;   remove temp sources after target made
  275.     InstallBin dir : sources ;         install binaries
  276.     InstallLib  dir : sources ;         install files
  277.     InstallShell dir : sources ;          install man pages
  278.  
  279. <HR ALIGN="CENTER">
  280. Figure 1 - Rules supplied with Jam
  281. </PRE>
  282. <P>The last act of Jambase is to include a file called Jamfile from the invoking user's current directory. Using
  283. the rules defined in Jambase, the user's Jamfile enumerates the source files and their relationship to the targets
  284. to be built.</P>
  285.  
  286. <P>The Jambase and Jamfile files share the same language; only their purposes distinguish them. It is possible
  287. to write a special-purpose replacement Jambase that is totally self-contained and needs no directory-specific Jamfile.
  288. It is also possible to use any Jam syntax - including conditionals, rule definitions, etc. - in a Jamfile.
  289. <PRE>
  290. <HR ALIGN="CENTER">
  291. Main prog : prog.c ;
  292.       Depends exe : prog ;
  293.       Link prog : prog.o ;
  294.         Depends prog : prog.o ;
  295.       Object prog.o : prog.c ;
  296.         Cc prog.o : prog.c ;
  297.           Depends prog.o : prog.c ;
  298.     Libs prog : libaux.a ;           
  299.       Depends prog : libaux.a ;
  300.       NEEDLIBS on prog = libaux.a ;           
  301.     Archive libaux.a : compile.c gram.y scan.c ;
  302.       Depends libaux.a : libaux.a(compile.o) libaux.a(gram.o) libaux.a(scan.o) ;
  303.       Depends libaux.a(compile.o) : compile.o ;
  304.       Object compile.o : compile.c ;
  305.         Cc compile.o : compile.c ;
  306.           Depends compile.o : compile.c
  307.       Depends libaux.a(gram.o) : gram.o ;
  308.       Object gram.o : gram.y ;
  309.         Cc gram.o : gram.c ;
  310.           Depends gram.o : gram.c
  311.         Yacc gram.c : gram.y ;
  312.           Depends gram.c gram.h : gram.y ;
  313.           Includes gram.c : gram.h ;          
  314.       Depends libaux.a(scan.o) : scan.o ;
  315.       Object scan.o : scan.c ;
  316.         Cc scan.o : scan.c ;
  317.           Depends scan.o : scan.c
  318.       Archive libaux.a : compile.o gram.o scan.o ; 
  319.       Temporary compile.o gram.o scan.o ;          ???
  320.     
  321.  
  322. <HR ALIGN="CENTER">
  323. Figure 2 - Rule execution for the example rule invocations
  324. </PRE>
  325. <H2>The Example</H2>
  326. <P>Returning to our earlier example:</P>
  327. <PRE>    Main prog : prog.c ;
  328.     Libs prog : libaux.a ;                            ??? Libs
  329.     Archive libaux.a : compile.c gram.y scan.c ;
  330. </PRE>
  331. <P>This example invokes three rules that instruct Jam to build an archive from three source files, to compile a
  332. fourth source file, and to link it against the archive. All these rules are defined by the Jambase file and do
  333. most of their work by invoking other rules defined in the Jambase.</P>
  334.  
  335. <P>Main calls Link to set up the relationship between <B>prog</B> and <B>prog.o</B>, and then calls Object to set
  336. up the relationship between <B>prog.o</B> and <B>prog.c</B>. Object calls a rule specific to the file suffix, in
  337. this case, Cc for <B>.c</B>. Along the way, the various rules invoke the built-in Depends rule to set up the dependency
  338. graph.</P>
  339.  
  340. <P><I>Libs</I> is a rule that arranges for <B>libaux.a</B> to become a dependency of <B>prog</B>, and it sets the
  341. target-specific variable NEEDLIBS to let the actions of <I>Link</I> know that <B>libaux.a</B> should be included
  342. on the link command line. <I>Libs</I> has no actions of its own.</P>
  343.  
  344. <P><I>Archive</I> is a rule that sets up the (somewhat complicated) dependencies between the archive <B>libaux.a</B>,
  345. its members, and the <I>Temporary</I> object modules that are to be its members. It calls <I>Object</I> to set
  346. up the relationship between each of the <I>Temporary</I> object modules and their source files. It also calls the
  347. <I>FArchive</I> rule to handle the archiving of the <I>Temporary</I> object modules into <B>libaux.a</B>.</P>
  348.  
  349. <P>A more complete list of rule invocations seen by Jam for this example is given in Figure 2.</P>
  350.  
  351. <P>Probably lost in this litany of rules are some important features: the <I>Object</I> rule, when presented with
  352. the task of making a <CITE>.o</CITE> file from a <CITE>.y</CITE> file, called both the <I>Cc</I> and <I>Yacc</I>
  353. rules. Note that this is considerably easier and more deterministic than <I>make</I>'s approach of making a <CITE>.o</CITE>
  354. from whatever happens to be available. Also, note that the <I>Yacc</I> rule took advantage of the <I>Includes</I>
  355. built-in, to ensure the dependencies on the generated file are accurately registered.</P>
  356.  
  357. <P>Actually, the rule definitions include a few more machinations to give special variables sensible defaults.
  358. For source code, $(SEARCH) is set to $(SEARCH_SOURCE); for object files, $(LOCATE) is set to $(LOCATE_OBJECT);
  359. for C source files, $(HDRSCAN) is set to the example pattern mentioned above, and $(HDRRULE) is set to  <CITE>HdrRule</CITE>
  360. , the generic header-handling rule defined in the Jambase file.</P>
  361.  
  362. <P>
  363. <H2>Implementation</H2>
  364. <P>The weight of Jam's implementation is evenly divided between its rule-processing subsystem (driven by a <I>yacc</I>(1)
  365. grammar), its recursive binding and scanning subsystem, and its recursive build subsystem.</P>
  366.  
  367. <P>The rule-processing subsystem is entirely system independent, only setting in-memory variables, building the
  368. dependency graph, and associating update actions with targets. The <I>yacc</I> grammar is less than 200 lines.</P>
  369.  
  370. <P>The recursive binding and scanning subsystem is mostly system independent, but calls system-dependent routines
  371. to time-stamp files and to manipulate file names.</P>
  372.  
  373. <P>The recursive build subsystem is mostly system independent, but calls system-specific routines to execute shell
  374. commands (which are system-specific as well).</P>
  375.  
  376. <P>The system dependencies are hidden through three interfaces: one to time-stamp files; one to manipulate file
  377. names; and one to execute shell commands.</P>
  378.  
  379. <P>The file time-stamp interface has two layers: a higher one that asks about individual files; and a lower one
  380. that scans directories and library archives whole. The latter is more efficient, and all current implementations
  381. (UNIX, NT) are coded against it.</P>
  382.  
  383. <P>The file name manipulation interface consists of two routines: one to break a file name down into its components
  384. and one to build a file name from its components. These are quite simple - except on VMS, where concatenating path
  385. names is black art.</P>
  386.  
  387. <P>The shell-command interface currently approximates the UNIX <I>system</I>(3) call interface, with an addition
  388. for catching interrupts.</P>
  389.  
  390. <P>Jam achieves its functionality while going sparingly on features. It has only four flags (mostly to do with
  391. debugging), six built-in rules (<I>Depends, Includes, Echo, NotFile, NoCare, Temporary</I>) and six special variables
  392. ($(&gt;), $(&lt;), $(SEARCH), $(LOCATE), $(HDRSCAN), $(HDRRULE)). The whole of Jam for UNIX is under 5,000 lines
  393. of code, exclusive of Henry Spencer's <I>regexp</I>(3) regular-expression code (about another 1,300 lines).</P>
  394.  
  395. <P>A design goal of Jam was portability, specifically so that the same mechanism could be used to build the same
  396. system on different platforms. Jam scores well in this category: the OS interface is constricted, leaving the bulk
  397. of the system dependencies in the Jambase file. Even the Jambase file is somewhat portable, with only the filename
  398. syntax and the actual update commands having to change between UNIX and NT. Jamfile files themselves usually contain
  399. nothing system-specific.</P>
  400. <H2>Performance</H2>
  401. <P>Used to build from scratch a large commercial software system (the INGRES relational DBMS), lapse time for Jam
  402. breaks down as follows (on an HP9000/710):</P>
  403. <PRE>    parsing 5,000 lines of Jamfiles                 16 seconds 
  404.     stat()'ing 12,000 source files                  1 minute
  405.     scanning 12,000 source files for headers        9 minutes 
  406.     actual building (compiling, linking, etc)       12 hours
  407. </PRE>
  408. <P>The simple conclusion is that Jam's performance in inconsequential. When everything is up-to-date, only few
  409. improvements could be made. <I>stat</I>()'ing files is essentially unavoidable without resorting to other techniques
  410. for determining outdated targets. Scanning source files could be avoided by caching header-file dependency information
  411. in state files. SunOS <I>make</I> and <I>nmake</I> use this approach. The only other recourse is to hammer on the
  412. <I>regexp</I> implementation.</P>
  413. <P>The real performance limitation is in actual building time. Jam does not yet support parallel command execution,
  414. which on a large SMP system can reduce build time by a factor of 5 or more. This feature is anticipated.</P>
  415. <P>
  416. <H2>Comparisons</H2>
  417. <P>Jam's per-target variables are a convenience approached only by SunOS <I>make</I>'s <CITE>target := macro =
  418. value</CITE> syntax. Both Jam and SunOS <I>make</I> make use of the value when updating the target, but Jam gets
  419. added mileage out of the facility by using the value during the binding and header-file scanning.</P>
  420. <P>Jam's searching mechanism is superior to VPATH in two ways: first, it provides not only searching for existing
  421. targets, but also binding for new targets; second, Jam's SEARCH and LOCATE variables can be set per-target. GNU
  422. <I>make</I> allows VPATH to be set selectively, using patterns, and the patterns could be full file names, but
  423. GNU <I>make</I> handles the degenerate case of separate values per file poorly. Jam's SEARCH and LOCATE mechanism
  424. can make the invoker's directory irrelevant, which amounts to a complete solution.</P>
  425.  
  426. <P>Jam's pattern-scanning method of header-file scanning is faster than those that offload the problem to separate
  427. programs (<I>dmake</I>, <I>cake</I>, GNU <I>make</I>). It is not strictly correct, like SunOS <I>make</I> and GNU
  428. <I>make</I>, which use the C preprocessor. Jam's mechanism, driven by per-target variables and user-defined relationships
  429. is, however, quite flexible. It can handle languages that don't offer a separate preprocessor, as well as languages
  430. where the result of a file being included is more than just a simple dependency. For example, when a <I>yacc</I>
  431. file <I>Includes</I> a C header-file, Jam can be made to understand that the generated C source file will include
  432. the generated C header-file. Jam supports these types of arrangements entirely in its language.</P>
  433.  
  434. <P>Jam is missing a few features cherished by some <I>make</I> users: the ability to run update commands concurrently
  435. and fancy variable editing. These may appear in future versions of Jam.</P>
  436. <P>
  437. <H2>Discussion</H2>
  438. <P>The comparison of Jam's language with <I>make</I>'s is somewhat subjective and complicated. As stated in the
  439. introduction, Jam is an attempt to replace <I>make</I>'s rule system with an expressive language that makes it
  440. possible to describe explicitly and cogently the compilation of programs.</P>
  441. <P>In this respect, Jam is a success. For small systems, the Jamfile file is often not larger than the three lines
  442. that made up our example. For large systems, any added complexity can be centralized in the Jambase file, while
  443. the Jamfile file(s) in source directories remain simple.</P>
  444. <P>Jam's rule semantics, that of expressing named relationships among files, is Jam's single biggest advantage
  445. over its contemporaries. Its power and economy of expression seem unmatched. There are two other approaches deserving
  446. mention. <I>nmake</I> and <I>dmake</I> allow new operators (replacements for the simple <CITE>:</CITE> dependency
  447. statement) to be defined as macros, and these can be used to create new relationship types. Unfortunately, the
  448. number of available <CITE>operator</CITE> characters is limited, and the coding of the macros would curl the eyebrows
  449. of even seasoned <I>sendmail</I> hackers. <I>cook</I>, <I>cake</I>, Plan 9 <I>mk</I>, and NET2 <I>make</I> promote
  450. a different approach: that of defining variables and then #including <CITE>recipes</CITE> (other <I>make</I> files)
  451. that define the relationships. The recipes are the approximate equivalent of Jam rules, using pass-by-name variables.
  452. This scheme works, but it is an ugly ordeal to try to recover with a preprocessor the functionality that is lacking
  453. in a language.</P>
  454. <P>The Jam language turns out to be fairly straightforward to program. With its reliance on keywords rather than
  455. special characters and its use of a <CITE>;</CITE> to terminate statements, it is easier reading than most <I>make</I>
  456. syntax.</P>
  457. <P>Abandoning <I>make</I> syntax was an easy decision: even those new <I>make</I>s that understand traditional
  458. <I>make</I> syntax get their added functionality through incompatible syntax. If compatibility with <I>make</I>
  459. is the priority, users can just use <I>make</I>. If users want greater functionality, they can't use vanilla <I>make</I>
  460. anyway.</P>
  461. <P>The proof of Jam is in the pudding (sorry...): it is worth mentioning that the timing information given above
  462. is for a single, non-recursive invocation of Jam to compile 12,000 source files scattered throughout 300 directories,
  463. producing 7,000 intermediate targets and 1,000 deliverable files. Each source directory contains a single Jamfile
  464. with an average of 1.5 words per source file (including the source file name). The author knows of no other <I>make</I>
  465. that can approach such completeness with such economy.</P>
  466. <H2>Availability</H2>
  467. <P>Jam is freely available from <A HREF="http://public.perforce.com/public/index.html">Perforce Public Depot</A>.
  468. It is known to compile and work on VMS (Alpha and VAX) and the following variants of UNIX: BSD/386, OSF/1, DGUX
  469. 5.4, HPUX 9.0, AIX, IRIX 5.0, PTX V2.1.0, SunOS 4, Solaris 2, Ultrix 4.2, Linux and NT.</P>
  470. <H2>Bibliography</H2>
  471. <P>[Brokken, 1994] Frank B. Brokken and Karel Kubat, <CITE>ICMAKE - the Intelligent C-like MAKEr, or the ICce MAKE
  472. utility</CITE>, Linux Sources, 1994</P>
  473. <P>[BSD, 1991] BSD NET2 make(1) manual page, BSD NET2 documentation, July 1991.</P>
  474. <P>[Feldman, 1986] S. I. Feldman, <CITE>Make - A Program for Maintaining Computer Programs</CITE>, BSD NET2 documentation,
  475. April 1986 (revision).</P>
  476. <P>[Flandrena] R. Flandrena, <CITE>Plan 9 Mkfiles</CITE>, available via anonymous FTP from plan9.att.com.</P>
  477. <P>[Fowler, 1985] Glenn Fowler, <CITE>The Fourth Generation Make</CITE>, Proceedings of the USENIX Summer Conference,
  478. June 1985.</P>
  479. <P>[Miller, 1993] Peter Miller, <CITE>Cook - A File Construction Tool</CITE>, Volume 26, comp.sources.unix archives,
  480. 1993.</P>
  481. <P>[Seiwald, 1993] Christopher Seiwald, Jam(1) and Jambase(5) manual pages, Volume 27, comp.sources.unix archives,
  482. 1993.</P>
  483. <P>[Spencer, 1986] Henry Spencer, Regexp code and comment, comp.sources.unix archives, 1986.</P>
  484. <P>[Stallman, 1991] Richard M. Stallman and Roland McGrath, <CITE>GNU Make - A Program for Directed Recompilation</CITE>,
  485. Free Software Foundation, 1991</P>
  486. <P>[Somogyi, 1987] Zoltan Somogyi, <CITE>Cake, a Fifth Generation Version of Make</CITE>, Australian Unix System
  487. User Group Newsletter, April 1987.</P>
  488. <P>[Sun, 1989] Sun Microssytems Corporation, SunOS make(1) manual page, SunOS 4.1.2 documentation, September 1989.</P>
  489.  
  490. <P>[Vadura, 1990] Dennis Vadura, dmake(1) manual page, Volume 27, comp.sources.misc archives, 1990.
  491.  
  492. </BODY>
  493.  
  494. </HTML>
# Change User Description Committed
#1 4362 Dick Dunbar Documentation updates. 21 years ago