Helix SwarmHelix Swarm

overview.html

  • //
  • guest/
  • paul_dymecki/
  • mondrian/
  • doc/
  • overview.html #1
  • View
  • Commits
  • Open Download .zip Download (25 KB) 
  1. <body>
  2. <html>
  3.  
  4. <head>
  5. <meta http-equiv="Content-Language" content="en-us">
  6. <meta name="GENERATOR" content="Microsoft FrontPage 5.0">
  7. <meta name="ProgId" content="FrontPage.Editor.Document">
  8. <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
  9. <title>Mondrian overview</title>
  10.  
  11. <style>
  12.  
  13. A:link { color:#000066; }
  14. A:visited { color:#666666; }
  15.  
  16. A.clsIncCpyRt,
  17. A.clsIncCpyRt:visited,
  18. P.clsIncCpyRt {
  19.   font-weight:normal; font-size:75%; font-family:verdana,arial,helvetica,sans-serif;
  20.   color:black;
  21.   text-decoration:none;
  22. }
  23.  
  24. A.clsLeftMenu,
  25. A.clsLeftMenu:visited {
  26.   color:#000000;
  27.   text-decoration:none;
  28.   font-weight:bold; font-size:8pt;
  29. }
  30.  
  31. A.clsBackTop,
  32. A.clsBackTop:visited {
  33.   margin-top:10; margin-bottom:0;
  34.   padding-bottom:0;
  35.   font-size:75%;
  36.   color:black;
  37. }
  38.  
  39. A:hover,
  40. A.clsBackTop:hover,
  41. A.clsIncCpyRt:hover,
  42. A:active { color:blue; }
  43.  
  44. A.clsGlossary {
  45.   font-size:10pt;
  46.   color:green;
  47. }
  48.  
  49. BODY { font-size:80%; font-family:verdana,arial,helvetica,sans-serif; }
  50.  
  51. BUTTON.clsShowme,
  52. BUTTON.clsShowme5 {
  53.   font-weight:bold; font-size:11; font-family:arial;
  54.   width:68; height:23;
  55.   position:relative; top:2;
  56.   background-color:#002F90;
  57.   color:#FFFFFF;
  58. }
  59.  
  60. DIV.clsBeta {
  61.   font-weight:bold;
  62.   color:red;
  63. }
  64.  
  65. DIV.clsDocBody { margin-left:10px; margin-right:10px; }
  66.  
  67. DIV.clsDocBody HR { margin-top:0; }
  68.  
  69. DIV.clsDesFooter { margin:10px 10px 0px 223px; }
  70.  
  71. DIV.clsFPfig { font-size:80%; }
  72.  
  73. DIV.clsHi {
  74. 	padding-left:2em;
  75. 	text-indent:-2em
  76. }
  77.  
  78. DIV.clsShowme { margin-bottom:.5em; margin-top:.5em; }
  79.  
  80. H1{
  81.   font-size:145%;
  82.   margin-top:1.25em; margin-bottom:0em;
  83. }
  84.  
  85. H2 {
  86.   font-size:135%;
  87.   margin-top:1.25em; margin-bottom:.5em;
  88. }
  89.  
  90. H3 {
  91.   font-size:128%;
  92.   margin-top:1em; margin-bottom:0em;
  93. }
  94.  
  95. H4 {
  96.   font-size:120%;
  97.   margin-top:.8em; margin-bottom:0em;
  98. }
  99.  
  100. H5 {
  101.   font-size:110%;
  102.   margin-top:.8em; margin-bottom:0em;
  103. }
  104.  
  105. H6 {
  106.   font-size:70%;
  107.   margin-top:.6em; margin-bottom:0em;
  108. }
  109.  
  110. HR.clsTransHR {
  111. 	position:relative; top:20;
  112. 	margin-bottom:15;
  113. }
  114.  
  115. P.clsRef {
  116.   font-weight:bold;
  117.   margin-top:12pt; margin-bottom:0pt;
  118. }
  119.  
  120. PRE {
  121.   background:#EEEEEE;
  122.   margin-top:1em;	margin-bottom:1em; margin-left:0px;
  123.   padding:5pt;
  124. }
  125.  
  126. PRE.clsCode, CODE.clsText { font-family:'courier new',courier,serif; font-size:130%; }
  127.  
  128. PRE.clsSyntax { font-family:verdana,arial,helvetica,sans-serif; font-size:120%; }
  129.  
  130. SPAN.clsEntryText {
  131.   line-height:12pt;
  132.   font-size:8pt;
  133. }
  134.  
  135. SPAN.clsHeading {
  136.   color:#00319C;
  137.   font-size:11pt; font-weight:bold;
  138. }
  139.  
  140. SPAN.clsDefValue, TD.clsDefValue { font-weight:bold; font-family:'courier new' }
  141.  
  142. SPAN.clsLiteral, TD.clsLiteral { font-family:'courier new'; }
  143.  
  144. SPAN.clsRange, TD.clsRange { font-style:italic; }
  145.  
  146. SPAN.clsShowme {
  147.   width:100%;
  148.   filter:dropshadow(color=#000000,OffX=2.5,OffY=2.5,Positive=1);
  149.   position:relative;
  150.   top:-8;
  151. }
  152.  
  153. TABLE { font-size:100%; }
  154.  
  155. TABLE.clsStd {
  156. 	background-color:#444;
  157. 	border:1px none;
  158. 	cellspacing:0;
  159. 	cellpadding:0
  160. }
  161.  
  162. TABLE.clsStd TH,
  163. BLOCKQUOTE TH {
  164. 	font-size:100%;
  165. 	text-align:left; vertical-align:top;
  166. 	background-color:#DDD;
  167. 	padding:2px;
  168. }
  169.  
  170. TABLE.clsStd TD,
  171. BLOCKQUOTE TD {
  172. 	font-size:100%;
  173. 	vertical-align:top;
  174. 	background-color:#EEE;
  175. 	padding:2px;
  176. }
  177.  
  178. TABLE.clsParamVls,
  179. TABLE.clsParamVls TD { padding-left:2pt; padding-right:2pt; }
  180.  
  181. #TOC { visibility:hidden; }
  182.  
  183. UL UL, OL UL { list-style-type:square; }
  184.  
  185. .clsHide { display:none; }
  186.  
  187. .clsShow { }
  188.  
  189. .clsShowDiv {
  190.   visibility:hidden;
  191.   position:absolute;
  192.   left:230px; top:140px;
  193.   height:0px; width:170px;
  194.   z-index:-1;
  195. }
  196.  
  197. .#pBackTop { display:none; }
  198.  
  199. #idTransDiv {
  200. 	position:relative;
  201. 	width:90%; top:20;
  202.   filter:revealTrans(duration=1.0, transition=23);
  203. }
  204.  
  205.  
  206. /*** INDEX-SPECIFIC ***/
  207.  
  208. A.clsDisabled {
  209.   text-decoration:none;
  210.   color:black;
  211.   cursor:text;
  212. }
  213.  
  214. A.clsEnabled { cursor:auto; }
  215.  
  216. SPAN.clsAccess { text-decoration:underline; }
  217.  
  218. TABLE.clsIndex {
  219.   font-size:100%;
  220.   padding-left:2pt; padding-right:2pt;
  221. 	margin-top: 17pt;
  222. }
  223.  
  224. TABLE.clsIndex TD {
  225.   margin:3pt;
  226.   background-color:#EEEEEE;
  227. }
  228.  
  229. TR.clsEntry { vertical-align:top; }
  230.  
  231. TABLE.clsIndex TD.clsLetters {
  232.   background-color:#CCCCCC;
  233.   text-align:center;
  234. }
  235.  
  236. TD.clsMainHead {
  237.   background-color:#FFFFFF;
  238.   vertical-align:top;
  239.   font-size:145%; font-weight:bold;
  240.   margin-top:1.35em; margin-bottom:.5em;
  241. }
  242.  
  243. UL.clsIndex { margin-left:20pt; margin-top:0pt; margin-bottom:5pt; }
  244.  
  245. LI OL { padding-bottom: 1.5em }
  246.  
  247.  
  248. /*** GALLERY/TOOLS/SAMPLES ***/
  249.  
  250. FORM.clsSamples { margin-bottom:0; margin-top:0; }
  251.  
  252. H1.clsSampH1 {
  253. 	font-size:145%;
  254.   margin-top:.25em; margin-bottom:.25em;
  255. }
  256.  
  257. H1.clsSampHead {
  258.   margin-top:5px; margin-bottom:5px;
  259.   font-size:24px; font-weight:bold; font-family:verdana,arial,helvetica,sans-serif;
  260. }
  261.  
  262. H2.clsSampTitle {
  263.   font-size:128%;
  264.   margin-top:.2em; margin-bottom:0em;
  265. }
  266.  
  267. TD.clsDemo {
  268.   font-size:8pt;
  269.   color:#00319C;
  270.   text-decoration:underline;
  271. }
  272.  
  273. .clsSampDnldMain { font-size:11px; font-family:verdana,arial,helvetica,sans-serif; }
  274.  
  275. .clsShowDesc { cursor:hand; }
  276.  
  277. A.clsTools {
  278.   color:#0B3586;
  279.   font-weight:bold;
  280. }
  281.  
  282. H1.clsTools, H2.clsTools {
  283.   color:#0B3586;
  284.   margin-top:5px;
  285. }
  286.  
  287. TD.clsToolsHome {
  288.   font-size:9pt;
  289.   line-height:15pt;
  290. }
  291.  
  292. SPAN.clsToolsTitle {
  293.   color:#00319C;
  294.   font-size:11pt; font-weight:bold;
  295.   text-decoration:none;
  296. }
  297.  
  298.  
  299. /*** DESIGN ***/
  300. P.cat {
  301. 	font-size:13pt;
  302. 	color:#787800;
  303. 	text-decoration:none;
  304. 	margin-top:18px;
  305. }
  306.  
  307. P.author {
  308. 	font-size:9pt; font-style:italic;
  309. 	line-height:13pt;
  310. 	margin-top:10px;
  311. }
  312.  
  313. P.date {
  314. 	font-size:8pt;
  315. 	line-height:12px;
  316. 	margin-top:0px;
  317. 	color:#3366FF;
  318. }
  319.  
  320. P.graph1 {
  321. 	line-height:13pt;
  322. 	margin-top:-10px;
  323. }
  324.  
  325. P.col {
  326. 	line-height:13pt;
  327. 	margin-top:10px; margin-left:5px;
  328. }
  329.  
  330. P.cal1 {
  331. 	text-decoration:none;
  332. 	margin-top:-10px;
  333. }
  334.  
  335. P.cal2 {margin-top:-10px; }
  336. P.photo { font-size:8pt; }
  337.  
  338.  
  339. /*** DOCTOP ***/
  340.  
  341. #tblNavLinks A {
  342. 	color:black;
  343. 	text-decoration:none;
  344. 	font-family:verdana,arial,helvetica,sans-serif;
  345. }
  346. #lnkShowText, #lnkSyncText, #lnkSearchText, #lnkIndexText { font-size:8pt; font-weight:bold; }
  347. #lnkPrevText, #lnkNextText, #lnkUpText { font-size:7.5pt; font-weight:normal; }
  348.  
  349.  
  350. DIV.clsBucketBranch {
  351. 	margin-left:10px; margin-top:15px; margin-bottom:-10pt;
  352. 	font-style:italic; font-size:85%;
  353. }
  354.  
  355. DIV.clsBucketBranch A,
  356. DIV.clsBucketBranch A:link,
  357. DIV.clsBucketBranch A:active,
  358. DIV.clsBucketBranch A:visited { text-decoration:none; color:black; }
  359. DIV.clsBucketBranch A:hover { color:blue; }
  360.  
  361.  
  362. /*** SDK, IE4 ONLY ***/
  363.  
  364. DIV.clsExpanded, A.clsExpanded { display:inline; color:black; }
  365. DIV.clsCollapsed, A.clsCollapsed { display:none; }
  366. SPAN.clsPropattr { font-weight:bold; }
  367.  
  368. #pStyles,	#pCode, #pSyntax, #pEvents, #pStyles {display:none; text-decoration:underline; cursor:hand; }
  369.  
  370. /*** jhyde added ***/
  371. CODE { color:maroon; font-family:'courier new' }
  372.  
  373. DFN { font-weight:bold; font-style:italic; }
  374.  
  375.  
  376. </style>
  377.  
  378. </head>
  379.  
  380. <!-- 
  381.  
  382. This sentence is here to fool javadoc (which is looking for a period, and otherwise finds one
  383. inside one of our header tables).
  384.  
  385.  -->
  386.  
  387. <table border="1" class="clsStd" width="100%">
  388.   <tr>
  389.     <td colspan="2"><a href="index.html">Top</a> |
  390.     <a href="http://public.perforce.com/guest/julian_hyde/mondrian/doc/index.html">Web home</a> |
  391.     <a href="http://sourceforge.net/projects/mondrian/">SourceForge home</a></td>
  392.     <td width="0" align="right" rowspan="2">
  393.     <a href="http://sourceforge.net"><img src="http://sourceforge.net/sflogo.php?group_id=35302&type=1" width="88" height="31" border="0" alt="SourceForge.net Logo"></a></td>
  394.   </tr>
  395.   <tr>
  396.     <td colspan="2"><em>$Id: //guest/paul_dymecki/mondrian/doc/overview.html#1 $</em></td>
  397.   </tr>
  398.   <tr>
  399.     <td colspan="3"><em>(C) Copyright 2002, Kana Software, Inc. and others</em></td>
  400.   </tr>
  401.   <tr>
  402.     <th align="right" width="30%">Author</th>
  403.     <td colspan="2">Julian Hyde (<a href="mailto:julian.hyde@mail.com">julian.hyde@mail.com</a>)</td>
  404.   </tr>
  405.   <tr>
  406.     <th align="right" width="30%">Created</th>
  407.     <td colspan="2">February 13<sup><font face="Verdana">th</font></sup>, 2002</td>
  408.   </tr>
  409. </table>
  410.  
  411. <h1>Mondrian overview</h1>
  412.  
  413. <p>Mondrian is an OLAP engine written in Java. It executes queries written in 
  414. the MDX language, reading data from a relational database (RDBMS), and presents 
  415. the results in a multidimensional format via a Java API. Let's go into what that 
  416. means.</p>
  417. <h2>Online Analytical Processing</h2>
  418. <p><dfn><font face="Verdana">Online Analytical Processing (OLAP)</font></dfn> 
  419. means analysing large quantities of data in real-time. Unlike Online Transaction 
  420. Processing (OLTP), where typical operations read and modify individual and small 
  421. numbers of records, OLAP deals with data in bulk, and operations are generally 
  422. read-only. The term 'online' implies that even though huge quantities of data 
  423. are involved  typically many millions of records, occupying several gigabytes 
  424.  the system must respond to queries fast enough to allow an interactive 
  425. exploration of the data. As we shall see, that presents considerable technical 
  426. challenges.</p>
  427. <p>OLAP employs a technique called <dfn><font face="Verdana">Multidimensional Analysis</font></dfn>. Whereas a 
  428. relational database stores all data in the form of rows and columns, a 
  429. multidimensional dataset consists of <dfn><font face="Verdana">axes</font></dfn> and 
  430. <dfn><font face="Verdana">cells</font></dfn>. Consider the dataset</p>
  431. <blockquote>
  432.   <table border="0" style="clsStd" id="AutoNumber1" cellpadding="2">
  433.     <tr>
  434.       <td nowrap><i>Year</i></td>
  435.       <th align="right" colspan="2">2000</th>
  436.       <th align="right" colspan="2">2001</th>
  437.       <th align="right" colspan="2">Growth</th>
  438.     </tr>
  439.     <tr>
  440.       <td nowrap><i>Product</i></td>
  441.       <th align="right">Dollar sales</th>
  442.       <th align="right">Unit sales</th>
  443.       <th align="right">Dollar sales</th>
  444.       <th align="right">Unit sales</th>
  445.       <th align="right">Dollar sales</th>
  446.       <th align="right">Unit sales</th>
  447.     </tr>
  448.     <tr>
  449.       <th nowrap>Total</th>
  450.       <td align="right">$17,165</td>
  451.       <td align="right">$2,825</td>
  452.       <td align="right">$18,867</td>
  453.       <td align="right">3,163</td>
  454.       <td align="right">10%</td>
  455.       <td align="right">12%</td>
  456.     </tr>
  457.     <tr>
  458.       <th nowrap> Books</th>
  459.       <td align="right">$12,845</td>
  460.       <td align="right">956</td>
  461.       <td align="right">$14,562</td>
  462.       <td align="right">1,121</td>
  463.       <td align="right">13%</td>
  464.       <td align="right">17%</td>
  465.     </tr>
  466.     <tr>
  467.       <th nowrap> Fiction</th>
  468.       <td align="right">$1,341</td>
  469.       <td align="right">424</td>
  470.       <td align="right">$1,202</td>
  471.       <td align="right">380</td>
  472.       <td align="right">16%</td>
  473.       <td align="right">37%</td>
  474.     </tr>
  475.     <tr>
  476.       <th nowrap> Non-fiction</th>
  477.       <td align="right">$1,412</td>
  478.       <td align="right">400</td>
  479.       <td align="right">$1,224</td>
  480.       <td align="right">386</td>
  481.       <td align="right">11%</td>
  482.       <td align="right">2%</td>
  483.     </tr>
  484.     <tr>
  485.       <th nowrap> Magazines</th>
  486.       <td align="right">$2,753</td>
  487.       <td align="right">824</td>
  488.       <td align="right">$2,426</td>
  489.       <td align="right">766</td>
  490.       <td align="right">-12%</td>
  491.       <td align="right">-7%</td>
  492.     </tr>
  493.     <tr>
  494.       <th nowrap>&mdash; Greetings cards</th>
  495.       <td align="right">$1,567</td>
  496.       <td align="right">1,045</td>
  497.       <td align="right">$1,879</td>
  498.       <td align="right">1,276</td>
  499.       <td align="right">20%</td>
  500.       <td align="right">22%</td>
  501.     </tr>
  502.     </table>
  503. </blockquote>
  504.  
  505.  
  506. <p>The rows axis consists of the members 'All products', 'Books', 'Fiction', and 
  507. so forth, 
  508. and the columns axis consists of the cartesian product of the years '2000' and 
  509. '2001', and the <font face="Verdana">calculation</font> 'Growth', and the 
  510. <dfn>measures</dfn> 'Unit sales' and 'Dollar sales'. Each cell 
  511. represents the sales of a product category in a particular year; for example, 
  512. the dollar sales of Magazines in 2001 were $2426.</p>
  513. <p>This is a richer view of the data than would be presented by a relational 
  514. database. The members of a multidimensional dataset are not always values 
  515. from a relational column. 'Total', 'Books' and 'Fiction' are members at 
  516. successive levels in a <dfn>hierarchy</dfn>, each of 
  517. which is rolled up to the next. And even though it is alongside the years '2000' 
  518. and '2001', 'Growth' is 
  519. a <dfn>calculated member</dfn>, which introduces a
  520. formula for computing cells from other 
  521. cells.</p>
  522. <p>The dimensions used here  products, time, and measures  are just three of 
  523. many dimensions by which the dataset can be categorized and filtered. The 
  524. collection of dimensions, hierarchies and measures is called a <dfn>
  525. <font face="Verdana">cube</font></dfn>.</p>
  526. <p>I hope I have demonstrated that multidimensional  is above all a way of <em>
  527. <font face="Verdana">presenting</font></em> data. Although some multidimensional 
  528. databases <em><font face="Verdana">store</font></em> the data in multidimensional format, 
  529. I shall argue that it is simpler to store the data in relational format. It's 
  530. time to look at the architecture of an OLAP system.</p>
  531. <h2>Architecture</h2>
  532. <p>A Mondrian OLAP System consists of four layers; working from the eyes of the 
  533. end-user to the bowels of the data center, these are the presentation layer, the 
  534. calculation layer, the aggregation layer, and the storage layer.</p>
  535. <p>The <dfn><font face="Verdana">presentation layer</font></dfn> determines what 
  536. the end-user sees on his or her monitor, and how he or she can interact to ask 
  537. new questions. There are many ways to present multidimensional datasets, 
  538. including pivot tables (an interactive version of the table shown above), pie, 
  539. line and bar charts, and advanced visualization tools such as clickable maps and 
  540. dynamic graphics. These might be written in Swing or JSP, charts rendered in 
  541. JPEG or GIF format, or transmitted to a remote application via XML. What all of 
  542. these forms of presentation have in common is the multidimensional 'grammar' of 
  543. dimensions, measures and cells in which the presentation layer asks the question 
  544. is asked, and OLAP server returns the answer.</p>
  545. <p>The second layer is the <dfn><font face="Verdana">calculation layer</font></dfn>. 
  546. The calculation layer parses, validates and executes MDX queries. A query is 
  547. evaluted in multiple phases. The axes are computed first, then the values of the 
  548. cells within the axes. For efficiency, the calculation layer sends cell-requests 
  549. to the aggregation layer in batches. A <dfn>
  550. <font face="Verdana">query transformer</font></dfn> allows the application to 
  551. manipulate existing queries, rather than building an MDX statement from scratch 
  552. for each request. And <dfn>
  553. <font face="Verdana">metadata</font></dfn> describes the the dimensional model, 
  554. and how it maps onto the relational model.</p>
  555. <p>The third layer is the <dfn><font face="Verdana">aggregation layer</font></dfn>. 
  556. An aggregation is a set of measure values ('cells') in memory, qualified by a 
  557. set of dimension column values. The calculation layer sends requests for sets of 
  558. cells. If the requested cells are not in the cache, or derivable by rolling up 
  559. an aggregation in the cache, the aggregation manager and sends a request to the 
  560. storage layer.</p>
  561. <p>The <dfn><font face="Verdana">storage layer</font></dfn> is an RDBMS. It is 
  562. responsible for providing aggregated cell data, and members from dimension 
  563. tables. I describe <a href="#Storage_and_aggregation_strategies">below</a> why I 
  564. decided to use the features of the RDBMS rather than developing a storage system 
  565. optimized for multidimensional data.</p>
  566. <p>All four of these components can exist on the same machine. Layers 2 and 3, 
  567. which comprise the Mondrian server, must be on the same machine. The storage 
  568. layer could be on another machine, accessed via remote JDBC connection. In a 
  569. multi-user system, the presentation layer would exist on each end-user's machine 
  570. (except in the case of JSP pages generated on the server).</p>
  571. <h3><a name="Storage_and_aggregation_strategies">Storage and aggregation 
  572. strategies</a></h3>
  573. <p>OLAP Servers are generally categorized according to how they store their 
  574. data:</p>
  575. <ul>
  576.   <li>A <font face="Verdana"><dfn>MOLAP (multidimensional OLAP)</dfn></font> 
  577.   server stores all of its data on disk in structures optimized for 
  578.   multidimensional access. Typically, data is stored in dense arrays, requiring 
  579.   only 4 or 8 bytes per cell value.</li>
  580.   <li>A <font face="Verdana"><dfn>ROLAP (relational OLAP)</dfn></font> server 
  581.   stores its data in a relational database. Each row in a fact table has a 
  582.   column for each dimension and measure.</li>
  583. </ul>
  584. <p>Three kinds of data need to be stored: fact table data (the transactional 
  585. records), aggregates, and dimensions.</p>
  586. <p>MOLAP databases store fact data in multidimensional format, but if there are 
  587. more than a few dimensions, this data will be sparse, and the multidimensional 
  588. format does not perform well. A <font face="Verdana"><dfn>HOLAP (hybrid OLAP)</dfn></font> 
  589. system solves this problem by leaving the most granular data in the relational 
  590. database, but stores aggregates in multidimensional format.</p>
  591. <p>Pre-computed aggregates are necessary for large data sets, otherwise certain 
  592. queries could not be answered without reading the entire contents of the fact 
  593. table. MOLAP aggregates are often an image of the in-memory data structure, 
  594. broken up into pages and stored on disk. ROLAP aggregates are stored in tables. 
  595. In some ROLAP systems these are explicitly managed by the OLAP server; in other 
  596. systems, the tables are declared as materialized views, and they are implicitly 
  597. used when the OLAP server issues a query with the right combination of columns 
  598. in the <code>group by</code> clause.</p>
  599. <p>The final component of the aggregation strategy is the cache. The cache holds 
  600. pre-computed aggregations in memory so subsequent queries can access cell values 
  601. without going to disk. If the cache holds the required data set at a lower level 
  602. of aggregation, it can compute the required data set by rolling up.</p>
  603. <p>The cache is arguably the most important part of the aggregation strategy 
  604. because it is <em><font face="Verdana">adaptive</font></em>. It is difficult to 
  605. choose a set of aggregations to pre-compute which speed up the system without 
  606. using huge amounts of disk, particularly those with a high dimensionality or if 
  607. the users are submitting unpredictable queries. And in a system where data is 
  608. changing in real-time, it is impractical to maintain pre-computed aggregates. A 
  609. reasonably sized cache can allow a system to perform adequately in the face of 
  610. unpredictable queries, with few or no pre-computed aggregates.</p>
  611. <p>Mondrian's aggregation strategy is as follows:</p>
  612. <ul>
  613.   <li>Fact data is stored in the RDBMS. Why develop a storage manager when the 
  614.   RDBMS already has one?</li>
  615.   <li>Read aggregate data into the cache by submitting <code>group by</code> 
  616.   queries. Again, why develop an aggregator when the RDBMS has one?</li>
  617.   <li><em><font face="Verdana">If</font></em> the RDBMS supports materialized 
  618.   views, <em><font face="Verdana">and </font></em>the database administrator 
  619.   chooses to create materialized views for particular aggregations, then 
  620.   Mondrian will use them implicitly. Ideally, Mondrian's aggregation manager 
  621.   should be aware that these materialized views exist and that those particular 
  622.   aggregations are cheap to compute. If should even offer tuning suggestings to 
  623.   the database administrator.</li>
  624. </ul>
  625. <p>The general idea is to delegate unto the database what is the database's. 
  626. This places additional burden on the database, but once those features are added 
  627. to the database, all clients of the database will benefit from them. 
  628. Multidimensional storage would reduce I/O and result in faster operation in some 
  629. circumstances, but I don't think it warrants the complexity at this stage.</p>
  630. <p>A wonderful side-effect is that because Mondrian requires no storage of its 
  631. own, it can be installed by adding a JAR file to the class path and be up and 
  632. running immediately. Because there are no redundant data sets to manage, the 
  633. data-loading process is easier, and Mondrian is ideally suited to do OLAP on 
  634. data sets which change in real time.</p>
  635. <p><i>Note to self</i>: The cache manager ought to distinguish between data which is being 
  636. pulled into the cache to be rolled up immediately into some other aggregation, 
  637. and an aggregation which is explicitly needed.</p>
  638. <h2>Components</h2>
  639. <h3>Query transformer</h3>
  640. <p>See {@link mondrian.olap.Parser}.</p>
  641. <h3>Metadata</h3>
  642. <p>It is represented as an XML file. The metadata is loaded into memory the 
  643. first time you reference a dimensional model. You can modify the model at 
  644. runtime by creating instances of classes such as <code>{@link 
  645. mondrian.rolap.RolapHierarchy}</code>.</p>
  646. <h3>Calculation layer</h3>
  647. <p><i>todo</i>: See {@link mondrian.olap.Query} and {@link mondrian.olap.Result}.</p>
  648. <p><i>todo</i>: The <code>package {@link mondrian.rolap}</code>. is the one and 
  649. only implementation of the API. The DriverManager (<code>class {@link 
  650. mondrian.olap.DriverManager}</code>) acts as class-factory.</p>
  651. <p><i>todo</i>: How members are calculated...</p>
  652. <p><i>todo</i>: How aggregations are batched...</p>
  653. <p><i>todo</i>: MDX functions. See <a href="#User_defined_functions">user-defined functions</a>.</p>
  654. <h3>Aggregation manager</h3>
  655. <p>Aggregations are based upon the relational model: as far as the aggregation 
  656. manager is concerned, there is no relationship between the columns <code>city</code> 
  657. and <code>state</code>. This means that all roll-ups are the same: you just drop 
  658. a column. Consider the 3 roll-ups possible by dropping a column from the 
  659. aggregation {<code>gender</code>, <code>city</code>, <code>state</code>}: 
  660. dropping <code>gender</code> is equivalent to removing the <code>[Gender]</code> 
  661. dimension; dropping <code>city</code> is equivalent to rolling up to a higher 
  662. level in the <code>[Geography]</code> hierarchy; and dropping <code>state</code> 
  663. is not even allowed in the dimensional model (no, sorry, you can't ask about 
  664. products sold in a cities called 'Portland'). This approach will also allow us 
  665. to implement 'drill anywhere'.</p>
  666. <p>An aggregation is defined by a search condition, for example, <code>{state in 
  667. ('CA', 'OR', 'WA'), city = <i>any</i>, gender = 'M', measure = 'Unit sales'}</code>. 
  668. The <i><code>any</code></i> value is important; if we had asked for a specific 
  669. set of cities, we would not later be able to roll-up by dropping the <code>city</code> 
  670. column.</p>
  671. <p>The caching strategy is to throw out the aggregation with the lowest 
  672. cost/benefit ratio. The 'benefit' of an item is the effort it took to produce 
  673. (effort which it is saving future queries) multiplied by its 'usefulness' which 
  674. declines exponentially if it is not used over time. The 'cost' of an item is its 
  675. size.</p>
  676. <h2>How do I use Mondrian in my application?</h2>
  677. <p>Something like this.</p>
  678. <ol>
  679.   <li>Install the JAR.</li>
  680.   <li>Create an XML mapping file.</li>
  681.   <li>Create a Mondrian connection, specifying the JDBC URL of the RDBMS, and the 
  682. URL of the mapping file.</li>
  683.   <li>Execute an MDX statement.</li>
  684.   <li>Render it. (There are currently no presentation tools which can render it.)</li>
  685.   <li>In response to user actions such as drill-down and pivot, use the query 
  686. transformer services to transform the query, and re-execute.</li>
  687. </ol>
  688. <h2>Why doesn't Mondrian use a standard API?</h2>
  689. <p>Because there isn't one. MDX is a component of Microsoft's OLE DB for OLAP 
  690. standard which, as the name implies, only runs on Windows. Mondrian's API is 
  691. fairly similar in flavor to ADO MD (ActiveX Data Objects for Multidimensional), 
  692. a API which Microsoft built in order to make OLE DB for OLAP easier to use.</p>
  693. <p>XML for Analysis is pretty much OLE DB for OLAP expressed in Web Services 
  694. rather than COM, and therefore seems to offer a platform-neutral standard for 
  695. OLAP, but take-up seems to be limited to vendors who supported OLE DB for OLAP 
  696. already.</p>
  697. <p>The other query vendors failed to reach consensus several years ago with the 
  698. OLAP Council API, and are now encamped on the JOLAP specification.</p>
  699. <p>I plan to provide a JOLAP API to Mondrian as soon as JOLAP is available.</p>
  700. <h2>How does Mondrian's dialect of MDX differ from MSOLAP's?</h2>
  701. <p>Not very much.</p>
  702. <ol>
  703.   <li>The <code>StrToSet()</code> and <code>StrToTuple()</code> functions take 
  704.   an extra parameter.</li>
  705.   <li>Parsing is case-sensitive.</li>
  706.   <li>Pseudo-functions <code>Param()</code> and <code>ParamRef()</code> allow 
  707.   you to create parameterized MDX statements.</li>
  708. </ol>
  709. <h2>How can Mondrian be extended?</h2>
  710. <p><i>todo</i>: <a name="User_defined_functions">User-defined functions</a></p>
  711. <p><i>todo</i>: Cell readers</p>
  712. <p><i>todo</i>: Member readers</p>
  713. <h2>Can Mondrian handle large datasets?</h2>
  714. <p>Yes, if your RDBMS can. We delegate the aggregation to the RDBMS, and if your 
  715. RDBMS happens to have materialized group by views created, your query will fly. 
  716. And the next time you run the same or a similar query, that will really fly, 
  717. because the results will be in the aggregation cache.</p>
  718. <h2>Where is Mondrian going in the future?</h2>
  719. <ol>
  720.   <li>Presentation layer</li>
  721.   <li>Complete implementation of MDX (not many functions implemented yet)</li>
  722.   <li>Tuning</li>
  723.   <li>Support JOLAP API.</li>
  724. </ol>
  725. <h2>Mondrian is fantastic! How can I possibly thank you?</h2>
  726. <p>Please send me an email, and let me know what you liked and didn't like about 
  727. it. If you can think of ways that Mondrian can be improved, roll up your sleeves 
  728. and help make it better. If you use Mondrian in your application, consider 
  729. sharing your work so that everyone can use it.</p>
  730.  
  731.  
  732.  
  733. <b>
  734.   <table border="1" width="100%" class="clsStd">
  735.     <tr>
  736.       <td>End <i>$Id: //guest/paul_dymecki/mondrian/doc/overview.html#1 $</i></td>
  737.     </tr>
  738.   </table>
  739.   <p>&nbsp;</p>
  740. </b>
  741.  
  742. </html>
  743.  
  744. </body>
# Change User Description Committed
#1 1820 Paul Robert Dymecki mondrian: Integrate latest from //guest/julian_hyde 23 years ago
//guest/julian_hyde/mondrian/doc/overview.html
#3 1501 Julian Hyde Mondrian:
generate MetaDef.java;
fix home page link. 		23 years ago
#2 1460 Julian Hyde mondrian: Add home page. 23 years ago
#1 1459 Julian Hyde mondrian: Add overview. 23 years ago