On modelling relative risks for longitudinal binomial responses: implications from two dueling paradigms ======================================================================================================== * Tuo Lin * Rongzhe Zhao * Shengjia Tu * Hao Wu * Hui Zhang * Xin M Tu ## Abstract Although logistic regression is the most popular for modelling regression relationships with binary responses, many find relative risk (RR), or risk ratio, easier to interpret and prefer to use this measure of risk in regression analysis. Indeed, since Zou published his modified Poisson regression approach for modelling RR for cross-sectional data, his paper has been cited over 7 000 times, demonstrating the popularity of this alternative measure of risk in regression analysis involving binary responses. As longitudinal studies have become increasingly popular in clinical trials and observational studies, it is imperative to extend Zou’s approach for longitudinal data. The two most popular approaches for longitudinal data analysis are the generalised linear mixed-effects model (GLMM) and generalised estimating equations (GEE). However, the parametric GLMM cannot be used for the extension within the current context, because Zou’s approach treats the binary response as a Poisson variable, which is at odds with the Bernoulli distribution for the binary response. On the other hand, as it imposes no mathematical model on data distributions, the semiparametric GEE is coherent with Zou’s modified Poisson regression. In this paper, we develop a GEE-based longitudinal model for binary responses to provide inference about RR. * OR * generalized estimating equations * relative risk * sandwich variance estimator * semiparametric generalized linear models ## Introduction Logistic regression is widely used to model binary responses. However, many find relative risk (RR), or risk ratio easier to interpret and prefer to model regression relationships with inference about RR, rather than odds ratio (OR) as in logistic regression. Indeed, since Zou1 published his modified Poisson regression approach for inference about RR, his paper has been cited 7 128 times, demonstrating the popularity of using RR in modelling binary responses. However, his approach isn’t applied to longitudinal data. Moreover, there is no one-to-one relationship between RR and OR for regression analysis.2 As longitudinal studies have become increasingly the standard in clinical trials and observational studies, it is imperative to develop statistical models for longitudinal binary responses with inference based on RR to fill the critical gap. The two most popular paradigms to extend models for cross-sectional data to longitudinal data are the generalised linear mixed-effects model (GLMM) and generalised estimating equations (GEE). The parametric GLMM explicitly models the within-subject correlation using random effects, while the semiparametric, or distribution-free GEE implicitly accounts for such correlations using sandwich variance estimates.3 Since Zou’s approach treats binary responses as count variables and derives estimators of RR under the Poisson distribution, GLMM cannot be used to extend his approach to longitudinal data within the current context. As his approach is essentially a semiparametric log-linear model, a simplified version of GEE for cross-sectional data, GEE provides a coherent paradigm to develop to extend his approach to longitudinal data. In the Models for Relative Risks for Longitudinal Binary Responses section, we first review semiparametric regression models for cross-sectional and longitudinal binary responses under the logit and log link for inference about the respective log of OR and log of RR. We then discuss a GEE-based approach for longitudinal binary responses for inference about RR by leveraging semiparametric log-linear models. In the Application section, we use real and simulated data to illustrate the proposed approach. In the Discussion section, we give our concluding remarks. ## Models for relative risks for longitudinal binary responses We start with a brief review of Zou’s approach for inference about RR when modelling binary responses in cross-sectional data. ### Cross-sectional data Consider a study with n subjects indexed by ![Formula][1] Let ![Formula][2] denote a binary response of interest and let ![Formula][3] with ![Formula][4] denote a ![Formula][5] vector of explanatory, or independent, variables from the i th subject ![Formula][6] . The popular logistic regression model is defined by a generalised linear model (GLM) with the logit link as Tang *et al* 3: ![Formula][7] (1) where ![Formula][8] denotes independently distributed, Bernoulli ![Formula][9] denotes the *Bernoulli* distribution with mean ![Formula][10] , logit denotes the logit link function and γ is the vector of model parameters or coefficients. Under logistic regression, each regression coefficient ![Formula][11] has the log OR interpretation per unit change in ![Formula][12] for ![Formula][13] 3 Inference about γ is generalised based on maximum likelihood.3 For ![Formula][14] to have the RR interpretation, we need to change the logit link to the log link function to express (1) as: ![Formula][15] (2) For differentiating log OR from log RR, we use a different symbol β in (2) to denote the model coefficients. Under (2), each coefficient ![Formula][16] has the log RR interpretation. For example, consider one unit increase in ![Formula][17] from ![Formula][18] to ![Formula][19] . Denote the change in the mean of ![Formula][20] in response to the change in ![Formula][21] by: ![Formula][22] Then, it follows from (2) that the log of RR, ![Formula][23] , for the unit change in ![Formula][24] from ![Formula][25] to ![Formula][26] is: ![Formula][27] The two GLMs in (1) and (2) are quite similar except for the different link functions. Under logit link in (1), the conditional mean ![Formula][28] is constrained between 0 and 1, while under the log link in (2), ![Formula][29] is confined only to positive values. Since ![Formula][30] may exceed 1, the upper bound for a probability quantity, estimates based on maximising the Bernoulli likelihood may not converge under the log link.4 5 To alleviate this problem, we may switch the Bernoulli distribution in (2) to the Poisson, that is, ![Formula][31] (3) Since the logic restriction of positive values on ![Formula][32] is consistent with the mean of the Poisson, fitting the model (3) to observed data will not be an issue. For rare diseases, ![Formula][33] will be close to 0 and ![Formula][34] may be viewed as a count, frequency, or response with mean ![Formula][35] , in which case the Poisson-based (3) is a reasonable approximation. In general, with increased ![Formula][36] , (3) may not provide valid inference, since the binary ![Formula][37] will not have a Poisson distribution in this case. Zou discussed the use of the sandwich variance estimator as an alternative to estimate the variance of the estimator of β . This approach is essentially a semiparametric regression, or restricted moment model, in which only the model for the conditional mean of ![Formula][38] given ![Formula][39] in (3) is assumed: ![Formula][40] (4) Thus, unlike (3), the semiparametric log-linear model above does not assume Poisson or any other parametric distribution for ![Formula][41] . Different from a parametric model, a semiparametric model leverages estimating equations to play the role of the likelihood to provide inference.3 Unlike maximum likelihood estimation, inference based on estimating equations is consistent regardless of the distribution of ![Formula][42] , so long as the assumed conditional mean in (4) is correct.3 Thus, even if ![Formula][43] does not have a Poisson distribution, inference about β in (4) is still correct when based on the estimating equations. Within the current context, the estimating equations for inference about β have the form: ![Formula][44] (5) where ![Formula][45] is the conditional variance of ![Formula][46] given ![Formula][47] . Under (4), ![Formula][48] and ![Formula][49] are readily evaluated. However, ![Formula][50] is not determined by the semiparametric log-linear model in (4), since it only specifies the conditional mean ![Formula][51] . Within the current context, ![Formula][52] follows the Bernoulli ![Formula][53] , in which case we have ![Formula][54] . We obtain the estimate ![Formula][55] of β by solving (5) for β . Unlike linear regression, ![Formula][56] cannot be evaluated in closed form but is readily computed numerically.3 The estimator ![Formula][57] has an asymptotically normal distribution with mean β and variance ![Formula][58] : ![Formula][59] (6) where ![Formula][60] denotes the inverse of B . We can estimate ![Formula][61] by the following sandwich variance estimator ![Formula][62] : ![Formula][63] (7) Note that unlike likelihood-based inference for parametric models, inference based on the estimating equations in (5) for semiparametric models is always valid, regardless of the distribution of ![Formula][64] . In particular, instead of ![Formula][65] , we may also set ![Formula][66] to any function of ![Formula][67] such as ![Formula][68] (by treating ![Formula][69] as a Poisson with mean ![Formula][70] ) for valid inference about β . This is why we can model a binary ![Formula][71] using a semiparametric log-linear model for a count response. ### Longitudinal data We now consider extending the semiparametric log-linear model above to longitudinal data. Suppose that the subjects are assessed repeatedly over T time points ![Formula][72] . Let ![Formula][73] and ![Formula][74] denote the same response and explanatory variables as in the cross-sectional data setting, but with t indicating their dependence on the time of assessment (![Formula][75] , ![Formula][76] ). By applying the semiparametric log-linear model in (4) to each assessment t , we obtain an extension of the semiparametric log-linear model for the association of longitudinal ![Formula][77] and ![Formula][78] : ![Formula][79] (8) Thus, we do not explicitly model correlations among the repeated ![Formula][80] ’s. Inference about β is based on extending the estimating equations in (5) to the correlated ![Formula][81] ’s. Let ![Formula][82] The estimating equations, which are often called the generalised estimating equations (GEE) in the literature, for inference about β have the form: ![Formula][83] (9) where ![Formula][84] is the conditional variance of ![Formula][85] given ![Formula][86] . As in the cross-sectional case, we can readily evaluate ![Formula][87] and ![Formula][88] under (8) and set ![Formula][89] for each ![Formula][90] . However, the conditional covariance between ![Formula][91] given ![Formula][92] is quite complex. In almost all applications of GEE, we use a working correlation ![Formula][93] to approximate the true correlation ![Formula][94] , where ![Formula][95] is a ![Formula][96] correlation matrix with its entries defined by a parameter vector α *.* 3 Popular choices of ![Formula][97] are the working independence, with ![Formula][98] , and working exchangeable, with ![Formula][99] , model, where ![Formula][100] denotes the ![Formula][101] identity matrix and ρ is a parameter. Under a specific ![Formula][102] , we have ![Formula][103] , where ![Formula][104] denotes a diagonal matrix with ![Formula][105] on its t th diagonal. As in the case of cross-sectional data, inference is always valid even if ![Formula][106] ![Formula][107] is not the true correlation (variance) of ![Formula][108] given ![Formula][109] . In (9), ![Formula][110] also depends on α , though we have suppressed this dependence to highlight the fact that (9) is the equation for estimating β . Thus, α must be estimated (except for the working independence model) to solve (9) for ![Formula][111] We can either assign a value to or estimate α together with β . For example, under ![Formula][112] , we may set ρ to any value between 0 and 1 or estimate ρ using correlated residuals ![Formula][113] , with ![Formula][114] . Inference about β is based on the asymptotic normal distribution of the GEE estimator ![Formula][115] , which has mean β and variance ![Formula][116] : ![Formula][117] (10) where ![Formula][118] denotes the transpose of B . We can estimate ![Formula][119] by the sandwich variance estimator ![Formula][120] , which is obtained by: ![Formula][121] (11) where ![Formula][122] , ![Formula][123] and ![Formula][124] denote substituting ![Formula][125] in place of β for the respective quantity ![Formula][126] , ![Formula][127] and ![Formula][128] . Popular software packages all support semiparametric regression models for both cross-sectional and longitudinal data. For example, PROC GEE in SAS and geeglm() in the geepack package in R6 can be used to fit the semiparametric log-linear models in (4) for cross-sectional and (8) for longitudinal data. ## Application We illustrate our considerations with both real and simulated data. In all the examples, we set the statistical significance at ![Formula][129] . All analyses are carried out using the geeglm() function in the geepack package in R.6 ### Simulation study We consider modelling regression associations of a single time-invariant binary explanatory variable ![Formula][130] with a binary response ![Formula][131] in a longitudinal study with three assessments. To simulate the correlated ![Formula][132] , we use a Gaussian copula with the marginal ![Formula][133] given ![Formula][134] following a Bernoulli7: ![Formula][135] (12) For our simulation, we set ![Formula][136] and ![Formula][137] and an exchangeable correlation ![Formula][138] in the trivariate normal with ![Formula][139] . We fit the semiparametric (8) to the data simulated, that is, ![Formula][140] (13) using the GEE in (9) under the working independent correlation model. Shown in table 1 are the estimates of β along with their standard errors (SEs) (both asymptotic and empirical), over 1 000 Monte Carlo (MC) replications under a sample size ![Formula][141] . The estimates ![Formula][142] were quite close to their true values, and the asymptotic SEs were quite close to their empirical counterparts. Also, shown in table 1 are type I error rates from testing the null hypothesis ![Formula][143] and ![Formula][144] . We estimate the type I errors using MC iterations. Let ![Formula][145] denotes the Wald statistic at the m th MC replication, the type I error rate for testing ![Formula][146] is estimated by: ![Formula][147] , where ![Formula][148] is the 95th percentile of a ![Formula][149] distribution, a χ2 distribution with 1 df. As seen, the type I error rates were close the normal values ![Formula][150] . View this table: [Table 1](http://gpsych.bmj.com/content/36/2/e100977/T1) Table 1 Parameter estimates, SEs (asymptotic and empirical) and type I errors from GEE model with 1 000 MC replications ### Real study Smoking is the chief avoidable cause of morbidity and mortality in the USA, exacting a substantive financial burden as well.8 Smoking rates among persons with serious mental illness are exceptionally high, contributing to significant medical morbidity and mortality in this population, with many unlikely to live beyond their 50th birthday. Persons with mental illness spend nearly one-third of their monthly public assistance income on cigarettes instead of buying needed food, clothing and shelter.9 A study was conducted to evaluate the effect of a multicomponent smoking cessation programme adapted to patients with serious psychiatric disorders within an outpatient psychiatric clinic at the University of Rochester Medical Center. This study, sponsored by the New York State Department of Health Tobacco Control Program, capitalises on packaging multiple evidence-based components to achieve a better outcome than when each practice is individually implemented in a number of clinical venues, for example, central line–associated bloodstream infections and ventilator-associated pneumonia.10 Among the 276 participating subjects, 99 also participated in a formal evaluation, in which interviews were conducted at the point of enrolment (baseline), prior to intervention and again at 3, 6 and 12 months. For illustrative purposes, we model the binary abstinence outcome, defined as the 7-day point prevalence (ie, abstinent from smoking for 7 days in a row), from preintervention at baseline, ![Formula][151] , to each of the three postintervention assessments, ![Formula][152] , at 3, 6 and 12 months, using data from 99 subjects. We create three time-varying dummy variables ![Formula][153] , ![Formula][154] and ![Formula][155] to indicate intervention effects at ![Formula][156] : ![Formula][157] Let ![Formula][158] if the i th subject is abstinent for 7 days consecutively and ![Formula][159] otherwise. The semiparametric GEE for change of abstinence rates over time is given by: ![Formula][160] (14) We fit (8) to the 7-day point prevalence data using the GEE in (9) under the working independent correlation model. Shown in table 2 are the estimates ![Formula][161] of ![Formula][162] and associated SEs, p values for testing the null ![Formula][163] and RRs (exponentiated ![Formula][164] ) at each assessment ![Formula][165] . The results show a RR greater than 1 for all three postintervention assessments, though only statistically significant at months 3 and 6. The intervention did have a significant effect on reducing smoking in this study sample, though the effect diminished 12 months after the intervention. View this table: [Table 2](http://gpsych.bmj.com/content/36/2/e100977/T2) Table 2 Estimates of parameters, SEs, p values and relative risks over time from GEE model to the Smoking Cessation Study data ## Discussion We extended the popular approach for modelling RRs for binary responses to longitudinal data by leveraging the semiparametric GEE. Like the original approach in Zou,1 the parameters of the proposed log-linear model have the log of RR interpretation and, thus, with appropriately defined explanatory variables, can be used for inference about RRs when modelling longitudinal regression relationships with binary responses. We also illustrated the proposed approach using both real and simulated longitudinal data. The proposed GEE-based approach provides valid inference under the missing completely at random (MCAR) mechanism.3 11 In many real studies, missing data follow the missing at random (MAR) mechanism,3 11 in which case the lowest patterns done by the proposed approach generally yield biased estimates of RR. We can readily extend the approach to provide valid inference under MAR by employing the weighted generalised estimating equations (WGEEs).11 Under WGEE, we also model the missingness of the binary response over time using GLMs for binary responses such as logistic regression and estimate its parameters and the parameters of the log-linear model in (8) together using a set of estimating equations that extend (9) to include the additional parameters.3 ### Supplementary data [[gpsych-2022-100977supp001.pdf]](pending:yes) ## Footnotes * Contributors All authors participated in the discussion of the statistical issues and worked together to develop this paper. HZ and XMT suggested the topic, and TL, RZ, ST and HW reviewed the literature. All authors discussed the conceptual and analytical issues with modelling RRs for longitudinal data using the parametric and semiparametric models. RZ, ST and HW developed the simulation settings, algorithms and associated R codes and performed the simulation study under the direction of TL. TL, HZ and XMT drafted the manuscript, while TL, RZ, ST and HW provided all the technical details and derivations, along with completing the application section. All authors worked together to finalise the manuscript. * Funding The project described was partially supported by the National Institutes of Health (grant UL1TR001442) of Georgia Clinical and Translational Science Alliance funding. * Competing interests None declared. * Provenance and peer review Commissioned; externally peer-reviewed. * Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise. * Received December 1, 2022. * Accepted February 10, 2023. * © Author(s) (or their employer(s)) 2023. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ. [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/) This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/). ## References 1. Zou G . A modified poisson regression approach to prospective studies with binary data. Am J Epidemiol 2004;159:702–6. [doi:10.1093/aje/kwh090](http://dx.doi.org/10.1093/aje/kwh090) [CrossRef](http://gpsych.bmj.com/lookup/external-ref?access_num=10.1093/aje/kwh090&link_type=DOI) [PubMed](http://gpsych.bmj.com/lookup/external-ref?access_num=15033648&link_type=MED&atom=%2Fgpsych%2F36%2F2%2Fe100977.atom) [Web of Science](http://gpsych.bmj.com/lookup/external-ref?access_num=000220484900009&link_type=ISI) 2. Feng C , Wang H , Wang B , et al . Relationships among three popular measures of differential risks: relative risk, risk difference, and odds ratio. Shanghai Arch Psychiatry 2016;28:56–60. [doi:10.11919/j.issn.1002-0829.216031](http://dx.doi.org/10.11919/j.issn.1002-0829.216031) 3. Tang W , He H , Tu XM . Applied categorical and count data analysis. Hall/CRC, FL: Chapman, 2012. [doi:10.1201/b12123](http://dx.doi.org/10.1201/b12123) 4. McNutt L-A , Wu C , Xue X , et al . Estimating the relative risk in cohort studies and clinical trials of common outcomes. Am J Epidemiol 2003;157:940–3. [doi:10.1093/aje/kwg074](http://dx.doi.org/10.1093/aje/kwg074) [CrossRef](http://gpsych.bmj.com/lookup/external-ref?access_num=10.1093/aje/kwg074&link_type=DOI) [PubMed](http://gpsych.bmj.com/lookup/external-ref?access_num=12746247&link_type=MED&atom=%2Fgpsych%2F36%2F2%2Fe100977.atom) [Web of Science](http://gpsych.bmj.com/lookup/external-ref?access_num=000182958100011&link_type=ISI) 5. Wallenstein S , Bodian C . Inferences on odds ratios, relative risks, and risk differences based on standard regression programs. American Journal of Epidemiology 1987;126:346–55. [doi:10.1093/aje/126.2.346](http://dx.doi.org/10.1093/aje/126.2.346) [CrossRef](http://gpsych.bmj.com/lookup/external-ref?access_num=10.1093/aje/126.2.346&link_type=DOI) [PubMed](http://gpsych.bmj.com/lookup/external-ref?access_num=3605061&link_type=MED&atom=%2Fgpsych%2F36%2F2%2Fe100977.atom) 6. R Core Team. R: a language and environment for statistical computing. R foundation for statistical computing. Vienna, Austria. Available: [www.R-project.org/](http://www.R-project.org/) [Accessed 15 Dec 2022]. 7. Yan J . Enjoy the joy of copulas: with a package copula. J Stat Softw 2007;21:1–21. [CrossRef](http://gpsych.bmj.com/lookup/external-ref?access_num=10.18637/jss.v021.i11&link_type=DOI) [PubMed](http://gpsych.bmj.com/lookup/external-ref?access_num=WOS:00025242&link_type=MED&atom=%2Fgpsych%2F36%2F2%2Fe100977.atom) 8. Fiore MC , Jaen CR , Baker TB , et al . Treating tobacco use and dependence: 2008 update. clinical practice guideline. the U.S. surgeon general’S world wide web web site. 2008. Available: [www.surgeongeneral.gov/tobacco/treating\_tobacco\_use08.pdf](http://www.surgeongeneral.gov/tobacco/treating_tobacco_use08.pdf) [Accessed 15 Dec 2022]. 9. Steinberg ML , Williams JM , Ziedonis DM . Financial implications of cigarette smoking among individuals with schizophrenia. Tob Control 2004;13:206. 10. Institute for Health Care Improvement. Introduction to evidence based practices and bundling. 2009. Available: [www.premierinc.com/quality-safety/tools-services/safety/topics/bundling](http://www.premierinc.com/quality-safety/tools-services/safety/topics/bundling) [Accessed 15 Nov 2022]. 11. Little RJA , Rubin DB . Statistical analysis with missing data. New York: Wiley, 1987. Tuo Lin is a fifth-year PhD student in Biostatistics at the University of California, San Diego (UCSD) in the USA. He obtained his master’s degree in Statistics at UCSD in 2018. He is currently working as a graduate student researcher in the division of Biostatistics and Bioinformatics of Herbert Wertheim School of Public Health and Human Longevity Science at UCSD. He has also been working at Altman Clinical and Translational Research Institute (ACTRI) in the USA for many years, helping with study designs and data analyses. His main research interests include survey sampling and methods, causal inference and longitudinal data analysis in psychiatry studies.
![][166] [1]: /embed/mml-math-1.gif [2]: /embed/mml-math-2.gif [3]: /embed/mml-math-3.gif [4]: /embed/mml-math-4.gif [5]: /embed/mml-math-5.gif [6]: /embed/mml-math-6.gif [7]: /embed/mml-math-7.gif [8]: /embed/mml-math-8.gif [9]: /embed/mml-math-9.gif [10]: /embed/mml-math-10.gif [11]: /embed/mml-math-11.gif [12]: /embed/mml-math-12.gif [13]: /embed/mml-math-13.gif [14]: /embed/mml-math-14.gif [15]: /embed/mml-math-15.gif [16]: /embed/mml-math-16.gif [17]: /embed/mml-math-17.gif [18]: /embed/mml-math-18.gif [19]: /embed/mml-math-19.gif [20]: /embed/mml-math-20.gif [21]: /embed/mml-math-21.gif [22]: /embed/mml-math-22.gif [23]: /embed/mml-math-23.gif [24]: /embed/mml-math-24.gif [25]: /embed/mml-math-25.gif [26]: /embed/mml-math-26.gif [27]: /embed/mml-math-27.gif [28]: /embed/mml-math-28.gif [29]: /embed/mml-math-29.gif [30]: /embed/mml-math-30.gif [31]: /embed/mml-math-31.gif [32]: /embed/mml-math-32.gif [33]: /embed/mml-math-33.gif [34]: /embed/mml-math-34.gif [35]: /embed/mml-math-35.gif [36]: /embed/mml-math-36.gif [37]: /embed/mml-math-37.gif [38]: /embed/mml-math-38.gif [39]: /embed/mml-math-39.gif [40]: /embed/mml-math-40.gif [41]: /embed/mml-math-41.gif [42]: /embed/mml-math-42.gif [43]: /embed/mml-math-43.gif [44]: /embed/mml-math-44.gif [45]: /embed/mml-math-45.gif [46]: /embed/mml-math-46.gif [47]: /embed/mml-math-47.gif [48]: /embed/mml-math-48.gif [49]: /embed/mml-math-49.gif [50]: /embed/mml-math-50.gif [51]: /embed/mml-math-51.gif [52]: /embed/mml-math-52.gif [53]: /embed/mml-math-53.gif [54]: /embed/mml-math-54.gif [55]: /embed/mml-math-55.gif [56]: /embed/mml-math-56.gif [57]: /embed/mml-math-57.gif [58]: /embed/mml-math-58.gif [59]: /embed/mml-math-59.gif [60]: /embed/mml-math-60.gif [61]: /embed/mml-math-61.gif [62]: /embed/mml-math-62.gif [63]: /embed/mml-math-63.gif [64]: /embed/mml-math-64.gif [65]: /embed/mml-math-65.gif [66]: /embed/mml-math-66.gif [67]: /embed/mml-math-67.gif [68]: /embed/mml-math-68.gif [69]: /embed/mml-math-69.gif [70]: /embed/mml-math-70.gif [71]: /embed/mml-math-71.gif [72]: /embed/mml-math-72.gif [73]: /embed/mml-math-73.gif [74]: /embed/mml-math-74.gif [75]: /embed/mml-math-75.gif [76]: /embed/mml-math-76.gif [77]: /embed/mml-math-77.gif [78]: /embed/mml-math-78.gif [79]: /embed/mml-math-79.gif [80]: /embed/mml-math-80.gif [81]: /embed/mml-math-81.gif [82]: /embed/mml-math-82.gif [83]: /embed/mml-math-83.gif [84]: /embed/mml-math-84.gif [85]: /embed/mml-math-85.gif [86]: /embed/mml-math-86.gif [87]: /embed/mml-math-87.gif [88]: /embed/mml-math-88.gif [89]: /embed/mml-math-89.gif [90]: /embed/mml-math-90.gif [91]: /embed/mml-math-91.gif [92]: /embed/mml-math-92.gif [93]: /embed/mml-math-93.gif [94]: /embed/mml-math-94.gif [95]: /embed/mml-math-95.gif [96]: /embed/mml-math-96.gif [97]: /embed/mml-math-97.gif [98]: /embed/mml-math-98.gif [99]: /embed/mml-math-99.gif [100]: /embed/mml-math-100.gif [101]: /embed/mml-math-101.gif [102]: /embed/mml-math-102.gif [103]: /embed/mml-math-103.gif [104]: /embed/mml-math-104.gif [105]: /embed/mml-math-105.gif [106]: /embed/mml-math-106.gif [107]: /embed/mml-math-107.gif [108]: /embed/mml-math-108.gif [109]: /embed/mml-math-109.gif [110]: /embed/mml-math-110.gif [111]: /embed/mml-math-111.gif [112]: /embed/mml-math-112.gif [113]: /embed/mml-math-113.gif [114]: /embed/mml-math-114.gif [115]: /embed/mml-math-115.gif [116]: /embed/mml-math-116.gif [117]: /embed/mml-math-117.gif [118]: /embed/mml-math-118.gif [119]: /embed/mml-math-119.gif [120]: /embed/mml-math-120.gif [121]: /embed/mml-math-121.gif [122]: /embed/mml-math-122.gif [123]: /embed/mml-math-123.gif [124]: /embed/mml-math-124.gif [125]: /embed/mml-math-125.gif [126]: /embed/mml-math-126.gif [127]: /embed/mml-math-127.gif [128]: /embed/mml-math-128.gif [129]: /embed/mml-math-129.gif [130]: /embed/mml-math-130.gif [131]: /embed/mml-math-131.gif [132]: /embed/mml-math-132.gif [133]: /embed/mml-math-133.gif [134]: /embed/mml-math-134.gif [135]: /embed/mml-math-135.gif [136]: /embed/mml-math-136.gif [137]: /embed/mml-math-137.gif [138]: /embed/mml-math-138.gif [139]: /embed/mml-math-139.gif [140]: /embed/mml-math-140.gif [141]: /embed/mml-math-141.gif [142]: /embed/mml-math-142.gif [143]: /embed/mml-math-143.gif [144]: /embed/mml-math-144.gif [145]: /embed/mml-math-145.gif [146]: /embed/mml-math-146.gif [147]: /embed/mml-math-147.gif [148]: /embed/mml-math-148.gif [149]: /embed/mml-math-149.gif [150]: /embed/mml-math-150.gif [151]: /embed/mml-math-153.gif [152]: /embed/mml-math-154.gif [153]: /embed/mml-math-155.gif [154]: /embed/mml-math-156.gif [155]: /embed/mml-math-157.gif [156]: /embed/mml-math-158.gif [157]: /embed/mml-math-159.gif [158]: /embed/mml-math-160.gif [159]: /embed/mml-math-161.gif [160]: /embed/mml-math-162.gif [161]: /embed/mml-math-163.gif [162]: /embed/mml-math-164.gif [163]: /embed/mml-math-165.gif [164]: /embed/mml-math-166.gif [165]: /embed/mml-math-167.gif [166]: /embed/graphic-1.gif