Review of Understanding psychology as a science: An introduction to scientific and statistical inference more

Editorial Manager(tm) for British Journal of Mathematical and Statistical Psychology Manuscript Draft Manuscript Number: BJMSP406R1 Title: Understanding psychology as a science: An introduction to scientific and statistical inference Article Type: Book / Software Review Keywords: philosophy, psychology, statistics, inference, Bayes, likelihood Corresponding Author: Dr Thom Baguley, PhD Corresponding Author's Institution: Nottingham Trent University First Author: Thom Baguley Order of Authors: Thom Baguley; Danny Kaye Manuscript Region of Origin: Author / title page Running head: Understanding psychology as a science Understanding psychology as a science: An introduction to scientific and statistical inference Thom Baguley1 and Danny Kaye2 1 Division of Psychology, Nottingham Trent University NIA, Nottingham Trent University 2 Word count (incl. references and appendices): 1,496 *Requests for reprints should be addressed to Professor Thom Baguley, Division of Psychology, Nottingham Trent University, Burton Street, Nottingham, NG1 4BU, United Kingdom (e-mail: Thomas.Baguley@ntu.ac.uk). *Main document (inc. abstract, figs and tables) Understanding psychology as a science: An introduction to scientific and statistical inference By Z. Dienes Basingstoke: Palgrave Macmillan. 2008. Paperback £15.99. ISBN 0-230-54231-0. If we had to teach a course on the philosophy of science tomorrow this is the book we’d use. It takes an unashamedly quantitative view of science – a corrective that is useful in a marketplace where many texts take a historical perspective and perhaps have a too narrow focus on ontology. Dienes’ central thread – through what first appears to be a rather odd set of topics – is the question of how to test a theory in psychology. His starting point is the philosophy of science, first through the eyes of Popper and subsequently the contrasting views of Lakatos and Kuhn. These first two chapters cover familiar ground to many psychologists, but greatly to his credit, Dienes departs from the conventional script. The result is an engaging and scholarly reappraisal of the fundamentals of each philosophical perspective. The dissection of common misunderstandings of Popper’s philosophy is particularly enjoyable. The author’s voice comes through clearly in the text, but there is a skillful balance between authority and opinion that students should respond to with an appropriately critical attitude. Dienes’ take on falsification is one of the strongest sections, and seems to fit closely with how many scientists – certainly experimental psychologists – seem to employ it. By invoking the comparison of quantitative predictions from competing theories at this early stage he primes readers for some of the core ideas in the later chapters. These later chapters contrast three different stances on the statistical comparison and testing of theories: significance testing, Bayesian inference and likelihood. This half of the book will challenge many students – not because it is poorly written, or because it is inherently harder than the sections on philosophy of science – but because Dienes doesn’t hesitate to use quantitative examples. Where the first half can be read for gist, the second half requires students to engage closely with the material to get any proper understanding of it. Mathematically able students will probably breeze through these numerical examples, but others will need encouragement to follow them. However, most of the crucial examples are sufficiently simple that they can be worked through in class (and some will, depending on the level of the student, be familiar from other work). The use of detailed numerical examples is also essential to the goal of the book. Without them the distinction between the three approaches to testing would remain mysterious. It is important that readers understand that, at its core, each approach is instantiated in routine arithmetic operations (though these can increase in complexity as other considerations are added in). Although the author has tried to make almost every passage as accessible as possible – and has been generally successful – there are a few passages which it is a struggle to understand. These include the explanation of how to determine one’s personal subjective probability of an event (p. 83). There are also a number of minor typographical errors that get in the way of the prose at times – though at least one, featuring “Freud’s central tenant [sic]” (p. 11) brought a smile to our faces. On a practical note, providing programming code for several of the procedures used in the text (e.g., for Bayes factors and likelihood ratios) is an excellent idea, as is the provision of the calculators on the author’s web site. However, we feel that the provision of a free version of the code written in an open source system such as R (R Core Development Team, 2008) would be useful to those who do not have access to Matlab. To this end we have written R functions for several of Dienes’ examples. Appendix A contains brief notes on how to apply the functions for the Bayes factor examples. The Bayes factor function itself is included in Appendix B with instructions for accessing the source code for the remaining examples. In terms of content, Dienes has covered a great deal of material in just over 150 pages. It therefore seems harsh to focus on areas that would benefit from greater detail. One point stands out. The coverage of Bayesian inference is almost exclusively focused on subjective Bayesian methods. This is probably understandable in terms of the philosophical emphasis of the book, but perhaps out of step with the methods gaining prominence in psychology. Two recent examples – one too recent to have made the book – stand out. These are proposals for Bayesian t tests (e.g., Rouder et al., 2009) and BIC (Wagemakers, 2007). The omission of BIC is slightly puzzling – because of its link to likelihood and Bayes factors (and because AIC is covered). Overall this is one of the most original teaching texts that we’ve had the pleasure of reading for some time. It is well-structured and very readable. For almost any level of psychology student it manages to present the philosophical basis of statistics in a highly approachable manner. This is all the more useful as the application of Bayesian and Likelihood methods in psychology seems to be growing rapidly. T. BAGULEY (Division of Psychology, Nottingham Trent University) and W. K. ‘DANNY’ KAYE (NIA, Nottingham Trent University). References R Development Core Team (2008). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3900051-07-0, URL http://www.R-project.org. Rouder J. N, Speckman P. L., Sun D., Morey R. D., & Iverson G. (2009). Bayesian ttests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16, 225-237. Wagenmakers, E.-J. (2007). A practical solution to the pervasive problems of p values. Psychonomic Bulletin & Review, 14, 779-804. Appendix A: Applying Dienes’ Bayes factor calculations in R The Bf() function calculates Bayes factors as described in Dienes (2008, p.100119). These examples use the following parameter values taken from the book: sd = 1.09 upper = 0 meanoftheory = 0 obtained= -2.8 lower = -600 sdtheory = 2 The function definition is: Bf(sd, obtained, uniform, lower=0, upper=1, meanoftheory=0, sdtheory=1, tail=2) It can be called in several ways. For the format Bf(sd, obtained, uniform, lower, upper) the term uniform = 1, calls the function to calculate a Bayes factor under the assumption that the distribution of p(population value | theory) is uniform. Thus the call: > Bf(1.09, -2.8, 1, lower = -600, upper = 0) returns: $LikelihoodTheory = 0.00166435 $Likelihoodnull = 0.01350761 $BayesFactor = 0.1232157 Note that the last two parameters (lower and upper) must be named. The alternative format is: Bf(sd, obtained, uniform, meanoftheory, sdtheory, tail) In this case uniform = 0 and tail = 1, call the function to calculate a Bayes factor under the assumption that the distribution of p(population value | theory) is non-uniform and the distribution is one-tailed (tail = 2 giving the corresponding two-tailed Bayes factor). Note that in this type of call, the last three parameters (meanoftheory, sdtheory and tail) must be named. Thus: > Bf(1.09, -2.8, 0, meanoftheory =0, sdtheory =2, tail =1) returns $LikelihoodTheory = 0.001955743 $Likelihoodnull = 0.01350761 $BayesFactor = 0.1447882 In contrast, the call > Bf(1.09, -2.8, 0, meanoftheory =0, sdtheory =2, tail =2) gives $LikelihoodTheory = 0.08227421 $Likelihoodnull = 0.01350761 $BayesFactor = 6.090951 Appendix B: Bayes factor function for R based on Dienes (2008) The following function is an example of a set of R functions that allow readers to reproduce and apply the calculations set out in Dienes’ book. Each function is written as the code in the book, variable names are the same where possible and the basic structure is maintained. One extra function, the 2x2 cross classification calculated as a ratio of odds is given as well as that calculated from probabilities, this is based on similar code available from the author’s home page or via a link from the publisher’s web site. The full set of functions and notes on their use are available from http://www.danny-kaye.co.uk/research.html The sample function below allows readers to reproduce Bayes factor calculations from the book, as set out in Appendix A above. Bf<-function(sd, obtained, uniform, lower=0, upper=1, meanoftheory=0, sdtheory=1, tail=2) { #test data can be found starting at p100 of Dienes (2008) # area <- 0 if(identical(uniform, 1)){ theta <- lower range <- upper - lower incr <- range / 2000 for (A in -1000:1000){ theta <- theta + incr dist_theta <- 1 / range height <- dist_theta * dnorm(obtained, theta, sd) area <- area + height * incr } }else{ theta <- meanoftheory - 5 * sdtheory incr <- sdtheory / 200 for (A in -1000:1000){ theta <- theta + incr dist_theta <- dnorm(theta, meanoftheory, sdtheory) if(identical(tail, 1)){ if (theta <= 0){ dist_theta <- 0 } else { dist_theta <- dist_theta * 2 } } height <- dist_theta * dnorm(obtained, theta, sd) area <- area + height * incr } } LikelihoodTheory <- area Likelihoodnull <- dnorm(obtained, 0, sd) BayesFactor <- LikelihoodTheory / Likelihoodnull ret <- list("LikelihoodTheory" = LikelihoodTheory, "Likelihoodnull" = Likelihoodnull, "BayesFactor" = BayesFactor) } ret