Simple Statistical Tests and P Values

Abstract

Having a general understanding of statistical tests , and how to interpret their results, is an important part of conducting, consuming, and applying evidence from the surgical literature. Unfortunately, statistical guides can often be focused on mathematical formulas, which maybe difficult for the average surgical researcher to understand. The goal of this chapter is to describe common statistical principles in English, leaving the mathematical symbolism to a minimum. This chapter utilizes a clinical trial from the surgical literature to explain simple statistical tests that are often seen in the surgical literature. These statistical tests are then reproduced using readily available statistical software, and compared to those results reported by the original authors. This chapter will introduce readers to common statistical ideas including, but not limited to chi-square tests, Student’s t test, Fisher exact tests, and sensitivity analysis . Additionally, readers will be introduced to the Cochrane risk of bias tool, and how this tool can be used to appraise a study. This chapter also acts as a reference to connect readers with additional resources, some more mathematical than others, that will offer additional information and different perspectives than this chapter.

Appendices

Appendix 1: Articles Identified in the Literature Search

• 1. Barberan-Garcia A, Ubré M, Roca J, Lacy AM, Burgos F, Risco R, Momblan D, Balust J, Blanco I, Martinez-Palli G. Personalized prehabilitation in high-risk patients undergoing elective major abdominal surgery: a randomized blinded controlled trial. Ann Surg. 2018;267(1):50–6. https://doi.org/10.1097/00000000000002293.

• 2. Berkel AEM, Bongers BC, vanKamp M-JS, Kotte H, Weltevreden P, deJongh FHC, Eijvogel MMM, Wymenga ANM, Bigirwamungu-Bargeman M, vander Palen J, van Det MC, van Meeteren NLU, Klasse JM. The effects of prehabilitation versus usual care to reduce postoperative complications in high-risk patients with colorectal cancer of dysplasia scheduled for elective colorectal resection: study protocol of a randomized controlled trial. BMC Gastroenterol. 2018;18:29. https://doi.org/10.1186/s12876-018-0754-6.

• 3. Mayo NE, Feldman L, Scott S, Zavorsky G, Kim DJ, Charlebois P, Stein B, Carli F. Impact of preoperative change in physical function on postoperative recovery: argument supporting prehabilitation for colorectal surgery. Surgery. 2011;150(3):505–14. https://doi.org/10.1016/j.surg.2011.07.045.

• 4. NCT02024776. Effectiveness of prehabilitation program for high-risk patients underwent abdominal surgery. [Registration]. Online Publication Date: 2018. Available from: http://clinicaltrials.gov/show/nct02024776.2013. [This is the registration for A1-1.]

• 5. NCT02934230. The prehabilitation study: exercise before surgery to improve patient function in people. [Registration]. Online Publication Date: 2018. Available from: http://clinicaltrials.gov/show/nct02934230.2016.

Appendix 2: Sensitivity Analysis for Table 27.3 Using Chi-Square Test and Minitab 18

• (a) Assume outcomes for missing patients were all NO, for both arms

Comps\Arm	I, %	C, %	Total, %	Δ	Pearson X2, LR X2	DF	P values	Report P2
Yes	19, 26	39, 55	58.40	29	12.499, 12.793	1	0.000, 0.000	< 0.001, < 0.001
No	54	32	86
Total	73	71	144

Comps = complications, Δ = %C − %I, LR = Likelihood Ratio , X² = computed chi-square statistic , DF = degrees of freedom, P2 = two-tailed P value ; also for b, c, and d.

• (b) Assume outcomes for missing patients were all NO for I and Yes for C

  Comps\Arm	I, %	C, %	Total, %	Δ	Pearson X2, LR X2	DF	P values	Report P2
  Yes	19, 26	47, 66	66.46	40	23.394, 24.077	1	0.000, 0.000	< 0.001, < 0.001
  No	54	24	78
  Total	73	71	144

• (c) Assume outcomes for missing patients were all NO for C and Yes for I

  Comps\Arm	I, %	C, %	Total, %	Δ	Pearson X2, LR X2	DF	P values	Report P2
  Yes	30, 41	39, 55	69.48	14	2.760, 2.769	1	0.097, 0.096	No change
  No	54	32	86
  Total	73	71	144

• (d) Assume outcomes for missing patients were all Yes, for both arms

  Comps\Arm	I, %	C, %	Total, %	Δ	Pearson X2, LR X2	DF	P values	Report P2
  Yes	30, 41	47, 66	77.53	25	9.115, 9.219	1	0.003, 0.002	No change
  No	43	24	67
  Total	73	71	144

• (e) Sensitivity analysis summary of all four tables above: think checkerboard! Using a 3 × 3 board

	I No		I Yes
C No	< 0.001, < 0.001, a	–	0.097, 0.096, c
	–	–	–
C Yes	< 0.001, < 0.001, b	–	0.003, 0.002, d

Entries are P2 values, letters indicate the tables above, and dashes (–) indicate parts of the sensitivity that we did not compute. Notice that the bolded cells, a, b, and d are all less than α = 0.05, so are statistically significant, while c cell that is not bolded so is not statistically significant.

Appendix 3: Risk of Bias Scoring of the Barberan-Garcia et al. [1] Paper

Risk of Bias

A	Was the method of randomization adequate?	Yes
B	Was the treatment allocation concealed?	No
Was knowledge of the allocated interventions adequately prevented during the study?
C	1. Was the patient blinded to the intervention ?	No
D	2. Was the care provider blinded to the intervention?	No
E	3. Was the outcome assessor blinded to the intervention?	Unsure
Were incomplete outcome data adequately addressed?
F	1. Was the dropout rate described and acceptable?	No, No
G	2. Were all randomized participants analyzed in the group to which they were allocated?	No
H	Are reports of the study free of suggestion of selective outcome reporting?	Unsure
Other sources of potential bias :
I	1. Were the groups similar at baseline regarding the most important prognostic indicators?	Unsure
J	2. Were co-interventions avoided or similar?	Unsure
K	3. Was the compliance acceptable in all groups?	Unsure
L	4. Was the timing of the outcome assessment similar in all groups?	Unsure
M	Is there a serious and fatal flaw with this study? (focus on the impact of selection bias, information bias, reporting errors, and confounding)	Flawed
	Are other sources of potential bias unlikely? - funding bias, conflict of interest statement - outcome measure not valid	Unsure

The Cochrane risk of bias tool is available from Higgins et al. [3].