, Volume 236, Issue 9, pp 2641–2652 | Cite as

Evaluating non-medical prescription opioid demand using commodity purchase tasks: test-retest reliability and incremental validity

  • Justin C. StricklandEmail author
  • Joshua A. Lile
  • William W. Stoops
Original Investigation



Non-medical prescription opioid use and opioid use disorder (OUD) present a significant public health concern. Identifying behavioral mechanisms underlying OUD will assist in developing improved prevention and intervention approaches. Behavioral economic demand has been extensively evaluated as a measure of reinforcer valuation for alcohol and cigarettes, whereas prescription opioids have received comparatively little attention.


Utilize a purchase task procedure to measure the incremental validity and test-retest reliability of opioid demand.


Individuals reporting past year non-medical prescription opioid use were recruited using the crowdsourcing platform Amazon Mechanical Turk (mTurk). Participants completed an opioid purchase task as well as measures of cannabis demand, delay discounting, and self-reported pain. A 1-month follow-up was used to evaluate test-retest reliability.


More intense and inelastic opioid demand was associated with OUD and more intense cannabis demand was associated with cannabis use disorder. Multivariable models indicated that higher opioid intensity and steeper opioid delay discounting rates each significantly and uniquely predicted OUD. Increased opioid demand intensity, but not elasticity, was associated with higher self-reported pain, and no relationship was observed with perceived pain relief from opioids. Opioid demand showed acceptable-to-good test-retest reliability (e.g., intensity rxx = .75; elasticity rxx = .63). Temporal reliability was lower for cannabis demand (e.g., intensity rxx = .53; elasticity rxx = .58) and discounting rates (rxx = .42–.61).


Opioid demand was incrementally valid and test-retest reliable as measured by purchase tasks. These findings support behavioral economic demand as a clinically useful measure of drug valuation that is sensitive to individual difference variables.


Behavioral economics Cannabis Demand Discounting mTurk Opioid Pain Purchase task Reliability 


Funding information

This research was supported by the National Science Foundation Grant 1247392, a Graduate Student Research Grant from the Psi Chi Psychology Honor Society, and Professional Development Funds from the University of Kentucky Department of Behavioral Science. These funding sources had no role in study design, data collection or analysis, or preparation and submission of the manuscript. The authors have no financial conflicts of interest in regard to this research.

Compliance with ethical standards

The University of Kentucky Institution Review Board approved all procedures and participants reviewed an informed consent prior to participation.

Supplementary material

213_2019_5234_MOESM1_ESM.docx (127 kb)
ESM 1 (DOCX 126 kb)


  1. Acuff SF, Murphy JG (2017) Further examination of the temporal stability of alcohol demand. Behav Process 141:33–41CrossRefGoogle Scholar
  2. Arner S, Meyerson B (1988) Lack of analgesic effect of opioids on neuropathic and idiopathic forms of pain. Pain 33:11–23CrossRefGoogle Scholar
  3. Ashburn MA, Staats PS (1999) Management of chronic pain. Lancet 353:1865–1869CrossRefGoogle Scholar
  4. Aston ER, Farris SG, MacKillop J, Metrik J (2017) Latent factor structure of a behavioral economic marijuana demand curve. Pyshcopharmacol 234:2421–2429CrossRefGoogle Scholar
  5. Aston ER, Metrik J, MacKillop J (2015) Further validation of a marijuana purchase task. Drug Alcohol Depend 152:32–38CrossRefGoogle Scholar
  6. Baker F, Johnson MW, Bickel WK (2003) Delay discounting in current and never-before cigarette smokers: similarities and differences across commodity, sign, and magnitude. J Abnorm Psychol 112:382–392CrossRefGoogle Scholar
  7. Berinsky AJ, Huber GA, Lenz GS (2012) Evaluating online labor markets for experimental research:’s Mechanical Turk. Polit Anal 20:351–368CrossRefGoogle Scholar
  8. Bickel WK, Landes RD, Christensen DR, Jackson L, Jones BA, Kurth-Nelson Z, Redish AD (2011) Single-and cross-commodity discounting among cocaine addicts: the commodity and its temporal location determine discounting rate. Pyshcopharmacol 217:177–187CrossRefGoogle Scholar
  9. Bickel WK, Snider SE, Quisenberry AJ, Stein JS (2017) Reinforcer pathology: the behavioral economics of abuse liability testing. Clin Phramacol Ther 101:185–187CrossRefGoogle Scholar
  10. Bidwell LC, MacKillop J, Murphy JG, Tidey JW, Colby SM (2012) Latent factor structure of a behavioral economic cigarette demand curve in adolescent smokers. Addict Behav 37:1257–1263CrossRefGoogle Scholar
  11. Brown JD, Goodin AJ, Talbert JC (2018) Rural and Appalachian disparities in neonatal abstinence syndrome incidence and access to opioid abuse treatment. J Rural Health 34:6–13CrossRefGoogle Scholar
  12. Bujarski S, MacKillop J, Ray LA (2012) Understanding naltrexone mechanism of action and pharmacogenetics in Asian Americans via behavioral economics: a preliminary study. Exp Clin Psychopharmacol 20:181–190CrossRefGoogle Scholar
  13. Carlson RG, Nahhas RW, Martins SS, Daniulaityte R (2016) Predictors of transition to heroin use among initially non-opioid dependent illicit pharmaceutical opioid users: a natural history study. Drug Alcohol Depend 160:127–134CrossRefGoogle Scholar
  14. Cassidy RN, Tidey JW, Colby SM, Long V, Higgins ST (2017) Initial development of an e-cigarette purchase task: a mixed methods study. Tob Regul Sci 3:139–150CrossRefGoogle Scholar
  15. Caulkins JP, Sussell J, Kilmer B, Kasunic A (2015) How much of the cocaine market are we missing? Insights from respondent-driven sampling in a mid-sized American city. Drug Alcohol Depend 147:190–195CrossRefGoogle Scholar
  16. Center for Behavioral Health Statistics (2018) 2017 National Survey on drug use and health: detailed tables. Substance Abuse and Mental Health Services Administration, Rockville, MDGoogle Scholar
  17. Cicchetti DV (1994) Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess 6:284–290CrossRefGoogle Scholar
  18. Chandler J, Shapiro D (2016) Conducting clinical research using crowdsourced convenience samples. Annu Rev Clin Psychol 12:53–81CrossRefGoogle Scholar
  19. Charlton SR, Fantino E (2008) Commodity specific rates of temporal discounting: does metabolic function underlie differences in rates of discounting? Behav Process 77:334–342CrossRefGoogle Scholar
  20. Chase HW, Mackillop J, Hogarth L (2013) Isolating behavioural economic indices of demand in relation to nicotine dependence. Pyshcopharmacol 226:371–380CrossRefGoogle Scholar
  21. Choo EK, Ewing SWF, Lovejoy TI (2016) Opioids out, cannabis in: negotiating the unknowns in patient care for chronic pain. JAMA 316:1763–1764CrossRefGoogle Scholar
  22. Compton WM, Jones CM, Baldwin GT (2016) Relationship between nonmedical prescription-opioid use and heroin use. N Engl J Med 374:154–163CrossRefGoogle Scholar
  23. Conrad C, Bradley HM, Broz D, Buddha S, Chapman EL, Galang RR, Hillman D, Hon J, Hoover KW, Patel MR (2015) Community outbreak of HIV infection linked to injection drug use of oxymorphone--Indiana, 2015. MMWR Morb Mortal Wkly Rep 64:443–444Google Scholar
  24. Cox DJ, Dallery J (2016) Effects of delay and probability combinations on discounting in humans. Behav Process 131:15–23CrossRefGoogle Scholar
  25. Cunningham JA, Godinho A, Kushnir V (2017) Using Mechanical Turk to recruit participants for internet intervention research: experience from recruitment for four trials targeting hazardous alcohol consumption. BMC Med Res Methodol 17:e156CrossRefGoogle Scholar
  26. Epstein LH, Stein JS, Paluch RA, MacKillop J, Bickel WK (2018) Binary components of food reinforcement: amplitude and persistence. Appetite 120:67–74CrossRefGoogle Scholar
  27. Few LR, Acker J, Murphy C, MacKillop J (2012) Temporal stability of a cigarette purchase task. Nicotine Tob Res 14:761–765CrossRefGoogle Scholar
  28. Hagman BT (2017) Development and psychometric analysis of the brief DSM-5 alcohol use disorder diagnostic assessment: towards effective diagnosis in college students. Psychol Addict Behav 31:797–806CrossRefGoogle Scholar
  29. Hedegaard H, Warner M, Miniño AM (2017) Drug overdose deaths in the United States, 1999–2016. US Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health StatisticsGoogle Scholar
  30. Hill KP (2015) Medical marijuana for treatment of chronic pain and other medical and psychiatric problems: a clinical review. JAMA 313:2474–2483CrossRefGoogle Scholar
  31. Huff C, Tingley D (2015) “Who are these people?” Evaluating the demographic characteristics and political preferences of MTurk survey respondents. Res Politics 2:e2053168015604648CrossRefGoogle Scholar
  32. Hursh SR, Roma PG (2013) Behavioral economics and empirical public policy. J Exp Anal Behav 99:98–124CrossRefGoogle Scholar
  33. Hursh SR, Silberberg A (2008) Economic demand and essential value. Psychol Rev 115:186–198CrossRefGoogle Scholar
  34. Jacobs EA, Bickel WK (1999) Modeling drug consumption in the clinic using simulation procedures: demand for heroin and cigarettes in opioid-dependent outpatients. Exp Clin Psychopharmacol 7:412–426CrossRefGoogle Scholar
  35. Johnson MW, Bickel WK (2006) Replacing relative reinforcing efficacy with behavioral economic demand curves. J Exp Anal Behav 85:73–93CrossRefGoogle Scholar
  36. Johnson MW, Bickel WK, Baker F (2007) Moderate drug use and delay discounting: a comparison of heavy, light, and never smokers. Exp Clin Psychopharmacol 15:187–194CrossRefGoogle Scholar
  37. Johnson MW, Bruner NR (2012) The sexual discounting task: HIV risk behavior and the discounting of delayed sexual rewards in cocaine dependence. Drug Alcohol Depend 123:15–21CrossRefGoogle Scholar
  38. Kaplan BA, Foster RN, Reed DD, Amlung M, Murphy JG, MacKillop J (2018) Understanding alcohol motivation using the alcohol purchase task: a methodological systematic review. Drug Alcohol Depend 191:117–140CrossRefGoogle Scholar
  39. Kim HS, Hodgins DC (2017) Reliability and validity of data obtained from alcohol, cannabis, and gambling populations on Amazon’s Mechanical Turk. Psychol Addict Behav 31:85–94CrossRefGoogle Scholar
  40. Kirby KN, Petry NM (2004) Heroin and cocaine abusers have higher discount rates for delayed rewards than alcoholics or non-drug-using controls. Addiction 99:461–471CrossRefGoogle Scholar
  41. Kirby KN, Petry NM, Bickel WK (1999) Heroin addicts have higher discount rates for delayed rewards than non-drug-using controls. J Exp Psychol Gen 128:78–87CrossRefGoogle Scholar
  42. Koffarnus MN, Bickel WK (2014) A 5-trial adjusting delay discounting task: accurate discount rates in less than one minute. Exp Clin Psychopharmacol 22:222–228CrossRefGoogle Scholar
  43. Koffarnus MN, Franck CT, Stein JS, Bickel WK (2015) A modified exponential behavioral economic demand model to better describe consumption data. Exp Clin Psychopharmacol 23:504–512CrossRefGoogle Scholar
  44. Kolodny A, Courtwright DT, Hwang CS, Kreiner P, Eadie JL, Clark TW, Alexander GC (2015) The prescription opioid and heroin crisis: a public health approach to an epidemic of addiction. Annu Rev Public Health 36:559–574CrossRefGoogle Scholar
  45. Lakens D (2017) Equivalence tests: a practical primer for t tests, correlations, and meta-analyses. Soc Psychol Personal Sci 8:355–362CrossRefGoogle Scholar
  46. Lucas P (2012) Cannabis as an adjunct to or substitute for opiates in the treatment of chronic pain. J Psychoactive Drugs 44:125–133CrossRefGoogle Scholar
  47. Mack KA, Jones CM, Ballesteros MF (2017) Illicit drug use, illicit drug use disorders, and drug overdose deaths in metropolitan and nonmetropolitan areas—United States. Am J Transplant 17:3241–3252CrossRefGoogle Scholar
  48. MacKillop J (2016) The behavioral economics and neuroeconomics of alcohol use disorders. Alcohol Clin Exp Res 40:672–685CrossRefGoogle Scholar
  49. MacKillop J, Murphy JG (2007) A behavioral economic measure of demand for alcohol predicts brief intervention outcomes. Drug Alcohol Depend 89:227–233CrossRefGoogle Scholar
  50. Mackillop J, Murphy JG, Tidey JW, Kahler CW, Ray LA, Bickel WK (2009) Latent structure of facets of alcohol reinforcement from a behavioral economic demand curve. Pyshcopharmacol 203:33–40CrossRefGoogle Scholar
  51. Mendoza T, Mayne T, Rublee D, Cleeland C (2006) Reliability and validity of a modified brief pain inventory short form in patients with osteoarthritis. Eur J Pain 10:353–353CrossRefGoogle Scholar
  52. Morris V, Patel H, Vedelago L, Reed DD, Metrik J, Aston E, MacKillop J, Amlung M (2018) Elevated behavioral economic demand for alcohol in co-users of alcohol and cannabis. J Stud Alcohol Drugs 79:929–934CrossRefGoogle Scholar
  53. Murphy JG, Dennhardt AA, Yurasek AM, Skidmore JR, Martens MP, MacKillop J, McDevitt-Murphy ME (2015) Behavioral economic predictors of brief alcohol intervention outcomes. J Consult Clin Psychol 83:1033–1043CrossRefGoogle Scholar
  54. Murphy JG, MacKillop J, Skidmore JR, Pederson AA (2009) Reliability and validity of a demand curve measure of alcohol reinforcement. Exp Clin Psychopharmacol 17:396–404CrossRefGoogle Scholar
  55. Necka EA, Cacioppo S, Norman GJ, Cacioppo JT (2016) Measuring the prevalence of problematic respondent behaviors among MTurk, campus, and community participants. PLoS One 11:e0157732CrossRefGoogle Scholar
  56. Paolacci G, Chandler J (2014) Inside the Turk: understanding Mechanical Turk as a participant pool. Curr Dir Psychol Sci 23:184–188CrossRefGoogle Scholar
  57. Pickover AM, Messina BG, Correia CJ, Garza KB, Murphy JG (2016) A behavioral economic analysis of the nonmedical use of prescription drugs among young adults. Exp Clin Psychopharmacol 24:38–47CrossRefGoogle Scholar
  58. Rasmussen EB, Lawyer SR, Reilly W (2010) Percent body fat is related to delay and probability discounting for food in humans. Behav Process 83:23–30CrossRefGoogle Scholar
  59. Schranz AJ, Barrett J, Hurt CB, Malvestutto C, Miller WC (2018) Challenges facing a rural opioid epidemic: treatment and prevention of HIV and hepatitis C. Curr HIV/AIDS Rep 15:245–254CrossRefGoogle Scholar
  60. Sehgal N, Manchikanti L, Smith HS (2012) Prescription opioid abuse in chronic pain: a review of opioid abuse predictors and strategies to curb opioid abuse. Pain Physician 15:ES67–ES92Google Scholar
  61. Shapiro DN, Chandler J, Mueller PA (2013) Using Mechanical Turk to study clinical populations. Clin Psychol Sci 1:213–220CrossRefGoogle Scholar
  62. Stein JS, Koffarnus MN, Snider SE, Quisenberry AJ, Bickel WK (2015) Identification and management of nonsystematic purchase task data: toward best practice. Exp Clin Psychopharmacol 23:377–386CrossRefGoogle Scholar
  63. Stein JS, Sze YY, Athamneh L, Koffarnus MN, Epstein LH, Bickel WK (2017) Think fast: rapid assessment of the effects of episodic future thinking on delay discounting in overweight/obese participants. J Behav Med 40:832–838CrossRefGoogle Scholar
  64. Strathdee SA, Beyrer C (2015) Threading the needle—how to stop the HIV outbreak in rural Indiana. N Engl J Med 373:397–399CrossRefGoogle Scholar
  65. Strickland JC, Lile JA, Stoops WW (2017) Unique prediction of cannabis use severity and behaviors by delay discounting and behavioral economic demand. Behav Process 140:33–40CrossRefGoogle Scholar
  66. Strickland JC, Stoops WW (2015) Perceptions of research risk and undue influence: implications for ethics of research conducted with cocaine users. Drug Alcohol Depend 156:304–310CrossRefGoogle Scholar
  67. Strickland JC, Stoops WW (2017) Stimulus selectivity of drug purchase tasks: a preliminary study evaluating alcohol and cigarette demand. Exp Clin Psychopharmacol 25:198–207CrossRefGoogle Scholar
  68. Strickland JC, Stoops WW (2018) Feasibility, acceptability, and validity of crowdsourcing for collecting longitudinal alcohol use data. J Exp Anal Behav 110:136–153CrossRefGoogle Scholar
  69. Strickland JC, Stoops WW (2019) The use of crowdsourcing in addiction science research: Amazon Mechanical Turk. Exp Clin Pyshcopharmacol 27:1–18CrossRefGoogle Scholar
  70. Tsukayama E, Duckworth AL (2010) Domain-specific temporal discounting and temptation. Judgm Decis Mak 5:72–82Google Scholar
  71. Van Handel M, Rose CE, Hallisey EJ, Kolling JL, Zibbell JE, Lewis B, Bohm MK, Jones CM, Flanagan BE, Siddiqi A-E (2016) County-level vulnerability assessment for rapid dissemination of HIV or HCV infections among persons who inject drugs, United States. J Acquir Immune Defic Syndr 73:323–331CrossRefGoogle Scholar
  72. Volkow ND, Collins FS (2017) The role of science in addressing the opioid crisis. N Engl J Med 377:391–394CrossRefGoogle Scholar
  73. Young AM, Havens JR (2012) Transition from first illicit drug use to first injection drug use among rural Appalachian drug users: a cross-sectional comparison and retrospective survival analysis. Addiction 107:587–596CrossRefGoogle Scholar
  74. Yurasek AM, Murphy JG, Clawson AH, Dennhardt AA, MacKillop J (2013) Smokers report greater demand for alcohol on a behavioral economic purchase task. J Stud Alcohol 74:626–634CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of PsychologyUniversity of Kentucky College of Arts and SciencesLexingtonUSA
  2. 2.Department of Behavioral ScienceUniversity of Kentucky College of MedicineLexingtonUSA
  3. 3.Department of PsychiatryUniversity of Kentucky College of MedicineLexingtonUSA
  4. 4.Center on Drug and Alcohol ResearchUniversity of Kentucky College of MedicineLexingtonUSA

Personalised recommendations