Methods and limitations
A critical consideration of Global Drug Survey methodology and limitations
Dr Adam R Winstock
Dr Monica J Barratt
Dr Jason A Ferris
Dr Larissa Maier
Global Drug Survey (GDS) runs the world’s biggest annual drug survey. Our last survey, GDS2016, ran for 6 weeks at the end of 2015, was translated into 10 languages and received over 100,000 responses from around the world. Over the last 3 years we have obtained data from over 300,000 people. By the end of 2016 we estimate our global database to be in excess of 600,000. In this section we provide more details on our methods including a critical review of its their limitations. For additional information we refer the reader to consider some of our recent scientific publications which can be accessed here.
“…Probability based surveys tell you the size of the drug use problem in your nation. GDS tells you what to do about it”
— Adam Winstock Director of GDS
For further information, please see Surveys
GDS runs large-scale anonymous online surveys through a secure encrypted website (https://www.globaldrugsurvey.com). The research tool and methods employed in current studies are based on previous work by the group conducted over the last decade, beginning with the first paper published in 2001 that has been cited over 250 times (Winstock et al 2001). Our team has successfully used this methodology to identify new drug trends before they reach the wider community and explore the relationship between patterns of substance use and experienced harms (Winstock et al 2010, Winstock et al 2011, Winstock et al 2012, Winstock et al 2013, Winstock et al 2014, Winstock et al 2015, Barratt et al 2014, Bellis et al 2015).
A link to the survey, hosted at https://www.globaldrugsurvey.com, is shared with our global media partners who promote the survey online, through social media and in print for a period of approximately 6 weeks from the time of launch in mid-November each year. Since 2011 the scope of our work has expanded beyond our original sampling focus on clubbers and ‘party drug use’. Our surveys now encompass the monitoring of alcohol, e-cigarettes, emerging drugs of abuse, as well as prescription medications. A core battery of questions facilitating time trend analyses (see below) addressing patterns of use, prevalence, cost and drug related harms are accompanied by a range of specialist topics, which are selected each year by the GDS Academic Network and GDS Core Research Group. Since 2012 we have published between 5-10 peer reviewed academic articles per year in diverse but high quality journals.
Overall respondent characteristics
A snapshot of our participants’ demographics over the last 5 years is provided below.
- The the ratio of male to female respondents is 2:1
- the mean age of the sample is around 25 years
- roughly 82% are heterosexual
- 93% are white
- and 65% have a university degree or higher
- over 70% have used an illicit drug in the last year
- while almost 3% have ever injected any drug
- over 50% report regular involvement in nightlife and clubbing.
For academic and public health reports we have one of the richest and most detailed sources of data that allows us to describe our sample robustly. Beyond the basic demographics we can also describe our sample by the following selection of variables:
- income levels
- frequency of exercise
- employment status
- 10 item AUDIT (Alcohol Use Disorders Identification Test) score
- levels of educational attainment
- wellbeing (Australian Personal Well Being assessment)
- body mass index (BMI)
- self-reported diagnosis of mental illness
- SAPAS scores (personality dysfunction measure GDS2012 and GDS2014)
- musical preferences
- and dietary preferences
- and much, much more.
The GDS database is huge, but its non-probability sample means analyses are suited to highlight differences among user populations. GDS is this thus best suited to answer comparison questions that are not dependent on probability samples. While GDS cannot and should not be used to provide point estimate prevalence associated with population usage or other population generalisations, GDS has been used to spot new drugs trends and describe the effect profile of novel drugs (Winstock et al 2010, 2012, 2014) and misuse of prescribed medications (Winstock et al 2013) as they enter the wider population. In fact, GDS is often the first international survey to provide information on new drugs and new ways of acquiring drugs, such as dark net or crypto markets (Barratt et al 2014 and 2016). The international nature of our sample and its size allow our network to explore the relationship between different patterns of consumption for common substances such as alcohol and the harms it causes (Bellis et al 2015).
Furthermore, GDS is often the only source containing data on drug drug-using groups that are large and heterogeneous enough to capture dose-response curves related to harms of less commonly used and researched drugs such as nitrous oxide (Kaar et al 2016) and synthetic cannabinoids (Winstock et al 2015). GDS is thus appropriately considered as an additional real time data source to complement other population based epidemiological databases such as national household surveys, treatment seeking statistics and police seizures. The value of GDS data is evidenced by our inclusion in the UNODC World Drug Report and the EMCDDA Annual Drugs Report, and our contribution to a long list of peer reviewed journal articles on new drug trends (see academic GDS articles here).
It is important to understand what GDS can and cannot do when interpreting our findings. GDS acknowledges that when compared with traditional epidemiological criteria for a good public health surveillance system, our approach has significant limitations. GDS utilises non-random, opportunistic sampling methods to recruit very large numbers of people who use drugs. The recruitment window is brief with the survey active for only 6 weeks. The sample representativeness is limited by response bias whereby there will be inherent differences between those who participate and those who do not. This survey is only available on-line and will therefore tend to miss those without easy online access and those with high levels of health literacy.
- Don’t look to GDS for national estimates.
- GDS is designed to answer comparison questions that are not dependent on probability samples.
- The GDS database is huge but its non-probability sample means analyses are suited to highlight differences among user populations.
- GDS recruits younger, more involved drug using populations. We spot emerging drugs trends before they enter into the general population.
The survey is targeted towards at people who have used any substance, with a focus on those who have used any substance within the last 12 months. It is more likely that individuals will respond to surveys if they see topics or items that are of interest to them, and thus by definition will differ from those who do not participate. Therefore, as participants in our survey may have a greater interest in or experience with drugs, they may not be representative of the wider non-drug using population. However, this limitation is somewhat alleviated when the focus of the research is on drug using behaviour rather than estimating prevalence of drug use in the community. Also, by sampling a wide cross-section of drug users, and encouraging large numbers of people to complete the survey, the data is able to examine drug using behaviour of more unusual forms of drug use, or new and emerging drugs.
Importantly our approach accesses sections of the populations, more hidden and hard-to-reach populations, that general household surveys do not, and we are able to explore drug patterns of use use-related harm especially in relation to less commonly used drugs in significantly more depth (Lawn et al 2014, Kaar et al 2016). Due to the nature of our recruitment methods, and using an online survey, our participants tend to be better educated than the general population. Also, although we recruit over 15% from the LGBT community, our access to diverse ethnic groups is limited with 90% of our sample identifying themselves as ‘white’.
What GDS can do for you
- GDS is an efficient approach to gain content content-rich data that explores diverse health outcomes associated with the use of drugs and alcohol across the population of your country.
- GDS helps you better understand the quantitative dynamics of personal decision-making about drug use, detects regional differences in patterns of drug use and related harm, and informs novel interventions.
- GDS provides essential, current data on the patterns of use, harms, health and well-being experienced by the full spectrum of users in your country.
GDS makes no claims that data from any year are representative of the wider population. We see our approach as better suited to identify potential new trends within our cohort that may be predictive of future trends within the wider population. Also, the richness of our data allows a better understanding of drug using behaviour for almost 150 drugs, an opportunity to explore cross-country comparisons associated with drug using behaviour, and changes over time for particular drugs or newly emerging drugs. Given our interest is in the early identification of new drug trends and drug related harm, having an overrepresentation of people who have extensive drug using experiences is a rather helpful (indeed purposeful) recruitment bias. Our samples comprise sentinel drug using groups who display both a higher level of interest in and use of a range of psychoactive substances than the population that they are recruited from. As such our data complements and even supplements other drug use data sets such as those from national household surveys and data obtained from treatment seeking databases and prescribing records. When it comes to assessing the risk related to commonly used drugs by non-treatment seeking persons, or describing new emerging drug trends, our methods are highly effective.
Because we conduct annual surveys and have a consistent core of questions, repeated measurements over time potentially allow inferences to be made on time trends where data collection procedures and other threats to the reliability of data can be shown to be constant or be effectively controlled. These trend analyses are most usefully pursued when we look at trends within subgroups e.g. age groups or association with clubbing. Following analysis of annual drug use prevalence data within specific populations, consideration of the statistical significance of the observed time trends is undertaken with logistic and multiple regression for prevalence and continuous data respectively. In our previous work (1999-2004) on patterns of stimulant and hallucinogen use over time the main effect of time (year 1 – 5) was the principal independent variable under study. Additional variables were also included in these models to control for potential confounding, such as included: age, gender, number of responses per year; and potential prior study participation (McCambridge et al 2005a 2005b 2005c, 2006). We plan on utilising this method again next year to provide a time trend analysis of the patterns of new drug trends and prescription drug use over a 5-year time frame in 2016 and 2017. Finally, given the vast number of countries represented, the data can capture the effects of country-based policy changes. For example, the NPS bill in NZ or the Novel Psychoactive Drugs Bill passed in May 2016 in the UK.
- GDS uses on-line survey methods and partnership with global media giants to recruit very large numbers of people who use drugs and alcohol from around the world.
- The GDS database is huge, but its non-probability sample means analyses are suited to highlight differences among user populations.
- GDS is designed to answer comparison questions that are not dependent on probability samples.
- GDS is translated into 10 languages and has partners in over 20 countries.
- GDS has access to thousands of real-life, real-time stories of people in your country as they make decisions about drug use.
- GDS lets you target, design, and justify intervention programs for different regions of your country, age/gender population segments, and types of harm.
- GDS reaches hidden, sentinel and hard to reach populations.
- GDS samples allows you to effectively compare population segments – young, old, males, females, gay, straight, clubbers, thin people, obese people, vegetarians, those with a current psychiatric symptoms and diagnoses, students, northerners, southerners.
- GDS can help add numbers and depth to the findings of more rigorous, though less detailed and smaller, survey findings.
- GDS puts you on top of emerging drug trends in your country and major cities.
Recent peer reviewed papers derived from GDS data
Barratt, M.J., Ferris, J.A. and Winstock, A.R., 2016. Safer scoring? Cryptomarkets, social supply and drug market violence. International Journal of Drug Policy. Barratt, M.J., Ferris, J.A. and Winstock, A.R., 2016. Safer scoring? Cryptomarkets, social supply and drug market violence. International Journal of Drug Policy.
Kaar, S.J., Ferris, J., Waldron, J., Devaney, M., Ramsey, J. and Winstock, A.R., 2016. Up: The rise of nitrous oxide abuse. An international survey of contemporary nitrous oxide use. Journal of Psychopharmacology, 30(4), pp.395-401. http://jop.sagepub.com/content/30/4/395.short
Hindocha, C., Freeman, T.P., Winstock, A.R. and Lynskey, M.T., 2016. Vaping cannabis (marijuana) has the potential to reduce tobacco smoking in cannabis users. Addiction, 111(2), pp.375-375. http://onlinelibrary.wiley.com/doi/10.1111/add.13190/abstract
Bellis, M.A., Quigg, Z., Hughes, K., Ashton, K., Ferris, J. and Winstock, A., 2015. Harms from other people’s drinking: an international survey of their occurrence, impacts on feeling safe and legislation relating to their control. BMJ open, 5(12), p.e010112. http://bmjopen.bmj.com/content/5/12/e010112.full
Winstock, A., Lynskey, M., Borschmann, R. and Waldron, J., 2015. Risk of emergency medical treatment following consumption of cannabis or synthetic cannabinoids in a large global sample. Journal of Psychopharmacology, 29(6), pp.698-703. http://jop.sagepub.com/content/29/6/698.short
Morley, K.I., Lynskey, M.T., Moran, P., Borschmann, R. and Winstock, A.R., 2015. Polysubstance use, mental health and high?risk behaviours: Results from the 2012 Global Drug Survey. Drug and alcohol review, 34(4), pp.427-437. http://onlinelibrary.wiley.com/doi/10.1111/dar.12263/abstract
Winstock, A., 2015. New health promotion for chemsex and ?-hydroxybutyrate (GHB). BMJ, 351, p.h6281. http://www.bmj.com/content/351/bmj.h6281.abstract
Uosukainen, H., Tacke, U. and Winstock, A.R., 2015. Self-reported prevalence of dependence of MDMA compared to cocaine, mephedrone and ketamine among a sample of recreational poly-drug users. International Journal of Drug Policy, 26(1), pp.78-83. http://www.sciencedirect.com/science/article/pii/S0955395914001728
Stevens, A., Barratt, M., Lenton, S., Ridout, M. and Winstock, A., 2015. Social Bias in the Policing of Illicit Drug Users in the UK and Australia: Findings from a Self-Report Study. Available at SSRN 2618393. http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2618393
Garnett, C., Crane, D., West, R., Michie, S., Brown, J. and Winstock, A., 2015. Normative misperceptions about alcohol use in the general population of drinkers: A cross-sectional survey. Addictive behaviors, 42, pp.203-206. http://www.sciencedirect.com/science/article/pii/S0306460314003827
Shiner, M. and Winstock, A., 2015. Drug use and social control: The negotiation of moral ambivalence. Social Science & Medicine, 138, pp.248-256.Shiner, M. and Winstock, A., 2015. Drug use and social control: The negotiation of moral ambivalence. Social Science & Medicine, 138, pp.248-256.
Freeman, T.P. and Winstock, A.R., 2015. Examining the profile of high-potency cannabis and its association with severity of cannabis dependence. Psychological medicine, 45(15), pp.3181-3189 . http://journals.cambridge.org/download.php?file=%2FPSM%2FPSM45_15%2FS0033291715001178a.pdf&code=ae3a71a70fa59432968140bc4db80943
Winstock, A.R., Lawn, W., Deluca, P. and Borschmann, R., 2015. Methoxetamine: An early report on the motivations for use, effect profile and prevalence of use in a UK clubbing sample. Drug and alcohol review. http://onlinelibrary.wiley.com/doi/10.1111/dar.12259/full
Lawn, W., Barratt, M., Williams, M., Horne, A. and Winstock, A., 2014. The NBOMe hallucinogenic drug series: patterns of use, characteristics of users and self-reported effects in a large international sample. Journal of Psychopharmacology, 28(8), pp.780-788. https://www.researchgate.net/profile/Will_Lawn/publication/260380686_The_NBOMe_hallucinogenic_drug_series_Patterns_of_use_characteristics_of_users_and_self-reported_effects_in_a_large_international_sample/links/55632b9d08ae9963a11c1862.pdf
Barratt, M.J., Ferris, J.A. and Winstock, A.R., 2014. Use of Silk Road, the online drug marketplace, in the United Kingdom, Australia and the United States. Addiction, 109(5), pp.774-783. http://espace.library.uq.edu.au/view/UQ:326444/UQ326444OA.pdf
Winstock, A.R., Borschmann, R. and Bell, J., 2014. The non?medical use of tramadol in the UK: findings from a large community sample. International journal of clinical practice, 68(9), pp.1147-1151.Winstock, A.R., Borschmann, R. and Bell, J., 2014. The non?medical use of tramadol in the UK: findings from a large community sample. International journal of clinical practice, 68(9), pp.1147-1151.
Winstock, A.R., Kaar, S. and Borschmann, R., 2013. Dimethyltryptamine (DMT): Prevalence, user characteristics and abuse liability in a large global sample. Journal of Psychopharmacology, p.0269881113513852. https://www.researchgate.net/profile/Rohan_Borschmann/publication/258957686_Dimethyltryptamine_DMT_Prevalence_user_characteristics_and_abuse_liability_in_a_large_global_sample/links/00463529f5480993a8000000.pdf
Archer, J.R.H., Dargan, P.I., Wood, D.M. and Winstock, A.R., 2013. Hospital and prehospital emergency service utilisation as an impact of acute recreational drug and ethanol toxicity. Journal of Substance Use, 18(2), pp.129-137 . http://www.tandfonline.com/doi/abs/10.3109/14659891.2012.707283#.V0wsm4-cHIU
Winstock, A.R. and Barratt, M.J., 2013. The 12?month prevalence and nature of adverse experiences resulting in emergency medical presentations associated with the use of synthetic cannabinoid products. Human Psychopharmacology: Clinical and Experimental, 28(4), pp.390-393. http://www.asknormen.co.uk/resources/uploads/files/nps11prevA_Ecannabiniods.pdf
Winstock, A.R. and Barratt, M.J., 2013. Synthetic cannabis: a comparison of patterns of use and effect profile with natural cannabis in a large global sample. Drug and alcohol dependence, 131(1), pp.106-111. https://www.infona.pl/resource/bwmeta1.element.elsevier-fc491828-deb7-33c4-94ce-023219075ab3
Winstock, A., Bell, J. and Borschmann, R., 2013. Friends, doctors, and tramadol: we might have a problem. BMJ, 347, p.f5599. http://www.bmj.com/content/347/bmj.f5599.short
Hughes, B. and Winstock, A.R., 2012. Controlling new drugs under marketing regulations. Addiction, 107(11), pp.1894-1899. https://www.researchgate.net/profile/Tom_Freeman3/publication/232064507_Supply_always_comes_on_the_heels_of_demand_what_effects_do_control_strategies_have_on_drug_users_themselves/links/02e7e52a72b3f77f76000000.pdf
Winstock, A.R. and Mitcheson, L., 2012. New recreational drugs and the primary care approach to patients who use them. BMJ, 344, p.e288. http://www.globaldrugsurvey.com/wp-content/uploads/New-recreational-drugs-and-the-primary-care-approach-to-patients-who-use-them.pdf
Winstock, A.R., Mitcheson, L., Gillatt, D.A. and Cottrell, A.M., 2012. The prevalence and natural history of urinary symptoms among recreational ketamine users. BJU international, 110(11), pp.1762-1766. http://onlinelibrary.wiley.com/doi/10.1111/j.1464-410X.2012.11028.x/full
Winstock, A. and Wilkins, C., 2011. ‘Legal highs’: the challenge of new psychoactive substances. TNI/IDPC Transnational Institute Series on Legislative Reform of Drug Policies, (16). https://www.tni.org/files/download/dlr16.pdf
Winstock, A.R., Mitcheson, L.R., Deluca, P., Davey, Z., Corazza, O. and Schifano, F., 2011. Mephedrone, new kid for the chop?. Addiction, 106(1), pp.154-161.Winstock, A.R., Mitcheson, L.R., Deluca, P., Davey, Z., Corazza, O. and Schifano, F., 2011. Mephedrone, new kid for the chop?. Addiction, 106(1), pp.154-161.
Winstock, A., Mitcheson, L., Ramsey, J., Davies, S., Puchnarewicz, M. and Marsden, J., 2011. Mephedrone: use, subjective effects and health risks. Addiction, 106(11), pp.1991-1996. http://www.biblioteca.cij.gob.mx/Archivos/Materiales_de_consulta/Drogas_de_Abuso/Articulos/66324670.pdf
Winstock, A., Mitcheson, L. and Marsden, J., 2010. Mephedrone: still available and twice the price. The Lancet, 376(9752), p.1537. https://www.researchgate.net/profile/Adam_Winstock/publication/47700682_Mephedrone_Still_available_and_twice_the_price/links/00b7d52a862e9c50a3000000.pdf
Winstock, A.R., Marsden, J. and Mitcheson, L., 2010. What should be done about mephedrone?. Bmj, 340, p.c1605. http://www.bmj.com/content/340/bmj.c1605.full
Winstock, A.R. and Ramsey, J.D., 2010. Legal highs and the challenges for policy makers. Addiction, 105(10), pp.1685-1687. http://onlinelibrary.wiley.com/doi/10.1111/j.1360-0443.2010.02934.x/full
Winstock, A.R. and Marsden, J., 2010. Mephedrone: Assessment of Health Risks and Harms. European Monitoring Centre for Drugs and Drug Addiction. Risk Assessment Report of a New Psychoactive Substance: 4-methylmethcathinone (mephedrone). EMCDDA: Lisbon.
Pre 2010 methods papers
McCambridge, J., Winstock, A., Hunt, N. and Mitcheson, L., 2007. 5-Year trends in use of hallucinogens and other adjunct drugs among UK dance drug users. European addiction research, 13(1), pp.57-64. http://neilhunt.org/pdf/2007-5-year-hallucinogens-mccambridge-et-al.pdf
McCambridge, J., McCambridge, J., Mitcheson, L., Hunt, N. and Winstock, A., 2006. The rise of Viagra among British illicit drug users: 5-year survey data. Drug and alcohol review, 25(2), pp.111-113. http://neilhunt.org/pdf/2006-viagra-mccambridge-mitcheson-hunt-winstock.pdf
McCambridge, J., Mitcheson, L., Winstock, A. and Hunt, N., 2005. Five?year trends in patterns of drug use among people who use stimulants in dance contexts in the United Kingdom. Addiction, 100(8), pp.1140-1149. http://neilhunt.org/pdf/2005-stimulants-mccambridge-mitcheson-winstock-and-hunt.pdf
Winstock, A.R., Griffiths, P. and Stewart, D., 2001. Drugs and the dance music scene: a survey of current drug use patterns among a sample of dance music enthusiasts in the UK. Drug and alcohol dependence, 64(1), pp.9-17. https://www.researchgate.net/profile/Paul_Griffiths3/publication/11872828_Drugs_and_the_dance_music_scene_A_survey_of_current_drug_use_patterns_among_a_sample_of_dance_music_enthusiasts_in_the_UK/links/00463524d2c7a61fbc000000.pdf