Introduction
The Internet is an important source of health information and a means of sharing personal experiences. In many countries, it is the most frequently used source of health information, and increasing numbers of people research their own medical problems on the Internet.
The user-generated content refers to content denoting or related to material that is voluntarily contributed to a website by members of the public. Koreans seem more familiar with user-created content (UCC). Any form of content, such as videos, photos, blogs, images, and other forms of media, can be considered UCC. Recently, video format UCC has emerged as an active means of sharing information, as it is more understandable than other monomodal media such as images, posts, and audio files. YouTube is the largest of all video-sharing Internet websites created in 2005 to provide free video streaming1). More than one billion unique users have been reported to visit the site, watching over six billion hours of content on a daily basis2). YouTube is a large participatory platform for sharing self-produced video clips containing various types of information. Therefore, most medical content on YouTube is produced by nonprofessionals. Although the Internet, particularly UCC, has become indispensable as a source of medical information, the quality of this information is difficult to determine, and healthcare professionals have expressed concern regarding the quality and integrity of health information obtained from Internet-based sources3).
Searching YouTube, people can easily identify cases that match their own health problems and access a considerable amount of medical information including details of symptoms, treatments, complications, and medications. With detailed information, there is a greater potential risk of inappropriate use by patients and their families. Further, the use of incorrect or irrelevant information and misunderstanding relevant and valid information can be physically, emotionally, and financially harmful4). Although, it is important that medical professionals warn patients about the potential risks of obtaining information from UCC, few studies have performed a scientific analysis of the content and quality of the UCC available on YouTube.
Precocious puberty is generally defined as the onset of secondary sexual characteristics before 8 years of age in girls and 9 years in boys. It continues to attract the interest and concern of medical practitioners and the public worldwide. According to data from the Korea Health Insurance Review and Assessment Service, the number of patients diagnosed with precocious puberty in 2006 was approximately 6,400, which quadrupled to 28,000 in 2010, and increased almost nine fold to 55,300 in 20125).
There are some studies that conducted to assess the quality and content of Internet-based medical information regarding several types of disease. However, only a few studies have evaluated the information available in UCC. To our knowledge, although precocious puberty is one of the fastest-growing pediatric diseases in South Korea, no studies have been conducted to analyze precocious-puberty-related UCC. Therefore, the aim of this study was to examine the quality and scientific accuracy of the information available on YouTube regarding precocious puberty.
Materials and methods
1. Materials
The YouTube website (YouTube, www.youtube.com, YouTube LLC, San Bruno, CA, USA) was searched for the following keywords: "precocious puberty," "early puberty," "sexual precocity," or "precocity" in Korean during June and July 2014. All videos uploaded before 20 July 2014 containing information on precocious puberty were included. More than 1,500 instances of UCC matched the keywords. We included 95 instances of UCC after UCC those were inaccessible, irrelevant, shorter than 1 minute in duration, or in a language other than Korean were excluded. Among them 4 instances of UCC were duplicated and 5 UCC were segmented into 40 parts. Aside from duplicate and segment UCC, 51 instances of UCC were ultimately identified for inclusion in the study; this included UCC with Korean subtitles and an English video (Fig. 1).
All videos were categorized into three groups by the information provider: medical, oriental, and commercial & others. UCC conducted by a doctor of traditional Korean medicine was classified as oriental. Videos that were designed to advertise a private hospital by medical personnel or not and provided by neither medical nor oriental users were classified as commercial & others.
2. Quality evaluation
We evaluated the quality of medical information on YouTube using the DISCERN instrument. Developed jointly by workers at Oxford University and the British Library, the instrument was designed for use by health consumers and does not require previous knowledge. It is a validated rating tool and freely available online (www.discern.org.uk)6). The DISCERN instrument contains a series of 15 questions concerning the content of health information. The user responds using a 5-point scale, with total scores ranging from 15 to 75 and higher scores indicating higher information quality (Table 1). The questions included in the DISCERN instrument are organized into three sections: section 1 (questions 1-8), which addresses the reliability of the publication; section 2 (questions 9-15), which focuses on the quality of information regarding treatment choices; and section 3, which measures overall quality ratings. According to the DISCERN instrument guidelines, if the response to the question is a definite yes, which indicates that the quality criterion has been fulfilled entirely, 5 points should be awarded. If users feel that a publication meets the criterion to some extent, 2-4 points are awarded. If the response to a question is a definite no, the quality criterion has not been fulfilled at all, and 1 point is awarded7).
3. Scientific accuracy evaluation
We adapted the information score modified from Libertiny et al.8) and Kim et al.9). The scoring system included 14 questions organized into 5 sections: definition, cause and classification, diagnosis, treatment, and prognosis. Cause and classification, diagnosis, and treatment were subdivided into 3, 3, and 6 questions respectively. The raw scores for subsections and questions were multiplied by a weighting factor ranging from 1 to 3 according to importance. Questions concerning definition, gonadotropin-releasing hormone (GnRH) stimulation test, GnRH agonist and prognosis were weighted at 3, 2, 3, and 2, respectively (Table 2). The maximum information score was 60. The definition subsection included clinical features of puberty (testicular enlargement in boys and breast buds in girls) and age of onset. The cause and classification subsection included activation of the hypothalamic-pituitary-gonad axis, central and peripheral precocious puberty, and influencing factors. The diagnosis subsection included clinical features (e.g., testicular enlargement in boys, breast buds in girls, and rapid growth), GnRH stimulation test, and bone age. The treatment subsection included GnRH agonist therapy, treatment of underlying disease, conservative management (e.g., exercise, diet, and lifestyle modification), duration, efficacy, and complications. Information substantiated by a medical textbook was awarded 3 points. Information that had not yet culminated in a solid theory but was of appreciable clinical evidence was awarded 2 points. Controversial information that lacked sufficient evidence or was impossible to verify was awarded 1 point. Medically erroneous information and information that had not been referred to elsewhere was awarded 0 point.
4. Analysis of views and duration of UCC
The duration of precocious-puberty-related UCC and the number of times that the UCC had been viewed were analyzed. On YouTube, a video is considered as having been viewed when a viewer intentionally plays it; each time this occurs, a view is counted, and view counts increase each time videos are loaded.
Results
In total, 51 UCC were categorized as medical (n=17), oriental (n=17), or commercial & others (n=17) by the information provider.
1. Quality evaluation
Quality evaluation was performed using the DISCERN instrument. The mean score was 2.8 out of 5.0 for overall quality, which indicated that the majority of UCC received moderate ratings regardless of the information provider. The mean overall score for DISCERN questions showed moderate to good ratings, with the exception of questions 5, 7, and 11, which showed poor ratings of less than 2.0 and targeted only 3, 12, and 8 of the 51 UCC complied with questions respectively. Questions 2 and 12 showed good ratings of 4.0 and higher.
The overall quality score for medical UCC (3.4) was significantly higher than those of oriental and commercial & others UCC (2.8 and 2.3, respectively) (P<0.001). With the exception of questions 4, 5, 7, 12, and 13, DISCERN ratings differed significantly according to category. For questions 3, 6, 8, 9, 11 and overall quality, ratings for medical UCC were significantly higher relative to those of oriental and commercial & others (P<0.05) (Table 3).
In the evaluation of scientific accuracy, the mean overall information score was 17.2 out of 60. Most UCC received poor ratings for each question of information score. Scores for most questions were low without weighting, and scores were 1.0 or higher for only 6 questions (Table 4).
2. Scientific accuracy
According to the information provider, the mean information score for medical UCC (30.7) was significantly higher relative to the mean overall score (17.2), almost double that of oriental UCC (15.9), and 6 times higher than that of commercial & others UCC (5.1) (P<0.001). Only medical group satisfied all the subsection and questions of information score. None of the commercial & others UCC satisfied with eight questions (hypothalamicpituitary-gonad axis, central and peripheral precocious puberty, bone age, treatment of underlying disease, duration and efficacy of treatment). With respect to hypothalamic-pituitary-gonad axis, duration, and efficacy of treatment, only 1, 3, and 2 instances of oriental UCC were satisfied respectively.
3. Viewing
The mean overall number of views was 1,696 and the number of views for medical UCC (n=4,344) was considerably higher relative to those of oriental (n=269) and commercial & others UCC (n=475). The mean duration of total 51 UCC was 14'04" (m's"). The mean duration of oriental UCC (29'44") was longer than those of medical and commercial & others UCC (7'50" and 4'38", respectively) (P<0.001), however, the number of views recorded for oriental UCC was lower relative to those of medical and commercial and other UCC (P=0.086) (Table 5).
Discussion
In this study, we aimed to evaluate the quality and scientific accuracy of precocious-puberty-related UCC on YouTube during June and July 2014. More than 1,500 instances of UCC matched the keywords and 51 UCC were selected and categorized as medical, oriental, or commercial & others by the information provider. The quality and accuracy of UCC were variable often unreliable. The quality and scientific accuracy of medical UCC were significantly higher relative to those of oriental and commercial & others UCC.
Vast expansion of the use of social media, such as YouTube, Facebook, and Twitter, has provided several benefits, such as easy and rapid communication, enhanced capacity to share information with large audiences, and the ability to interact regardless of physical distance, for medical students and doctors. The spread of the song, "Gangnam Style," is an example of how people around the world are influenced, both rapidly and ubiquitously, by the world's most popular website, YouTube. In August 2012, the song and its accompanying music video went viral after being uploaded to YouTube, and as of 2014, it was the most watched video in history10). YouTube has grown rapidly in size and influence in recent years. It contains not only music, movies, and other entertaining content but also medical information.
Not only people in medical field but also patients use social media. As more people use social media, particularly YouTube, with increases in the speed with which information is transmitted, the potential risk of dissemination of inappropriate medical content has increased relative to that of traditional means of communication11).
Only one previous study has evaluated the quality and scientific accuracy of Internet-based information regarding precocious puberty9). However, examination of the issue was restricted to website use and did not involve the evaluation of UCC. As UCC is an emergent means of information transmission, we aimed to evaluate the quality and scientific accuracy of UCC on precocious puberty.
Several methods could be used to assess the quality of Internet-based health information. These methods include the Health On the Net Code of Conduct, which is the oldest and most definitive health information tool on the Internet and produced by health professionals12); information quality tools, which are automated data quality evaluation tools operating from a central query and evaluating data stored on individual geographic information system data servers across the country and at the United States Department of Agriculture's National Information Technology Center that ensures compliance with national standards13); and the DISCERN instrument, which was used in this study. The Health On the Net Code of Conduct and information quality tools are used by health organizations and professionals, while the DISCERN instrument is designed for the Internet user. The greatest advantage of the DISCERN instrument is that both experts and laymen are able to perform systematic and comprehensive assessments, even if they have never used health evaluation tools6). Park et al.14) demonstrated the validity of the instrument and consistency of results in both types of user.
Overall mean DISCERN scores were 1.9 in a study that analyzed a Korean health information website14) and 1.92 in a study that analyzed an allergic rhinitis website15). The scores for the current study were generally higher relative to those observed in other studies. While other studies conducted website-based evaluations, the current study focused on UCC. UCC is usually created by editing public TV shows and includes a description of the topic, which is shown at the beginning of the video to attract viewers' attention. Questions 1 and 2, concerning explicit aims and aims achieved, respectively, scored higher. As most of medical and oriental UCC originated from TV shows (15 and 16 instances of UCC, respectively), this could explain the higher scores observed in this study.
Ahn16) evaluated the quality of UCC concerning atopic dermatitis on YouTube and found low ratings (1.8) using the DISCERN instrument. The higher scores observed in this study may have occurred not only because of differences in websites and UCC between Ahn's study16) and the current study but also because of the extended criteria adopted in the current study. In the original DISCERN instrument, question 8, which concerns areas of uncertainty, is designed to evaluate uncertainty regarding treatment; however, we also included uncertainty regarding the cause of disease.
The reliability of information provided by medical UCC was significantly more accurate relative to that of oriental and commercial and other UCC, but the quality of information was similar across UCC types. Unlike health professionals, patients who lack knowledge estimate and deliver the value of medical information according to the quality of that information. Therefore, poor-quality UCC could be underestimated despite providing highly accurate and reliable information. To avoid devaluation, it is important to provide detailed information that is required by patients and their parents.
The mean overall information score was poor. According to the information provider, the mean information score for medical UCC was significantly higher relative to those of oriental and commercial& others. In all of the subsections and questions, medical UCC received the highest score and commercial & others UCC received the lowest score. The mean information scores for oriental and commercial & others UCC were significantly lower relative to that of medical UCC, which indicates that the information provided by oriental and commercial & others UCC was scientifically inaccurate.
Scores were lowest for questions concerning the duration and efficacy of treatment. The following causes of precocious puberty, which were suggested in oriental UCC, received a score of 0: air pollution, eggs, omega 3, berries, chocolate, allergic diseases, and high-calorie foods. In addition, global warming, lack of sleep, electromagnetic wave, heavy metal, and animal protein received a score of 1; environmental hormone and high soy consumption received a score of 2; and premature birth, low birth weight, use of female hormones containing certain products, and exposure to sexual stimulus received a score of 3. The following complications of precocious puberty received a score of 0: learning disorders, drug abuse, scoliosis, myopia, and sexually transmitted disease. In addition, dysmenorrhea received a score of 1, and breast cancer, endometrial cancer, and premature menopause received a score of 2. Most oriental UCC lower the quality of the information provided by suggesting unscientific and experimentally unproven causes and treatments.
The mean number of views for medical UCC was 4,344 which was 10-15 times higher than those of oriental and commercial & others UCC. This may have occurred because medical UCC was positioned at the top of the first page and linked to a large amount of related UCC. The mean duration of oriental UCC was longer relative to that of other providers. Differences between the TV programs involved may explain this; the majority of medical of UCC consisted of edited TV news, while most oriental UCC consisted of full-duration TV shows. As the view counts and duration of UCC are affected by many factors and UCC varies widely, there was no correlation with the reliability or accuracy.
In the assessment of the quality and accuracy of information, medical UCC received the highest score relative to those of other groups and showed overall moderate to good ratings in both the DISCERN instrument and information scores. Therefore, with a little extra effort, UCC on YouTube could serve as a good source of high-quality medical information. In contrast, most UCC uploaded by oriental and commercial & others contained unproven or fabricated information regarding diagnosis and treatment. Oriental UCC tended to focus on factors such as stress, obesity, and environmental hormones without any mention of the hypothalamic-pituitary-gonad axis, particularly with respect to the causes of precocious puberty. This pattern was also observed in other studies. Some oriental UCC even introduced herbal medicine, acupuncture, and lifestyle modification as the first line of treatment of precocious puberty. Application of inappropriate treatment not only leads to serious deterioration but also causes patients miss optimal treatment times; these adverse effects may be worse for young children than they are for adults.
The study was subject to some limitations. As a large amount of UCC is continually uploaded to YouTube, and this was a cross-sectional study, which reflected a particular moment, the numbers of UCC, views, and other information could differ from time to time. The DISCERN instrument and information scores were developed to evaluate websites rather than UCC; therefore, several problems were experienced in the evaluation of UCC. UCC has constraints of time to meet all the DISCERN questions about the quality of information. Furthermore, because scoring was dependent on the medical aspects of information, oriental UCC may have been underappreciated.
As UCC becomes a popular source of health information, it is important to provide reliable and scientifically accurate information. The development of an appropriate tool with which to evaluate health related UCC is required in the near future.
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
Full text via PMC
Download Citation
Print
