A comprehensive analysis of the quality of online health-related information regarding schizophrenia.

A comprehensive analysis of the quality of online health-related information regarding schizophrenia. In the past 15 years, the Internet has become a major source ofhealth information, including information about major mental healthdisorders such as schizophrenia (Eysenbach, Powell, Kuss, & Sa,2002; Murphy, Frost, Webster, & Schmidt, 2004). The Internet is arelatively open medium. Some sites provide medical information to sell aproduct; other sites offer inaccurate or outdated information that mightcomplicate an already difficult situation (Kummervold et al., 2002;Murphy et al., 2004). Studies have looked at various medical information sites (Anderson,Nikzad-Terhune, & Gaugler, 2009; Godin, Truschel, & Singh,2005), but few have reviewed sites with specific information aboutpsychiatric conditions such as schizophrenia. Consumers and familymembers have complained that mental health professionals provide littleinformation regarding schizophrenia (Lefley, 1998; National Alliance onMental Illness [NAMI], 2003). Consequently, the Internet might be aprimary resource to obtain information on schizophrenia. The presentstudy sought to do two things: (1) comprehensively assess the quality ofinformation about schizophrenia on the Internet at two time pointsacross a one-year period and (2) provide a simple, easy to use, rankordering of quality sites for social workers to use with consumers andfamily members. BACKGROUND Three million (1 percent) people in the United States hve withschizophrenia (National Institute of Mental Health [NIMH], 2008; U. S.Department of Health and Human Services (HHS), 2001). If left untreated,schizophrenia can result in inadvertent harmful consequences such aslack of self-care (including medical care), homelessness, substanceabuse problems, increased vulnerability, poverty, and suicide (NIMH,2008). NIMH reported that the impact of schizophrenia for consumers,their loved ones, and the communities they live in is substantial (HHS,2001; NIMH, 2008). Thus, care and intervention for those living withschizophrenia is a major public health concern. A critical component of intervention is information aboutschizophrenia that is accurate, up-to-date, and understandable. This isparticularly true regarding diagnosis, possible treatments (and risksinvolved), and resources available to consumers and family members. Asnoted, consumers and family members have complained of not receivingadequate information about schizophrenia and available treatments(Lefley, 1998; NAMI, 2003). Social workers can refer consumers andfamilies to quality online medical information as one option to decreasethe information gap. Quality of Online Medical Information A recent study surveying online users about their consumption ofhealth-related information found that the majority do not consistentlyevaluate the quality of the information they get online. This suggeststhat the majority of Americans do not ensure that the online informationthey obtain is accurate (Pew Internet & American Life Project,2006). Indeed, available studies have demonstrated that the quality ofonline health information varies widely (Eysenbach et al., 2002). At thesame time, the state of the literature in assessing the quality ofonline medical information is at an early stage, especially for specificpsychiatric disorders (Eysenbach et al., 2002; Murphy et al., 2004).Social workers can assist consumers and families by having a reliableresource that highlights quality online medical information. Researchers and professional health organizations have tried tooperationalize "quality" information retrieved from theInternet (Eysenbach et al., 2002). Typically, quality information hasincluded data from the peer-reviewed literature that are presented in aclear and understandable way and provide a broad range of informationabout the medical condition (that is, information is comprehensive)(Eysenbach et al., 2002; Pew Internet & American Life Project, 2006;Provost, Koompalum, Dong, & Martin, 2006). The Health on the NetFoundation (HNF) created a list of principles for health sites to followwhen providing health information (HNF, 2009). The principles are basedon the following "gold standard": the currency of information(peer-reviewed journals, recency of updates), the credentials of writersand editors, and the comprehensiveness of the information. In addition,the HNF principles include policies regarding editorial content; thepurpose of the information (that is, opinions versus facts); thepresence of advertisers, sponsors, or both; and privacy policies (HNF,2009). These efforts, however, pertain to general online healthinformation sites. The need remains to evaluate online information forspecific serious disorders such as schizophrenia. Some researchers haveevaluated the quality of online health information for other disorders,such as Alzheimer's and eating disorders (Anderson et al., 2009;Godin et al., 2005; Murphy et al., 2004). One study assessed the qualityof information regarding mental health issues and disorders (Godin etal., 2005). To our knowledge, the present study is the first study toevaluate online health information specific to schizophrenia. Purpose of the Study The present study focused on Web sites that provide informationabout schizophrenia. The main purpose was to answer this question: Whatis the quality of information about schizophrenia on the Internet? Theobjective is to contribute to the relatively new field of research thatanalyzes online health information in a standardized and comprehensivefashion. The second purpose was to provide a simple guide for socialworkers to use with consumers and their loved ones. METHOD Research shows that the general public uses a search engine asopposed to a specific uniform resource locator to locate desired healthinformation (Pew Internet & American Life Project, 2006). Researchalso shows that the general public uses search engines on the basis oftheir familiarity and past experiences with these, which results incertain search engines being favored over others (Pew Internet &American Life Project, 2006). The present study chose the three mostpopular search engines (Lewis, 2009) to develop a sample of healthinformation sites about schizophrenia: Google (google. com), Yahoo(yahoo. com), and MSN (msn.com). The study used the search terms"schizophrenia, "schizoaffective," and"psychosis" on the basis of diagnostic categories establishedfor schizophrenia spectrum disorders in the DSM-IV-TR (AmericanPsychiatric Association, 2000). Only English-language sites wereincluded. A major study also found that people chose only the top one ortwo search results they received from a search engine (iProspect, 2006);thus, the present study selected only sites on the first page of eachsearch. The search resulted in a total of 30 sites. Three of these siteswere secondary pages to one of the previous sites already selected, sothey were dropped from the analysis. The final total was 27 sites.Because of the fluid nature of information on the Internet, the studyevaluated the sites within a narrow time frame to reduce the influenceof content change. Sites were accessed two times across a one-yearperiod, the first series of rating occurring between March 30 and April9, 2009, and the second occurring between May 11 and May 25, 2010. Measure The literature discusses the relative benefits and drawbacks of adisease-specific assessment device versus a generically designed scaleto assess online health-related information (Anderson et al., 2009; Bath& Bouchier, 2003; Provost et al., 2006). There is no knowndisorder-specific Web site evaluation scale available for schizophrenia;thus, this study used a generic scale that offered a comprehensiveapproach to assessing quality online health information: the WebMedQualscale (Provost et al., 2006). The main purpose of the WebMedQual scale is to offer acomprehensive assessment of the quality of online health information, assuggested by organizations such as the HNF (2009). Provost et al. (2006)used the HNF principles to create the WebMedQual. They used severalphases of item development, including a health expert review of items.The final content analysis resulted in 95 items. The authors did notprovide other psychometric properties for the scale, and no further workhas been done to assess these properties to date, although there wereplans to do so sometime in the future (personal communication with M.Provost, clinical research associate, Novartis Pharmaceuticals Canada,Inc., Toronto, April 4, 2009). The scale has been used in other studiesevaluating disease-specific online health information (Anderson et al.,2009). The items are divided into eight subcategories: content (currencyand accuracy of information), authority of source (disclosure ofexpertise, training, and possible financial interests of contributingauthors), design of Web site (navigation experience of site),accessibility and availability of information (regular availability ofWeb site, ease in functionality),links (quality and number of links),user support (technical support information), confidentiality andprivacy (privacy and confidentiality policies), and e-commerce (productsor services offered by site) (Provost et al., 2006). The e-commercesubcategory was dropped as the focus of this study was information thatwas free and readily available online. The reader is invited to reviewAppendix A for more detailed information regarding each of thesubcategories. The items are answered in a "yes" or "no"checklist format. All items are summed into a final score, with a higherscore indicating a higher quality of online health information. Thescale is scored as follows: a site receives a score of 2("yes") if the information is present and a score of 1("no") if the information is not present. In addition, an itemis scored as I for "not sure" (that is, the reviewer isuncertain whether the site has the requested information). Severalquestions required reversed scoring to avoid potential scoring bias. Wedropped five items because of their apparent obsolescence. For example,we found no site that included the following features: a text-onlyoption, easy-to-find information about the platforms and browsers thatpermit optimal viewing, and user onscreen notification when entering orleaving a secure site. This resulted in use of 84 of the original 95items of the scale. We found that a substantial portion of sites did not include anyadvertisements due to their nonprofit status. Sites that did not includeadvertisements would automatically receive lower scores on a number ofitems pertaining to advertisements and corporate sponsorships (forexample, "Is funding or other sponsorship for any specific contentclearly indicated?") and, thus, would receive lower total scores,suggesting lower quality of information. Thus, the present study dividedsites into two groups: nonprofit and for-profit. Twelve items regardingthe use of advertisements were dropped for nonprofit sites, resulting in72 items for this group. We used the generic top-level domain, as usedin the Internet's domain name system, to determine a site'snonprofit (.org) or for-profit (.com/.net) status. Nonetheless, onenonprofit site (familydoctor. org) was moved to the for-profit groupbecause it included numerous ads typical of a for-profit site. Thisresulted in an n of 12 for nonprofit sites and an n of 15 forfor-profits. Joseph Guada, who has over 20 years of psychiatric social work andresearch experience with families and schizophrenia, was the mainassessor. Victoria Venable, who has several years of practice experiencewith child and adolescent mental health, assessed the same sites to testfor interrater reliability. The clinical backgrounds of both authorshelped in identifying sites that targeted nonprofessionals for inclusionin the study. Finally, neither author had any professional relationshipor financial interests with any of the organizations whose sites wereassessed. Both authors rated all 27 sites during the specified periodsnoted earlier. Analysis Subcategory and full-scale summations of nonprofit and for-profitWeb sites are presented separately, given the difference in number ofitems used from the WebMedQual scale. Descriptive statistics were runfor each subcategory as well as for the full scale within each group ofsites. An interrater reliability was run to test for consistency acrossthe two raters at both time periods, and a test-retest reliability wasrun to test for consistency of the overall scale across both timeperiods. In addition, because each group had a different range ofpossible scores, we converted the full and subscale scores into"percentage of items endorsed." In this way, group andsubscale scores were comparable (on a scale of 0 percent to 100 percent,similar to that of a grading system). This would help in interpretingand understanding the rank order of sites. RESULTS The results of interrater reliability for the full scale were goodacross both raters for the for-profit sites (.84). However, the resultswere less satisfactory for the nonprofit sites at time 1 (.59) and time2 (.68). Review of the results for time 1 showed four sites for whichthe two raters had differences in ratings (MedlinePlus, NIMH, SubstanceAbuse and Mental Health Services Administration [SAMHSA], and Wikipedia:Schizophrenia). The raters reviewed their ratings for these four sitesand conferred on items where there was disagreement. The interraterreliability was rerun for the nonprofit group at time 1, which provideda satisfactory reliability (.90). The same process was used at time 2.There was a greater distribution of which sites had substantialdifferences across raters, depending on the subscale in question. Thethree subscales with the lowest interrater reliability were links, usersupport, and confidentiality. After reviewing our ratings, the overallinterrater reliability for nonprofits was recalculated and increased to.87 at time 2. Test-retest reliabilities across the two time points forthe for-profit sites were .85 and .74 for nonprofit sites, respectively,suggesting a reasonable amount of reliability for the scale (Anastasi,1988). For-Profit Sites The mean score for the for-profit sites was 137 (SD = 12) at time 1and 132 (SD = 13) at time 2, with a possible range of 84 to 168. Theaverage scores indicated that approximately 82 percent of the items(time 1) and 79 percent endorsement rate (time 2), suggesting that thesites did a good job of providing information, features, and resourcesregarding schizophrenia. The item endorsement rate for subcategories wason average 75 percent (time 1) to 79 percent (time 2), with a range of73 percent (confidentiality) to 90 percent (design) at time 1 and 60percent (accessibility) to 97 percent (design) at time 2. The rank order of the sites is presented in Table 1. Scores rangedfrom a high of 153 (WebMD) to a low of 116 (The Experience ofSchizophrenia), with a 27-point difference between the highest andlowest ranking sites at time 1. The WebMD subsection onschizophrenia's (http://www.webmd. com/schizophrenia/default.htm)91 percent endorsement rate suggests that it does a very good job ofproviding features and information. MedicineNet.com's section onschizophrenia (http://www.medicinenet.com/schizophrenia/ article.htm)had an 89 percent endorsement rate, and Psych Central's sectioncalled "Schizophrenia and Psychosis"(http://psychcentral.com/disorders/ schizophrenia/) had an 88 percentendorsement rate. The scores suggest that the sites offered good qualityinformation and features on the disorder. This rank order changedsomewhat at time 2. Scores ranged from a high of 147 for eMedicine.comto low of 107 for Docguide.com, with a 40-point spread. Essentially, thesame top five sites from the first series of ratings were the top fivein the second series of ratings, although the exact order changedsomewhat (see Table 1). Once again, the endorsement rates for the topfive sites (84 percent to 87 percent) (time 2) suggest an overall goodquality of information and features regarding schizophrenia. Nonprofit Sites The overall mean score for nonprofits was 111 (SD = 6; range: 101to 123) at time 1 and 111 (SD = 8; range: 94 to 123) at time 2. Theaverage scores indicated that 77 percent of the items were endorsed (forboth times), suggesting that the sites did an average job of providinginformation, features, and resources regarding schizophrenia. Scores onthe subcategories ranged from 53 percent to 90 percent at time 1. Two ofthe subcategories (authority and confidentiality) had less than 75percent of the items endorsed. Authority had only 67 percent of theitems endorsed, and confidentiality had only 53 percent of the itemsendorsed. In comparison, the for-profits did noticeably better than thenonprofits in these subcategories (authority: 78 percent;confidentiality: 73 percent) at time 1. Nonetheless, the range ofendorsement of items across subscales increased somewhat, such that thelowest endorsement rating was 67 percent (authority) and the highest was93 percent (design) at time 2. Both confidentiality and accessibilityhad endorsement rates of 70 percent; all other subscales had anendorsement rate greater than 75 percent. Overall, these results suggestthat nonprofits did a somewhat poorer job of providing informationregarding the qualifications of authors, providing clear informationabout confidentiality policies, and providing accessibility features. As shown in Table 1, scores at time 1 ranged from a high of 123(NAMI) to a low of 101 (Wikipedia: Schizophrenia), with a 22-pointdifference between the highest and lowest ranking sites. The NAMIsection on schizophrenia(http://www.nami.org/Template.cfm?Section=By_Illness&Template=/TaggedPage/TaggedPageDisplay.cfm&TPLID=54&ContentID=23036) was the optimal site, with an 85 percent endorsement rate,suggesting a good quality of information and features. Mental HealthAmerica's page on schizophrenia(http://www.nmha.org/go/schizophrenia) had an 82 percent endorsementrate, also suggesting a relatively good quality of information andfeatures; Helpguide. org's page on schizophrenia(http://www.helpguide.org/mental/schizophrenia_symptom.htm) had an 80percent endorsement rate, suggesting a similar level of qualityinformation and features. Scores at time 2 ranged from a high of 124 forMedlinePlus to a low of 94 for Successful Schizophrenia. As listed inTable 1, the top five sites changed at time 2: Both the Mental HealthAmerica and SAMHSA sites dropped out, and the site for the NationalAlliance for Research on Schizophrenia and Depression (NARSAD) moved up(only one site moved up given that time 1's list included six sitesin the top five, owing to a tie between two sites for the rank order offive). A brief summary of top=rated sites across both groups based onthe 2010 ratings is presented in Appendix B. DISCUSSION An increasing number of people use the Internet as a main source ofhealth information (Eysenbach et al., 2002; iProspect, 2006; PewInternet & American Life Project, 2006); thus, there is a criticalneed to ensure that such information is accurate and based on the bestavailable research. This is particularly salient for consumers of mentalhealth services and their families, who historically have beendissatisfied with the lack of information they receive fromprofessionals. The present study sought to add to the literatureregarding quality online health information by assessing sites aboutschizophrenic spectrum disorders. In addition, we sought to provide aresource for both social workers and consumers regarding the best ofthese sites (see Appendix B). Overall Findings on Quality of Information and Site Features The study found that the majority of Web sites included most of thefeatures and information assessed by the WebMedQual scale, which isbased on criteria from the HNF. The results offer preliminary data thatthe majority of sites strive to provide accurate, comprehensive, andhelpful information about a potentially devastating disorder (HNF, 2009;Pew Internet & American Life Project, 2006). This was found at bothrating time points. These results are in marked contrast to those ofrecent studies done to evaluate the quality of information about eatingdisorders (Murphy et al., 2004) and self-help sites (Godin et al.,2005). A more recent study on the quality of information for caregiversof someone with Alzheimer's found similar results to thisstudy's in that the overall quality was adequate to very good, withsome variation across individual sites (Anderson et al., 2009). There were a few common problems across sites. Most did not offerthe flexibility to customize interactivity for individuals with adisability, did not offer another language choice, or did not offer textenlargement. In addition, the nonprofits did a poorer job of providinginformation about confidentiality. This may have been because thesesites did not collect either personally identifying or aggregateinformation (see the For-Profits versus Nonprofits section).Nonetheless, nonprofits should strengthen their disclosure regarding thecollection (or lack thereof) of any personal identifying information andhow that information is used. An interesting finding is that none of the sites actually provideda stand-alone site that offered information specifically geared toconsumers and family members about schizophrenic spectrum disorders.Typically, the information was a secondary part of the site. Asubstantial number of for-profit sites were what we called "generalpurpose" medical information Web sites that offered a wide range ofinformation, from preventive care and prescription medicationinformation to sections specific to the needs of children, women, men,and older adults. Thus, schizophrenia was a subsection of a largermental health section, which itself was a section among numerous othermedical conditions. Although it is encouraging to see that mental healthdisorders are seen as a part of the larger notion of "health"on general purpose sites, it also made finding the information moredifficult or, at least, less convenient. For-Profits versus Nonprofits In general, the study found that for-profit sites' ratingswere better than nonprofit sites' and that this persisted from time1 to time 2. Despite having sponsors and advertisers, the for-profitshad a higher percentage of items endorsed across all of thesubcategories. The ratings indicated that they did a better job with thecurrency and accuracy of health information and the disclosure ofexpertise and possible financial interests of authors, and they provideduser-friendly navigational features, technical support, and clearerprivacy and confidentiality policies. This suggests that the merepresence of advertisements or sponsors did not necessarily compromisethe quality or availability of features on a site. The results raise the question of why for-profit sites tended to dobetter than nonprofit sites. One possibility is that for-profit sitesare motivated to keep their sites up-to-date and information easilyaccessible for potential consumers of products and services offered onthe site. Functionality and user friendliness can provide a morepositive experience for a visitor while increasing the likelihood of theperson returning in the future. Likewise, for-profit sites are moremotivated to streamline their sites to facilitate navigation to as manyportions of the site as possible, which can mean more "clicks"on pages where advertisers place their ads. For-profit sites alsoextensively used "cookies" to track users. As explained byvirtually all of the sites that used cookies, the information obtainedis provided to potential advertisers in aggregate form. This practicehelps to create a more personalized experience not only in features andinformation for the visitor, but also in targeted advertising based on avisitor's previous navigation of the site (Ha, Al Shaar, Inkpen,& Hdeib, 2006). However, nonprofits probably did not have theresources to more closely manage their sites: The existence of theirsites was independent of any possible income earned through them.Likewise, there was a larger amount of change regarding which sites wererated in the top 5 across the two time periods. This was mostly due to asubstantial revamping of the NARSAD, the Brain and Behavior ResearchFund site (http://www.narsad.org), which rebranded itself and includedseveral new pages, features, and policies about using the site. Limitations The present study selected sites that, at the time of the ratings,were listed as top search results for two time periods. It is likelythat many of these sites will remain at the top of search engine resultsat least in the near future. However, given the ongoing, changing natureof the Internet, the present study's results could quickly becomeobsolete. As previously noted, this proved in part to be the case withthe nonprofit sites at time 2. We hope that the results from the presentstudy will serve as a starting point for the ongoing evaluation of thequality of health-related information online regarding schizophrenia. Interrater reliability was good in regard to the for-profit sites,but it is unclear why it was poorer for the nonprofit sites, althoughthe rating improved somewhat at time 2. The nature of the nonprofitsites raises questions regarding what it was about these sites that madeit difficult for us to endorse in like fashion at least whenindependently rated. Both raters reported that it was difficult todetermine what features and information was located on some nonprofitWeb sites because they had atypical designs. The Wikipedia page inparticular has a unique design as well as a rapidly changing format forinformation updates. Likewise, although Medline Plus had many of thedesign features typical of others Web pages (for example, help,frequently asked questions, contact information), the topicalinformation was provided across numerous separate pages so that locatinga specific topic or site policy was sometimes difficult. This impliesthat the nature of some sites may make it difficult to locateinformation, at least initially. In addition, at present, the WebMedQual scale has limitedpsychometric properties. Any findings based on the scale are limiteduntil further psychometric testing is done. Although the test-retestreliabilities were good for both the for-profit and nonprofit sites, theconsistently lower interrater reliabilities for the nonprofits suggestthat the scale may have limited utility for such sites. Thus, thepresent study's findings are provisional at best. CONCLUSION The results provide the first evaluation of the quality of onlinehealth information about schizophrenic spectrum disorders. Theinformation can be used as a resource guide for social workers servingindividuals seeking information about this often debilitating andserious disorder. In addition, the present study offers a starting pointfor the continuation of a process for evaluating Web-based informationregarding schizophrenic spectrum disorders. Systematically evaluatingthe online health information about specific mental illnesses cancontribute to further clarification regarding a gold standard forInternet-based health information, in particular for schizophrenia. And,as previously noted, it provides another source of information forsocial workers to provide to consumers and their loved ones. Given the constantly changing nature of the Internet, it isprobable that the sites included in any study of this nature maysubstantially change in content and site organization. It is likewisepossible that other new sites, unavailable at the time of this study,will appear, necessitating ongoing evaluation of the quality ofinformation related to schizophrenia on the Internet. We hope that thiswill be the first step toward an ongoing process of evaluating onlinehealth information regarding one of the major psychiatric disordersaffecting people today so that social workers have yet one more valuableresource to assist consumers and their families.APPENDIX A: WEBMEDQUAL SUBCATEGORY DESCRIPTIONSSubcategory Description No. of itemsContent Items deal with the currency 19 and accuracy of health information provided by the site. How timely is the information? How thorough and complete is the information?Authority of Items deal with the disclosure 18source of expertise, training, and possible financial interests of contributing authors. Who are the site's authors, and do they have potential conflicts of interest if the site has advertisers or corporate sponsorship?Design of Web Items deal with the user's 17site navigation experience of the site. How appealing and intuitive are the site's features so that information can be easily found?Accessibility Items deal with the regular 6and availability of Web siteavailability of content and ease ininformation functionality (including for those with sight impairments or whose first language is not English).Links Items deal with the quality 4 and number of links offered by the site so that a user can get more information if desired.User support Items deal with both technical 8 support and requests for more information. How easy is it to locate and contact technical support? How easy is it to locate and contact site administrators with question regarding the health information provided by the site?Confidentiality Items deal with the site's 17and privacy privacy and confidentiality policies. Does the site clearly explain what personal information, if any, is collected and how this information might be used in the future?E-commerce Items deal with the user's 6 experience of buying products or services offered by the site (Provost et al., 2006). APPENDIX B: INTERNET SITES ON SCHIZOPHRENIA FOR FAMILIES,CONSUMERS, AND PROFESSIONALS For-Profit Web Sites (In No Particular Order) WebMD Schizophrenia http://www.webmd.com/schizophrenia/default.htmIn-depth information on schizophrenia plus discussion boards and blogs.Registration may be required for some features. MedicineNet.com: Schizophreniahttp://www.medicinenet.com/schizophrenia/article.htm Provides content onschizophrenia, medications, and treatment. Discussion pages are alsoavailable. Psych Central: Schizophrenia and Psychosishttp://psychcentral.com/disorders/schizophrenia/ Fact sheets, chatrooms, and in-depth information about schizophrenia. Registration may berequired to access some features. Nonprofit Web Sites (In No Particular Order) National Alliance on Mental Illness (NAMI) http://www.nami.org/ Provides general information on schizophrenia as well as othermajor mental health disorders. Also provides information about gettinghelp and doing advocacy at the local and national levels. NARSAD--The Brain and Behavior Research Fund http://www.narsad.org/ Provides general information about schizophrenia as well as othermajor mental health disorders. Also provides the opportunity to donatetoward its general fund to fund promising research regarding thesedisorders.. Helpguide.org http://www.helpguide.org/ Provides general information regarding mental health issues andcoping strategies. Note: This is a partial listing of Web sites reviewed for content,source authority, and confidentiality. For a full listing, pleasecontact Joseph Guada (guada.1@osu.edu) at Ohio State University, Collegeof Social Work. Original manuscript received October 19, 2009 Final revision received June 17, 2010 Accepted July 28, 2010 REFERENCES American Psychiatric Association. (2000). Diagnostic andstatistical manual of mental disorders (4th ed., text rev.). Washington,DC: Author. Anastasi, A. (1988). Psychological testing (6th ed.). New York:Macmillan. Anderson, K. A., Nikzad-Terhune, K. A., & Gaugler, J. E.(2009). A systematic evaluation of online resources for dementiacaregivers. Journal of Consumer Health on the Internet, 13, 1-13. Bath, P. A., & Bouchier, H. (2003). Development and applicationof a tool designed to evaluate Web sites providing information onAlzheimer's disease. Journal of Information Science, 29, 279-297. Eysenbach, G., Powell, J., Kuss, O., & Sa, E. (2002). Empiricalstudies assessing the quality of health information for consumers on theWorld Wide Web: A systematic review. JAMA, 287, 2691-2700. Godin, S., Truschel, J., & Singh, V. (2005). Assessing qualityassurance of self-help sites on the Internet. Journal of Prevention& Intervention in the Community, 29, 67-84. Ha, V., Al Shaar, F., Inkpen, K., & Hdeib, L. (2006). Anexamination of user perception and misconception of Internet cookies. InCHI extended abstracts (pp. 833-838). New York: SIGCHI. Health on the Net Foundation. (2009). The HONCode in brief.Retrieved from http://www.hon.ch/HONcode/Webmasters/Conduct.html iProspect. (2006). Search Engine Behavior Study [Electronicversion]. Retrieved fromhttp://www.iprospect.com/premiumPDFs/WhitePaper_2006_SearchEngineUserBehavior.pdf Kummervold, P. E., Gammon, D., Bergvik, S., Johnsen, J. K.,Hasvold, T., & Rosenvinge, J. H. (2002). Social support in a wiredworld: Use of online mental health forums in Norway. Nordic Journal ofPsychiatry, 56, 59-65. Lefley, H. P. (1998). Families, culture, and mental illness:Constructing new realities. Psychiatry: Interpersonal & BiologicalProcesses, 61, 335-355. Lewis, E. (2009). Top ten search engines. Retrieved fromhttp://www.seoconsultants.com/search-engines/ Murphy, R., Frost, S.,Webster, P., & Schmidt, U. (2004). Anevaluation of Web-based information. International Journal of EatingDisorders, 35, 145-154. National Alliance on Mental Illness. (2003). Family to family:Recent studies of family-to-family. Retrieved fromhttp://www.nami.org/Template.cfm?Section=Family-to-Family&template=/ContentManagement/ContentDisplay.cfm&ContentID=36366 National Institute of Mental Health. (2008). National Institute ofMental Health strategic plan. Washington, DC: Author. Pew Internet & American Life Project. (2006). Online healthsearch 2006. Retrieved fromhttp://www.pewinternet.org/~/media//Files/Reports/2006/PIP_Online_Health_2006.pdf.pdf Provost, M., Koompalum, D., Dong, D., & Martin, B. C. (2006).The initial development of the WebMedQual Scale: Domain assessment ofthe construct of quality of health Web sites. International Journal ofMedical Informatics, 75, 42-57. U.S. Department of Health and Human Services. (2001). Mentalhealth: Culture, race, and ethnicity--A supplement to mental health: Areport of the surgeon general. Rockville, MD: U.S. Department of Healthand Human Services, Substance Abuse and Mental Health ServicesAdministration, Center for Mental Health Services. Joseph Guada, PhD, MSW, is assistant professor, and VictoriaVenable, MSW, LISW-S, is research assistant, College of Social Work,Ohio State University, Columbus. Address correspondence to Joseph Guada,College of Social Work, Ohio State University, 1947 College Road, 325GStillman Hall, Columbus, OH 43210; e-mail: guada.1@osu.edu.Table 1: Rank Order Ratings of Sites Based on Overall Score onWebMedQual Scale 2009 2010 Rank RankFor-Profit Order OrderWeb Site (a) (Score) (Score)WebMD Schizophrenia 1 (153) 2 (146)MedicineNet.com: 2 (150) 2 (146)SchizophreniaPsych Central: 3 (147) 3 (145)Schizophrenia andPsychosiseMedicine: 4 (146) 1 (147)SchizophreniaMayoClinic.com: 5 (145) 4 (141)SchizophreniaFamilydoctor.org: (145) 5 (139)SchizophreniaEverydayHealth.com: 7 (142) 6 (137)Schizophrenia CenterSchizophrenia.com 8 (138) 9 (132)Medical News Today: 9 (134) 10 (129)SchizophreniaMerck Manual Home 10 (133) 7 (135)Edition:SchizophreniaHealth Information-- 133 12 (118)SchizophreniaMentalhelp.net: 12 (129) 8 (133)SchizophreniaBehaveNet: 13 (120) 11 (124)SchizophreniaDocguide.com: 14 (119) 14 (107)SchizophreniaThe Experience of 15 (116) 13 (110)SchizophreniaFor-ProfitWeb Site (a) URLWebMD Schizophrenia http://www.webmd.com/schizophrenia/ default.htmMedicineNet.com: http://www.medicinenet.com/schizophrenia/Schizophrenia article.htmPsych Central: http://psychcentral.com/disorders/Schizophrenia and schizophrenia/PsychosiseMedicine: http://emedicine.medscape.com/article/Schizophrenia 805988 overviewMayoClinic.com: http://www.mayoclinic.com/health/Schizophrenia schizophrenia/DS00196Familydoctor.org: http://familydoctor.org/online/famdocen/Schizophrenia home/common/mental health/ treatment/ 266.htmlEverydayHealth.com: http://www.everydayhealth.com/emotional-Schizophrenia Center health/schizophrenia/index.aspxSchizophrenia.com http://www.schizophrenia.com/index.phpMedical News Today: http://www.medicalnewstoday.com/Schizophrenia sections/schizophrenia/Merck Manual Home http://www.merck.com/mmhe/sec07/Edition: ch107/ch1076.htmlSchizophreniaHealth Information-- http://www.cmellc.com/topics/Schizophrenia schiz.htmlMentalhelp.net: http://www.mentalhelp.net/poc/center-Schizophrenia index. php?id=7BehaveNet: http://behavenet.com/capsules/Schizophrenia disorders/schiz.htmDocguide.com: http://www.docguide.com/news/Schizophrenia content.nsf/ PatientResAllCateg/ Schizophrenia?OpenDocumentThe Experience of http://www.chovil.com/Schizophrenia 2009 2009 Rank RankNonprofit Order OrderWeb Site (b) (Score) (Score)NAMI 1 (123) 3 (119)Mental Health 2 (118) 8 (111)America:SchizophreniaHelpguide.org: 3 (115) 4 (114)SchizophreniaNeuroscience for 4 (114) 4 (114)Kids--SchizophreniaMedlinePlus: 5 (112) 1 (123)SchizophreniaSAMHSA: (112) 7 (112)SchizophreniaAACAP: Schizophrenia 7 (111) 9 (108)In ChildrenNARSAD--The BrainFund and BehaviorResearch (formerly 8 (110) 2 (120)known as NationalAlliance forResearch onSchizophrenia andDepression)NIMH--Schizophrenia 9 (109) 6 (113)Mental Health: AReport of the 10 (105) 11 (100)Surgeon General--Chapter 4(Schizophrenia)Successful 11 (103) 12 (94)SchizophreniaWikipedia: 12 (101) 10 (107)SchizophreniaNonprofitWeb Site (b) URLNAMI http:www.nami.org/Template.cmf?Section-By_ Illness&Template=/TaggedPage/TaggedPageDisplay. cfm&TPLID=54&ContentID=23036Mental Health http://www.nmha.org/go/schizophreniaAmerica:SchizophreniaHelpguide.org: http://www.helpguide.org/mental/Schizophrenia schizophrenia_symptom.htmNeuroscience for http://faculty.washington.edu/Kids--Schizophrenia chudler/schiz.htmlMedlinePlus: http://www.nlm.nih.gov/medlineplus/Schizophrenia schizophrenia.htmlSAMHSA: http://mentalhealth.samhsa.gov/Schizophrenia publications/allpubs/KEN98-0052/ default.aspAACAP: Schizophrenia http://www.aacap.org/cs/root/factsIn Children for families/schizophrenia-in- childrenNARSAD--The Brain http://www.narsad.org/Fund and BehaviorResearch (formerlyknown as NationalAlliance forResearch onSchizophrenia andDepression)NIMH--Schizophrenia http://www.nimh.nih.gov/health/ topics/schizophrenia/index.shtmlMental Health: AReport of the http://www.surgeongeneral.gov/Surgeon General-- library/mental health/chapter4/Chapter 4 sec4.html(Schizophrenia)Successful http://www.successfulSchizophrenia schizophrenia.org/Wikipedia: http://en.wikipedia.org/wiki/Schizophrenia SchizophreniaNote: URL= uniform resource locator; NAMI = National Alliance onMental Illness; 5AMHSA= Substance Abuse and Mental HealthServices Administration; AACAP =American Academy of Child andAdolescent Psychiatry: NARSAD = National Alliance for Research onSchizophrenia and Depression; NIMH = National Institute of MentalHealth.(a) Possible range: 84-168.(b) Possible range: 72-144.