Ethical issues about qualitative studies especially in this digital era is complex and ambiguous sometimes. Thanks to Tim’s reflection, I reviewed the 2012 report from AoIR (Association of Information Researchers) ethics working committee: Ethical Decision-Making and Internet Research. It is a good supplement to what I learned from Markham’s article about fabrication.

The ethical related decision making is even harder than recalling the ethical principles such as privacy and confidentiality. It seriously intertwines with researchers’ ideological beliefs,  political and academic culture environment (IRB for example), one’s disciplinary assumptions, and one’s methodological approaches. To consider about internet ethnographic research, the first ethical concern is the privacy. For my online forum studies, the posts can be viewed publicly without login information, and thus IRB approval, according to their policy, should not be a big deal.

The guidelines provided by the report is quite useful, and I’d like to share with my dear classmates. Although the ambiguous debate keeps on about private/public, human/textual message, and who is human subject, the major concern should be still given to people who are vulnerable.

In practice, following questions need to be kept in mind when conducting relative online research:

1. how is the context conceptualized? what are ethical expectations related to that context?

It is very important that how participants view the venue you are investigating. If it is a closed virtual community with rigorous terms of use, more ethical concerns may be probably expected. By contrast, if encountering public data with no personal information nor stakeholders, less ethical issues need to be in mind.

2. how is the context accessed? how researcher and participants are situated in the context? Is there any need to accommodate to “perceived privacy” in ethical concerns even if it is public data?

3. What are different ethical expectations for participants and researchers in the context?

4. What is the ethical expectation for similar studies you are going to investigate?

5. How data will be securely managed, stored and represented?

6. How are finding presented?

7. Any potential harms nor grades to participants within the group?

8.  How are we recognizing the autonomy of others and acknowledging that they are of equal worth to ourselves and should be treated so?

This update is about how to apply content analysis to analyze a discussion board of a deaf virtual community: alldeaf.com. A virtual community is the connection between members which is created, maintained, and developed via computers that are connected to networks (Granit & Nathan, 2000). Or groups of people with shared interests who conduct regular, organized interactions online, by means of a common location or mechanism (Ridings, Gefen, & Arinze, 2002). D/deaf and hard of hearing populations particularly, rely more on communication through Internet or online forum. AllDeaf. com is one of the largest virtual community for D/deaf and hard of hearing people. And cochlear implant & hearing aid is one of hottest topic on that forum.

Three research questions are posed for this study:

1. What are the main subjects people are talking about cochlear implant and hearing aids?

2. Does the forum include behaviors that express social support or beyond social support?

3. How Deaf cultural conflict involves into online discussions?

Data for six months will be collected for analysis. Research data were processed by means of content analysis: The messages posted on the forum will be assigned to thematic content categories. A code book needs to be conducted before the formal procedure of content analysis.

The reflection site I chose is AllDeaf.com. This is an online forum for Deaf and Hard of hearing population. The qualitative content analysis for the forum is about the information flow within discussion on Cochlear Implant. The disciplinary context for this part is about audiology, CI and Deaf Culture. The social context for this part is about dispute between medical and social model of deafness: should deafness be cured? The conceptual context for that part is about CI usage and acceptance. There are 4273 posts from 2003 to 2015. The themes can be derived from all posts using content analysis methods.

The grounded theory and coding practice in this Class is fun. After reading through all the grounded theory guidelines, I felt that coding is a systematic procedure which requires a lot of time and close reflections. And different person has different ideas when they tried to code the same materials in that 10 minutes.

For me, when I started reading through the material. I first thought about what research questions I held to analyze the data. To understand how information was posted for online courses? Second, I went through and found only one user were present, which means a lot of contextual information might be missing. Then I was thinking how I could analyze the data. From topics covered? Emotional change? Time of posts? How topics are related to the course? However, I didn’t notice about the funny or weird pieces as commented by my classmates. And it makes me to think about the reliability and validity issues in grounded theory coding. We all have our barriers so the coding results cannot get rid of its subjectivity. Another concern for me is whether personal judgment or comment are appropriate as the codes in analyzing the data. Shouldn’t us keep the objective perspective towards our data and keep ourselves as neutral?

Applying Qualitative Content Analysis to Study Online Communities

In this update, basic ideas about qualitative content analysis are reviewed and a bibliographic list is provided related to application of qualitative content analysis in online community studies.

Introduction of Qualitative Content Analysis

• Concept
  • Content analysis is a research technique for making replicable and valid inferences from texts (or other meaningful matter) to the contexts of their use (Krippendorff, 2012, p. 24). In other words, it is a method to understand the manifest and latent content of the texts, images, moving images, or other meaningful materials by categorizing, evaluating, or tabulating symbols in materials. The aim is to figure out meanings or probable effect behind the texts.
  • From communication perspective, Weber (1990, p. 9) pinpointed that valid inferences from text includes senders of messages, audience of messages, and message itself.
• Qualitative Content Analysis
  • The literature yields both quantitative and qualitative approaches of content analysis.
  • Quantitative approach, first brought up by Lasswell (1938, 1943, & 1952), focused on word frequency, space measurements, time counts (for video or television time) and keyword frequency. After reducing data into numbers and frequencies, statistical analysis will be applied to describe the phenomenon or test some hypotheses. The assumption is that words and phrases mentioned most are those reflecting important concerns in every communication.
  • By overcoming the rigorous nature and superficial analysis of quantitative content analysis (Kracauer, 1952), qualitative content analysis focuses on informal content and comprehensive understanding of a text (Hsieh & Shannon, 2005).
  • The nature of qualitative content analysis is to cope with interpretation problem of textual matters, which is rooted in hermeneutics (Janzuihof, 2013). According to Schleiermacher (1998), interpretation is the art of individuality. Each text is determined by individual historical context and its author. Content dependency makes each single “word” in hermeneutics has no permanent meaning, since factors of context cannot be fully understood. Therefore, full objective understanding can therefore never be achieved.
  • According to Krippendorff (2012, p. 32), qualitative content analysis has following characteristics:
    • Requires close reading of relatively small amounts of textual matter
    • Interpretation of given texts into new narratives, which are accepted within particular scholar communities
    • The systematic interpretation is required, and analysts should acknowledge that their own socially and culturally understandings constitutively participate in the interpretation.
  • Finding themes or systematic categories are the central task of qualitative content analysis (Krippendorff, 1980, p.76).
  • Mayring (2000) provided four basic rules about qualitative content analysis:
    • Fit the materials into a model of communication: analysts should be aware which aspect they’d like to focus on the inferences: to aspects of communicator (their feelings, experiences or opinions), situation of text production, to the social-cultural background, text itself, or the effect of the message.
    • Step by step rule: a systematic procedure should be followed to devise the material into content analytical units.
    • Category is the center of analysis, the aspects of text interpretation will be put into categories by following research questions, which is carefully founded and revised within the process of analysis
    • Reliability and validity concern: reliability will be maintained by comparing with other studies and inter-coder reliability. Validity will be guaranteed by trained project team members.
    • Three Approaches of Qualitative Content Analysis
      • Inductive and deductive (Graneheim & Lundman, 2004; Mayring, 2000)
        • Inductive approach starts from category criterion definition, which derives from theoretical background and research question. Following the derived criterion, texts will be worked through and categories will be tentatively step-by-step achieved. Feedback loop will be applied to revise and evaluate derived categories and their reliability.
        • Deductive approach starts from theoretical assumptions of categories from theories, and work through the texts to see if they fit the derived theoretical categories.
      • Conventional, directed, or summative (Hsieh & Shannon, 2005).
        • The three approaches differ in their ways to extract coding schemes and final coding categories
        • In conventional content analysis, researchers avoid using categories in previous theories or studies. Instead, they allow categories flow totally from the text. It requires researchers to emerge themselves into the data until insights come out.
        • Directed content analysis is based on previous research or theory, which is regarded as deductive approach in Marying’s classification
        • Summative approach is to understand contextual use of words or content frequencies. This approach usually starts from counting words or interested contents, and then interprets the context of those words (why they happen in certain ways), or evaluate quality of the content.
      • Discourse analysis, rhetorical analysis, ethnographic content analysis, and conversation analysis (Krippendorff, 2012)
        • Discourse analysis tends to focus on how particular phenomena are presented through texts. For example, how movies and TV shows manifest the stereotypical image of librarians.
        • Rhetorical analysis focuses on how message are delivered and with what effects. Structural elements, tropes, styles of argumentation, speech acts, and the like are researchers’ focus in analysis. Efforts to study negotiations to see what works and what doesn’t work is one example of this.
        • Ethnographic content analysis is an approach to encourage analysis emerged from the text. It doesn’t only work with categories and their narrative descriptions, but also deals with situations, settings, styles, images, meanings, and nuances presumed to be recognizable by the participants
        • Conversation analysis tends to analyze transcripts as records of conversational moves toward a collaborative construction of conversations.
      • Procedures of Qualitative Content Analysis
        • Zhang and Wildemuth (2009) proposed 8 steps of qualitative content analysis in library and information science from a particularly inductive approach with a focus of finding out themes and categories:
          • Prepare the data, for instance, transcribing interviews to analyzed data
          • Define the unit of analysis. Usually, individual themes rather than a sentence or paragraph is a unit for analysis.
          • Develop categories and a coding scheme, both inductive and deductive approaches can be applied. Constant comparative method will be applied in the inductive approach.
          • Test the coding scheme on a sample of text
          • Code all the text.
          • Assess coding consistency: usually, inter-coder consistency will be assessed.
          • Draw conclusions from the coded data
          • Report methods and findings

In this week’s interview, the research question I chose is “information behavior related to buying your laptop”. Accordingly, non-structured interview question is “How would you decide to buy your laptop?”; semi-structured questions are: 1) What information do you consider when you decide to buy your laptop? 2) Where do you get those information? 3) Are there any other information related behaviors you’d like to talk about related to your laptop decision?; critical incident interview question is: Could you describe your last experience to buy your laptop?

In order to talk about later information searching or inquiring, most of my interviewees spent a lot of time describing their criteria and preference in laptop decision, which leaves little time for information behavior questions. Maybe because people would become talkative when it relates to their own preferences? It seems hard to drag them back to “not that interested” information related questions.

I found critical incident question is much easier for me. Because the steps of the incident can be tracked, conversations can be led to the way I want. Not sure if anyone else have the same feeling:)

Methods Statement: Content Analysis

1. Definition:

A research technique for making replicable and valid inferences from texts (or other meaningful matter) to the contexts of their use. (Krippendorff, 2012, p. 24)

2. Basic Procedure/Framework

What are the texts to analyze?
What is your research question?
What is the population or context that texts are involved?
What is the coding book or analytical construct?
What are the inferences to retrieve from the texts?
How can you validate your results?

(Krippendorff, 2012, p. 35)

3. Differences about quantitative and qualitative approaches in content analysis:
Quantitative: systematic, word counts, computer assisted technique, analytical path (Franzosi, 2008)
Qualitative: require close reading of relatively small amounts of textual matter (Krippendorff, 2012)

3.1 When should we apply quantitative and when qualitative approach?
(“QuantContentAnalysis” n.d.)

4. Qualitative approaches:
A. inductive and deductive (Graneheim & Lundman, 2004)
B. conventional, directed, or summative (Hsieh & Shannon, 2005).
C. discourse analysis, rhetorical analysis, ethnographic content analysis, and conversation analysis (Krippendorff, 2012)

5. What is the analyzing unit?
6. Sampling
7. Coding construct
8. Analytical techniques
9. Reliability and validity measurements
10. Examples about content analysis in online community settings.

References

Graneheim, U. H., & Lundman, B. (2004). Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Education Today, 24(2), 105–112. doi:10.1016/j.nedt.2003.10.001

Hsieh, H.-F., & Shannon, S. E. (2005). Three Approaches to Qualitative Content Analysis. Qualitative Health Research, 15(9), 1277–1288. doi:10.1177/1049732305276687

Krippendorff, K. (2012). Content Analysis: An Introduction to Its Methodology. SAGE.

QuantContentAnalysis < MoM < digitalmethods.net. (n.d.). Retrieved February 16, 2015, from https://www.digitalmethods.net/MoM/QuantContentAnalysis

Franzosi, R. (2008). Content analysis: Objective, systematic, and quantitative description of content. SAGE benchmarks in social research methods: Content analysis, 2-43.

Elo, S., & Kyngäs, H. (2008). The qualitative content analysis process. Journal of Advanced Nursing, 62(1), 107–115. doi:10.1111/j.1365-2648.2007.04569.x

Krippendorff, K. (2004). Reliability in Content Analysis. Human Communication Research, 30(3), 411–433. doi:10.1111/j.1468-2958.2004.tb00738.x

Berelson, B. (1952). Content analysis in communication research. New York, NY, US: Free Press.

Gleave, E., Welser, H. T., Lento, T. M., & Smith, M. A. (2009). A Conceptual and Operational Definition of “Social Role” in Online Community. In 42nd Hawaii International Conference on System Sciences, 2009. HICSS ’09 (pp. 1–11). doi:10.1109/HICSS.2009.6

Marra, R. M., Moore, J. L., & Klimczak, A. K. (2004). Content analysis of online discussion forums: A comparative analysis of protocols. Educational Technology Research and Development, 52(2), 23–40. doi:10.1007/BF02504837

Shea, P., Hayes, S., Vickers, J., Gozza-Cohen, M., Uzuner, S., Mehta, R., … Rangan, P. (2010). A re-examination of the community of inquiry framework: Social network and content analysis. The Internet and Higher Education, 13(1–2), 10–21. doi:10.1016/j.iheduc.2009.11.002

Kožuh, I., Hintermair, M., & Debevc, M. (2014). Examining the Characteristics of Deaf and Hard of Hearing Users of Social Networking Sites. In K. Miesenberger, D. Fels, D. Archambault, P. Peňáz, & W. Zagler (Eds.), Computers Helping People with Special Needs (pp. 498–505). Springer International Publishing. Retrieved from http://link.springer.com/chapter/10.1007/978-3-319-08599-9_74

Shoham, S., & Heber, M. (2012). Characteristics of a virtual community for individuals who are d/deaf and hard of hearing. American Annals of the Deaf, 157(3), 251–263.

Here is the note about my field observation in Sweet Shop.

It is a shining afternoon at 2:43 on Thursday. The weather and sunshine is wonderful for people to stay outside even in early cold Feb. The observation site I chose is the sweet shop. There are 22 people there, 7 outside, 15 inside; 5 males and 17 females. 3 are eating their lunch or snacks, 5 brings their own drinks, and others are consuming drinks there. Inside the shop, 8 people are finishing their personal work individually, two ladies are chatting casually inside the couch near the reception desk, two ladies are collaborating something on one table, and one couple are sitting near the door having sweet chatting. Outside the sweetshop, only one 40-year-old women is typing something on her laptop and listening to music. Another guy with black jacket, black glasses, and black hat seems watching something in front of him, not clear. Two males are chatting, standing up and exchanging information. One male, and two ladies are chatting and bursting some laughters beside them.

REFLECTION
First of all, it is very hard to guarantee “unobtrusive observation” in a restaurant. Although I tried to keep my research questions in advance, and quickly glanced surrounding environments, still people there noticed my “somehow awkward” behavior to look around. There were at least 2 ladies looking at me for two seconds. I think the difficulties to make it “unobtrusive” are also reflected in Cheryl’s and Anna’s observations.

Another reflection I have from our readings is the systematic techniques related to ethnography. You have limited time and the information in the environment is unlimited. It is interesting to see that everyone has different focuses on what they’d like to describe in their field notes. Are there some common rules to cover what should be and shouldn’t be included in the field notes?

Biyang YU

I first leaned about qualitative methods is in one of my anthropology class. And I was amazed and impressed when the professor told me how he researched mysterious local experiences about how an uneducated child can sing an lost long epic in Mongolia. The stories he provided were so live  to me, and it somehow broadens my worldview, as well as arises my respect to human nature. Isn’t it a cool job to listen to write down different stories?

My first qualitative research experience is my undergraduate thesis about serendipity around scholars: sometimes researchers will encounter information that they eagerly want but cannot find originally. It’s a quite interesting experience, I listened to 11 stories and coded them using grounded theory. I was amazed about the richness of my final findings! Just from 11 stories, I began to understand the active and passive information searching procedures. And I understand why some people encounter information more often than others. So cool!

For quantitative research, you need to know the story in advance. And just try to prove or disprove your hypothesis. The surprise from quantitative approach (at least to me) is much less than the qualitative one.

More important, what I want to study now is an unrepresented population (i.e. deaf). Their intentions and culture are always misunderstood by public. And I believe qualitative approach will provide more authentic, holistic, and meaningful stories about that group.