Theme 6: Qualitative and case study research (After)

For this weeks theme I didn't do much additional reading apart from the article about case studies and the two articles that I chose. I read up on a couple of methods that was in one of my articles. I guess I felt like this theme really built upon all of the previous themes and that the article on case studies was otherwise sufficient. It was nice to have a theme at the end of the course that tied all of the concepts of earlier themes together. I think that I have a clear understanding of what a case study means and how it should be implemented because of my previous understanding of the other themes about theory and method. It really tied together nicely.

The case study article that I chose made me a bit unsure of the boundaries between a study with extensive qualitative methods and a case study. I guess it kind of balances between them because of the fact that so much of the environment which they studied was constructed by the researchers. The approach did seem beneficial for them though and was good as a first study to be followed up by a study that results in a bit more theory.

It was interesting for me to choose an article that had methods I didn't quite know about which I did for the qualitative methods part of this theme. Understanding more about different methods is interesting and learning about discourse analysis and ethnography was helpful to grasp how case studies work. The thing that seems to me like the hardest part of a case study is how to choose which data to gather and how to process it afterwards. Understanding two additional methods made it easier to understand how more types of data can be gathered and analyzed. 

Theme 5: Design research (After)

This weeks theme made me think a lot more concrete about what to do and think about when you are about to conduct a study. I have been thinking a lot about what I should do for my master thesis and looked upon the articles for this theme with that perspective. The lecture by Haibo Li surprised me a bit by not talking so much about using prototypes as part of research but rather about stepping outside of the box to look at your problem in a new way. It made me more confident in an observation I made about the research by Sellnäs. What if she would have skipped including the haptical interface in her study abou group work? The haptical interfaces have already been studied themselves and putting her research in a future scope where these interfaces might be a lot more complex I think the research might just as well have been conducted with a "real" physical interface. She could have had an actual wooden box with wooden geometrical figures in it and if the blind student needed guiding the sighted student could just grab the actual figure that the blind student is holding on to to lead the way. I am not perfectly sure that this would be a good way to do it but thinking about what Haibo Li said about not making your experiments too complex makes me feel like there is at least something to it.

The idea that Haibo Li talked about, that sometimes you are trying to solve the wrong problem was also something I found interesting. I often feel like I am locked down by my presumptions when thinking about a problem and then after a while, sometimes a long time, I come upon something that changes my  perspective and suddenly the solution just appears. Oftentimes what happens then is that I exclude something from the problem or regard something to be less important. I've learned in a project course a few years back that it is a good approach to "kill your darlings" when coming up with a great solution to a problem. That means that you after you've worked on an idea that you think is great but is not really taking you all the way or progressing to slowly you completely abandon that idea and start again from scratch. I guess that what happens a lot of times when you do that is that you force yourself to look at a problem from a completely different angle and redefine the problem. Maybe trying to come up with different, contradictory, sets of definitions for a problem could be a good approach to more easily find these different solutions to a problem.

Theme 6: Qualitative and case study research (Before)

Qualitative methods paper

"Living the VirtuReal: Negotiating Transgender Identity in Cyberspace", Marciano, Avi; Journal of Computer-Mediated Communication (2014)

This paper is about how transgender individuals use the internet to extend their social network and/or the space within which they lead their lives. The author uses discourse analysis and virtual ethnography as analysis methods. I think this paper is interesting because of its emphasis on the implications of technology and I thought it would be interesting to read a paper with qualitative methods which I am not confident with.

The author studied content and commentary on a popular webpage and forum respectively. One of the foremost benefits of doing so is that the author could get information on what people say and think when they are acting within a natural context. The author won't affect the study "participants". However, there is no possibility of asking for clarification. Further, observing multiple instances of the same kind might prove hard. Specific questions might have to be abandoned or accepted to have unspecific answers based on a small representation of subjects. 

For me it was interesting to understand that even with methods that seem purely analytical at first, there is often an element of quantifying. Like the categorization in the discourse analysis method. I am starting to see that there is a relatively large grey zone where qualitative and quantitative meet or maybe overlap.

I think this paper could have been improved by making a survey or interview people to see more of how they potentially differ in personality between offline and online and to provide deeper understanding of some of the issues that come up. However that would be beyond the aim for this study. 

------------------

Case study paper

1. A case study is a broad range, often open ended study that involves several methods of gathering data. The study is conducted by choosing a few places where the topic of interest can be observed. The study may include experiments that are set up by the researchers but it might also just be the "natural" situation that is being observed in several ways by the researchers. It is good to chose different places that represents different categories or qualities of the topic of interest, each of these are one case. The gathering of data may be anything from structured to unstructured  or both and may change during the course of the study as new phenomena are observed. 

I chose the paper:

"Student experiences of virtual reality: A case study in learning special relativity", McGrath, Dominic and Wegener, Margaret and McIntyre, Timothy J. and Savage, Craig and Williamson, Michael; American Journal of Physics, 78, 862-868 (2010)

This paper is about using a visual software to help university students to understand special relativity phenomenon. The study is divided into two major cases, one at each of two universities, but is partially subdivided by the different categories of students that are taking the two similar courses (the setup of the courses differ a bit between universities).

This is one of the weaknesses that I observed. Since they basically only have to cases they cannot compare the impact on different styles of education. It would have been good with one or two more universities or other courses of similar content at the same university. However their results do not present complex theory but rather binary theory, basically if their visual software will or will not help students in their learning process. They do use the fact that they have different categories of students to help them to their conclusion. But stronger theory could possibly have been achieved with more cases.

They use multiple ways of gathering data like observation, questionnaires and interviews in a structured way with questions integrated into the laboratory task. They also compare results on the final exam between students who used their software and those who didn't. This combination is one of the strengths of this study I think, since they are able to verify the preliminary results from the laboratory sessions with the comparison of exam results. Some of the data is quantifiable and is therefor subject to statistical analysis which helps them further in building their theory.

They do very little to extend their theory by investigating reasons behind the fact that student seem to benefit from using their software. They retrieve some data that points to how the learning process continues for students who participate but no attempt is made to understand the mechanisms. Neither do they extend their theory by referencing to previous work that might explain why visualization, or some other part of their experimental setup, will help the students.

Theme 4: Quantitative research (After)

For this weeks theme I read the articles "Perceptual evaluation of violins: A quantitative analysis of preference judgments by experienced players" and "Physical Activity, Stress, and Self-Reported Upper Respiratory Tract Infection". I also read about different qualitative and quantitative methods and approaches on the course web page and extensively on wikipedia.

I found the seminar of this week to be the most valuable part of this theme. It was a fun and good way to learn a lot more about how to relate to quantitative research. What I feel like is the most important lesson is that the research design down to the smallest detail is crucial for the results obtained. There are so many factors to consider when gathering data and the result is bound to be biased or useless if only a few factors are taken into account. It seems to me like it is a good idea not only to test your study in small scale before deploying it, as we were told in the seminar, but also to use some sort of focus group to come up with problems and ideas for conduct when designing your study. By discussing benefits, disadvantages, possibilities and limitations in group in the seminar we came up with a lot of ideas that I wouldn't have come up with on my own.

The article I chose had implemented a combination of qualitative and quantitative methods which came up in the articles and reflections for next weeks theme. I think this was another valuable lesson. When conducting a study based on qualitative methods adding a quantitative part will greatly help in discussing and clarifying the results.

Some issues regarding how to design a questionnaire also came up during the seminar. I have many times been irritated when answering questionnaires that do not give me the options I'd like to have or where answering is simply to tedious. I've thought that this is just a natural part of questionnaires in general and that designing these is at least as tedious as answering them. But I'm starting to think that it might be really interesting to design a questionnaire. They rely on so many psychological factors about stuff like how to create unambiguous questions, driving motivation for participants to answer and what questions to include. What is relevant, necessary and sufficient questions in a questionnaire?

Theme 5: Design research (Before)

Relating to the article by Haibo Li et. al

For a lot of the research in media technology I believe that there is an intended recipient, or user. In other fields the technology is not inherently user centered to the extent that is the case for media technologies. So when researching media technologies the best approach is often to retrieve qualitative data from relevant research participants. Usability, attitude, human perception and cognitive abilities are the key concepts of research in media technology I think. So research will have to take these into account to successfully gain new knowledge. A good approach to attaining the data for this kind of research will a lot of times be to present a prototype or concept idea that can be directly evaluated by participants in a study. The research can be designed to test a small part of an upcoming prototype or the whole prototype itself. The idea will however be to identify attitudes and constraints to further base the research on. The data collected when evaluating media technologies will, I think, always be connected to the concepts above mentioned. In a combination of observing, asking questions and collecting usage data from the participants a broad range of qualities can be investigated.

The main advantage of designing a prototype for research is probably that it makes it possible to test for multiple attributes at the same time and also to test in an environment that is similar to that of the intended use. Like for qualitative methods in general a prototype can give unpredicted or unintended information. It is hard to foresee all possible obstacles or constraints in a research so by using a prototype this unintended information may be valuable for adapting the research or for further research. It is a natural part of an iterative process but that might also be seen as a limitation to using prototypes as a method. When starting off a research in this fashion the iterative process must be embraced and every research step must be expected to incrementally work it's way toward the goal. Of course every step will provide valuable information but one cannot be sure that it is generalizable.

Relating to the articles by Eva-Lotta Sallnäs et. al

I found the conclusion that introducing audio to a haptical visual interface improves performance interesting, but not surprising. It seems quite natural that an interface that more closely replicates the "real world" will be easier to grasp. I wonder what kind of performance boost they would get by implementing actual 3d sound. It seem quite possible to do so I think. If the collaborating participants wear headsets they could talk to each other through those will still hearing 3d sound. They could even position the voice of the collaborator in the virtual audio space.

It was also interesting to see that visually impaired and sighted people effectively could collaborate. But it seems that the sighted people had little to gain from this group exercise. So I am wondering, which could be questions for the lecture, to what extent is the haptical interfaces relevant for group work? Are the results to be seen as implications for how to perform group work in general or only in the situation with a haptical interface?

The collaborative experimental setup will make it different from an individual setup in a few ways. First of all there are some technical and measurement constraints to this approach. Communication in the group, especially in this case, is important and thus participants cannot be hindered in their communication by for example headphones. Also they will be positioned at different places in the experiment scene which might affect the results. The gathering of data must take all participants into account and gathering data on the communication between them is essential to explain the result. Also, people may act differently in a group than individually which will have to be accounted for.

By combining qualitative and quantitative methods one will acquire a setup that will be able to show a correlation and provide the information needed to explain that correlation. The quantitative method will act as support when formulating the main theories which in turn are derived from the qualitative data. In the articles by Sallnäs et. al they used timing and event data to show correlations and then explained them using the information they gathered from interviews/comments and the observation of participants. To extend this it will also mean that as the research is conducted the qualitative methods will provide deeper understanding and the possibility of adapting the study, eg. follow up questions, and to identify new factors that are of interest. If the experimental design and the gathering of data is more extensive than to account only for the expected results than the new factors identified will also be able to be supported by quantitative data. So by combining subjective and objective methods a study with greater dynamic is achieved.

Theme 3: Research and theory (After)

For this theme I have read the articles assigned by the course, the article of my choice and general "stuff" on the webpage of the journal which I chose to write about. I based my understanding of the article I chose on my previous knowledge of the acoustical field, so no further reading was necessary.

What I have learned after this weeks theme is that theory seems to be everywhere and that theory in a scientific context is not just the theory which is proposed but also all the underlying theories on which it is built. When I was reading my article I was frantically looking for theories and with some struggle concluded what was their main theory. It was much harder than I thought to distinguish hypothesis from theory. Also, in my struggle I failed to recognize all the underlying theories at work in the article. Things that are taken for granted in the argumentation and the formation of their hypotheses. Acoustical theory, signal theory, theories regarding perception and regarding human evolution were some of the theories that I disregarded since it was not what they were emphasizing.

A thought occurred during the seminar that I think will help me to further understand theory and that will "keep it in the loop" in my mind. That theories are a natural part of language. Concepts are in a way theories of that which it conceptualize and so in that way we speak in theories.

In the process of trying to understand the essence of theory I made an attempt to define theory. I think it was quite helpful too in my learning process. I am by no means completely satisfied with this definition but I believe it at least provided me with a framework for thinking about theory.

My def: of theory;

Logically sound argumentation proving causality between events based on premises that are fairly assumed to be relevant and sufficient for the argument.

Theme 4: Quantitative research (Before)

Article of my choice:

Perceptual evaluation of violins: A quantitative analysis of preference judgments by experienced players

Saitis, Charalampos and Giordano, Bruno L. and Fritz, Claudia and Scavone, Gary P., The Journal of the Acoustical Society of America, 132, 4002-4012 (2012), DOI:http://dx.doi.org/10.1121/1.4765081

In this study they conducted a set of two experiments. Both where based on observing violin players in the process of evaluating different violins. In the first experiment the violin players were asked to freely play a set of violins and order them by preference. They repeated the ordering process multiple times and on multiple occasions. This way a lot of data could be gathered and then both individual consistency and agreement among participants could be analyzed. In addition a qualitative content analysis was performed. Participants where asked to describe what features they thought where important for their evaluation.

For the second experiment the features which were mentioned by most participants, plus some features common in the scientific literature on violins, were the only ones allowed for evaluation. Participants were asked to grade violins for each of the specified features. Again multiple trials were made.

The result of these experiments showed that violin players are very consistent in their own evaluation and that they agree on what features, or characteristics, are important in a violin, but that the perceptual evaluation differed among them so that no consistency could be achieved. 

I think that they nested the relatively qualitative procedure with their quantitative methods in a clever way. However the method in both experiments might be lacking by having no consistent way of playing the violins among all participants. A study before this one on what ways of playing gives greater agreement among participants, without losing overall individual consistency, might have been a better start. Letting the players play in whatever fashion they liked might have produced the inconclusive result of this study, since perceptual evaluation might differ with playing style. However, forcing a playing style or repertoire on the participants seems unnatural in the sense that preference of a violin likely is linked to the participants preferred way of playing.

A further investigation on the importance of each of the characteristics specified might also be interesting. It is possible that one or more characteristics can be broken down into a set of features and that agreement among participants could be acquired for one or more of these features.

Physical Activity, Stress, and Self-Reported Upper Respiratory Tract Infection

The result that I found most interesting was that it seems like if you exercise (extensively) it is more beneficial to be stressed than to not be stressed for reducing the risk of getting URTI. My initial thought was that stress would be positively correlated with URTI and negatively correlated with physical activity, thereby reducing URTI with an increasing amount of exercise.

Perceived stress was said to have a small confounding effect on the risk of URTI. I guess however that this was analyzed in general, so with the people who reported high stress having higher incidence rate than low stress for low exercise and lower incidence rate than low stress for high exercise the effect overall is the same as for low stress. Then couldn't the main result be compromised by a psychological factor? People who are highly stressed and high intensity exercisers might be people who are less likely to report URTI, due to the fact that they are high performers who don't have time to listen to their bodies. This also correlates well with the fact that they ".. saw little effect of physical activity for URTI with systemic symptoms", since a fever would be hard to ignore for anyone.

Which are the benefits and limitations of using quantitative methods?

They give more precise and reproducible results. Designing the study is based on identifying factors which you can measure and then measure them many times. This can make the study easier to design but might also make it harder to conduct since gathering extensive data might take a lot of work and/or be costly. Quantitative methods also gives statistically more significant results, because of a greater number of observations, and the possibility of discarding a hypothesis if no correlation is found. If a correlation is found, however, explaining that correlation might prove harder than with a qualitative method since less information generally is available.

Which are the benefits and limitations of using qualitative methods?

Qualitative methods have the major benefit of giving more depth in the answers to the questions asked. They may produce results that could not be predicted or give a greater nuance in the answer than expected. Since interpretation and reasoning are inherent in a qualitative study it will be colored by the observer in a way that may steer the results further from the truth than what would happen in a quantitative study. A qualitative study may be harder to reproduce and will generally give a statistical significance with less power, mainly due to less data and the complications of adjusting for confounding factors.