Research methodology workshop

Archer: future DDS plans, based on DDS survey, talk with Magia and Stephanie
Saw a number of requests for biweekly DDS- they leaned away from that, thinking there are too many topics to be biweekly
It was very helpful to see the relative usefulness of topic according to students, but an outside perspective is useful- some topics may not seem useful now, but will be in the future
Next year: 10:30-12, but could revert to 9:30 if that creates course conflicts
Tweaks for next year: second year students will become the session organizers, which fits with the goal of the seminar in terms of training. One senior student will be there as well
Ten sessions planned per semester in the future
Identified 15/16 core topics, with others as less core- can be substituted for others, replaced, or covered every other year (handout given)
So, 16 core sessions out of 20, will be left up to student organizers to decide what will fill in the remaining 4 per year
Want to continue with the wiki, although it will look different in future years, now that topics are established, hope it will become more topic than date based. First year students will be more responsible for wiki content
Brett: has the survey influenced any of this? Is it a final decision?
Archer: this is a working decision, modifications are welcome. The survey largely dealt with topics, and was influential in identifying the core topics. We should meet to talk more later, in depth (those of us who are interested in this).
Need to find a 3rd or 4th year student to supervise, to determine the topics over the summer for doctoral committee approval (the additional 4 & 5). So people who will be 2nd year students need to start talking about it. Talk to me later with additional thoughts and questions. I don’t want to take up to much time for today’s session.

Presenters:
Emilee, Rick, Marianne, Cory, Jude, Jina, Yong-Mi, Xiaomu

Split up into 4 clusters, rotate through the groups

Emilee and Rick:
Emilee: getting data from videos- she watches them all and writes a little synopsis for each, but in terms of pulling out data, just grabs a couple of things, because its so time consuming.
Rick: large data sets- getting the data takes a little technical skill, but not too much. Important to look at terms of service for website- facebook prohibits use of data (but app developers can get data from their apps).
Emilee: you’ll never have the entire database for any of this- some people protect their stuff
Rick: you have to think about biases that may result from this- what are people making visible and why
Emilee: for qual data analysis, talk to Jina. We did quantitative stuff
Rick: the hardest part with big data sets is the large amount of noise. We spent a year throwing methods at our data sets- all that work never got published because we spent so long trying to figure out what was in it
Emilee: I have a bunch of CTools data, there are many people who made sites and only used them for a few weeks - what can it tell me without talking to people? You are looking at traces of people’s activity. Might not have all of the context you need to interpret things
I saw a paper at CHI of Youtube videos of iphones. He analyzed comments but didn’t read them all- stopped when he started to see repeated themes. this may be a good methodological point. You may also want to think about sampling at different times.
May also want to look for certain keywords. Use a database to search through it
Rick: we used mySQL, which made it easier to search and try out things- you have a lot of dead ends before you find something that works. Its also good to have others to bounce ideas off of.

Jina and Yong-Mi:
Yong-Mi: experience sampling method: bug people several times a day to see how they respond.
We are emailing people 5 times a day over 3 days, they have to respond to at least 3 of these.
We are screening for heavy internet users to look at credibility related behavior. We will have 9 to 15 responses from each person
Ted: we are doing work with medical students looking at self-regulated learning. Asking them every 2-3 hours, what have they been looking at, what have they done, may be a good way to go.
Yong-Mi: there were some recent CHI papers on this method
Jina: I am still being explorative with methods- there are a lot of pros and cons- it motivates me in terms of research, but in terms of publication it can be hard to justify in writing if there is not a lot of literature to support it. On the other hand, it can make you a more interesting person for your community. My use of methods is more research than course driven
With a method I am using now, I ask people what computational methods they use to manage personal information and they draw it. I piloted it with a few variations on that question to learn which question worked best. In interviews, you have to be linear, but in drawing, you get a holistic view of what people are doing. One person in my study drew a “desktop” while another drew cloud computing resources. From the drawing, I ask reflective questions- what did you mean by this? Brenda Dervin has a critical incident technique where you ask a broad question, draw a timeline of critical incidents, then triangulate. I think this is a similar technique.

Cory and Marianne:
Cory: an implication of this is you can’t do a method before methodology- it is an implicit choice of methodology
A fundamental choice: are you a methodologist or a phenomenologist—if you have a preferred method, you go to the world looking for things through that perspective.
Marianne: or you could say, I am interesting in certain things and this is how I think I can get it.
Cory: ask yourself, why are you interested, why do you think its important, do you think this event is an example of a larger class of phenom- why is this a good example?
Marianne: think about, what is my particular path to this data- ask yourself, decide this stuff before you start
Cory: the famous 10 questions in the doctoral student handbook are a good place to start, especially the first few questions
Thinking about these positionings can lead to methodological ideas.
“Situational Analysis: Grounded Theory after the Postmodern Turn” by Clarke, a book they recommend. Published in 2005, and has been used in 3 dissertations since then. Looking at the use of methods in different places is a good idea.
Cory: I read Steve Jackson’s dissertation for structure, since he and I do similar research, and it gave me a better sense of how to put my own work together- especially to see how long sections were—I can easily write 2 single spaced pages about x. This educates you about the structure of what you are doing, and helps with cognitive understanding of, this is something I can do.

Xiaomu:
In my observations, I don’t really regret not taking very detailed notes at the beginning, since I was gaining an understanding of the field. If it is a pattern, it will recur later. Most work comes out of analytical notes rather than detailed observation notes. In the first few weeks, observe as much as you can, record data in great detail later, once you have a clear idea of your research questions. Mark pushed me to record everything at first because you did not know what you were going to focus on later. At first I tried to record everything, like when nurses prepare medication, because it was very interesting to me, but then I realized I wasn’t going to use it. There is one area I would have liked to collect data, but its too late now because the artifacts are no longer being produced. But I still have observational data about how nurses used them.