Home » research
Category Archives: research
We need more instructional design cases written by those who did not actually do the design, but where to start?
I suggest interviewing the designer. Elizabeth Boling did the same in her case about the Alcatraz cell-house audio walking tour. I have done interviews twice now to begin design cases about designs I did not create, and I share the protocol I developed here. This protocol was originally developed from reviewer questions in an article where I belabor all the difficulties I went through doing my own first design case. Of course, the instructional design case doesn’t begin and end with the interview. A good case will also have a few images of the design, the authors experience of it, maybe a few quotes from others’ user experiences, and the author’s reflection about why they were originally drawn to write the case. But of course, start with interviewing he designer.
Interview Protocol for Instructional Design Cases
- Telling them you are recording AFTER the recorder starts, so you get their agreement on tape
- Explaining that a member check will go out to them prior to the article’s submission for publication.
- Mentioning the time it takes– 90 minutes seems about right in most cases.
- Explaining that the markers and paper are there for exploring, please use them when needed. The notes are drawing from teh designer used to explain their product or process are important parts of the interview!
- This meeting is about the instructional design, not necessarily about the success of the design.
- Most importantly, Thank the designers for participation in this process.
Situating the design context and process:
- What were changes in context which motivated the design? Something must have happened that brought this design about.
- Who was the design team and what were their influences? Can we assume that the different members of the design team had different goals? Was that discussed? How were those decisions made?
- Did you initially intend to have students create a game? What were those key decisions? When did they happen?
- Can you describe the process by which you came to the initial formulation of the design?
- As you reflect on how you created these learning opportunities, what were the pivotal moments during the formulating of the instruction, the ah-ha moments or innovations, that you would want to tell someone else, who might be considering doing something similar for their learners?
Describing the design:
- Can you map out all the parts, especially the invisible ones, which someone viewing this teaching intervention might not see from the game itself? [Point to markers / pencils / paper]
- What is particularly interesting about this instruction?
- If you were to name the instructional design, NOT THE GAME, what would that name be?
Depicting the experience of the design:
- Can you describe the user experience? / How was learning measured, or not?
- Can you tell me about any unforeseen obstacles or aspects of the design that needed revisions that you only found out about after decisions were made?
- Did you try anything out, or consider anything, that was deemed in the end to be a bad idea in retrospect?
- How has this instructional design created complexities or challenges in your teaching? Has the instructional design failed anyone? TA’s, students, not met your goals?
- Have you skipped anything for simplicity’s sake? This can often trip up a design case because often what was skipped may be rationale for design decisions.
Designing identity is what we do when we talk about education. (not talking about instruction– that’s something different) Once we dig underneath a collection of performance objectives, and the billions of exceptions and complexity that accompany them, what we are really doing when it comes down to it, is designing an identity that we hope learners will assume. PhD programs help people become scholars, ID programs help people become instructional designers. EFL programs teach people to identify as English language learners. In truth, underneath much of educational research, is the true objective of just trying to figure out how to help people to become someone they were not when we first encountered them. Of course saying that would be controversial, so we say other things.
This process is everywhere, down to the simplest experiences that turn children into functioning adults. My daughter is afraid of slugs, and is afraid of riding a bike. It seems there is not much I can do about it at the moment, nor does it bother me in any way that she is afraid of slugs and can’t ride a bike at age 6. It’s cute. At age 24, I don’t think it will really be so cute anymore. I like her for who she is now, and I want her to be who she is now. For as long as she is 6, i’d like he to be happy with being 6. But no matter how much I love the girl she is now, she’s going to be someone else tomorrow– and I want that person to be someone who is not afraid of slugs and can ride a bike. That’s an important change; and most of it is identity. As soon as she decides she isn’t afraid of slugs and can ride a bike (I know she can ride a bike because she didn’t fall over until she realized I had let go) she will be those things. As long as she thinks she is x, she will be x. We’ve got lots of words like “self-efficacy” to mask the truth of what we’re doing when we’re teaching.
I see this process of change in our graduate learners. I recently wrapped up the final course for a new batch of graduating instructional designers. One candidate I saw present on an LMS implementation (Fred Kelly) just today and another has already moved on to the ID job she came to the program to get, Janelle Galbraith. At one point in this paste semester I could see the change happening before my eyes. In a talk about revising websites, Janelle recounted her design process. There was a point in which a certain instructional image had to be redesigned. I could see it had gone through extensive development. I took a guess at how many iterations, I guessed four. She replied, “HA! Try 18.” The next part of the interchange is what has been eating at me for weeks. I asked, “Wow, what kept you going beyond four?” and she answered, “because it didn’t work yet and that’s just what you do.” It’s just what you do when you are a designer, but it sure isn’t what they did when they started the program. I would argue that keeping going when it’s not working is surely not what you do when you are an eye surgeon. It’s a good thing I don’t teach eye surgery.
I went through the same identity change myself. In 1990 I inherited a tutoring gig from Steven Pfaff as he left Karl-Franzens Universität Graz, Austria, to go to NC Chapel Hill to start on his doctoral study. He had been teaching using magazines, but that wasn’t my style really. I started using comics. If you have ever tried turning comics into cloze activities, you would quickly realize this is likely the worst textual choice for foreign language learning strategies one could imagine. Comics hinge on providing unexpected turns of phrase, not the ones the reader is prepped for. As discouraging for the serious learner as it might sound, I had a good time teaching, I think my learners had fun talking about the jokes, and I had a fair bit of fun choosing which words I would block with white out before I made the copies. What I didn’t do is see myself as a teacher, and surely not as an instructional designer. It was too much fun to really view it as work, per se. It took six years, a few student teaching placements, a comps exam, a number of academic papers, a whopping student loan, and then a full time teaching job to finally see myself as a teacher. Then, years later, it took a PhD and a stint with Microsoft to see myself as an instructional designer. All that to get back to where I was at 19, albeit with some identity. Would I still make a cloze activity out of a life is hell comic by Matt Groening? Hell yes. But I would do it over 18 times until it worked right. All this schooling and learning didn’t in effect change what I was doing; it changed how I was doing it and how I thought about what I was doing. The end goal of school is to make you someone new, and that’s painful because, now I would totally suck as an eye surgeon. So yeah Mr Groening, school is hell.
Years ago Elizabeth Boling mentioned this classic instructional design, The Nutshell Studies, and I just had not run into a thorough discussion of it for years. It is an instructional design intended to support the learning of forensic science for detectives. The most remarkable part of the design are 19 miniature dioramas which make lifelike crimes scenes available for viewing by multiple detectives in training. A perfect instructional design case for a historical issue of IJDL, but the dioramas, and the life of their creator Frances Glassner Lee, were recently featured on an NPR show called Sidedoor, and in all honesty, Tony Cohn from PRX does an excellent job, hitting all the required components of a design case. As AECT is just around the corner again, I am sure I will be talking about instructional design cases again in the near future, and this Sidedoor episode is a case worth discussing.
Frances Glessner Lee and The Nutshell Studies of Unexplained Death, is actually the name of an art exhibit, but is spawns from an instructional design, from which Cohn crafted his audio instructional design case. Mrs. Lee created complex dioramas with such detail that they could tell the story of an entire murder, well, almost the entire story. A curious aspect to her dioramas was their purposefully unsolved-ness. There is no solution offered to the detectives in training. In other words, despite their intricacy, they will always remain, unsolved. This 1940’s design choice of creating unsolved murder scenes, fodder for discussion but not for resolution, speaks to the persistence of discussion, rather than resolution, as being an ever-present aspect of higher order thinking. The key is not to select the right answer, but to untangle the complexity and multiple possibilities that could play into each scene. Of course, the hope is that with practice, learning the process of disentanglement will result in better investigations, leading to a higher chance of getting things right eventually. Once we let go of the idea of always getting is right this time, and focus on getting it right most of the time, our chances go way up each and every time.
Learners in the Master’s program in IT here at UT chuckle about my choices of terms: grapple, dig in, go deeper, dissect, take apart. But these terms do tell what I am trying to accomplish. I don’t think we’re ever going to learn definitively how to teach (sadly) and I don’t think we’re ever going to definitively determine the right way to design instruction (woefully), and , that’s not the point (joyously). The point is, to see a design like the Nutshell Studies for more than just the dioramas, though, for sure, the dioramas were pretty damn cool.
Since stopping this blog in February, ideas have come and gone, and my apprehensions about leaving Texas A&M worked on me to focus and prepare for a new start. That start has come; it’s September and I am teaching in a new context. While I am very happy to be here in Tennessee, I am also trying to work out exactly where is the best place to invest my time, thus the new banner image and a renewed concern over the clock. One thing that is not in question is the purpose of this blog. It’s a place to work ideas out in a more public venue, allowing for comments from anywhere, and from anyone. This month’s post is such an opportunity. It is inspired by a dialogue that came about organically from a posted comment by a reader.
The issue I am dealing with here is an old one, but as Marshal Poe writes, our perspectives on right and wrong are very much tied to the times- the way we communicate and who gets to communicate to whom about what and how often (2011). Had this issue arose at my workplace, I probably would not find out about it because of the nature of the issue; some topics can be discussed on a blog at another university that would not be welcome at one’s own institution. That’s one area where the blog excels. It creates a fruitful venue to discuss topics outside of one’s own institution.
This discussion started as a comment to Publishing as a graduate student and developed into something worth a little more space than the comment trail, I think. What follows is the email discussion with a few headers added in after the fact. Eric Hufford, the author of these questions and the unfortunate graduate student in the pickle described in this post, is a senior software engineer in Maryland with 10 years of experience in the field of software engineering. He holds a B.S. in Computer Science and is pursuing a Masters in Software Engineering as well as an MBA.
Eric: Thank you for this fantastic information. I do have one question for you though, as I am struggling with this before I submit my paper. I wrote a paper for a graduate course. The paper I plan on submitting is exactly as I turned it in to the course with absolutely zero input from or collaboration with the professor. In fact, his entire involvement with the paper was approving the topic (it was a term paper) and grading it. Yet he is telling me that since I wrote it for his course, he needs to be listed as an author. I strongly disagree with his statement. Is this something I should be concerned about? Do I really need to add him as an author?”
Craig: Wonderful question and thank you for submitting it. I will use headers to organize and emphasize my points, because this response is lengthy.
Know the authoring conventions for your field
There are a few things that need to be clarified prior to saying your professor is outright wrong. First off, this blog post was meant for graduate learners in Instructional Technology, but might also be broadly applicable to social science research in general. I don’t know your field of study. Much depends on the conventions in your field. If your paper was developed from someone’s course which included access to a lab where data was collected, it is very likely that authorship listing the lab leader is the most appropriate convention. Also, if the paper topic was developed iteratively with the professor, that in itself may very well constitute input worthy of authorship. Your professor might also be refereeing to content in the paper that was derived from teachings in the course, in which case the professor would be making a legitimate claim. Both contributions and conventions should be discussed prior to any submission of the article anywhere. Graduate students should inform themselves of some of the more general conventions using authorship guidelines published for their appropriate field. There are guidelines out there for determining authorship. In my field, Instructional Technology, AECT has some, but more generally, there is an APA guide online. Read it closely. Knowing these conventions will help you learn more from your discussion about authorship with your professor. This is a key learning step in graduate school and one often overlooked in the process from admission to graduation, and possibly the root cause behind the uncomfortable feelings expressed in the comment.
Course papers are never journal articles, they are, at best, first drafts of journal articles
Secondly, one phrase in your question strikes in me a note of caution, “The paper I plan on submitting is exactly as I turned it in to the course.” This strategy runs the risk of getting a swift and straightforward rejection, needlessly uses the time of an editor at the journal you submit to, and moreover, runs you the risk of sparking a negative image of your work in the eyes of the editor. No paper should go to submission raw. Now that the course is over, re-read the writer’s guidelines of the proposed venue, and make sure you are familiar with the prospective journal’s most recent publications on the topic you cover. I expect there’s a fair bit of editing to do. I have heard students complain that established professors can submit less polished drafts than students and get away with it. I am not fully convinced this is true. Scholars that have been dealing with a topic for many years may have a battery of key phrases and can use shorthand to express concepts and other ideas. A learner’s text faces the formidable obstacle of being a relatively new topic, and without the crafted language at the tips of your fingers, it must be well crafted though scrutiny and revision. You haven’t been reading/writing for enough years to know them all the phrases you need, and a facility with them only comes with extended exposure.
Collaborations must be mutually beneficial
More strategically, it would be wise to rethink your take on collaboration in the field. From the perspective of the student, credit for a single-authored publication is paramount. However, from the perspective of scholars in the field, it is likely not so. The co-authored paper is worth more, not less. When a potential candidate’s cv evidences multiple collaborations that resulted in publications, it sends a far stronger message of competence than a cv with a few single authored published papers. Scholars publish dozens of articles over their career, and their co-authoring relationships evidence the breadth of their participation with topics of interest to others doing work worthy of publication. There is much to gain from collaboration and showing that you work well with your professors, even if this particular professor does not spur in you the desire for a long term identity of association. You have far more to gain from the professor’s name being on the byline than you have to gain from being sole-author, even if you plan on never co-authoring with this professor again. Your paper is also more likely to be read because inclusion in the professor’s online archive may draw readers surfing your professor’s publications, and it may also heighten your rank in internet searches if your professor is more published than you are—which is more often the case than it is not.
At this point, your task is to get yourself informed about the appropriate conventions for authorship, make strides in the development of the paper for publication, and bring the draft back to your professor to have a more informed conversation about the draft and target venue. It may feel like eating humble pie, but it’s not. Even if you are vastly more experienced and it is a junior faculty member you’re dealing with (which does happen as grad students come from all walks of life and faculty can come straight from successive years of schooling) what you do not want is to publish the paper without the professor’s approval, especially with verbatim sections from the paper you submitted in the course. That runs the risk of you being marked as someone who does not give credit where credit is due, could alienate you from other scholars in the future, and potentially endanger your completion of the program where you are now.
Eric: The paper in question was an end-of-term paper for a Software Engineering course, and I have scoped out a few different journals and honed in on two that I feel may be a good fit. I plan on re-tooling the paper to fit within the journal guidelines, but other than that, I plan on submitting the same basic paper. That is to say, I do not intend on taking any input from the professor’s review before I submit it. It sounds as if he is attempting to take credit for his student’s work even though he had no involvement in the process of creating the paper. [He only taught the course in which the paper was submitted.] Do I really need to add him as an author?
Craig: Yes, but more importantly, why fight it when the outcome (s)he proposed actually favors you? Nevertheless, the issues you raise here are endemic to graduate school, and actually, really important. Every year this issue pops up somewhere- often between a junior faculty and a senior doc student. I really think pushing this to a richer discussion might uncover something valuable for other grad students and those who teach grad students. So, let me ask you this– do you think it was something in the classroom dynamic that lead you to want to publish the paper without his name on it? Is there something the unnamed faculty member didn’t do that s/he should have done? What brought you to so fervently want to publish it without his name on it?
Eric: I’ll answer your last question first before going back to your first two. “What made me feel like I didn’t need to include him as an author?” I suppose it is relevant to provide some background information on myself. I am a professional software engineer with about 10 years’ experience in the industry. I finally decided to pursue a masters’ degree and an MBA. This particular course was entirely online so I never really developed a rapport with the professor. While he is a senior faculty member and appeared to be a fairly competent fellow, the course content was nothing new to me. I never opened the book, nor did I ever have any conversations with the professor beyond those required to complete the course (like approving my term paper topic). In other words, the class was a breeze, I received an A and moved on with my life. With that said, I felt that it would not be necessary to add his name as an author because he and I had a very minimal amount interaction, and it was only ever via email. There was never a relationship or any sense of loyalty. However, I try to remain open minded as best I can, which is why I came across your post and asked my question. I simply didn’t want to do the wrong thing and have it come back to bite me later. I was also under the impression that if I shared authorship, that my involvement in the paper would be diluted, which I assumed would reflect less upon me. In your response, what convinced me to include his name on the paper regardless of how little he did, was your argument that it looks better on me to have a co – authored paper. You mentioned that it would make it look like I perform well with others and, more importantly, if he is well published, my paper will get more visibility than it ever would under my single authorship. This may sound heartless and blunt, but these are the things that matter most to me about publishing my paper. Your answer surprised me, it wasn’t what I was expecting at all, but it made a ton of sense.
Craig: The online dynamic raises lots of curious questions.
Unspoken information lost in the lean media
Low-interaction online courses can be a breeding ground for conflicting perspectives, especially if they appear early on in the program of study. In order to create a smoother pedagogical experience for both learners and the instructor, some programs require foundational courses that may fall below what an advanced learner is expecting. In such cases, the indebtedness of the learner to the knowledge of the professor is a stretch for advanced learners to recognize, and surprisingly often, that lack of indebtedness is completely justified, especially in online classes where the direct instruction is limited. In some online classes, it’s not hard to imagine just anyone teaching the course if it basically runs by itself with little instructor-student interaction. A close working relationship is implied by a shared byline, but in a course where the interaction has been so little, it is bound to feel weird to share credit. At the same time, it is difficult for the instructor to recognize the insights did not come from the course, especially if those insights could have been inferred from the materials assembled by the professor. I wonder if such a misunderstanding is the case in your context.
The lean media of online instruction, often limited to text interactions, often requires a seasoned online instructor to recognize insights that break from course materials into new ground for the learner. Junior faculty with limited online experience, and even seasoned faculty who are new to the online format, may be relying on cues that simply are not there in online interactions to recognize when a learner is breaking from simply what they have been taught, into original research. He may well have thought you were simply building from the given materials. Having not opened the book puts you at a significant disadvantage when arguing that you didn’t learn x or y from a course of study, and at the same time, sometimes it takes a seasoned online teacher to recognize truly original work.
Differing expectations, none of which are written down
On top of this, these instructors may be working with a hidden curriculum that is never made explicit to the learner. Online faculty may have tacit expectations of a set of rules of interaction leading to co-authorship with grad students, and at least in my experience, rarely are these understandings made explicit in a course syllabus or other materials that the online learner is exposed to. With online learning we generally default back to the face to face rules until people figure out a better way to proceed and new conventions are adopted, but we rarely make it as explicit as it should be. Who wants to read, nonetheless write, a set of conduct rules? In graduate programs, for all the documents we produce, there are still topics navigate via intuition instead of explicit guidelines. How often do graduate students read the handbook anyway? I know I did not labor of over such a document until an issue arose. While I would say it is the responsibility of the instructor to make those rules transparent as best they can, the large number of exceptions makes that goal pretty far reaching and arguably infeasible, especially for someone who is not used to teaching online where there is far more emphasis placed on documenting procedures.
Advanced students may also not be aware that in the context of the professor, they may feel they are working at a disadvantage to their peers. If a professor teaches exclusively online, working relationships with students may be very challenging to construct. I know that in my experience at Texas A&M, it was definitely a challenge. In many schools, professors are expected to publish with their students, and for a faculty member teaching a significant portion online, I could foresee a claim that a paper resulting from a course they teach is a byline they “deserve.” But I am not so sure that the claim is warranted, and I am not so sure professors who teach exclusively online do in fact uniformly feel this way. But, this perspective makes sense to me.
Discussion is part of scholarship
The more important thing here is your intuition to go out and run the idea by somebody else who has grappled with the topic before you jumped to a conclusion as to right and wrong ways to proceed. That’s wise, and actually says a lot about your instincts about how scholarship works. Everything is discussed. But the discussion does not stop with the third party.
While you’re not wholly wrong in your assumption that there’s something diluted about the nature of co-authorship, understanding how authorship is credited in your field is what’s paramount here. The authorship discussion is an assumed part of scholarship and one that the publication venue trusts that you have completed. (Institutional Review is another research component that publishers often handle on an honor system.) For a scholar, this authorship discussion appears in other locations as well. In education, each tenure portfolio contains a document where you list a percentile authorship for each work on your cv for which you are claiming is evidence for tenure. Thus for each article, there will be percentage numbers next to each author’s name. I think it is assumed that the tenure candidate checks with the other authors that the numbers jive with how they felt. I have been asked this in the past — did you contribute 25 or 30%? Absurd, yes, better that other alternatives, also yes. In truth, I find that even the most diligent scholars are somewhat lax about this trivial task because it is hard to actually figure out who did what percentage when each person is working as a team, but they don’t skip the discussion altogether.
The important part is that the discussion happened, not whether 29% or 30.5 % was in fact the most accurate representative figure for authorship/research contribution. Reasonable scholars know that authorship is hard to quantify. For example– how would you and I divide credit for this blog post? I wrote more words, but you wrote the important questions. Word count is not a realistic reflection of value here, nor in research, but that’s how it’s done, along with a little negotiation. The percentile contribution is a make-shift fix for a problem we haven’t figured out how to reliably solve, and I don’t see a resolution coming apart from the fact that the discussion is required. I predict that as collaborations become ever more prominent, the percentile distribution document will become more of a guestimate, and at the same time, more important for professors to show they are in fact, working collaboratively with their peers.
Eric: My only response is a more personal one in that I do not intend to enter the academic community professionally any time soon. I quite enjoy working on products more so than research. However, I would like to have an academic body of work to fall back on 30 or so years in the future. Someday I’d like to get out of the development game and “retire” to academia. I am currently unpublished so it seems to me that adding him as a coauthor is my best option. Seeing that I have plenty of time (30 or so years) to build this, getting my foot in the door and putting my name out there is all I am looking to do.
Craig: This is an easy answer, but one you would be wise to note, and note quickly.
The unlikelihood of falling back on an academic career
I can’t tell you how many times I have heard this strategy. Contrary to popular opinion outside of the academe (those that know about it), an academic career to fall back on does not exist. Of course adjunct work abounds, but just read the Chronicle of Higher Ed to understand just how below slave labor contingent faculty suffer. The only people who ever “retire” to academic positions are intellectual celebrities. For example, Henry Kissenger, now at John Hopkins University, entered his post after a long career in politics, and there are occasional superstars from each field who find they have something to say that schools are open to hear. In their role, these celebrities in their fields do not necessarily do the work of typical professor either. I don’t know how they are paid, but I imagine it is worked out on an individual celebrity basis, and they are doubtfully paid as typical professor. I am not saying it is impossible to enter the academe after a career in another field, but these individuals are not falling back on an academic career Some may argue that such people are actually not entering the academe. Rather, they have something to say to members of their discipline earlier on in the career and the faculty is giving the celebrity an opportunity. For a detailed discussion of the struggle it takes to move from even the best PhD programs into a professorial position, read The Professor is In blog moderated by Dr Karen Kelsky.
While the hiring value of a publication generally dissipates into oblivion after five years, publication does hold a lot of value for professionals not intending to enter the professoriate. Publishing is concrete evidence that someone is up to date in a field of practice and can collect their ideas in a coherent and persuasive manner. The skills to put forward a polished argument is a very attractive feature for anyone looking to be recognized as a professional. So I would say it is darn good goal to have, just not for the reason you stated.
Acknowledgement: I thank Eric for humoring my excessive questions, writing frankly, and reviewing the final draft of this post prior to its publication on this blog.
Reference: Marshall T. Poe (2011). A History of Communications: Media and Society from the Evolution of Speech to the Internet. Cambridge: Cambridge University Press. xi + 275 pp. ISBN 9780521179447.
I see research groups get stuck all the time. Graduate students, and those setting out to do research in groups, can get bogged down and tripped up at dozens of spots in the process from conception of an idea to publication of a completed study. Collective intelligence matters less in the process of research than strategy does in seeing the task through to completion. I’ve put together a few thoughts to help research groups strategize their process of working together.
Disclaimer: Putting these ideas down in a document was inspired by two blogs I follow. I am a big fan of two online blogs where senior faculty tell it like it is to doc students: Karen Kelsky’s the Professor is In and Matt Might’s Computer Science Blog. Both of those blogs offer great tips for doc students and others trying to do research, but I haven’t found this particular topic on those blogs—some of these ideas might be there under different terms than the one I use- strategizing group research.
1. Write down a strategy. I am surprised by how few groups seem to actually have a research strategy written and shared among group members. I think some of the fault lies in the terms we use to do research. Research strategies are comprised of more than just research methods. Of course, without the knowledge of methods a study will never get off the ground, but the knowledge of a research method can give a group a false sense of security that they know exactly how to get a study done. Coursework in a research method simulates a research environment much like an aviation simulation mirrors really flying a real plane. In a methods course, errors will not kill you, nor kill the study; they will just provide opportunities for learning. Errors in the real world can stop a study in its tracks. Research methods are so complex that having a controlled environment is often the only way to start in on the process of learning how to apply a method of data collection and analysis, but coursework takes place in an artificially controlled environment, unlike “real” research that happens in the real world. So this blog post is about those other areas that deter studies from finding their way into publication. A research strategy is usually a shared document that delineates who will do what, by when, and is all-inclusive, from IRB approvals to drafting tasks, from start to finish. It’s not set in stone, it’s a living documents that gets a group from start to finish on each project. I offer a few tips for creating a strategy to get the job done beyond the methods.
2. Document reasons for group membership. Students who have completed coursework in research methods often overlook the roles different people play in a research group. Perhaps this is because the experience of group work in methods coursework sets a stage where everyone is equal, and that aspect of the simulation is far from the truth of doing real research in the real world. In the real world, each member of a research group is there for a different reason, and those reasons are important. They significantly impact how group members are likely to behave. Each person needs a certain thing out of the project. Perhaps it is learning a new method, or perhaps it is getting a pub. Perhaps it is some stake in a research grant. Perhaps it is exposure to the experience. Whatever it is, it needs to be written down. Often, graduate students are not taught to do this in their coursework. These reasons are not addressed in methods coursework because they are not methods, but they are part of research in a group. Recognizing these roles is part of a solid research strategy. Methods classes rarely tell you to solicit and document the reasons people have come to a research group, but the practice informs all the members of the group and facilitates good will and understanding as the group moves forward with a project.
3. Recognize power dynamics in the beginning. Very often faculty and students are mixed. Sometimes junior and senior faculty are also mixed. When these contexts occur, power dynamics can get confusing. Students may solicit guidance from a faculty member who is not the leader on that particular study and their subsequent advice may be taken in undesirable directions for the leader of the group, or they may not recognize junior faculty are not exactly full equals to other members of the group. Sometimes a faculty member will refuse guidance as they might feel out of place, or if the leader is using the study as a teaching mechanism, for either a junior faculty or for a student. Recognizing positions of authority can help everyone navigate awkward moments and move on.
4. Discuss authorship and roles prior to the onset of the study. This is probably the most critical of the points I am making here. To be honest, the onus for this rests on those in lesser positions of power because they have the most to lose. At the same time, I am yet to encounter a faculty member anywhere who doesn’t want the process spelled out clearly – so do not be hesitant in approaching any faculty member leading or participating in research with questions about roles and authorship. It often happens is that the group is so busy in doing the work that they forget to stop and make some decisions about the strategy to get each aspect of the project done, or he needs of certain members are overlooked. There are dozens of websites out there with guidelines to determine who is first author, but the truth is, it is often the highest ranking faculty member who sets the rules. For my own work with students, my policy is always the same: If you have written the first draft of the paper, you are the first author. If I am invited to be a subsequent author, my name will appear last no matter how many students are included in the byline, and I will read and edit the draft that is submitted as well as the final proof, and those are the only drafts I will read from start to finish. I think most faculty members have some sort of general policy but rarely put it in writing because of the many different contexts they encounter. My own circumstances are pretty limited so I can put forward a blanket policy at this point in my career. While a discussion of roles and authorship may seem trivial, I have seen plenty of cases where studies develop in one direction or another where the roles and tasks were not clearly spelled out at the start, confusion ensues, papers never make it to submission, and there’s a fair bit of “I thought you were going to do that part.” Everybody loses in such a case. I suggest that all groups doing research have the authorship discussion up front to.
5. Have a contingency plan. A contingency plan is a plan B should members not be able to complete up their assumed role. The plan should ensure that the study makes it through the process even when things pop up, and thing generally do pop up. From my experience of multiple-author research, the contingency plan generally rests on the shoulders of the first author. For example, when there is a deadline ASAP, it’s the first author who sucks up the final touches and provides the momentum to get the product out the door, even on tasks agreed to by others. If the policy is addressed openly, feelings are not hurt and members can proceed, if nothing else, at least informed. The plan especially comes into play in cases where one is waylaid by illness, taking kids little league games, a death in the family, or any of a number of reasons that put obstacles in the path to publication/presentation. From group to group the first author position may means something different. In my experiences, the first author position had a specific meaning that was to-often unspoken– it’s not only (and not necessarily) the one who did the bulk of the work. It signifies who was the driving force behind the research itself, and who is taking responsibility for its completion. Of course, the mores of practices change among fields and institutions, so having the contingency plan discussed ahead of time can be a welcomed insurance policy for guilt and hurt feelings should the unexpected arise. When members are from a variety of cultures, straightforward addressing of the circumstances of contingency becomes even more important.
6. Avoid assuming that formulating the research design determines first authorship. While practices may vary among institutions and fields, the authorship discussion may also bring to light some important nuances of research we rarely get in methods courses. Just because one person came up with one idea does not mean the study itself is “theirs.” There’s a lot that goes into research. When groups meet, dialogue often results in new ideas. Group members new to a group may feel slighted if their assumptions about credit are not fulfilled in the practices of the group. In my own experience, faculty do not require first author position for every study they guide; however, in some cases, a particular study may be of particularly importance, either to a long-term trajectory or for some other reason. As long as the role of the study and value of the credit for it are addressed early on, the authorship discussion is a comfortable one. Faculty accused of “stealing” students research ideas are less likely research thieves than they are guilty of not having the authorship discussion up front. A lot of this could be mitigated by avoiding assumptions about how group work results in credit. Unfortunately, as it is often the responsibility of the student, and not the faculty member, rarely does the faculty member initiate the discussion or put forward these ideas at the start.
7. Several studies can be written on the same data, but this means a separate strategy is needed for each. Planning ahead and deciding who is writing which study from a given project is needed in order to do this. Few projects are at this phase in the early stages. Simply designating one person as “doing qualitative” and another as “doing quantitative” doesn’t avoid conflict in the future because the terms are not accurate descriptions of research. In my experience, “qualitative” is too vague. Simply equating “quantitative” with anything study that contains numbers is also too vague. I would avoid the both terms in deciding who is doing what. If a project is in the early stages, and multiple studies are planned, some things may need to be worked out prior to the strategy and authorship discussions. In those cases, it is helpful to anticipate revising the strategy as you move forward.
The final three points might be specific to scholars in education. I don’t know enough about other fields to speak to how research groups might work most productively in those fields.
8. Look beyond the quasi-experimental study. One of the problems research groups have is that newer members may only see as far as the experimental aspect of the research. While many research designs in education will fall into this category of quasi-experimental study, there is much more to look at in most projects. There is the design of the intervention, close inspections of learner experiences, stakeholders’ accounts of a practice, and methods articles, just to name a few. Very often, these other aspects of the study are the most interesting part, once you dig into them. And the experimental study itself may face larger obstacles to publication than new members are ready to tackle: quasi-experimental studies are often closely read for precise usage of research terms, such as the proper labeling of the data, the correct usage of words like “control group” and necessary treatment of what makes them “quasi” as opposed to true experiments, such as having a placebo. The literature review is perhaps the most foundational aspect of any study that gets overlooked, yet group members stepping forward excited to do the literature review are rare. The person who does the literature review is an important and often overlooked role.
9. Revisit the strategy when major changes occur. One of the most difficult things to grasp about the act of actually doing research is that research always reads linearly, but is almost never so in practice. Rather, the process is iterative. I might also add that is it not a bad idea to revisit the strategy after the initial grappling with the data by the group. Research questions are a work in progress until the proofs go to publication, in most cases. Initially viewing data under a general driving concept is great, but keeping in mind that the RQs can change as the project evolves can really empower a research group, and if all the members are aware that the strategy can be amended at any point, people can be encouraged to follow fruitful paths. Avoid getting stuck in “causes” and proposed calculations in the beginning. They can be elusive, especially in early stages when data has just been collected and is not fully prepared. Also, some calculations may or may not be worth the time to calculate until the group has grappled with some of the main ideas. You can always add calculations later at reviewers’ request. If you haven’t yet dug into the data, and done a few passes through with your analysis scheme, you can’t truly foresee what that data really means; and therefore, the initial strategy may have been naive. Seasoned scholars are not deterred by this, but newer members may feel discouraged when data shows the unexpected.
10. Share the strategy document when new members arrive, and revisit it at the end. The research strategy is more than a Gannt chart. New members need to see the progress that has been made to catch up quickly and contribute, and simply sharing a strategy document with new members can bring them up to speed in a non-biased way, especially is sub-studies have moved in directions of their own. It has value up until the final publication is sent out, and perhaps even after that. A group research strategy can function as a collaborative narrative of the twists and turns of the project. Group projects that last over extended periods of time can lose track of the rationale behind important decisions and expected outcomes. Revisiting a strategy after a projects nears the final stages can offer a lot in retrospect about how decisions were made and how those decisions played out in the end.
On 11/16: I gave a talk entitled Instructional design & design thinking: How we develop knowledge in instructional design and used some slides that students might find handy later on. I covered a definition of instructional design, what is and is not design knowledge, and how we might build design knowledge for learning contexts. This is contrasted to how we build knowledge about theories related to instructional interventions.
On 10/22: I also gave a talk at Texas A&M University about my dissertation, but focused on the research methods. The links I used are below, along with the main points of what I had to say:
- My dissertation: Higher Order Thinking in Collaborative Video Annotations: Investigating discourse modeling and the staggering of participation. (2012)
- The new media created and studied I named Collaborative Video Annotations. I provide an example here
- I focused this talk on three terms underlined here: This was a quasi-experimental content analysis of learner discourse.
- Every study has some big findings and some lesser findings. My big ones were: “Learners in the staggered conditions groups produced twice as much higher order thinking than learners in the basic condition. Learners in the modeled discourse condition produced three times as much higher order thinking. The staggering of learner participation also required no effort and thus should be a requisite design feature in online discussions.”
Addendum: Pay-for-publication and self-publishing, in my field, are the same thing. It is a dangerous proposition for pre-tenure faculty to take part in these venues as neither count for tenure, and neither should count for tenure. It’s been just over a year since I posted about this and new predatory publishers are still soliciting me and trying to get me to pay them for publishing my papers. Some have even promised 2 weeks from submission to publication for a fee. New solicitations include:
- International Journal of Humanities and Social Science. Fees disclosed on this page: http://XXX.ijhssnet.com/index.php/faq.html
- Journal of Curriculum and Teaching. Fees disclosed on this page: http://XXX.ijhssnet.com/index.php/faq.XXX
The darker side of academic publishing is now part of my teaching thanks to a few solicitations over the past few months from somewhat questionable publishers. These publishers wanted to add me to their reviewer list or their “Editorial Boards.” I use quotes there for a reason. I have to wonder just what these editorial boards do. I am quickly learning that a knowledge of predatory publishing is required to understand educational research in an internet-connected world. One must be able to pick out what is worthy of citation and what is not. An email this morning reminded me to write this post.
While Macrothink Inc. is to credit on this occasion, previous solicitations to the most recent one have been similar. The business model is easy enough to understand given the context. Since faculty need to publish research to make tenure, there is a demand out there for the quick publishing of less-than-polished research, for cash payments. This means the authors pay the journal and the journal publishes the article. I have to question the quality of the research if there is financial incentive to publish it. Pay-for-publishing venues immediately raise concerns when there is such a dynamic in the process. What’s the barrier to publication if there is money behind the acceptance? I don’t find acceptance rates on their homepages. Every couple of weeks I get a request to review for such a journal- sometimes it is a request to be listed as a “Board Member.” With so many requests, I have become even more skeptical. We also need to ask just what role the reviewers have in donating their time for others’ financial gain. Is it ethical to ask faculty to donate their time towards a service that is financially driven?
The dynamic inherent in pay-for academic publishing hammers home the need for information literacy– a point Howard Rheingold makes at length in his new book- Netsmart: How to thrive online, which calls again for more information literacy. His recent talk at AECT 2013 brought up some familiar themes. He calls it a crap detector, but the skill set does not propose any new questions that others such as Damico’s critical web reader have been propoising for years.
The email however provides excellent course material for an activity, because it violates basic tenants of information literacy– the context of the communication should be studied before consuming the content. Below you will find a copy of the solicitation email from Macrothink. It is curious that the email signature is from “John White,” the email address is unrelated – “jet@macrothink,” and the registered name is “Leonard Bai.” These are great indicators of the nuances of reading skills that are required to make sense of email these days, a topic I addressed in An instructional paradigm for the teaching of computer-mediated communication here. The tone of this solicitation struck me as the academic equivalent of the queen of Nigeria emailing me for a business transition and bank account information. If anyone is interested in doing a lit review on predatory publishing in Ed Tech or Inst Design, the field dearly needs it. There should be more out there about how to teach this in effective ways.
All is not lost; we have a more general resource to help us. Jeffrey Beal at UC Denver keeps a running log of predatory publishers. Macrothink, the originators of this email, are listed on Jeffrey Beal’s blacklist. I have copied the solicitation email I received yesterday into the space below. I have also removed their links (by adding XXX in the http space) because on the internet, links have meaning. I don’t want to appear as endorsing this publisher.
Dear Dr. Craig D. Howard,
I have had the opportunity to read your paper “An instructional paradigm for the teaching of computer-mediated communication” in Instructional Science and can tell from your work that you are an expert in this field. We are recruiting reviewers for the journal. If you are interested in this position of reviewing submissions, we welcome you to apply for. Please find further details at XXXmacrothink.org/recruitment/
I am John White, the editor of Journal of Education and Training (JET). Journal of Education and Training (JET) is an international, peer-reviewed online journal published by Macrothink Institute, USA. This Journal publishes research papers cover the whole range of education and training, aims to provide an international forum for the exchange of ideas, opinions, innovations and research on topics related to education and teaching. Professionals and researchers are encouraged to contribute their high quality, original works of the field.
It is a great pleasure to invite you to contribute your best knowledge and research. Your contribution will help us to establish a high standard. We use a double-blind system for peer reviews. The paper will be peer-reviewed by at least two experts. The review process may take two to three weeks. If you are interested, please find the journal’s profile at: XXXmacrothink.org and submit your manuscripts online. Or you can send your paper directly to the e-mail: XXXmacrothink.org. If you have any questions, please feel free to contact the editorial assistant at firstname.lastname@example.org.
We would appreciate it if you could share this information with your colleagues and associates.
Journal of Education and Training
5348 Vegas Dr. #825
Phone: 1-702-953-1852 ext.534
Update March 2014, they keep coming:
Dear Dr. Craig D. Howard,
We are reaching you because of your paper entitled, ‘Designerly Talk in Non-Pedagogical Social Spaces’, which was published in Journal of Learning Design, 2014 Vol. 7 No. 1, and were very impressed at its scope and contents. We know you are an expert in your research area.
Our journals named ‘World Journal of Education’ and ‘Journal of Curriculum and Teaching’, peer-review, published by Sciedu Press. They are devoted to publishing research papers in various aspects, fields and scope of the education, teaching, learning and other relevant subjects.
It is our great honor to invite you to submit your new manuscripts to us as one of the ‘Authors’ in our next publication.
If you are interested to be a member of our editorial board, please find the application form and details at: http://web.sciedu.ca/recruitment.html and send the application form to: email@example.com or firstname.lastname@example.org
We would appreciate if you could share this information with your colleagues and associates who might be interested in joining us as a ‘Reviewer’ or submit their manuscripts to us as ‘Authors’.
Thank you and we hope to hear from you and/or your colleagues and associates soon.
Ms. Sara M. Lee
Mailing Add: 1120 Finch Avenue West, Suite 701-309, Toronto, ON., M3J 3H7, Canada
Tel: 1-416-479-0028 ext. 218
here’s the fine print:
This journal charges the following author fees.
Article Publication Fee – World Journal of Education: 300.00 (USD)
If the paper is accepted for publication, you will be asked to pay an Article Publication Fee. Please find payment information at: http://www.sciedu.ca/payment