Home » research
Category Archives: research
Years ago Elizabeth Boling mentioned this classic instructional design, The Nutshell Studies, and I just had not run into a thorough discussion of it for years. It is an instructional design intended to support the learning of forensic science for detectives. The most remarkable part of the design are 19 miniature dioramas which make lifelike crimes scenes available for viewing by multiple detectives in training. A perfect instructional design case for a historical issue of IJDL, but the dioramas, and the life of their creator Frances Glassner Lee, were recently featured on an NPR show called Sidedoor, and in all honesty, Tony Cohn from PRX does an excellent job, hitting all the required components of a design case. As AECT is just around the corner again, I am sure I will be talking about instructional design cases again in the near future, and this Sidedoor episode is a case worth discussing.
Frances Glessner Lee and The Nutshell Studies of Unexplained Death, is actually the name of an art exhibit, but is spawns from an instructional design, from which Cohn crafted his audio instructional design case. Mrs. Lee created complex dioramas with such detail that they could tell the story of an entire murder, well, almost the entire story. A curious aspect to her dioramas was their purposefully unsolved-ness. There is no solution offered to the detectives in training. In other words, despite their intricacy, they will always remain, unsolved. This 1940’s design choice of creating unsolved murder scenes, fodder for discussion but not for resolution, speaks to the persistence of discussion, rather than resolution, as being an ever-present aspect of higher order thinking. The key is not to select the right answer, but to untangle the complexity and multiple possibilities that could play into each scene. Of course, the hope is that with practice, learning the process of disentanglement will result in better investigations, leading to a higher chance of getting things right eventually. Once we let go of the idea of always getting is right this time, and focus on getting it right most of the time, our chances go way up each and every time.
Learners in the Master’s program in IT here at UT chuckle about my choices of terms: grapple, dig in, go deeper, dissect, take apart. But these terms do tell what I am trying to accomplish. I don’t think we’re ever going to learn definitively how to teach (sadly) and I don’t think we’re ever going to definitively determine the right way to design instruction (woefully), and , that’s not the point (joyously). The point is, to see a design like the Nutshell Studies for more than just the dioramas, though, for sure, the dioramas were pretty damn cool.
Since stopping this blog in February, ideas have come and gone, and my apprehensions about leaving Texas A&M worked on me to focus and prepare for a new start. That start has come; it’s September and I am teaching in a new context. While I am very happy to be here in Tennessee, I am also trying to work out exactly where is the best place to invest my time, thus the new banner image and a renewed concern over the clock. One thing that is not in question is the purpose of this blog. It’s a place to work ideas out in a more public venue, allowing for comments from anywhere, and from anyone. This month’s post is such an opportunity. It is inspired by a dialogue that came about organically from a posted comment by a reader.
The issue I am dealing with here is an old one, but as Marshal Poe writes, our perspectives on right and wrong are very much tied to the times- the way we communicate and who gets to communicate to whom about what and how often (2011). Had this issue arose at my workplace, I probably would not find out about it because of the nature of the issue; some topics can be discussed on a blog at another university that would not be welcome at one’s own institution. That’s one area where the blog excels. It creates a fruitful venue to discuss topics outside of one’s own institution.
This discussion started as a comment to Publishing as a graduate student and developed into something worth a little more space than the comment trail, I think. What follows is the email discussion with a few headers added in after the fact. Eric Hufford, the author of these questions and the unfortunate graduate student in the pickle described in this post, is a senior software engineer in Maryland with 10 years of experience in the field of software engineering. He holds a B.S. in Computer Science and is pursuing a Masters in Software Engineering as well as an MBA.
Eric: Thank you for this fantastic information. I do have one question for you though, as I am struggling with this before I submit my paper. I wrote a paper for a graduate course. The paper I plan on submitting is exactly as I turned it in to the course with absolutely zero input from or collaboration with the professor. In fact, his entire involvement with the paper was approving the topic (it was a term paper) and grading it. Yet he is telling me that since I wrote it for his course, he needs to be listed as an author. I strongly disagree with his statement. Is this something I should be concerned about? Do I really need to add him as an author?”
Craig: Wonderful question and thank you for submitting it. I will use headers to organize and emphasize my points, because this response is lengthy.
Know the authoring conventions for your field
There are a few things that need to be clarified prior to saying your professor is outright wrong. First off, this blog post was meant for graduate learners in Instructional Technology, but might also be broadly applicable to social science research in general. I don’t know your field of study. Much depends on the conventions in your field. If your paper was developed from someone’s course which included access to a lab where data was collected, it is very likely that authorship listing the lab leader is the most appropriate convention. Also, if the paper topic was developed iteratively with the professor, that in itself may very well constitute input worthy of authorship. Your professor might also be refereeing to content in the paper that was derived from teachings in the course, in which case the professor would be making a legitimate claim. Both contributions and conventions should be discussed prior to any submission of the article anywhere. Graduate students should inform themselves of some of the more general conventions using authorship guidelines published for their appropriate field. There are guidelines out there for determining authorship. In my field, Instructional Technology, AECT has some, but more generally, there is an APA guide online. Read it closely. Knowing these conventions will help you learn more from your discussion about authorship with your professor. This is a key learning step in graduate school and one often overlooked in the process from admission to graduation, and possibly the root cause behind the uncomfortable feelings expressed in the comment.
Course papers are never journal articles, they are, at best, first drafts of journal articles
Secondly, one phrase in your question strikes in me a note of caution, “The paper I plan on submitting is exactly as I turned it in to the course.” This strategy runs the risk of getting a swift and straightforward rejection, needlessly uses the time of an editor at the journal you submit to, and moreover, runs you the risk of sparking a negative image of your work in the eyes of the editor. No paper should go to submission raw. Now that the course is over, re-read the writer’s guidelines of the proposed venue, and make sure you are familiar with the prospective journal’s most recent publications on the topic you cover. I expect there’s a fair bit of editing to do. I have heard students complain that established professors can submit less polished drafts than students and get away with it. I am not fully convinced this is true. Scholars that have been dealing with a topic for many years may have a battery of key phrases and can use shorthand to express concepts and other ideas. A learner’s text faces the formidable obstacle of being a relatively new topic, and without the crafted language at the tips of your fingers, it must be well crafted though scrutiny and revision. You haven’t been reading/writing for enough years to know them all the phrases you need, and a facility with them only comes with extended exposure.
Collaborations must be mutually beneficial
More strategically, it would be wise to rethink your take on collaboration in the field. From the perspective of the student, credit for a single-authored publication is paramount. However, from the perspective of scholars in the field, it is likely not so. The co-authored paper is worth more, not less. When a potential candidate’s cv evidences multiple collaborations that resulted in publications, it sends a far stronger message of competence than a cv with a few single authored published papers. Scholars publish dozens of articles over their career, and their co-authoring relationships evidence the breadth of their participation with topics of interest to others doing work worthy of publication. There is much to gain from collaboration and showing that you work well with your professors, even if this particular professor does not spur in you the desire for a long term identity of association. You have far more to gain from the professor’s name being on the byline than you have to gain from being sole-author, even if you plan on never co-authoring with this professor again. Your paper is also more likely to be read because inclusion in the professor’s online archive may draw readers surfing your professor’s publications, and it may also heighten your rank in internet searches if your professor is more published than you are—which is more often the case than it is not.
At this point, your task is to get yourself informed about the appropriate conventions for authorship, make strides in the development of the paper for publication, and bring the draft back to your professor to have a more informed conversation about the draft and target venue. It may feel like eating humble pie, but it’s not. Even if you are vastly more experienced and it is a junior faculty member you’re dealing with (which does happen as grad students come from all walks of life and faculty can come straight from successive years of schooling) what you do not want is to publish the paper without the professor’s approval, especially with verbatim sections from the paper you submitted in the course. That runs the risk of you being marked as someone who does not give credit where credit is due, could alienate you from other scholars in the future, and potentially endanger your completion of the program where you are now.
Eric: The paper in question was an end-of-term paper for a Software Engineering course, and I have scoped out a few different journals and honed in on two that I feel may be a good fit. I plan on re-tooling the paper to fit within the journal guidelines, but other than that, I plan on submitting the same basic paper. That is to say, I do not intend on taking any input from the professor’s review before I submit it. It sounds as if he is attempting to take credit for his student’s work even though he had no involvement in the process of creating the paper. [He only taught the course in which the paper was submitted.] Do I really need to add him as an author?
Craig: Yes, but more importantly, why fight it when the outcome (s)he proposed actually favors you? Nevertheless, the issues you raise here are endemic to graduate school, and actually, really important. Every year this issue pops up somewhere- often between a junior faculty and a senior doc student. I really think pushing this to a richer discussion might uncover something valuable for other grad students and those who teach grad students. So, let me ask you this– do you think it was something in the classroom dynamic that lead you to want to publish the paper without his name on it? Is there something the unnamed faculty member didn’t do that s/he should have done? What brought you to so fervently want to publish it without his name on it?
Eric: I’ll answer your last question first before going back to your first two. “What made me feel like I didn’t need to include him as an author?” I suppose it is relevant to provide some background information on myself. I am a professional software engineer with about 10 years’ experience in the industry. I finally decided to pursue a masters’ degree and an MBA. This particular course was entirely online so I never really developed a rapport with the professor. While he is a senior faculty member and appeared to be a fairly competent fellow, the course content was nothing new to me. I never opened the book, nor did I ever have any conversations with the professor beyond those required to complete the course (like approving my term paper topic). In other words, the class was a breeze, I received an A and moved on with my life. With that said, I felt that it would not be necessary to add his name as an author because he and I had a very minimal amount interaction, and it was only ever via email. There was never a relationship or any sense of loyalty. However, I try to remain open minded as best I can, which is why I came across your post and asked my question. I simply didn’t want to do the wrong thing and have it come back to bite me later. I was also under the impression that if I shared authorship, that my involvement in the paper would be diluted, which I assumed would reflect less upon me. In your response, what convinced me to include his name on the paper regardless of how little he did, was your argument that it looks better on me to have a co – authored paper. You mentioned that it would make it look like I perform well with others and, more importantly, if he is well published, my paper will get more visibility than it ever would under my single authorship. This may sound heartless and blunt, but these are the things that matter most to me about publishing my paper. Your answer surprised me, it wasn’t what I was expecting at all, but it made a ton of sense.
Craig: The online dynamic raises lots of curious questions.
Unspoken information lost in the lean media
Low-interaction online courses can be a breeding ground for conflicting perspectives, especially if they appear early on in the program of study. In order to create a smoother pedagogical experience for both learners and the instructor, some programs require foundational courses that may fall below what an advanced learner is expecting. In such cases, the indebtedness of the learner to the knowledge of the professor is a stretch for advanced learners to recognize, and surprisingly often, that lack of indebtedness is completely justified, especially in online classes where the direct instruction is limited. In some online classes, it’s not hard to imagine just anyone teaching the course if it basically runs by itself with little instructor-student interaction. A close working relationship is implied by a shared byline, but in a course where the interaction has been so little, it is bound to feel weird to share credit. At the same time, it is difficult for the instructor to recognize the insights did not come from the course, especially if those insights could have been inferred from the materials assembled by the professor. I wonder if such a misunderstanding is the case in your context.
The lean media of online instruction, often limited to text interactions, often requires a seasoned online instructor to recognize insights that break from course materials into new ground for the learner. Junior faculty with limited online experience, and even seasoned faculty who are new to the online format, may be relying on cues that simply are not there in online interactions to recognize when a learner is breaking from simply what they have been taught, into original research. He may well have thought you were simply building from the given materials. Having not opened the book puts you at a significant disadvantage when arguing that you didn’t learn x or y from a course of study, and at the same time, sometimes it takes a seasoned online teacher to recognize truly original work.
Differing expectations, none of which are written down
On top of this, these instructors may be working with a hidden curriculum that is never made explicit to the learner. Online faculty may have tacit expectations of a set of rules of interaction leading to co-authorship with grad students, and at least in my experience, rarely are these understandings made explicit in a course syllabus or other materials that the online learner is exposed to. With online learning we generally default back to the face to face rules until people figure out a better way to proceed and new conventions are adopted, but we rarely make it as explicit as it should be. Who wants to read, nonetheless write, a set of conduct rules? In graduate programs, for all the documents we produce, there are still topics navigate via intuition instead of explicit guidelines. How often do graduate students read the handbook anyway? I know I did not labor of over such a document until an issue arose. While I would say it is the responsibility of the instructor to make those rules transparent as best they can, the large number of exceptions makes that goal pretty far reaching and arguably infeasible, especially for someone who is not used to teaching online where there is far more emphasis placed on documenting procedures.
Advanced students may also not be aware that in the context of the professor, they may feel they are working at a disadvantage to their peers. If a professor teaches exclusively online, working relationships with students may be very challenging to construct. I know that in my experience at Texas A&M, it was definitely a challenge. In many schools, professors are expected to publish with their students, and for a faculty member teaching a significant portion online, I could foresee a claim that a paper resulting from a course they teach is a byline they “deserve.” But I am not so sure that the claim is warranted, and I am not so sure professors who teach exclusively online do in fact uniformly feel this way. But, this perspective makes sense to me.
Discussion is part of scholarship
The more important thing here is your intuition to go out and run the idea by somebody else who has grappled with the topic before you jumped to a conclusion as to right and wrong ways to proceed. That’s wise, and actually says a lot about your instincts about how scholarship works. Everything is discussed. But the discussion does not stop with the third party.
While you’re not wholly wrong in your assumption that there’s something diluted about the nature of co-authorship, understanding how authorship is credited in your field is what’s paramount here. The authorship discussion is an assumed part of scholarship and one that the publication venue trusts that you have completed. (Institutional Review is another research component that publishers often handle on an honor system.) For a scholar, this authorship discussion appears in other locations as well. In education, each tenure portfolio contains a document where you list a percentile authorship for each work on your cv for which you are claiming is evidence for tenure. Thus for each article, there will be percentage numbers next to each author’s name. I think it is assumed that the tenure candidate checks with the other authors that the numbers jive with how they felt. I have been asked this in the past — did you contribute 25 or 30%? Absurd, yes, better that other alternatives, also yes. In truth, I find that even the most diligent scholars are somewhat lax about this trivial task because it is hard to actually figure out who did what percentage when each person is working as a team, but they don’t skip the discussion altogether.
The important part is that the discussion happened, not whether 29% or 30.5 % was in fact the most accurate representative figure for authorship/research contribution. Reasonable scholars know that authorship is hard to quantify. For example– how would you and I divide credit for this blog post? I wrote more words, but you wrote the important questions. Word count is not a realistic reflection of value here, nor in research, but that’s how it’s done, along with a little negotiation. The percentile contribution is a make-shift fix for a problem we haven’t figured out how to reliably solve, and I don’t see a resolution coming apart from the fact that the discussion is required. I predict that as collaborations become ever more prominent, the percentile distribution document will become more of a guestimate, and at the same time, more important for professors to show they are in fact, working collaboratively with their peers.
Eric: My only response is a more personal one in that I do not intend to enter the academic community professionally any time soon. I quite enjoy working on products more so than research. However, I would like to have an academic body of work to fall back on 30 or so years in the future. Someday I’d like to get out of the development game and “retire” to academia. I am currently unpublished so it seems to me that adding him as a coauthor is my best option. Seeing that I have plenty of time (30 or so years) to build this, getting my foot in the door and putting my name out there is all I am looking to do.
Craig: This is an easy answer, but one you would be wise to note, and note quickly.
The unlikelihood of falling back on an academic career
I can’t tell you how many times I have heard this strategy. Contrary to popular opinion outside of the academe (those that know about it), an academic career to fall back on does not exist. Of course adjunct work abounds, but just read the Chronicle of Higher Ed to understand just how below slave labor contingent faculty suffer. The only people who ever “retire” to academic positions are intellectual celebrities. For example, Henry Kissenger, now at John Hopkins University, entered his post after a long career in politics, and there are occasional superstars from each field who find they have something to say that schools are open to hear. In their role, these celebrities in their fields do not necessarily do the work of typical professor either. I don’t know how they are paid, but I imagine it is worked out on an individual celebrity basis, and they are doubtfully paid as typical professor. I am not saying it is impossible to enter the academe after a career in another field, but these individuals are not falling back on an academic career Some may argue that such people are actually not entering the academe. Rather, they have something to say to members of their discipline earlier on in the career and the faculty is giving the celebrity an opportunity. For a detailed discussion of the struggle it takes to move from even the best PhD programs into a professorial position, read The Professor is In blog moderated by Dr Karen Kelsky.
While the hiring value of a publication generally dissipates into oblivion after five years, publication does hold a lot of value for professionals not intending to enter the professoriate. Publishing is concrete evidence that someone is up to date in a field of practice and can collect their ideas in a coherent and persuasive manner. The skills to put forward a polished argument is a very attractive feature for anyone looking to be recognized as a professional. So I would say it is darn good goal to have, just not for the reason you stated.
Acknowledgement: I thank Eric for humoring my excessive questions, writing frankly, and reviewing the final draft of this post prior to its publication on this blog.
Reference: Marshall T. Poe (2011). A History of Communications: Media and Society from the Evolution of Speech to the Internet. Cambridge: Cambridge University Press. xi + 275 pp. ISBN 9780521179447.
I see research groups get stuck all the time. Graduate students, and those setting out to do research in groups, can get bogged down and tripped up at dozens of spots in the process from conception of an idea to publication of a completed study. Collective intelligence matters less in the process of research than strategy does in seeing the task through to completion. I’ve put together a few thoughts to help research groups strategize their process of working together.
Disclaimer: Putting these ideas down in a document was inspired by two blogs I follow. I am a big fan of two online blogs where senior faculty tell it like it is to doc students: Karen Kelsky’s the Professor is In and Matt Might’s Computer Science Blog. Both of those blogs offer great tips for doc students and others trying to do research, but I haven’t found this particular topic on those blogs—some of these ideas might be there under different terms than the one I use- strategizing group research.
1. Write down a strategy. I am surprised by how few groups seem to actually have a research strategy written and shared among group members. I think some of the fault lies in the terms we use to do research. Research strategies are comprised of more than just research methods. Of course, without the knowledge of methods a study will never get off the ground, but the knowledge of a research method can give a group a false sense of security that they know exactly how to get a study done. Coursework in a research method simulates a research environment much like an aviation simulation mirrors really flying a real plane. In a methods course, errors will not kill you, nor kill the study; they will just provide opportunities for learning. Errors in the real world can stop a study in its tracks. Research methods are so complex that having a controlled environment is often the only way to start in on the process of learning how to apply a method of data collection and analysis, but coursework takes place in an artificially controlled environment, unlike “real” research that happens in the real world. So this blog post is about those other areas that deter studies from finding their way into publication. A research strategy is usually a shared document that delineates who will do what, by when, and is all-inclusive, from IRB approvals to drafting tasks, from start to finish. It’s not set in stone, it’s a living documents that gets a group from start to finish on each project. I offer a few tips for creating a strategy to get the job done beyond the methods.
2. Document reasons for group membership. Students who have completed coursework in research methods often overlook the roles different people play in a research group. Perhaps this is because the experience of group work in methods coursework sets a stage where everyone is equal, and that aspect of the simulation is far from the truth of doing real research in the real world. In the real world, each member of a research group is there for a different reason, and those reasons are important. They significantly impact how group members are likely to behave. Each person needs a certain thing out of the project. Perhaps it is learning a new method, or perhaps it is getting a pub. Perhaps it is some stake in a research grant. Perhaps it is exposure to the experience. Whatever it is, it needs to be written down. Often, graduate students are not taught to do this in their coursework. These reasons are not addressed in methods coursework because they are not methods, but they are part of research in a group. Recognizing these roles is part of a solid research strategy. Methods classes rarely tell you to solicit and document the reasons people have come to a research group, but the practice informs all the members of the group and facilitates good will and understanding as the group moves forward with a project.
3. Recognize power dynamics in the beginning. Very often faculty and students are mixed. Sometimes junior and senior faculty are also mixed. When these contexts occur, power dynamics can get confusing. Students may solicit guidance from a faculty member who is not the leader on that particular study and their subsequent advice may be taken in undesirable directions for the leader of the group, or they may not recognize junior faculty are not exactly full equals to other members of the group. Sometimes a faculty member will refuse guidance as they might feel out of place, or if the leader is using the study as a teaching mechanism, for either a junior faculty or for a student. Recognizing positions of authority can help everyone navigate awkward moments and move on.
4. Discuss authorship and roles prior to the onset of the study. This is probably the most critical of the points I am making here. To be honest, the onus for this rests on those in lesser positions of power because they have the most to lose. At the same time, I am yet to encounter a faculty member anywhere who doesn’t want the process spelled out clearly – so do not be hesitant in approaching any faculty member leading or participating in research with questions about roles and authorship. It often happens is that the group is so busy in doing the work that they forget to stop and make some decisions about the strategy to get each aspect of the project done, or he needs of certain members are overlooked. There are dozens of websites out there with guidelines to determine who is first author, but the truth is, it is often the highest ranking faculty member who sets the rules. For my own work with students, my policy is always the same: If you have written the first draft of the paper, you are the first author. If I am invited to be a subsequent author, my name will appear last no matter how many students are included in the byline, and I will read and edit the draft that is submitted as well as the final proof, and those are the only drafts I will read from start to finish. I think most faculty members have some sort of general policy but rarely put it in writing because of the many different contexts they encounter. My own circumstances are pretty limited so I can put forward a blanket policy at this point in my career. While a discussion of roles and authorship may seem trivial, I have seen plenty of cases where studies develop in one direction or another where the roles and tasks were not clearly spelled out at the start, confusion ensues, papers never make it to submission, and there’s a fair bit of “I thought you were going to do that part.” Everybody loses in such a case. I suggest that all groups doing research have the authorship discussion up front to.
5. Have a contingency plan. A contingency plan is a plan B should members not be able to complete up their assumed role. The plan should ensure that the study makes it through the process even when things pop up, and thing generally do pop up. From my experience of multiple-author research, the contingency plan generally rests on the shoulders of the first author. For example, when there is a deadline ASAP, it’s the first author who sucks up the final touches and provides the momentum to get the product out the door, even on tasks agreed to by others. If the policy is addressed openly, feelings are not hurt and members can proceed, if nothing else, at least informed. The plan especially comes into play in cases where one is waylaid by illness, taking kids little league games, a death in the family, or any of a number of reasons that put obstacles in the path to publication/presentation. From group to group the first author position may means something different. In my experiences, the first author position had a specific meaning that was to-often unspoken– it’s not only (and not necessarily) the one who did the bulk of the work. It signifies who was the driving force behind the research itself, and who is taking responsibility for its completion. Of course, the mores of practices change among fields and institutions, so having the contingency plan discussed ahead of time can be a welcomed insurance policy for guilt and hurt feelings should the unexpected arise. When members are from a variety of cultures, straightforward addressing of the circumstances of contingency becomes even more important.
6. Avoid assuming that formulating the research design determines first authorship. While practices may vary among institutions and fields, the authorship discussion may also bring to light some important nuances of research we rarely get in methods courses. Just because one person came up with one idea does not mean the study itself is “theirs.” There’s a lot that goes into research. When groups meet, dialogue often results in new ideas. Group members new to a group may feel slighted if their assumptions about credit are not fulfilled in the practices of the group. In my own experience, faculty do not require first author position for every study they guide; however, in some cases, a particular study may be of particularly importance, either to a long-term trajectory or for some other reason. As long as the role of the study and value of the credit for it are addressed early on, the authorship discussion is a comfortable one. Faculty accused of “stealing” students research ideas are less likely research thieves than they are guilty of not having the authorship discussion up front. A lot of this could be mitigated by avoiding assumptions about how group work results in credit. Unfortunately, as it is often the responsibility of the student, and not the faculty member, rarely does the faculty member initiate the discussion or put forward these ideas at the start.
7. Several studies can be written on the same data, but this means a separate strategy is needed for each. Planning ahead and deciding who is writing which study from a given project is needed in order to do this. Few projects are at this phase in the early stages. Simply designating one person as “doing qualitative” and another as “doing quantitative” doesn’t avoid conflict in the future because the terms are not accurate descriptions of research. In my experience, “qualitative” is too vague. Simply equating “quantitative” with anything study that contains numbers is also too vague. I would avoid the both terms in deciding who is doing what. If a project is in the early stages, and multiple studies are planned, some things may need to be worked out prior to the strategy and authorship discussions. In those cases, it is helpful to anticipate revising the strategy as you move forward.
The final three points might be specific to scholars in education. I don’t know enough about other fields to speak to how research groups might work most productively in those fields.
8. Look beyond the quasi-experimental study. One of the problems research groups have is that newer members may only see as far as the experimental aspect of the research. While many research designs in education will fall into this category of quasi-experimental study, there is much more to look at in most projects. There is the design of the intervention, close inspections of learner experiences, stakeholders’ accounts of a practice, and methods articles, just to name a few. Very often, these other aspects of the study are the most interesting part, once you dig into them. And the experimental study itself may face larger obstacles to publication than new members are ready to tackle: quasi-experimental studies are often closely read for precise usage of research terms, such as the proper labeling of the data, the correct usage of words like “control group” and necessary treatment of what makes them “quasi” as opposed to true experiments, such as having a placebo. The literature review is perhaps the most foundational aspect of any study that gets overlooked, yet group members stepping forward excited to do the literature review are rare. The person who does the literature review is an important and often overlooked role.
9. Revisit the strategy when major changes occur. One of the most difficult things to grasp about the act of actually doing research is that research always reads linearly, but is almost never so in practice. Rather, the process is iterative. I might also add that is it not a bad idea to revisit the strategy after the initial grappling with the data by the group. Research questions are a work in progress until the proofs go to publication, in most cases. Initially viewing data under a general driving concept is great, but keeping in mind that the RQs can change as the project evolves can really empower a research group, and if all the members are aware that the strategy can be amended at any point, people can be encouraged to follow fruitful paths. Avoid getting stuck in “causes” and proposed calculations in the beginning. They can be elusive, especially in early stages when data has just been collected and is not fully prepared. Also, some calculations may or may not be worth the time to calculate until the group has grappled with some of the main ideas. You can always add calculations later at reviewers’ request. If you haven’t yet dug into the data, and done a few passes through with your analysis scheme, you can’t truly foresee what that data really means; and therefore, the initial strategy may have been naive. Seasoned scholars are not deterred by this, but newer members may feel discouraged when data shows the unexpected.
10. Share the strategy document when new members arrive, and revisit it at the end. The research strategy is more than a Gannt chart. New members need to see the progress that has been made to catch up quickly and contribute, and simply sharing a strategy document with new members can bring them up to speed in a non-biased way, especially is sub-studies have moved in directions of their own. It has value up until the final publication is sent out, and perhaps even after that. A group research strategy can function as a collaborative narrative of the twists and turns of the project. Group projects that last over extended periods of time can lose track of the rationale behind important decisions and expected outcomes. Revisiting a strategy after a projects nears the final stages can offer a lot in retrospect about how decisions were made and how those decisions played out in the end.
On 11/16: I gave a talk entitled Instructional design & design thinking: How we develop knowledge in instructional design and used some slides that students might find handy later on. I covered a definition of instructional design, what is and is not design knowledge, and how we might build design knowledge for learning contexts. This is contrasted to how we build knowledge about theories related to instructional interventions.
On 10/22: I also gave a talk at Texas A&M University about my dissertation, but focused on the research methods. The links I used are below, along with the main points of what I had to say:
- My dissertation: Higher Order Thinking in Collaborative Video Annotations: Investigating discourse modeling and the staggering of participation. (2012)
- The new media created and studied I named Collaborative Video Annotations. I provide an example here
- I focused this talk on three terms underlined here: This was a quasi-experimental content analysis of learner discourse.
- Every study has some big findings and some lesser findings. My big ones were: “Learners in the staggered conditions groups produced twice as much higher order thinking than learners in the basic condition. Learners in the modeled discourse condition produced three times as much higher order thinking. The staggering of learner participation also required no effort and thus should be a requisite design feature in online discussions.”
Addendum: Pay-for-publication and self-publishing, in my field, are the same thing. It is a dangerous proposition for pre-tenure faculty to take part in these venues as neither count for tenure, and neither should count for tenure. It’s been just over a year since I posted about this and new predatory publishers are still soliciting me and trying to get me to pay them for publishing my papers. Some have even promised 2 weeks from submission to publication for a fee. New solicitations include:
- International Journal of Humanities and Social Science. Fees disclosed on this page: http://XXX.ijhssnet.com/index.php/faq.html
- Journal of Curriculum and Teaching. Fees disclosed on this page: http://XXX.ijhssnet.com/index.php/faq.XXX
The darker side of academic publishing is now part of my teaching thanks to a few solicitations over the past few months from somewhat questionable publishers. These publishers wanted to add me to their reviewer list or their “Editorial Boards.” I use quotes there for a reason. I have to wonder just what these editorial boards do. I am quickly learning that a knowledge of predatory publishing is required to understand educational research in an internet-connected world. One must be able to pick out what is worthy of citation and what is not. An email this morning reminded me to write this post.
While Macrothink Inc. is to credit on this occasion, previous solicitations to the most recent one have been similar. The business model is easy enough to understand given the context. Since faculty need to publish research to make tenure, there is a demand out there for the quick publishing of less-than-polished research, for cash payments. This means the authors pay the journal and the journal publishes the article. I have to question the quality of the research if there is financial incentive to publish it. Pay-for-publishing venues immediately raise concerns when there is such a dynamic in the process. What’s the barrier to publication if there is money behind the acceptance? I don’t find acceptance rates on their homepages. Every couple of weeks I get a request to review for such a journal- sometimes it is a request to be listed as a “Board Member.” With so many requests, I have become even more skeptical. We also need to ask just what role the reviewers have in donating their time for others’ financial gain. Is it ethical to ask faculty to donate their time towards a service that is financially driven?
The dynamic inherent in pay-for academic publishing hammers home the need for information literacy– a point Howard Rheingold makes at length in his new book- Netsmart: How to thrive online, which calls again for more information literacy. His recent talk at AECT 2013 brought up some familiar themes. He calls it a crap detector, but the skill set does not propose any new questions that others such as Damico’s critical web reader have been propoising for years.
The email however provides excellent course material for an activity, because it violates basic tenants of information literacy– the context of the communication should be studied before consuming the content. Below you will find a copy of the solicitation email from Macrothink. It is curious that the email signature is from “John White,” the email address is unrelated – “jet@macrothink,” and the registered name is “Leonard Bai.” These are great indicators of the nuances of reading skills that are required to make sense of email these days, a topic I addressed in An instructional paradigm for the teaching of computer-mediated communication here. The tone of this solicitation struck me as the academic equivalent of the queen of Nigeria emailing me for a business transition and bank account information. If anyone is interested in doing a lit review on predatory publishing in Ed Tech or Inst Design, the field dearly needs it. There should be more out there about how to teach this in effective ways.
All is not lost; we have a more general resource to help us. Jeffrey Beal at UC Denver keeps a running log of predatory publishers. Macrothink, the originators of this email, are listed on Jeffrey Beal’s blacklist. I have copied the solicitation email I received yesterday into the space below. I have also removed their links (by adding XXX in the http space) because on the internet, links have meaning. I don’t want to appear as endorsing this publisher.
Dear Dr. Craig D. Howard,
I have had the opportunity to read your paper “An instructional paradigm for the teaching of computer-mediated communication” in Instructional Science and can tell from your work that you are an expert in this field. We are recruiting reviewers for the journal. If you are interested in this position of reviewing submissions, we welcome you to apply for. Please find further details at XXXmacrothink.org/recruitment/
I am John White, the editor of Journal of Education and Training (JET). Journal of Education and Training (JET) is an international, peer-reviewed online journal published by Macrothink Institute, USA. This Journal publishes research papers cover the whole range of education and training, aims to provide an international forum for the exchange of ideas, opinions, innovations and research on topics related to education and teaching. Professionals and researchers are encouraged to contribute their high quality, original works of the field.
It is a great pleasure to invite you to contribute your best knowledge and research. Your contribution will help us to establish a high standard. We use a double-blind system for peer reviews. The paper will be peer-reviewed by at least two experts. The review process may take two to three weeks. If you are interested, please find the journal’s profile at: XXXmacrothink.org and submit your manuscripts online. Or you can send your paper directly to the e-mail: XXXmacrothink.org. If you have any questions, please feel free to contact the editorial assistant at email@example.com.
We would appreciate it if you could share this information with your colleagues and associates.
Journal of Education and Training
5348 Vegas Dr. #825
Phone: 1-702-953-1852 ext.534
Update March 2014, they keep coming:
Dear Dr. Craig D. Howard,
We are reaching you because of your paper entitled, ‘Designerly Talk in Non-Pedagogical Social Spaces’, which was published in Journal of Learning Design, 2014 Vol. 7 No. 1, and were very impressed at its scope and contents. We know you are an expert in your research area.
Our journals named ‘World Journal of Education’ and ‘Journal of Curriculum and Teaching’, peer-review, published by Sciedu Press. They are devoted to publishing research papers in various aspects, fields and scope of the education, teaching, learning and other relevant subjects.
It is our great honor to invite you to submit your new manuscripts to us as one of the ‘Authors’ in our next publication.
If you are interested to be a member of our editorial board, please find the application form and details at: http://web.sciedu.ca/recruitment.html and send the application form to: firstname.lastname@example.org or email@example.com
We would appreciate if you could share this information with your colleagues and associates who might be interested in joining us as a ‘Reviewer’ or submit their manuscripts to us as ‘Authors’.
Thank you and we hope to hear from you and/or your colleagues and associates soon.
Ms. Sara M. Lee
Mailing Add: 1120 Finch Avenue West, Suite 701-309, Toronto, ON., M3J 3H7, Canada
Tel: 1-416-479-0028 ext. 218
here’s the fine print:
This journal charges the following author fees.
Article Publication Fee – World Journal of Education: 300.00 (USD)
If the paper is accepted for publication, you will be asked to pay an Article Publication Fee. Please find payment information at: http://www.sciedu.ca/payment
The International Journal of Designs for Learning and the differences among Design Research, Design-based research, and Design Science
I was very happy to receive an invitation to join the advisory board of the International Journal of Designs for Learning (IJDL) yesterday. IJDL started in 2010 with a purpose to fill a very specific niche in scholarship in instructional design. Until that time, there was no venue exclusively set aside for design cases, rigorous analyses and reflection on specific designs for learning (Boling 2010; Smith 2010; Howard 2011). I was in the right place at the right time. I had been a volunteer assistant editor for Tech Trends’ Instructional Design Portfolio feature, a regular column allowing authors to share designs. The ID Portfolio was a fantastic first step and had expanded the field’s awareness of this type of research. However, the venue limited the length of the articles and was print media despite authors’ and reviewers’ desires for more space and oftentimes animated or interactive expressions of the designs. Design cases in IJDL are essentially unlimited in length (the guidelines say 9,000 words, but more than 11,000 is common), and the format is flexible—multimedia is the preferred choice of expression for approximately 20% of the authors who contribute to IJDL. While my dissertation was not a design case, I became very interested in this form of scholarship and subsequently wrote my own case. I also wrote an analysis of the difficulties authors find in publishing this form of design research, and a definition of this form of knowledge building. What an opportunity it was to work with Gordon Roland, Kennon Smith, and Elizabeth Boling, and all the advisory board members in those first two years the journal was up and running. Recognition for this type of research is growing, and it should be. Both practitioners and scholars of instructional design need to understand just what the goals of the different forms of research are in order to make practical decisions about where they invest their reading time. Alan Foley presents it well in his graphic that he uses to depict to graduate students the different types of research in their field that include the term “design” in their title. See the graphic below.
For quick reference, here’s the consensus that’s growing around these terms: Design research has two types—one that speaks to how we design using various traditional methods of research, and another type, the design case, which speaks to the precedent a designer, or someone close to a design, has drawn out of the process and product of a specific design (Boling 2010; Howard 2012). Design-based research (Squire & Barab 2008) speaks to theories using design as the means for making those insights. Research in Design or Formative Evaluation is a type of research where the goal is to improve a specific design. Design and Development Research aims to establish empirical grounds for design practices. And finally, Design Science, a term Kevin Williams introduced to me, is parallel to design-based research placing emphasis on abstraction and theory building via the act of design in Information Systems. Check out this link to see just how similar the two terms are defined between Information Systems research and Instructional Technology research.
Most discussion about MOOCs (Massively Open Online Courses) seems to miss the point entirely. I have been sent a solicitation to talk about an informational graphic which lauds MOOCs, but have also discovered an excellent discussion of the dangers of MOOC-ing. Both seem to make the same error. We get a lot of discussion about these technologies from people outside of education. Some even start their positions with statements that they question their own legitimacy. Let me back up and provide context to any discussion of MOOCs that will demystify the topic and situate it among the larger scope of what we know about learning.
The MOOC is an instructional strategy that has arrived precisely as was predicted over 2 decades ago. In 2006, Ted Frick, PhD, now an emeritus faculty member at Indiana University, made a statement to a group of incoming PhD students. He said that it takes the educational community about 30 years to figure out how exactly to use a new technology for teaching and learning. Tim Berners-Lee created the web browser in 1990, and the first web browser to muster widespread use was Mosaic, which appeared in use in 1993. 2013 is exactly the date educators like Ted Frick expected to see the educational community grasp and run with the potential of web technology, but of course we expect 10 more years of significant developement. Instructional designers and educators who have made careers in distance learning are not surprised by MOOCs. They expected that by now, the community at large should understand the web well enough to apply it to teaching and learning in systematic ways which realize its potential. At the same time, the hype surrounding “Web 2.0” is not all forgotten. In the late 2000’s, Tim Berners-Lee said that Web 2.0 is jargon aimed at creating hype around aspects of HTML that have been there since its inception. So is the story with MOOCs. There is nothing technologically different with today’s MOOCs in comparison with regular technology that has been around for quite some time. Therefore, one can conclude nothing other than that MOOCs are an instructional strategy, not a technology.
There are many instructional strategies. Some encompass whole curricular designs and have multiple sub-parts, some focus on discrete points. Some are designed to teach process (for example, apprenticeship), some teach concepts (cloze activities), and other strategies teach attitudes or mindset (boot camps). Here are some instructional strategies you might be familiar with: the multiple choice test, the lecture, listen and repeat, group work, e-learning, and flash cards. Here are some instructional strategies you might not be familiar with: collaborative or pair testing (Mark Evan Nelson), the Silent Way (Gattengo), airborne instruction (MPATI). It is the very rare instance that one instructional strategy encompasses the entirety of an educational system. Can you imagine an entire course of study containing only lectures? The multiple choice test may be a sufficient strategy for certain aspects of teaching, but one would surely not want to do an entire curriculum via multiple choice tests. Using one instructional strategy for everything is rarely prescribed by those who design instruction on a regular basis.
Yet we find the arguments surrounding MOOCs to be primarily based on the assumption that the MOOC is a form of education, or that the MOOC is a new technology. Neither of these notions are true, and the majority of educators I have encountered understand this. One indicator that these discussions have little to do with education is that very few educators are taking part in the discussion. Some educators who focus on learning technologies, for example see Curt Bonk’s extensive website, put MOOCs in line with the many other technologies which have changed the landscape of distance education. But on the other hand, one of our nation’s leaders in education has raised the topic of teaching and learning via the exact opposite of the MOOC (Arthur Levine, example article here). He asks, instead of one teacher teaching many, what if we had a many teachers teaching an individual for longer periods of time? Sure, Arthur Levine is talking about technology in education, but he’s putting it in the larger scope, and considering a number of perspectives. When educators talk about education, this is how they do it. They consider a multitude of perspectives, and contexts, in light of the world we live in now, and the larger lives of the learners in our charge. Levine is not saying one instructional strategy fits all cases, and Bonk is viewing MOOCs as not a solution– rather an option among many. Both are saying we need to re-examine how, what and why we teach the way we do.
In this light, let’s look at some of the rhetoric surrounding MOOCs. Allison Morris of onlinecollegecourses.com has presented me with an informational graphic to help me understand MOOCs. In her graphic she makes some assertions that may warrant a closer look. She writes that “They’re growing at a rapid rate.” This would be true of any instructional strategy that is entering the main stream after many years of development and distribution. I would be more interested in seeing the growth in popularity of the multiple choice test because at this point, I think the educational community knows something about how multiple choice tests work. Binet got it down as a true proxy of reason back in 1905, and Thorndike got the multiple choice test into the form we now recognize, about 10 years later circa 1915. I would imagine that we started to get good at creating and applying multiple choice tests in the 1950’s. Where is the multiple choice test now?—everywhere. The multiple choice test has had its day and now its popularity is waning. Debates surrounding the common core reveal the limits of the instructional strategy. The limits of the MOOC are, as of yet, still murky. If Morris is correct with her assertion, the implication is that we’re going to see a lot more development of the MOOC. Growth in popularity means the strategy is not mature, and just like any other strategy it can be applied well, or poorly. What I do like about Morris’s informational graphic is the way she has laid out the content — a powerful method of conveying an argument quickly. But what we have in the end is an argument that holds no bearing on MOOCs; popularity, demographics, and a description of the designers and participants, does not have any bearing on the efficacy of an instructional strategy.
Fig 1: Allison Morris, The Mind Behind the MOOCs
The discussion would not be complete without the dystopian perspective that MOOCs are horrible. I have seen this theme reappear in the Chronicle of Higher Education repeatedly over the past 12 months. Early last year one scholar warned the Chronicle’s readership that content delivery does not equate to education, and that this error in reasoning would make real education a luxury for the rich (Graham, 2012). Another seasoned scholar wrote more recently that MOOCs are a product of the time, to blame for part of our collective degradation of educational practice, making today’s education a re-instantiation of Wal-mart (Patrick Deneen, June 7). I don’t think it is unreasonable to say that the reason we see positive perspectives of MOOcs on literally thousands of blog posts (Blog + MOOC retrieves over a millions entries in Google searches) and dytopian perspectives on MOOCs in articles in the Chronicle is that those who support the practice are generally not scholars—they are internet users who are involved in online education, often exclusively in at-a-distance education, and the former are scholars who are quite adept at writing for publication. Deneen’s argument that MOOCs are the Wal-mart of education is well constructed, elegant, and at times even funny. I suggest everyone partaking in the discussion read it because the article raises relevant topics and is well written. But, the argument does not stand. Here’s why.
The fact that educational strategies are changing is not evidence that one strategy is wrong. Education is always changing, and should change, as Arthur Levine argues, because society is always changing. As educators we must keep education relevant because it is directly linked to society, and learning that is out of synch with life is always lacking. Signs of the times such as initiatives for “measurable outcomes” and “assessments” (that are also under debate, by the way) are not an indication that designers of instruction, or institutions that are adopting those designs, are down a path of doom. MOOCs may promote a monoculture, as Deneen argues, but we have seen no indications that the monocultures of MOOCs are actually any different from the monoculture of required textbooks, or other standardized content which gains popularity across nationwide curricula. (For example, is English 101 really different from institution to institution?) While a Wal-mart of education may not be all to attractive, who is to say that the Wal-marts of the world are not a driving force behind the popularity of farmers markets which also did not exist a few years ago. MOOCs may just be the strategy which raises our awareness of the diversity of perspectives and backgrounds that enrich our institutions. Distinctions are now easier to draw between one instructional approach and another because they are more visible via MOOCs. However, this does not credit nor discredit an instructional strategy. James Paul Gee may have had a more insightful observation when he wrote about the rise of the distinction between uniform and nuanced curricular approaches, making boutique colleges and universities an educational landscape apart from the commodity institutions. Visibility itself is not a criteria for quality practice.
Thank you to Allison Morris for prompting the conversation I have wanted to post about for about a year. I hope this post will prompt good discussion from students of instructional design, and lay to rest from of the misguided rhetoric that looks at MOOCs in black and white terms without seeing it as another strategy that will have a life of its own like any other, for better or for worse. I welcome comments that bring to light my errors in this post and take the conversation in a positive direction.
The July 19, 2013 Issue of the Chronicle review contains an informed response by Clay Shirky to Patrick Deneen’s rant against MOOCs. He seems to be taking a similar position as mine; he writes, “The effect of MOOCs on the academy is no more likely to be about pedagogy than the effect of MP3’s on the music industry was about audio quality.” His argument is primarily based on the economics, but he brings a larger understanding of how educations functions to the discussion of MOOCs.