craig dennis howard

The rhetoric of citations in Learning, Design and Technology

Low_res_1984-chevrolet-citation

This is the 1984 Chevy Citation. This is a matched non-example of something else that is generally out of place as a citation in a LDT paper. Photo credit goes to myclassicgarage.com, and if they ever back link check my use of this image, they are gong to be seriously confused about why this image is here.

Citations are rhetoric, plain and simple. Each time you cite something, you are employing a rhetorical device. When you employ the device erroneously, you make the case to your reader that you don’t understand what you are trying to say. The most common rhetorical citation device I see is the consensus citation. A consensus citation is when you make the case that a whole bunch of scholars agree about something. For example, multiple citations on a given point express how pervasive the consensus is among scholars. Here is an example: Scholars agree that asynchronous communication is democratized (Chun & Plass, 2000; Gunawardena, 2004; Howard 2012; Johnson, 2006; Stockwell, 2007; Warschauer, 1997).

The same sentence with only one citation is not just a case of fewer citations, it’s actually a different rhetoric altogether. It suggests the author is the one make the point, the first one to coin the term, or the only you happened to encounter that mentions such a thing. For example, consider if you had the same sentence from above with just one citation, “Scholars agree that asynchronous communication is democratized (Howard, 2012)”. This sentence means that Howard (2012) claims that scholars agree about this, but you as the author of the piece, may or may not agree. Contrast this to the meaning of three or more citations in the same space. These are two different rhetorical moves, both made via the use of a citation.

Sometimes we find a writer will inadvertently credit a concept when they mean to credit an explanation. A direct citation or a paraphrased citation credits an author with a concept, but some scholars inadvertently credit an author with a concept when the author has simply used the concept or explained it more elegantly than others. For this second case, we have the rhetorical device “see” within the citation space. If the author simply explains the idea and doesn’t originate the concept, add “see” to make that clear. Thus the sentence, “This will be a Criterion-Referenced Assessment (Mager, 1997) to measure my students’ knowledge of each subject area based on the Common Core Standards.” Gives credit to Mager (1997) and the sentence, “This will be a Criterion-Referenced Assessment (see Mager, 1997) to measure my students’ knowledge of each subject area based on the Common Core Standards.” Lets the reader know that Mager (1997) is a good place to find a discussion of this concept, though Mager is likely not original contributor of this insight.  In other words, this first citation says the author (Mager) coined the term, or that the author is using an interpretation of the term that is drawn from Mager because there are other ways to interpret the term. And in the second example, Mager simply explained the term, as suggested by the rhetorical “see” prior to the citation.

The rhetoric of citations can also backfire on you if the relationships between the written work and the argument are unclear. If you have used a section in a way that exposes faulty logic, you can evidence that you don’t understand what you are talking about. For example, I sometimes find advanced learners use citations in such a way as to imply an implausible logical relationship, such as a causal relationship that actually does not exist. For example, look at this construction, “Participation has been argued to be an intrinsic part of learning,” (Wenger, 1998; Hrastinski, 2008) therefore the videos provide clients and their staff opportunities to learn about the features of the software during the course of integrated authentic performance assessment.” This means that because Wenger said it, the learners will learn. But f course the learner neither care nor heed what some obscure educational researchers say, so the reader dismisses the writer as misinformed, or worse. What a tragedy when the student is actually an insightful designer explaining complex designs! Had the writer phrased structure the rhetoric of the citation more accurately, their design insight would have shown through. For example, read the revised statement: “There is moderate consensus that participation is intrinsic to learning (Wenger, 1998; Hrastinski, 2008). Therefore, with this in mind, we designed the videos to provide clients and their staff opportunities to learn about the features of the software during the course of integrated authentic performance assessment.” With the questionable logic removed, the writer’s insight becomes far clearer. For this reason, I ask that none of my learners attempt to finish direct quotes in their writing. The rhetoric can too easily run amuck.  The same goes with direct quotes with two authors, which is impossible, and unfortunately, more common than I would hope.

Sometimes, the citation is the right rhetorical move. The rhetorical value of a citation does not trump a first-hand account; rather the citation is another perspective altogether. For example, “I noticed learners tended to complete online discussions only in the final hours prior to the deadline (Howard 2012).” By adding the citation, you have effectively told the reader that you are crediting someone else with what you saw. This makes no sense and is at the same time, a common error. If you witnessed it, that is evidence enough that it happened. If the phenomenon is also witnessed by another researcher, you’re free to say so. However, one separate the two perspectives or your point will be lost and the reader confused.

The date is part of the rhetoric of a citation. Furthermore, citations must always have a date; otherwise, they are not citations. The rhetorical value of the date has meaning on many different levels. For example, by citing a much older publication, a researcher can show that a concept has been churning through the mind of other scholars for quite some time. For example, “Design precedent (Lawson 1990) is a concept integral to our understanding of design Learners might grow in their understanding of how to design (Boling 2010).” Scholars often change their position on something over the course of their career, so the date is as important as the name. Also, if there really is no date, it is wisest not to include the reference unless you’re employing a rather complex rhetorical strategy, such as mocking public opinion.  For a detailed discussion of this, see my blog post on citing Wikipedia. But in general, I advise my learners that if there is no date, it is not worth citing at this point in their learning.

I often find myself suggesting to authors not to put citations in the first sentence of a paragraph. The reason I advise this is because, in such a position, the rhetorical value being employed by the citation is unclear. The topic sentence needs to lead the reader to the evidence behind the argument. When we have a citation in the topic sentence location, the typical role of citation as evidence can’t be used, so the reader goes searching for what the paragraph is actually about. But the topic sentence is supposed to say what the paragraph is about. This confuses the reader. I am sure there may be cases where putting a citation in the first line of a paragraph could be employed advantageously, but in 20 years of teaching writing, I am yet to see someone pull it off.  A citation in the first line is simply not a wise move.

Lastly, citing websites’ about pages, or product pages a firm produced about its own products, is very poor rhetoric. It makes the claim that you, as the researcher, trust an advertisement as much a scholarly paper. I can’t think of any scholarly context in which this strategy could be viewed as rhetorically sound. For example, the reaction one might fetch from this citation is easy to predict, “YouTube is arguably the best website ever known to mankind (YouTube, 2015).” Other citations from product literature, be they from web sources or elsewhere, are effectively the same though they may be less obvious to the learner. I strongly encourage my learners to question the source and not take a company’s literature at face value. I would not trust any corporation to give me an unbiased evaluation of its products in an advertisement, which is basically what an about page is.

If you have other citation rhetoric insights, I would love to hear them. Please send me a comment.

Scaffolding the five finger filet

Instructional graphic designed to teach the Five Finger Filet game. Notice the five dots below the thumb depicting the pattern. Erena Furuya Howard, pencil on paper, 2017.

There is a game called the Five Finger Filet. One holds a knife and stabs between thine outstretched fingers, risking a dangerous errant stab. Personally, I never had any attraction to games like this. I would surely chop off a finger and regret my own stupidity in a matter of moments. This however doesn’t dissuade my 5 year old daughter from trying to get me to play such a game. Of course, I show that I will get it wrong and continually mess up, stabbing ridiculously far-off and in the wrong pattern. To instruct her father, Erena designed this quick instructional scaffold. The location of the dots is what is important. Notice the string of five dots under her thumb. That’s the pattern, to each finger and back. I actually had not known what the real pattern was. But what caused me pause is how natural her behavior was. If a five-year old so naturally creates visual instruction, what does it say about the way we teach and learn that teachers and masters students in education rarely chose that avenue? Is any one else getting the idea that the pendulum has swung too far toward understanding learning as simply reading, writing, and nothing else? Maybe I am just chopping off another finger.

The Nutshell Studies: An instructional design case worth listening to, again

Frances Glessner Lee, Parlor (detail), about 1946-48. Collection of the Harvard Medical School, Harvard University, Cambridge, MA, courtesy of the Office of the Chief Medical Examiner, Baltimore, MD. Photograph by Susan Marks, Courtesy of Murder in a Nutshell documentary. I believe this is how my students feel during the academic crunch time in November, but that’s not why the picture is here.

Years ago Elizabeth Boling mentioned this classic instructional design, The Nutshell Studies, and I just had not run into a thorough discussion of it for years. It is an instructional design intended to support the learning of forensic science for detectives. The most remarkable part of the design are 19 miniature dioramas which make lifelike crimes scenes available for viewing by multiple detectives in training. A perfect instructional design case for a historical issue of IJDL, but the dioramas, and the life of their creator Frances Glassner Lee, were recently featured on an NPR show called Sidedoor, and in all honesty, Tony Cohn from PRX does an excellent job, hitting all the required components of a design case. As AECT is just around the corner again, I am sure I will be talking about instructional design cases again in the near future, and this Sidedoor episode is a case worth discussing.

Frances Glessner Lee and The Nutshell Studies of Unexplained Death, is actually the name of an art exhibit, but is spawns from an instructional design, from which Cohn crafted his audio instructional design case. Mrs. Lee created complex dioramas with such detail that they could tell the story of an entire murder, well, almost the entire story. A curious aspect to her dioramas was their purposefully unsolved-ness. There is no solution offered to the detectives in training. In other words, despite their intricacy, they will always remain, unsolved. This 1940’s design choice of creating unsolved murder scenes, fodder for discussion but not for resolution, speaks to the persistence of discussion, rather than resolution, as being an ever-present aspect of higher order thinking. The key is not to select the right answer, but to untangle the complexity and multiple possibilities that could play into each scene. Of course, the hope is that with practice, learning the process of disentanglement will result in better investigations, leading to a higher chance of getting things right eventually.  Once we let go of the idea of always getting is right this time, and focus on getting it right most of the time, our chances go way up each and every time.

Learners in the Master’s program in IT here at UT chuckle about my choices of terms: grapple, dig in, go deeper, dissect, take apart.  But these terms do tell what I am trying to accomplish. I don’t think we’re ever going to learn definitively how to teach (sadly) and I don’t think we’re ever going to definitively determine the right way to design instruction (woefully), and , that’s not the point (joyously). The point is, to see a design like the Nutshell Studies for more than just the dioramas, though, for sure, the dioramas were pretty damn cool.

New digs and old issues

Since stopping this blog in February, ideas have come and gone, and my apprehensions about leaving Texas A&M worked on me to focus and prepare for a new start. That start has come; it’s September and I am teaching in a new context. While I am very happy to be here in Tennessee, I am also trying to work out exactly where is the best place to invest my time, thus the new banner image and a renewed concern over the clock. One thing that is not in question is the purpose of this blog. It’s a place to work ideas out in a more public venue, allowing for comments from anywhere, and from anyone. This month’s post is such an opportunity. It is inspired by a dialogue that came about organically from a posted comment by a reader.

graphic of a child's artwork

古矢愛怜夏 / Erena Howard
Daddy at his new office, 2015
Crayon and multimedia on paper
18 in x 12 in

The issue I am dealing with here is an old one, but as Marshal Poe writes, our perspectives on right and wrong are very much tied to the times- the way we communicate and who gets to communicate to whom about what and how often (2011). Had this issue arose at my workplace, I probably would not find out about it because of the nature of the issue; some topics can be discussed on a blog at another university that would not be welcome at one’s own institution. That’s one area where the blog excels. It creates a fruitful venue to discuss topics outside of one’s own institution.

This discussion started as a comment to Publishing as a graduate student and developed into something worth a little more space than the comment trail, I think. What follows is the email discussion with a few headers added in after the fact. Eric Hufford, the author of these questions and the unfortunate graduate student in the pickle described in this post, is a senior software engineer in Maryland with 10 years of experience in the field of software engineering. He holds a B.S. in Computer Science and is pursuing a Masters in Software Engineering as well as an MBA.

Eric: Thank you for this fantastic information. I do have one question for you though, as I am struggling with this before I submit my paper. I wrote a paper for a graduate course. The paper I plan on submitting is exactly as I turned it in to the course with absolutely zero input from or collaboration with the professor. In fact, his entire involvement with the paper was approving the topic (it was a term paper) and grading it. Yet he is telling me that since I wrote it for his course, he needs to be listed as an author. I strongly disagree with his statement. Is this something I should be concerned about? Do I really need to add him as an author?”

Craig: Wonderful question and thank you for submitting it. I will use headers to organize and emphasize my points, because this response is lengthy.

Know the authoring conventions for your field
There are a few things that need to be clarified prior to saying your professor is outright wrong. First off, this blog post was meant for graduate learners in Instructional Technology, but might also be broadly applicable to social science research in general. I don’t know your field of study. Much depends on the conventions in your field. If your paper was developed from someone’s course which included access to a lab where data was collected, it is very likely that authorship listing the lab leader is the most appropriate convention. Also, if the paper topic was developed iteratively with the professor, that in itself may very well constitute input worthy of authorship. Your professor might also be refereeing to content in the paper that was derived from teachings in the course, in which case the professor would be making a legitimate claim. Both contributions and conventions should be discussed prior to any submission of the article anywhere. Graduate students should inform themselves of some of the more general conventions using authorship guidelines published for their appropriate field. There are guidelines out there for determining authorship. In my field, Instructional Technology, AECT has some, but more generally, there is an APA guide online. Read it closely. Knowing these conventions will help you learn more from your discussion about authorship with your professor. This is a key learning step in graduate school and one often overlooked in the process from admission to graduation, and possibly the root cause behind the uncomfortable feelings expressed in the comment.

Course papers are never journal articles, they are, at best, first drafts of journal articles
Secondly, one phrase in your question strikes in me a note of caution, “The paper I plan on submitting is exactly as I turned it in to the course.” This strategy runs the risk of getting a swift and straightforward rejection, needlessly uses the time of an editor at the journal you submit to, and moreover, runs you the risk of sparking a negative image of your work in the eyes of the editor. No paper should go to submission raw. Now that the course is over, re-read the writer’s guidelines of the proposed venue, and make sure you are familiar with the prospective journal’s most recent publications on the topic you cover. I expect there’s a fair bit of editing to do. I have heard students complain that established professors can submit less polished drafts than students and get away with it. I am not fully convinced this is true. Scholars that have been dealing with a topic for many years may have a battery of key phrases and can use shorthand to express concepts and other ideas. A learner’s text faces the formidable obstacle of being a relatively new topic, and without the crafted language at the tips of your fingers, it must be well crafted though scrutiny and revision. You haven’t been reading/writing for enough years to know them all the phrases you need, and a facility with them only comes with extended exposure.

Collaborations must be mutually beneficial
More strategically, it would be wise to rethink your take on collaboration in the field. From the perspective of the student, credit for a single-authored publication is paramount. However, from the perspective of scholars in the field, it is likely not so. The co-authored paper is worth more, not less. When a potential candidate’s cv evidences multiple collaborations that resulted in publications, it sends a far stronger message of competence than a cv with a few single authored published papers. Scholars publish dozens of articles over their career, and their co-authoring relationships evidence the breadth of their participation with topics of interest to others doing work worthy of publication. There is much to gain from collaboration and showing that you work well with your professors, even if this particular professor does not spur in you the desire for a long term identity of association. You have far more to gain from the professor’s name being on the byline than you have to gain from being sole-author, even if you plan on never co-authoring with this professor again. Your paper is also more likely to be read because inclusion in the professor’s online archive may draw readers surfing your professor’s publications, and it may also heighten your rank in internet searches if your professor is more published than you are—which is more often the case than it is not.

Strategize
At this point, your task is to get yourself informed about the appropriate conventions for authorship, make strides in the development of the paper for publication, and bring the draft back to your professor to have a more informed conversation about the draft and target venue. It may feel like eating humble pie, but it’s not. Even if you are vastly more experienced and it is a junior faculty member you’re dealing with (which does happen as grad students come from all walks of life and faculty can come straight from successive years of schooling) what you do not want is to publish the paper without the professor’s approval, especially with verbatim sections from the paper you submitted in the course. That runs the risk of you being marked as someone who does not give credit where credit is due, could alienate you from other scholars in the future, and potentially endanger your completion of the program where you are now.

Eric: The paper in question was an end-of-term paper for a Software Engineering course, and I have scoped out a few different journals and honed in on two that I feel may be a good fit. I plan on re-tooling the paper to fit within the journal guidelines, but other than that, I plan on submitting the same basic paper. That is to say, I do not intend on taking any input from the professor’s review before I submit it. It sounds as if he is attempting to take credit for his student’s work even though he had no involvement in the process of creating the paper. [He only taught the course in which the paper was submitted.] Do I really need to add him as an author?

Craig: Yes, but more importantly, why fight it when the outcome (s)he proposed actually favors you? Nevertheless, the issues you raise here are endemic to graduate school, and actually, really important. Every year this issue pops up somewhere- often between a junior faculty and a senior doc student. I really think pushing this to a richer discussion might uncover something valuable for other grad students and those who teach grad students. So, let me ask you this– do you think it was something in the classroom dynamic that lead you to want to publish the paper without his name on it? Is there something the unnamed faculty member didn’t do that s/he should have done? What brought you to so fervently want to publish it without his name on it?

Eric: I’ll answer your last question first before going back to your first two. “What made me feel like I didn’t need to include him as an author?” I suppose it is relevant to provide some background information on myself. I am a professional software engineer with about 10 years’ experience in the industry. I finally decided to pursue a masters’ degree and an MBA. This particular course was entirely online so I never really developed a rapport with the professor. While he is a senior faculty member and appeared to be a fairly competent fellow, the course content was nothing new to me. I never opened the book, nor did I ever have any conversations with the professor beyond those required to complete the course (like approving my term paper topic). In other words, the class was a breeze, I received an A and moved on with my life. With that said, I felt that it would not be necessary to add his name as an author because he and I had a very minimal amount interaction, and it was only ever via email. There was never a relationship or any sense of loyalty. However, I try to remain open minded as best I can, which is why I came across your post and asked my question. I simply didn’t want to do the wrong thing and have it come back to bite me later. I was also under the impression that if I shared authorship, that my involvement in the paper would be diluted, which I assumed would reflect less upon me. In your response, what convinced me to include his name on the paper regardless of how little he did, was your argument that it looks better on me to have a co – authored paper. You mentioned that it would make it look like I perform well with others and, more importantly, if he is well published, my paper will get more visibility than it ever would under my single authorship. This may sound heartless and blunt, but these are the things that matter most to me about publishing my paper. Your answer surprised me, it wasn’t what I was expecting at all, but it made a ton of sense.

Craig: The online dynamic raises lots of curious questions.

Unspoken information lost in the lean media
Low-interaction online courses can be a breeding ground for conflicting perspectives, especially if they appear early on in the program of study. In order to create a smoother pedagogical experience for both learners and the instructor, some programs require foundational courses that may fall below what an advanced learner is expecting. In such cases, the indebtedness of the learner to the knowledge of the professor is a stretch for advanced learners to recognize, and surprisingly often, that lack of indebtedness is completely justified, especially in online classes where the direct instruction is limited. In some online classes, it’s not hard to imagine just anyone teaching the course if it basically runs by itself with little instructor-student interaction. A close working relationship is implied by a shared byline, but in a course where the interaction has been so little, it is bound to feel weird to share credit. At the same time, it is difficult for the instructor to recognize the insights did not come from the course, especially if those insights could have been inferred from the materials assembled by the professor. I wonder if such a misunderstanding is the case in your context.

The lean media of online instruction, often limited to text interactions, often requires a seasoned online instructor to recognize insights that break from course materials into new ground for the learner. Junior faculty with limited online experience, and even seasoned faculty who are new to the online format, may be relying on cues that simply are not there in online interactions to recognize when a learner is breaking from simply what they have been taught, into original research. He may well have thought you were simply building from the given materials. Having not opened the book puts you at a significant disadvantage when arguing that you didn’t learn x or y from a course of study, and at the same time, sometimes it takes a seasoned online teacher to recognize truly original work.

Differing expectations, none of which are written down
On top of this, these instructors may be working with a hidden curriculum that is never made explicit to the learner. Online faculty may have tacit expectations of a set of rules of interaction leading to co-authorship with grad students, and at least in my experience, rarely are these understandings made explicit in a course syllabus or other materials that the online learner is exposed to. With online learning we generally default back to the face to face rules until people figure out a better way to proceed and new conventions are adopted, but we rarely make it as explicit as it should be. Who wants to read, nonetheless write, a set of conduct rules? In graduate programs, for all the documents we produce, there are still topics navigate via intuition instead of explicit guidelines. How often do graduate students read the handbook anyway? I know I did not labor of over such a document until an issue arose. While I would say it is the responsibility of the instructor to make those rules transparent as best they can, the large number of exceptions makes that goal pretty far reaching and arguably infeasible, especially for someone who is not used to teaching online where there is far more emphasis placed on documenting procedures.

Advanced students may also not be aware that in the context of the professor, they may feel they are working at a disadvantage to their peers. If a professor teaches exclusively online, working relationships with students may be very challenging to construct. I know that in my experience at Texas A&M, it was definitely a challenge. In many schools, professors are expected to publish with their students, and for a faculty member teaching a significant portion online, I could foresee a claim that a paper resulting from a course they teach is a byline they “deserve.” But I am not so sure that the claim is warranted, and I am not so sure professors who teach exclusively online do in fact uniformly feel this way. But, this perspective makes sense to me.

Discussion is part of scholarship
The more important thing here is your intuition to go out and run the idea by somebody else who has grappled with the topic before you jumped to a conclusion as to right and wrong ways to proceed. That’s wise, and actually says a lot about your instincts about how scholarship works. Everything is discussed. But the discussion does not stop with the third party.

While you’re not wholly wrong in your assumption that there’s something diluted about the nature of co-authorship, understanding how authorship is credited in your field is what’s paramount here. The authorship discussion is an assumed part of scholarship and one that the publication venue trusts that you have completed. (Institutional Review is another research component that publishers often handle on an honor system.) For a scholar, this authorship discussion appears in other locations as well. In education, each tenure portfolio contains a document where you list a percentile authorship for each work on your cv for which you are claiming is evidence for tenure. Thus for each article, there will be percentage numbers next to each author’s name. I think it is assumed that the tenure candidate checks with the other authors that the numbers jive with how they felt. I have been asked this in the past — did you contribute 25 or 30%? Absurd, yes, better that other alternatives, also yes. In truth, I find that even the most diligent scholars are somewhat lax about this trivial task because it is hard to actually figure out who did what percentage when each person is working as a team, but they don’t skip the discussion altogether.

The important part is that the discussion happened, not whether 29% or 30.5 % was in fact the most accurate representative figure for authorship/research contribution. Reasonable scholars know that authorship is hard to quantify. For example– how would you and I divide credit for this blog post? I wrote more words, but you wrote the important questions. Word count is not a realistic reflection of value here, nor in research, but that’s how it’s done, along with a little negotiation. The percentile contribution is a make-shift fix for a problem we haven’t figured out how to reliably solve, and I don’t see a resolution coming apart from the fact that the discussion is required. I predict that as collaborations become ever more prominent, the percentile distribution document will become more of a guestimate, and at the same time, more important for professors to show they are in fact, working collaboratively with their peers.

Eric: My only response is a more personal one in that I do not intend to enter the academic community professionally any time soon. I quite enjoy working on products more so than research. However, I would like to have an academic body of work to fall back on 30 or so years in the future. Someday I’d like to get out of the development game and “retire” to academia. I am currently unpublished so it seems to me that adding him as a coauthor is my best option. Seeing that I have plenty of time (30 or so years) to build this, getting my foot in the door and putting my name out there is all I am looking to do.

Craig: This is an easy answer, but one you would be wise to note, and note quickly.

The unlikelihood of falling back on an academic career
I can’t tell you how many times I have heard this strategy. Contrary to popular opinion outside of the academe (those that know about it), an academic career to fall back on does not exist. Of course adjunct work abounds, but just read the Chronicle of Higher Ed to understand just how below slave labor contingent faculty suffer. The only people who ever “retire” to academic positions are intellectual celebrities. For example, Henry Kissenger, now at John Hopkins University, entered his post after a long career in politics, and there are occasional superstars from each field who find they have something to say that schools are open to hear. In their role, these celebrities in their fields do not necessarily do the work of typical professor either. I don’t know how they are paid, but I imagine it is worked out on an individual celebrity basis, and they are doubtfully paid as typical professor. I am not saying it is impossible to enter the academe after a career in another field, but these individuals are not falling back on an academic career Some may argue that such people are actually not entering the academe. Rather, they have something to say to members of their discipline earlier on in the career and the faculty is giving the celebrity an opportunity. For a detailed discussion of the struggle it takes to move from even the best PhD programs into a professorial position, read The Professor is In blog moderated by Dr Karen Kelsky.

While the hiring value of a publication generally dissipates into oblivion after five years, publication does hold a lot of value for professionals not intending to enter the professoriate. Publishing is concrete evidence that someone is up to date in a field of practice and can collect their ideas in a coherent and persuasive manner. The skills to put forward a polished argument is a very attractive feature for anyone looking to be recognized as a professional. So I would say it is darn good goal to have, just not for the reason you stated.

Acknowledgement: I thank Eric for humoring my excessive questions, writing frankly, and reviewing the final draft of this post prior to its publication on this blog.
Reference: Marshall T. Poe (2011). A History of Communications: Media and Society from the Evolution of Speech to the Internet. Cambridge: Cambridge University Press. xi + 275 pp. ISBN 9780521179447.

An organic peer review of an online class, and the Quality Matters rubric

anamorphic art by  UTISZ (Istvan Orosz)

Anamorphic Art by Istvan Orosz. Anamorphic art depends on how you look at it, and without the perspective, you might not see the image, much like a teaching observation of an online class. This Hungarian artist’s website is at: http://web.axelero.hu/utisz/page.htm

The last teaching task I completed for the year 2014 was a two-part peer teaching observation with Dr Sara Lawrence. But, unlike the teaching observations I have done in the past, this one was for an online class. I had never done a teaching observation for an online class before. We had a standard checklist for peer review, but the questions in it did little to guide us, so we made our way via a somewhat organic discussion, facilitated by the course materials stored in the completed versions of Blackboard shells for two separate courses. Here’s our process, boiled down to four parts that we experienced organically through the process of two, one and a half hour peer reviews. The first of these peer reviews was held in November, the other after classes ended in December 2014.

  1. We began by the person whose teaching was being reviewed expressing their values and goals, before we set out to look at materials.
  2. Then the person whose teaching was being reviewed discussed how the design of the course reflected those goals, or didn’t, and what struggles (design tensions) they had already discovered.
  3. Then we moved through course materials and some manifested discussions that took place in the course as. Looking at student work branched into a lot of discussion and ideation. I think we both found it difficult to stop from explaining details to the other while one looked through the online interactions.
  4. We then discussed nuances of interpretation that students might gather and divergent strategies we might be taking in our different approaches. These talks meandered quite a bit to the relationships between teaching content, learner context, institutional resources, materials and past experiences with the design.
  5. We concluded with itemized lists of revisions that might improve our instructional designs, we each added some plausible, and new, instructional tactics that might specifically serve these unique groups of students we have in these courses.

As it was my first peer observation for an online class, I didn’t know how our process stacked up against others. I have always taught online while also teaching other courses face to face, and this was my first formal review of an online class. The institution where I teach is going through a process of implementing review standards for online classes. I think standards are a good thing, if they promote learning in the end. So, after these reviews were completed, I went and read some of the literature surrounding the commercial materials being implemented by Quality Matters, the organization contracted to ensure a process of quality improvement for online course design. I found these materials from their website, and focused on one document (Introduction to the Quality Matters Program) in particular because I thought it might give me decent insight into how our ad hoc process compared to the initiative. That document provided an overview of the process and the focus areas, and I supplemented it with reading a white paper literature review published by Quality Matters.

What struck me was that our organic process and the areas of focus as stated in the Quality Matters literature seemed to outline identical discussions. The following table maps out the organic discussion as organized by the QM rubric.

Quality Matters Rubric Item Organic discussion among two peers
1. The Course Overview and Introduction  We both focused on getting to those materials, but I remember there was a fair bit of explaining verbally. This lead to a rather elaborated discussion about the role of the syllabus in online education, just how long it should and could be, and what value sub-documents might have as linked components of an online syllabus.
2.Learning Objectives and Competencies  Both Sarah and I related learning objectives when explaining materials we has added to our courses; competencies were are part of that discussion. What I found curious about this discussion is that each of us had more holistic goals that go beyond performance objectives. In one case, the competencies were state-mandated, which introduced a design tension: at the graduate level, the mandated competency is vaguely stated as required content of the syllabus. Is the instructor free to elaborate on that? Does that break the rules?
3.Assessment and Measurement  Both teachers elaborated on the rationale behind the point systems used, the rationale behind the tasks, and the values expressed by the design of the assessment systems as a whole. What struck me as curious about this discussion was the relationship between context and grading, revealing a design tension on one case. If a course is only offered once every two years, then with each instantiation materials must change, requiring changes in assessment practices, and limiting the design to an ongoing process rather than a fixed entity that can be decided on prior to the launch of the course. How would this context stack up against a rubric designed for courses with fixed and permanent content? Thus the design tension lies in the assessment of the assessment, rather than in the assessment of the students in their performances in the course.
4.Instructional Materials  I gathered from these segments of the discussion that materials’ discussions are pervasive within any discussion of online teaching. Our discussion lead to a recognition that all of the course is effectively course material. For example, in the discussion of navigation, which would be an access issue I assume, where content items are stored and how they are linked up (inside or outside of the LMS) is in fact teaching material in its own right. The design of how materials are delivered sends a message. A video linked outside the shell carries a message of don’t trust the system. Thus the design itself is a component of the course materials, a nuance I had not originally been cognizant of when I designed the course.
5.Learner Interaction and Engagement  Both teachers iterated the importance of interaction and the struggles with supporting it online. This lead to discussions of timing; e.g. when tasks are due during the week and throughout the semester. Views on interaction also lead to a discussion of teaching values. One instructor placed value on catering to the learners’ convenience by making everything phone-enabled. The other expressed a value of expressing the gravity of the task by pushing learners to experience the course via a full sized screen, were they can interact more fully, less distracted, and deeper. The contrasting values in this regard were some of my most valuable take-aways from the experience. Again, the configuration of learner interaction with both other learners and the instructor was both an expression of teaching values and a materials issue at the same time.
6.Course Technology  Discussions of technological choices were organic and pervasive through both talks. A number of persistent obstacles emerged. For example, Blackboard does not transfer links to purchased content (Atomic Learning models) resulting in dead links for students despite a number of proofreads of the course since the icons for the modules do in fact transfer over, albeit with outdated links. Thus each commercial content link, and there could be several in a design, must be revisited. In proofing a course before its launch, this can be difficult since catching all of them is not automated like we might find in a web authoring tool. Another point of concern was that Blackboard does not support a traditional wiki environment, in the sense that learners normally define the term. Supporting collaborative writing is then challenging despite the assumption that this technology supports that type of interaction.
7.Learner Support  I found it curious that our discussion of learner support actually hinged on a definition of our learners, who they are, and the broader goals behind what is meant to be taught in the course. Again, the topic itself cannot be disentangled from the discussion of materials. For example, one instructor felt that a course requirement asking learners to search out answers to technical topics prior to submitting questions was a move toward empowerment, and the other focused on curating materials because of the large amount of misinformation on the web. Both perspectives present a logical and reasoned position from a teaching standpoint. This discussion revealed an important aspect to understanding how a whole course fits together; something we would hope a viewer would know prior to evaluating an online course. It struck me that answers to the “correct” way one might empower learners in an environment that places so much emphasis on self-accessed learning hinges on each teacher’s perspective, and knowledge of their specific population of learners.
8.Accessibility  This term in the QM literature would need to be interrogated to be fully understood because accessibility can range from providing multimodal access to learning materials in a traditional sense (captioned videos for example), to values related to the design technology-enhanced learning more broadly. For example, I had not previously noticed the cluttered nature of my courses landing page. I had not discovered some of the tools on the landing page could be removed or hidden. This is actually an access issue. With so much on a single screen, is the instructor aware of how much of a burden they are putting on learners’ ability to find what they are looking for? Also, I had not included redundant links in convenient locations within the course. In retrospect, these are as much accessibility issues as providing captions on videos. We discovered this through the course of our talk. The same goes for providing free alternatives to software that I had assumed learners would have, an assumption I now question.

 

The literature on observation in teacher education makes some very clear points about how teachers are best supported. Some key considerations are that teachers are best supported by non-judgmental observations (Fanselow, Bullock, Van Es), observation of larger constructs such as course design and lesson plans reveal very different aspects of teaching expertise and perspective than close inspections of specific teacher moves (Preston, Rosean), and “effective” practice cannot be defined without the input of the teacher (Owston). I wonder if some of these ideas do not get lost when we try and standardize peer review of online teaching. The Quality Matters materials do seem quite tied to the current research in online learning, which is a good thing, so, it strikes me as a beneficial aid to a novice teacher. But I have to wonder if the rubric wasn’t created from discussions like the one I had with Sara. The mapping of the discussion over the rubric strikes me as too precise to be coincidental. Perhaps the rubric only tells you what *should* be in a professional discussion; things competent professionals would talk about anyway.

Perhaps the QM initiative is an attempt to combat the issue of teacher obsolesce in the face of new technology. I think what our conversation suggests to me is that teaching online is less about being online than it is about teaching. The QM website goes to great lengths to make it clear that it has a rigorous approach to development, informed by the most current research, and includes a lit review of some of the trends in online learning research. What I think the approach may be missing in this endeavor is a key recognition that teaching and learning are in fact two different things, albeit interrelated. Research into online learning can only take us so far into learning how to teach online. Once someone grasps the concepts behind the available tools, the discussion goes much deeper into values, perspectives, and deeply held notions about learning a specific set of content items. The relationship between strategy and content is not one to one, but rather stochastic and deeply tied to values the teacher holds (Reigeluth). We have a comparatively meager body of research related to online teaching strategies than we do on online learning. I came away from these talks with the notion that peer-review observation of an online course is less about technology than it is about what it means to teach, less about the machine and more about the people we are trying to connect with and nurture.

This experience left me somewhat suspect of decontextualized observations from someone who is not in touch with our learners. There comes a point where the context of the learner reigns supreme over any instructional strategy that a teacher is assuming, and those unique contexts surface through discussion with a colleague about how the materials and design came to be as they are. I welcome any peer review with a colleague from my institution, but I wonder how much common ground I might find with someone from somewhere else. Perhaps this is related to why we grant tenure based on teaching at the institution giving tenure and not elsewhere. The differences in foci among teachers at different institutions may parallel differences among teachers within one institution, but they may also not. What one teacher is trying to accomplish may, or may not, be what the rubric supposes the teacher is trying to accomplish, and may or may not be needed for these particular students given the circumstances of the program as a whole. I remember more than once coming upon the phrase, “no, that’s already in another course.” This may be why MJ Bishop (University System of Maryland) says we don’t do courses as much as we do degrees; the course and the teachers have to fit together. The conversation about what should be taught is equally as important as the conversation about how it should be taught, and I am not so sure the two can be extricated from each other in a meaningful way, as we found in the multiple locations in our talk where the discussion of materials could not be torn from the other topics that our meandering conversation unearthed. I would feel more comfortable discussing curriculum choices with a teacher whose approach I know, than one whose approach is still a black box. Professional familiarity has an important value on the context of a school, a value the QM perspective seems to overlook.

The value of professional knowledge of a colleague hints on something more profound, and something related to improving teaching growth on personal level as well as on a program or school level: the importance of trust and comradery. The importance of trust is pervading into many fields of late, including medicine. All of the components of teacher observation that I found in my in-person observations were present in this discussion of our teaching, and I don’t know how comfortable I feel about intimate discussions of my teaching rationale with someone who may function on very different belief system about what teaching is, is not, and what is should be. Nor do I see an incentive to want a third party to see my course through my eyes. Sharing that with a colleague I foresee working with for a long time does hold that value for sharing. Working with people you know and trust is not intimidating nor trivial; it is a pleasurable and feels like time well invested in a colleague and one’s own teaching. It strikes me that this will pay off for students as well; teachers who know each other and how each other teach can make better decisions about what policies to implement. Also, accomplishments can be shared, and setbacks can be supported. A peer review with a trusted colleague makes for a better working environment.

I came out of this experience with the conclusion that a peer observation of an online class is not wholly different peer observation of a classroom teaching– the most rewarding parts are the discussion that happens after looking at teaching choices, and in discovering why a trusted colleague teaches the way they do, sharing why I do what I do, and reflecting on how this might be done better for the both of us, and our learners.

Supporting healthy failure in online education

Edo is the formed name of the city that is now known as Tokyo

“Write this sentence in the past” (credit: 本気の発想 translated poorly by Craig as “serious creativity” from Twitter user tag: @Honki_Honki.

Teachers are told to make learning fun. Yeah! However, there is a fair percentage of learning that includes rejection and supporting that online is hard. Educational psychologists manage to gloss over the emotional side of things and come up with statements in research jargon that a learner is adjusting mental schema according to new information. Adjusting one’s mental schema to accommodate a big whopping REWRITE on a paper may have positive correlations on psychometric evaluations of learning, but calling it fun is a stretch. Many teacher training programs and educational psychology texts make it seem as if the learner simply processes information, without any pain. But adjusting one’s own “schema of the world” includes a key realization, I was wrong the first time.

Confronting one’s own being wrong is decidedly not fun, but it is effective. Sometimes it is so effective that we remember the lessons for a lifetime. Here are few of mine: Mr Savery, my highschool German teacher who I admired, told me to guess on the German language article declensions on my advanced placement exam. “Look, declensions in German are ‘en’ 70% of the time, if you try and figure it out, which is what you generally try to do and I know this because I have watched you take my tests for 4 years, you’ll lose time you could have spend getting to the questions that you are better at answering. Just guess all your declension items and move to vocabulary items as fast as you can.” This made me rethink just how much I knew and strategize my learning down the road. I went on to survive a year in Austria with this knowledge and pass a year’s worth of coursework delivered entirely in German. Dr George Hole’s comment on a final paper, “you didn’t think as much on this paper” was devastating. This forced me to rethink my writing and shortly thereafter I won a scholarship to Columbia University based on my application essay. Dr Carol Numrich’s similar comments on graduate papers were the same. Her feedback played into how I approached my doctoral application where I secured a fellowship to Indiana University. Ginette Delandshere handed me down a whopping REWRITE. Susan Herring’s 300+ requests for revision on the post-defense edit of my dissertation spun my head. How could I possibly write that poorly? Elizabeth Boling’s comment to my doctoral quals was eye-opening, in a more holistic way than just school, “righteous indignation does not serve you well in this context.” This idea played into my search strategy on the job market and helped me survive my dissertation. None of these rejections were fun, but they were all effective. All were in school, and none were online.

Healthy failures, as opposed to failures that lack transformative impact, also share something in common that is hard to accomplish online: care. The memory of the failure is coupled with the memory of the person who delivered it. It is the failures that are handed down from specific people that we remember. The failures I remember were all handed down from people who cared, or at least I felt that they cared for me. Imparting that feeling of being cared for is part of being a professional educator. While a missed objective question on an impersonal test may just require one to revisit the chapter, failures associated with more transformative learning all revolved around larger efforts: writing longer papers, preparing for bigger tests that seemingly predict a future, or attaining a significant life changing milestone–like finishing the Phd and getting my first tenure track job. Multiple choice, short answer and like approaches to formative assessments do not offer access to this kind of learning. Occasionally I have seen phrases on job descriptions in order of “genuine care for students,” but I have yet to see that operationalized into required teaching practices, especially online teaching practices. Failures on the big things may need to come from someone who has, or seems to have, a general interest in your well being in order to make that change from simple procedural and concept learning to transformational learning. The online instructor is not equipped with the same tools to express this care, and generally has had few examples to fall back on.

A large percentage of our online teachers have not experienced how care is expressed or attained online and have few experiences themselves of how failure is supported online, myself included. The online courses I have taken have all seemed somewhat removed from me; a kind of no frills instructional endeavor that gets right to the point, stays there, and ends. Read the materials, watch the video, listen to the podcast, post twice or three times a week, take the test, submit via the dropbox, finished. There were few pats on the back and little encouragement. Given this context, there was little opportunity for the instructors of these courses to impact real transformative change in my being. Any instructor who attempts incorporating support for failure is breaking new ground because, not only did they not experience failure support themselves, the media which they are working likely did not even exist a few years ago. Online instructors are not only learning to teach online, they are learning a whole gamut of digital ways to communicate. The best they can hope to do is try out some tips they saw at a conference, and learn from their mistakes over a long period of time, which I am trying to do.

Given the significance of these experiences in the longer term perspective of the experience of learning, I find it remarkable that I don’t find more emphasis on teaching failure well. Andrew Tawfik is starting to scratch the surface here, but we have a long way to go. In many years of teacher training I don’t remember ever focusing on how to deliver and support failure in learning, and I don’t remember any of these experiences that happened at-a-distance. Now that I am teaching online via an LMS, I feel even more handicapped at delivering this kind of caring for failure via the machine. Perhaps this is the greater implication of Joni Dunlap and Patrick Lowenthal’s advocacy for high touch low tech in online learning. But this is far easier said than done in contexts where learners have little time for synchronous communication and all their experiences come filtered through the lean media of a learning management system.

Healthy failure is difficult to administrate. It is a tricky business to put requirements for failure into programs, course designs, or teaching requirements. In the dynamics of instructional roles among teachers, few instructors enjoy being the one who tells learners that they were flat out wrong, the paper failed to meet certain standards, the learner is unlikely to pass given the present course of action, and so on. In graduate education there is a second ethical responsibility of telling a learner that, while the student may pass the course, from the state of the learners’ progress, it appears unlikely that the course of study will prove a prudent use of the student’s time, energy and money. This may just be the dirty work of teaching, especially online teaching where the message comes stripped of all tone and affection expressing care. Every job has some dirty work. Handing out the required rewrite may just be the dirty work of online education. Including precautionary statements in the omnipresent, all powerful, contractual agreement syllabus, seems to have little effect in mitigating the emotional impact. The statement “If the required criteria are not met in your final paper you may get a rewrite” is little consolation to the learner who actually does get a rewrite. Explanations that the editing process includes some of the most important aspects of learning the course entails can add salt to the wound even though the intension of the statement is to express care.

In our effort to make learning fun we sometimes handicap teachers, and learners, into thinking it is supposed to be fun all the time. If our most effective teaching strategies include experiences that are not fun, we may be handicapping our teachers from using the best tools they have to teach, especially online.

In-school but out of class, CATPALS at A&M Texarkana and why you should care

CATPALS this year focuses on Environmental Issues and all TAMUT registered students can pick up their free copy of Allen's The Good Food Revolution at the bursar. Go get it and get involved!

CATPALS this year focuses on Environmental Issues and all TAMUT registered students can pick up their free copy of Allen’s The Good Food Revolution at the Student Success Center-UC330. Go get it and get involved!

Universities, colleges and other types of learning institutions often pursue projects that initially start as non-credit bearing activities until they grow into accepted pedagogical practices and can become courses, or components of courses, and carry credit. These initiatives are how educational institutions transform. These are often the kinds of initiatives that make news, and often affect real change, even if that change is not readily apparent at the onset of the project. In each case, it takes multiple players to make institutional change into institutional reality.

For example, one such design is the reading program started by Alejandro Gac-Artigas featured recently on PBS here. This is instructional design, (by the way, it’s a design case, albeit a rather superficial one) teachers implemented a cooperative summer reading program with parents, a design genre that has been implemented dozens of times before (McCarthey 2000). Innovative or not, this type of attention on an in-school but out of class activity is important for gaining support and buy-in from those whose cooperation is needed to make initiatives succeed. What’s not mentioned in the video segment, but is vitally important, is the cooperative support from others at the institution. If you look closely at the PBS video segment, there are a number of indicators that this intervention is not a single-source project. The sessions happen during summer break; thus administration played a part in getting summer funding to keep doors open. There are multiple teachers teaching this curriculum, thus some other teacher volunteered to help with the teaching. There is non-junk food on the tables to keep parents and kids revved up during the activity; cafeteria staff also had a role. Before I talk about A&M Texarkana’s CATPALs, I will mention two other transformative projects that enlisted many players to transform teaching and learning at institutions where I saw this same type of in-school but out of class intervention transform the institution, in ways large and small.

Kanda University’s Self-Access Learning Centre: In 1998-99, Lucy Cooker wanted to create a space for learners that was less antiseptic, more inviting, and supportive of the motivated learner to really excel though self-paced learning. She talks about in this video. Notice she focuses on institutional support in her talk about the centre, which has now boomed to serving hundreds of learners daily from just a small 2-classroom project. Dozens of instructors supported the SALC during its fledgling years, though not all believed in the idea that a self-motivated learner could do what she was proposing. The SALC now employs several learning advisors and has transformed Kanda University from a school where learning was lock-step, to an institution that can support literally limitless language learning— in fact, several languages if a learner is so motivated. It truly transformed Kanda University in Chiba, Japan.

Indiana University’s Anti-plagiarism Test: in the early 2000’s, a number of teachers and teaching assistants at Indiana University endeavored to create a sustainable pedagogical response to help combat plagiarism. (Key members were: Theodore Frick, Elizabeth Boling, Andrew Barrett, Cesur Dagli, Rod Myers, Meltem Albayrak-Karahan, Joseph Defazio, and Noriko Matsumura). The test was supported by assistant instructors and other teachers who assigned the test, students who volunteered their skills in design, programming, or a myriad of other skills, and the university who supported the overhead. The IU anti-plagiarism tutorial (and test) now serves literally millions of students and teachers all over the world every year.

CATPALS at Texas A&M Texarkana: Committee for Annual Thematic Program and Lecture Series (CATPALS) is a whole-campus initiative designed to integrate multidisciplinary learning and community development at Texas A&M University – Texarakana. It has been spear-headed by Michal Perri, an Associate Professor of History. This year the theme is environmental issues and the University will be having lectures and providing reading materials to go along with the lectures. I will attend all the lectures I can. The projects started off with a free book (way to get professors on board!). You can pick up a copy of the book at Elizabeth Patterson’s office in the Student Success Center, UC330.

  • Allen, W. (2012) The good food revolution. Gotham Books (254 pages)

Lecture dates will be publicly announced via the TAMUT website: Search CATPALS @ http:tamut.edu

Here is my challenge: Any student who is taking any course I am teaching can gain a 5% increase on their final grade by creating instruction to go with any lecture material covered in CATPALS. It can be an online quiz, a tutorial, a video (vlog, etc…), or any other form of mediated instruction, but the experience must be accessible remotely in order to get credit. We can host your developed material on our student work webpage. I will share the media with the CATPALS committee. This is a great opportunity to put what you learn in Instructional technology to good use, become a part of the Texas A&M larger community of learners, and create a portfolio item which just could be that missing link that takes you from being a job applicant to an employed instructional designer.

References:

McCarthey, S. J. (2000). Home–school connections: A review of the literature. The Journal of Educational Research, 93(3), 145-153.