craig dennis howard

Home » teaching

Category Archives: teaching

What they told you is wrong: The prospectus, proposal, and the dissertation

image of child's artwork

If your to-do list contains a row for “write one page of dissertation,” you can move your little pony magnet to the mommy happy column on the fridge when you get it done. (Artwork by 古矢英玲奈 mixed media and magnets, 2019)

There is a lot of misinformation surrounding the prospectus, the proposal, and the dissertation. This may be so because this task is one left to advisement and mentorship rather than formal instruction. An open and critical discussion on the purposes of these documents rarely happens among scholars for good reason. The interpretation of these documents are unique to the research context, and the mentor. Even so, I feel there is much to be gained from an open discussion about the meaning of these documents; I welcome any comments on this. When doctoral students do not understand the purposes behind these documents, their completion may appear like administrative hoop-jumping rather than learning.  While I do not wish to dictate to other faculty how to view these three documents, clarification around the purposes allow a mentor and mentee smoother communication, and hopefully less frustration. I have written this explanation for my own doc students, Katie and Sam, and share it here.

The prospectus

The prospectus is a plan to study something. This plan is written down on paper for the purposes of facilitating discussions with faculty one might want on their committee. Writing the prospectus supports the candidate’s muddling through complex methodological and topical areas to form a team whereby a research strategy can be collaboratively orchestrated by the doctoral candidate. Of course, the writing process always helps clarify plans, but that’s not the immediate practical value of the task. If you assembled a team, you had a successful prospectus.

One thing a prospectus is not—an abstract of the dissertation. While sentences from the prospectus may in fact appear in the proposal or even in the final dissertation, the prospectus is not to be written as an attempt to capture a study in its entirety. Rather, the purpose of the prospectus is to start the discussion enough to assemble the team. The team is going to want to know what your plan is about so they can see where they might fit in, and that the topic is going to jive well enough with your life plans such that there’s a reasonable expectation that you’re going to finish the study. Red flags include academic jargon that seems unconnected to the learner’s larger picture of life goals, or text written purely to satisfy the interests of the faculty member. Effectively, you are looking for their buy-in, on your study, not theirs. Assembling a team requires that buy-in, and among scholars, buy-in is achieved only when ideas meet paper (or digital paper, you know what I mean).

The prospectus is not a written form of your dissertation elevator pitch. Instead, it’s a sales pitch and with that, all the subtle intricacies of sales play into it.  There are a fair number of subconscious elements at work here, including your presentation evidencing that you’re organized enough and cooperative enough to make this working together experience relatively enjoyable for the faculty member. Having your ideas written down makes them real- both in what is said and how it is said. The prospectus need not capture all perspectives, but it should convey enough to the reader knows why you’re in it and how they fit into your study. There is also risk on the side of the learner that the faculty member does not become invested,  and at the same time, the faculty member risks that the student never completes the study. These issues are being felt out in the prospectus process. In real and concrete terms, the prospectus should outline the plan in no fewer than two pages and no more than ten. Having sections mirroring the sections one might find in a dissertation is helpful. These guidelines surely vary among mentors.

The proposal

The proposal is a refined plan of action. It should capture the conversations supported in the prospectus, yet none of the language that appeared in the prospectus need be in the proposal. There is a common adage that the proposal is the first 3 chapters of your dissertation. After I completed my dissertation, going back and changing much of what I had in my original proposal, I felt this was untrue. However, this did not stop me from telling other people that the proposal was the first three chapters of the dissertation; it’s an easy description to fall back on when time is short and one need not get into a longwinded blog post about the purposes of a proposal. This refined plan of action’s purpose is to support the candidate both in laying out directions, and support the candidate’s self-efficacy surrounding the study.

An approved proposal guides the research process. The proposal should support even the most frustrated dissertating candidate because it is a document that, when approved, evidences that the candidate came up with a reasonable, manageable, and worthwhile plan for study and that plan was approved by those who should be able to foresee any issues that can be foreseen. Think bumper rails. Candidates should recognize that part of the purpose of the proposal is to get the scope of the study under control, and manageable. If you’re bring entertained by collecting and analyzing more data beyond what was in your proposal, it is not likely that such activities will be welcomed y the committee unless you check with them. Research studies can organically grow too large to complete. The committee’s oversight is there to make sure the project does not become too large, or wander into territory outside of the reasonable expectations of a first independent study (which is a dissertation after all). I advocate catching up with your committee every once-in-a-while to stay on track. If we look at the proposal as a confirmation that the candidate has the support of four learned scholars who agreed that the candidate had created worthwhile study that is manageable in scope, then the document should support self-efficacy.  The proposal is also to support and scaffold the learner through self-doubt and frustrating times when things don’t go as planned. That leads to the well intended by logically false contract adage.

While logically wrong, there is another adage that the proposal is a contract. A proposal cannot be a contract because a contract outlines what happens when certain foreseen events happen; and unforeseen events void the contract. Unexpected results don’t void your dissertation proposal though. Research is by definition new territory, so not everything can be predicted.  Therefore, a proposal cannot be a contract; the logic simply does not hold. However, the adage is a confidence builder for the student because it offers reliability that the student will not be left in the lurch should things not pan out as expected. It is a way of telling the student that if they in fact do what they planned to do, the process will result in a worthwhile dissertation no matter what the results show. Contracts do not plan for the unexpected, but in the context of a dissertation, there are often feelings of self-doubt, and at times, panic. The contract adage works against the panic, even if the adage is false. A proposal is a detailed plan, but no plan is perfect, and contracts are not plans. While contracts are not written for unknown entities, this way of thinking about the process is a good one; don’t bother arguing the semantics here. Just go with it.

In real and concrete terms, the prospectus should range from approximately 30 pages to no more than 100, and 100 is still really long. These guidelines surely vary among mentors. A proposal of less than 30 pages may raise concerns; it would for me. Not because committees measure quality by word count, but rather because detail is so important. The lit review in the proposal is important ground work. The methods need to be spelled out or cited extensively so procedures are obvious. The committee needs to know why you’re going to do the things you’re going to do, and how you’re planning on doing them requires detailed planning that needs to be discussed. A little working of real data goes a long way in playing out the plan; good to have, but it doesn’t need to be in the proposal. Sometimes that data is not available, but usually, there’s a way to get at least a small sample, enough to make that proposal a really good guess. Of course, you can use previous data, but the closer your data is to the data you will use for the dissertation itself, the better your guess will be. You may well see important things that change your analytical plan in major ways. The task of the committee is to help you get to your goal, and without sufficient planning, you might lose a lot of time in rethinking things.

A proposal of more than 100 pages will raise concerns. Not because committee members are getting old and feeble, making it difficult for us to read for long stretches, but excessive writing suggests you’re over planning, or can get sucked into a rabbit hole. It is important to let go of the writing process and submit it at some point. No writing project is ever truly finished, by life does march on by if you don’t submit it.

The dissertation

The dissertation is a show-all-work, independent, original research task. It’s not a magnum opus. And no matter what you have been told, your dissertation does not define you as a scholar. I have also heard people call the dissertation simply a book, but it’s not the same as a monograph. A dissertation must please 4 committee members. Books don’t necessarily have to do that. Rather, a dissertation places you in a field of study as someone who has contributed. It should connect you to the field you want to be in when you finish it, but if it doesn’t, you’re not going to be doomed. I think strategizing the dissertation takes more time than actually writing it. I would encourage all doc students to start thinking about the dissertation from the start of doc study. Big ideas don’t change; it’s the details of how to get it done that change. I came to doctoral study wanting to study the way people learn language because I was convinced that the way you talk has enormous influence on the quality of life that you lead. After a few years of more graduate study I learned what types of language I wanted to study, and some nifty tools that gave me access to study it. My dissertation topic was the same the day I left as it was the day I arrived. The tech I used to do it was not all that important. What I learned was how to do research.

Long awaited conclusion to this dry blog post

The relationships between these three documents are actually not linear, though they may appear so. One need not finish the prospectus to start the proposal. Each of these documents serve different purposes. There are some tricks that help the writing process. For example, for each research question you write, imagine the possible answers and write down what meaning those outcomes would have. This will make writing the discussion chapter much easier. Try not to be too hard on yourself. Make small, accomplish-able writing goals. For me, I had each chapter in a separate word file, and if I wrote a page a day, on any of them, I considered it a good day. I decided to be happy with myself for that day, even if it was just one page. I wrote a page (500 words), I had worked. Of course, there were days where I wrote a lot more, but I found the key was having no days where I wrote nothing. A day where you write nothing, or do nothing, is your enemy. Those days can pile up fast, as can days where you write just 500 words. You’ll find each chapter done in about 3 three weeks.

I also advocate being cognizant of the people skills beneath the dissertation process. How to navigate scholars and communicate with a committee of very different minded individuals is important learning and part of the process. Lastly, the doctoral research experience is not meant to be uniform. All PhDs do not serve the same purpose nor the same learning. It should do what you want it to do for you, and revisiting that at some point during each of these documents is wise. Each new scholar finds value in different places. For some, the people skills are the most important and the most crucial. For some, it is in fact the writing. For some, it is the defense itself. Some PhD’s are pursued to get a job, some to enter a field of research, some to enter a career in teaching. I believe that in all these cases, the dissertation serves as an exercise, albeit a very formalized, careful, and important exercise. In the end, it is training, an education. The dissertation is an exercise in creating knowledge and convincing others who have also consumed a fair bit of related knowledge to agree in the value of this knowledge that you created.

While everyone’s dissertation experience is different, the three stages of prospectus, proposal, and dissertation unify the experience among all scholars. I think most who have earned a PhD will understand what one means by these three documents though they may not actually have the time to explain the nuances, and will often take the easy outs provided by the adages. The misinformation is not borne of malice, rather of trepidation that their experience is unique, which it is. At the same time, I do not believe anyone who actually went through this process knew the differences among these docs when they started. I sure didn’t.

An Interview Protocol for Instructional Design Cases

image of sewrch results

Google image search results for keywords “interview protocol.” There are many, but this one is for instructional design cases.

We need more instructional design cases written by those who did not actually do the design, but where to start?

I suggest interviewing the designer. Elizabeth Boling did the same in her case about the Alcatraz cell-house audio walking tour. I have done interviews twice now to begin design cases about designs I did not create, and I share the protocol I developed here. This protocol was originally developed from reviewer questions in an article where I belabor all the difficulties I went through doing my own first design case. Of course, the instructional design case doesn’t begin and end with the interview. A good case will also have a few images of the design, the authors experience of it, maybe a few quotes from others’ user experiences, and the author’s reflection about why they were originally drawn to write the case. But of course, start with interviewing he designer.

 

 

Interview Protocol for Instructional Design Cases

Start by:

  • Telling them you are recording AFTER the recorder starts, so you get their agreement on tape
  • Explaining that a member check will go out to them prior to the article’s submission for publication.
  • Mentioning the time it takes–  90 minutes seems about right in most cases.
  • Explaining that the markers and paper are there for exploring, please use them when needed. The notes are drawing from teh designer used to explain their product or process are important parts of the interview!
  • This meeting is about the instructional design, not necessarily about the success of the design.
  • Most importantly, Thank the designers for participation in this process.

Situating the design context and process:

  • What were changes in context which motivated the design? Something must have happened that brought this design about.
  • Who was the design team and what were their influences? Can we assume that the different members of the design team had different goals? Was that discussed? How were those decisions made?
  • Did you initially intend to have students create a game? What were those key decisions? When did they happen?
  • Can you describe the process by which you came to the initial formulation of the design?
  • As you reflect on how you created these learning opportunities, what were the pivotal moments during the formulating of the instruction, the ah-ha moments or innovations, that you would want to tell someone else, who might be considering doing something similar for their learners?

Describing the design:

  • Can you map out all the parts, especially the invisible ones, which someone viewing this teaching intervention might not see from the game itself? [Point to markers / pencils / paper]
  • What is particularly interesting about this instruction?
  • If you were to name the instructional design, NOT THE GAME, what would that name be?

Depicting the experience of the design:

  • Can you describe the user experience? /  How was learning measured, or not?

Transparency:

  • Can you tell me about any unforeseen obstacles or aspects of the design that needed revisions that you only found out about after decisions were made?
  • Did you try anything out, or consider anything, that was deemed in the end to be a bad idea in retrospect?
  • How has this instructional design created complexities or challenges in your teaching? Has the instructional design failed anyone? TA’s, students, not met your goals?
  • Have you skipped anything for simplicity’s sake? This can often trip up a design case because often what was skipped may be rationale for design decisions.

 

Professional Dispositions in IDT

Special thanks to Duren Thompson, Synthia Clark and Lisa Shipley who were the student and staff contributors to this Policy for the LDT and IT Online programs. This policy developed the programs to better serve the field and to better enable our learners to navigate the expectations of practicing professionals in Instructional Design and Technology, both as a field of scholarship and one of practicing design.

Program Participant Professional Dispositions
University of Tennessee, Knoxville
Last Updated April 20, 2018

Preamble
Instructional design and technology is a professional field with common values and professional ethics that influence the way professionals interact with clients, peers, and mentors. During a learner’s time in the Instructional Technology, (IT Online) Learning, Design and Technology (LDT) or the Online Teaching and Learning Graduate Certificate (OTL) Programs, it is imperative for learners’ future success that academic ability grows in unison with the professional dispositions outlined by the Association for Educational Communications and Technology Standards (https://www.aect.org/docs/AECTstandards2012.pdf). Listed below are the core values by which we expect students in these programs to abide. Multiple or egregious violations of these core values will lead to the evaluation of a students’ status in the program, as well as the development of an improvement and retention plan, and thereafter, possible dismissal from the program. The plan procedures and expected consequences when students violate these core values are provided below. In addition to following the shared core values in IT Online, LDT, and OTL learners are expected to follow the Student Code of Conduct in Hilltopics (https://hilltopics.utk.edu/) as well as the Academic Policies and Requirements for Graduate Students in the Graduate Catalog (http://tiny.utk.edu/grad-catalog).

Core Values
1. Show commitment to the profession
• Learners must develop the skills and knowledge to become a productive member of the instructional design and technology professional community. This may include a new identity of professional conduct, which, while it can be intimidating and overwhelming, is a standard expectation. A learner’s professional transformation must evidence a voice that shows commitment to the profession. Remarks to peers, clients, or mentors suggesting a position against the profession as a legitimate entity would violate this core value.

2. Show academic integrity and honesty in your work
• Leaners will abide by the University Tennessee Honor Statement, which is listed in the Hilltopics as “An essential feature of the University of Tennessee is a commitment to maintaining an atmosphere of intellectual integrity and academic honesty. As a student of the University, I pledge that I will neither knowingly give nor receive any inappropriate assistance in academic work, thus affirming my own personal commitment to honor and integrity.” In particular, the ethical use of copyrighted materials is an important issue within the field of instructional design and technology. This includes the use of text, images, audio, video, and other media retrieved online. The use of published work created by others must be properly credited, and, in some cases, used only with express written permission. Learners should not only practice academic integrity, but also encourage and support peers to do so as well. When there are suspected cases of academic dishonesty, faculty and the student will follow the guidelines set for by the Office of Student Conduct and Community Standards (http://web.utk.edu/~osja/index.php) to identify the best course of action.

3. Maintain a healthy rapport with your adviser
• Learners must seek help with course selection and remain in contact with their academic advisor regarding progress of their program completion. The IT Online, LDT, and OTL program assumes that learners will take ownership of their learning. This includes following instructions as to how to schedule meetings, the integrity of reporting the outcomes of advisory meetings, pursing shared academic opportunities, and the documenting of advisement. Positive, regular communication with your advisor can improve the overall learning experience and provide important opportunities and connections for after graduation. Ignoring advice or communications from your advisor is not only a violation of this value, but may delay program completion.

4. Be a reflective practitioner
• Learners must evidence that they enter all learning experiences, both independent and collaborative, in good faith as a reflective practitioner. The process of instructional design and technology learning requires reflection. Learners will learn from providing feedback to others, as well as receiving feedback. Program instructors support learners in this process while creating opportunities for them to learn how to work with feedback. While receiving feedback on work, whether positive or negative, can be difficult. Additionally, statements evidencing a desire not to learn the content put forward by faculty and peers is unacceptable. As a professional, feedback on your work should be celebrated as part of the iterative design process. It is the learner’s responsibility to learn how to evaluate feedback and address it in future iterations of design. The outright refusal of feedback as well as Ad hominin attacks or unexplained criticism would constitute a violation of this core value.

5. Be open to diverse ideas, approaches, abilities, and learning needs
• Learners must nurture an ability to work with divergent perspectives and ideation as a means to learning in instructional design and technology. Sharing of diverse ideas, approaches, and reflections lead to innovative practices. Learners may experience discomfort while being challenged by ideas, abilities, and learning needs not encountered prior to entering this program; however, all learners must learn to be open and respectful in sharing ideas and approaches with clients, peers, and mentors. Instructors can support learners to increase knowledge and skills for working with diverse ideas, however, learners may not refuse to listen to collaborators, or accuse others of willful obstruction to the learning. Ignoring or refusing requests for reasonable learning or communication accommodations (from peers, clients or within assignment parameters) also constitutes a violation of this value.

6. Accept the challenge to learn
• Learners must put forward an attitude which views failures as learning opportunities and accept the challenge to learn. Many students who enter the IT Online, LDT, and OTLprograms have exhibited qualities of a successful learner in traditional learning environments. The experience of becoming a professional learner can be quite different from prior experiences. When confronting difficulties while adjusting to new ways of learning, refusals to participate in class, participate in required components of course activities, or to collaborate with others, constitute a violation of this value.

7. Treat oneself and others with respect
• Program participants (both instructional designers and technologists-in-training) engage in numerous project-based course activities in teams and with real-world clients. This experience may be difficult at times. The difficulties themselves are not a problem; in fact, course activities often include authentic challenges that prepare learners as highly-qualified professionals. Behaviors which embody respect for the knowledge and skills of the field lead to becoming a successful team member. It is paramount that all learners show respect to themselves and others throughout this learning process. Disrespect to either oneself, one’s team, other learners, or clients would violate this core value.

8. Be a productive member of our community
• Learners must contribute to the community of practice in which they are engaged, in synchronous and asynchronous whole class discussions (online or face-to-face) as well as in smaller group work scenarios. In whole class activities, learners are expected to both contribute materially and support others in making their own contributions. In teams and group work, the negotiation of roles must accommodate a fair distribution of work. Fair distribution means that learners must contribute equally and equitably as determined by the team and cannot do less than what is agreed upon, nor do more by taking over the group work. In addition, it is a violation of this core value for a learner to refuse to participate in group activities or attempt to claim exception from peer evaluations due to conflicts in teams. Engaging in group processes productively as a member of a community involves skills and knowledge that may be new to them If learners find difficulties in such processes, they should view the situation as an opportunity to learn and reach out to the instructor of the course for assistance. The instructor can function as a mediator to assist team members with how to work together effectively. Refusing to contribute or monopolizing whole class activities would violate this core value.

9. Show integrity, honesty, and inclusivity in collaborative work
• As previously noted, many projects in which instructional design and technology professionals engage are shared by a team, with team members sharing credit for publication. Learners in these programs will give credit to the contributions of others for all completed work and document their own participation with integrity and honesty. In addition, The University of Tennessee, Knoxville values inclusive teaching and learning practice, and this extends to collaboration. Learners must practice inclusivity as a regular part of their collaboration. While learning to become a professional, learners must gain skills and knowledge for assigning work equitably among team members, develop a mechanism for monitoring contributions made by all members, and allow allmembers access to the learning taking place. Failing to properly and fairly credit collaborative work, or purposefully excluding group members from learning, are both violations of this core value.

Professional Disposition Evaluation and Participant Retention
Program faculty review student progress on a regular basis to assist students as necessary. While our main goal is to celebrate learner successes in becoming a highly-qualified Instructional Design and Technology professional, there will be times that faculty need to communicate to learners about issues relating to retention and improvement to ensure that learners are meeting both academic and dispositional expectations.
Program faculty are aware that becoming a professional is a learning experience of its own, and when participants show initial signs of dispositional issues, learners will be provided warning(s) to address them. These warning(s) will be communicated to learners during academic advising sessions. When dispositional issues are not remediated through early warning(s), faculty will follow the steps described below to address them.

Step 1: Students will be notified through a letter sent to their UT email account regarding an academic or dispositional issue that came to light and will be held to a probationary status as a student in the program.
Step 2: Students will schedule a meeting with their academic advisor no later than 2 weeks from the date of the notification letter.
Step 3: During the student/advisor meeting, the student and faculty will jointly construct an improvement plan that clearly lists the identified issue(s), strategies addressing the issue(s), expected outcomes of how student academic performance/dispositional issue will improved, and a date for the improvement plan outcome review. This plan will be kept in the student’s permanent record.
Step 4: The Academic and Dispositions Improvement and Retention Committee will review the improvement plan. This committee will consist of the advisor and two other faculty. Upon review the improvement plan, the committee will notify the student through a letter sent to his/her UT email account whether the plan was accepted or not. If not, students must meet with faculty advisor and address revision to ensure the plan is accepted by the committee.
Step 5: Students will take action according to the approved improvement plan.
Step 6: After the date that is listed as the improvement plan outcome review, the faculty committee will evaluate the student’s improvement according to the criteria of the plan. The committee will deliberate on whether the student did or did not meet the improvement plan expectations. If the students did not meet expectations, it will result in dismissal from the program.

Scaffolding the five finger filet

Instructional graphic designed to teach the Five Finger Filet game. Notice the five dots below the thumb depicting the pattern. Erena Furuya Howard, pencil on paper, 2017.

There is a game called the Five Finger Filet. One holds a knife and stabs between thine outstretched fingers, risking a dangerous errant stab. Personally, I never had any attraction to games like this. I would surely chop off a finger and regret my own stupidity in a matter of moments. This however doesn’t dissuade my 5 year old daughter from trying to get me to play such a game. Of course, I show that I will get it wrong and continually mess up, stabbing ridiculously far-off and in the wrong pattern. To instruct her father, Erena designed this quick instructional scaffold. The location of the dots is what is important. Notice the string of five dots under her thumb. That’s the pattern, to each finger and back. I actually had not known what the real pattern was. But what caused me pause is how natural her behavior was. If a five-year old so naturally creates visual instruction, what does it say about the way we teach and learn that teachers and masters students in education rarely chose that avenue? Is any one else getting the idea that the pendulum has swung too far toward understanding learning as simply reading, writing, and nothing else? Maybe I am just chopping off another finger.

An organic peer review of an online class, and the Quality Matters rubric

anamorphic art by  UTISZ (Istvan Orosz)

Anamorphic Art by Istvan Orosz. Anamorphic art depends on how you look at it, and without the perspective, you might not see the image, much like a teaching observation of an online class. This Hungarian artist’s website is at: http://web.axelero.hu/utisz/page.htm

The last teaching task I completed for the year 2014 was a two-part peer teaching observation with Dr Sara Lawrence. But, unlike the teaching observations I have done in the past, this one was for an online class. I had never done a teaching observation for an online class before. We had a standard checklist for peer review, but the questions in it did little to guide us, so we made our way via a somewhat organic discussion, facilitated by the course materials stored in the completed versions of Blackboard shells for two separate courses. Here’s our process, boiled down to four parts that we experienced organically through the process of two, one and a half hour peer reviews. The first of these peer reviews was held in November, the other after classes ended in December 2014.

  1. We began by the person whose teaching was being reviewed expressing their values and goals, before we set out to look at materials.
  2. Then the person whose teaching was being reviewed discussed how the design of the course reflected those goals, or didn’t, and what struggles (design tensions) they had already discovered.
  3. Then we moved through course materials and some manifested discussions that took place in the course as. Looking at student work branched into a lot of discussion and ideation. I think we both found it difficult to stop from explaining details to the other while one looked through the online interactions.
  4. We then discussed nuances of interpretation that students might gather and divergent strategies we might be taking in our different approaches. These talks meandered quite a bit to the relationships between teaching content, learner context, institutional resources, materials and past experiences with the design.
  5. We concluded with itemized lists of revisions that might improve our instructional designs, we each added some plausible, and new, instructional tactics that might specifically serve these unique groups of students we have in these courses.

As it was my first peer observation for an online class, I didn’t know how our process stacked up against others. I have always taught online while also teaching other courses face to face, and this was my first formal review of an online class. The institution where I teach is going through a process of implementing review standards for online classes. I think standards are a good thing, if they promote learning in the end. So, after these reviews were completed, I went and read some of the literature surrounding the commercial materials being implemented by Quality Matters, the organization contracted to ensure a process of quality improvement for online course design. I found these materials from their website, and focused on one document (Introduction to the Quality Matters Program) in particular because I thought it might give me decent insight into how our ad hoc process compared to the initiative. That document provided an overview of the process and the focus areas, and I supplemented it with reading a white paper literature review published by Quality Matters.

What struck me was that our organic process and the areas of focus as stated in the Quality Matters literature seemed to outline identical discussions. The following table maps out the organic discussion as organized by the QM rubric.

Quality Matters Rubric Item Organic discussion among two peers
1. The Course Overview and Introduction  We both focused on getting to those materials, but I remember there was a fair bit of explaining verbally. This lead to a rather elaborated discussion about the role of the syllabus in online education, just how long it should and could be, and what value sub-documents might have as linked components of an online syllabus.
2.Learning Objectives and Competencies  Both Sarah and I related learning objectives when explaining materials we has added to our courses; competencies were are part of that discussion. What I found curious about this discussion is that each of us had more holistic goals that go beyond performance objectives. In one case, the competencies were state-mandated, which introduced a design tension: at the graduate level, the mandated competency is vaguely stated as required content of the syllabus. Is the instructor free to elaborate on that? Does that break the rules?
3.Assessment and Measurement  Both teachers elaborated on the rationale behind the point systems used, the rationale behind the tasks, and the values expressed by the design of the assessment systems as a whole. What struck me as curious about this discussion was the relationship between context and grading, revealing a design tension on one case. If a course is only offered once every two years, then with each instantiation materials must change, requiring changes in assessment practices, and limiting the design to an ongoing process rather than a fixed entity that can be decided on prior to the launch of the course. How would this context stack up against a rubric designed for courses with fixed and permanent content? Thus the design tension lies in the assessment of the assessment, rather than in the assessment of the students in their performances in the course.
4.Instructional Materials  I gathered from these segments of the discussion that materials’ discussions are pervasive within any discussion of online teaching. Our discussion lead to a recognition that all of the course is effectively course material. For example, in the discussion of navigation, which would be an access issue I assume, where content items are stored and how they are linked up (inside or outside of the LMS) is in fact teaching material in its own right. The design of how materials are delivered sends a message. A video linked outside the shell carries a message of don’t trust the system. Thus the design itself is a component of the course materials, a nuance I had not originally been cognizant of when I designed the course.
5.Learner Interaction and Engagement  Both teachers iterated the importance of interaction and the struggles with supporting it online. This lead to discussions of timing; e.g. when tasks are due during the week and throughout the semester. Views on interaction also lead to a discussion of teaching values. One instructor placed value on catering to the learners’ convenience by making everything phone-enabled. The other expressed a value of expressing the gravity of the task by pushing learners to experience the course via a full sized screen, were they can interact more fully, less distracted, and deeper. The contrasting values in this regard were some of my most valuable take-aways from the experience. Again, the configuration of learner interaction with both other learners and the instructor was both an expression of teaching values and a materials issue at the same time.
6.Course Technology  Discussions of technological choices were organic and pervasive through both talks. A number of persistent obstacles emerged. For example, Blackboard does not transfer links to purchased content (Atomic Learning models) resulting in dead links for students despite a number of proofreads of the course since the icons for the modules do in fact transfer over, albeit with outdated links. Thus each commercial content link, and there could be several in a design, must be revisited. In proofing a course before its launch, this can be difficult since catching all of them is not automated like we might find in a web authoring tool. Another point of concern was that Blackboard does not support a traditional wiki environment, in the sense that learners normally define the term. Supporting collaborative writing is then challenging despite the assumption that this technology supports that type of interaction.
7.Learner Support  I found it curious that our discussion of learner support actually hinged on a definition of our learners, who they are, and the broader goals behind what is meant to be taught in the course. Again, the topic itself cannot be disentangled from the discussion of materials. For example, one instructor felt that a course requirement asking learners to search out answers to technical topics prior to submitting questions was a move toward empowerment, and the other focused on curating materials because of the large amount of misinformation on the web. Both perspectives present a logical and reasoned position from a teaching standpoint. This discussion revealed an important aspect to understanding how a whole course fits together; something we would hope a viewer would know prior to evaluating an online course. It struck me that answers to the “correct” way one might empower learners in an environment that places so much emphasis on self-accessed learning hinges on each teacher’s perspective, and knowledge of their specific population of learners.
8.Accessibility  This term in the QM literature would need to be interrogated to be fully understood because accessibility can range from providing multimodal access to learning materials in a traditional sense (captioned videos for example), to values related to the design technology-enhanced learning more broadly. For example, I had not previously noticed the cluttered nature of my courses landing page. I had not discovered some of the tools on the landing page could be removed or hidden. This is actually an access issue. With so much on a single screen, is the instructor aware of how much of a burden they are putting on learners’ ability to find what they are looking for? Also, I had not included redundant links in convenient locations within the course. In retrospect, these are as much accessibility issues as providing captions on videos. We discovered this through the course of our talk. The same goes for providing free alternatives to software that I had assumed learners would have, an assumption I now question.

 

The literature on observation in teacher education makes some very clear points about how teachers are best supported. Some key considerations are that teachers are best supported by non-judgmental observations (Fanselow, Bullock, Van Es), observation of larger constructs such as course design and lesson plans reveal very different aspects of teaching expertise and perspective than close inspections of specific teacher moves (Preston, Rosean), and “effective” practice cannot be defined without the input of the teacher (Owston). I wonder if some of these ideas do not get lost when we try and standardize peer review of online teaching. The Quality Matters materials do seem quite tied to the current research in online learning, which is a good thing, so, it strikes me as a beneficial aid to a novice teacher. But I have to wonder if the rubric wasn’t created from discussions like the one I had with Sara. The mapping of the discussion over the rubric strikes me as too precise to be coincidental. Perhaps the rubric only tells you what *should* be in a professional discussion; things competent professionals would talk about anyway.

Perhaps the QM initiative is an attempt to combat the issue of teacher obsolesce in the face of new technology. I think what our conversation suggests to me is that teaching online is less about being online than it is about teaching. The QM website goes to great lengths to make it clear that it has a rigorous approach to development, informed by the most current research, and includes a lit review of some of the trends in online learning research. What I think the approach may be missing in this endeavor is a key recognition that teaching and learning are in fact two different things, albeit interrelated. Research into online learning can only take us so far into learning how to teach online. Once someone grasps the concepts behind the available tools, the discussion goes much deeper into values, perspectives, and deeply held notions about learning a specific set of content items. The relationship between strategy and content is not one to one, but rather stochastic and deeply tied to values the teacher holds (Reigeluth). We have a comparatively meager body of research related to online teaching strategies than we do on online learning. I came away from these talks with the notion that peer-review observation of an online course is less about technology than it is about what it means to teach, less about the machine and more about the people we are trying to connect with and nurture.

This experience left me somewhat suspect of decontextualized observations from someone who is not in touch with our learners. There comes a point where the context of the learner reigns supreme over any instructional strategy that a teacher is assuming, and those unique contexts surface through discussion with a colleague about how the materials and design came to be as they are. I welcome any peer review with a colleague from my institution, but I wonder how much common ground I might find with someone from somewhere else. Perhaps this is related to why we grant tenure based on teaching at the institution giving tenure and not elsewhere. The differences in foci among teachers at different institutions may parallel differences among teachers within one institution, but they may also not. What one teacher is trying to accomplish may, or may not, be what the rubric supposes the teacher is trying to accomplish, and may or may not be needed for these particular students given the circumstances of the program as a whole. I remember more than once coming upon the phrase, “no, that’s already in another course.” This may be why MJ Bishop (University System of Maryland) says we don’t do courses as much as we do degrees; the course and the teachers have to fit together. The conversation about what should be taught is equally as important as the conversation about how it should be taught, and I am not so sure the two can be extricated from each other in a meaningful way, as we found in the multiple locations in our talk where the discussion of materials could not be torn from the other topics that our meandering conversation unearthed. I would feel more comfortable discussing curriculum choices with a teacher whose approach I know, than one whose approach is still a black box. Professional familiarity has an important value on the context of a school, a value the QM perspective seems to overlook.

The value of professional knowledge of a colleague hints on something more profound, and something related to improving teaching growth on personal level as well as on a program or school level: the importance of trust and comradery. The importance of trust is pervading into many fields of late, including medicine. All of the components of teacher observation that I found in my in-person observations were present in this discussion of our teaching, and I don’t know how comfortable I feel about intimate discussions of my teaching rationale with someone who may function on very different belief system about what teaching is, is not, and what is should be. Nor do I see an incentive to want a third party to see my course through my eyes. Sharing that with a colleague I foresee working with for a long time does hold that value for sharing. Working with people you know and trust is not intimidating nor trivial; it is a pleasurable and feels like time well invested in a colleague and one’s own teaching. It strikes me that this will pay off for students as well; teachers who know each other and how each other teach can make better decisions about what policies to implement. Also, accomplishments can be shared, and setbacks can be supported. A peer review with a trusted colleague makes for a better working environment.

I came out of this experience with the conclusion that a peer observation of an online class is not wholly different peer observation of a classroom teaching– the most rewarding parts are the discussion that happens after looking at teaching choices, and in discovering why a trusted colleague teaches the way they do, sharing why I do what I do, and reflecting on how this might be done better for the both of us, and our learners.

Supporting healthy failure in online education

Edo is the formed name of the city that is now known as Tokyo

“Write this sentence in the past” (credit: 本気の発想 translated poorly by Craig as “serious creativity” from Twitter user tag: @Honki_Honki.

Teachers are told to make learning fun. Yeah! However, there is a fair percentage of learning that includes rejection and supporting that online is hard. Educational psychologists manage to gloss over the emotional side of things and come up with statements in research jargon that a learner is adjusting mental schema according to new information. Adjusting one’s mental schema to accommodate a big whopping REWRITE on a paper may have positive correlations on psychometric evaluations of learning, but calling it fun is a stretch. Many teacher training programs and educational psychology texts make it seem as if the learner simply processes information, without any pain. But adjusting one’s own “schema of the world” includes a key realization, I was wrong the first time.

Confronting one’s own being wrong is decidedly not fun, but it is effective. Sometimes it is so effective that we remember the lessons for a lifetime. Here are few of mine: Mr Savery, my highschool German teacher who I admired, told me to guess on the German language article declensions on my advanced placement exam. “Look, declensions in German are ‘en’ 70% of the time, if you try and figure it out, which is what you generally try to do and I know this because I have watched you take my tests for 4 years, you’ll lose time you could have spend getting to the questions that you are better at answering. Just guess all your declension items and move to vocabulary items as fast as you can.” This made me rethink just how much I knew and strategize my learning down the road. I went on to survive a year in Austria with this knowledge and pass a year’s worth of coursework delivered entirely in German. Dr George Hole’s comment on a final paper, “you didn’t think as much on this paper” was devastating. This forced me to rethink my writing and shortly thereafter I won a scholarship to Columbia University based on my application essay. Dr Carol Numrich’s similar comments on graduate papers were the same. Her feedback played into how I approached my doctoral application where I secured a fellowship to Indiana University. Ginette Delandshere handed me down a whopping REWRITE. Susan Herring’s 300+ requests for revision on the post-defense edit of my dissertation spun my head. How could I possibly write that poorly? Elizabeth Boling’s comment to my doctoral quals was eye-opening, in a more holistic way than just school, “righteous indignation does not serve you well in this context.” This idea played into my search strategy on the job market and helped me survive my dissertation. None of these rejections were fun, but they were all effective. All were in school, and none were online.

Healthy failures, as opposed to failures that lack transformative impact, also share something in common that is hard to accomplish online: care. The memory of the failure is coupled with the memory of the person who delivered it. It is the failures that are handed down from specific people that we remember. The failures I remember were all handed down from people who cared, or at least I felt that they cared for me. Imparting that feeling of being cared for is part of being a professional educator. While a missed objective question on an impersonal test may just require one to revisit the chapter, failures associated with more transformative learning all revolved around larger efforts: writing longer papers, preparing for bigger tests that seemingly predict a future, or attaining a significant life changing milestone–like finishing the Phd and getting my first tenure track job. Multiple choice, short answer and like approaches to formative assessments do not offer access to this kind of learning. Occasionally I have seen phrases on job descriptions in order of “genuine care for students,” but I have yet to see that operationalized into required teaching practices, especially online teaching practices. Failures on the big things may need to come from someone who has, or seems to have, a general interest in your well being in order to make that change from simple procedural and concept learning to transformational learning. The online instructor is not equipped with the same tools to express this care, and generally has had few examples to fall back on.

A large percentage of our online teachers have not experienced how care is expressed or attained online and have few experiences themselves of how failure is supported online, myself included. The online courses I have taken have all seemed somewhat removed from me; a kind of no frills instructional endeavor that gets right to the point, stays there, and ends. Read the materials, watch the video, listen to the podcast, post twice or three times a week, take the test, submit via the dropbox, finished. There were few pats on the back and little encouragement. Given this context, there was little opportunity for the instructors of these courses to impact real transformative change in my being. Any instructor who attempts incorporating support for failure is breaking new ground because, not only did they not experience failure support themselves, the media which they are working likely did not even exist a few years ago. Online instructors are not only learning to teach online, they are learning a whole gamut of digital ways to communicate. The best they can hope to do is try out some tips they saw at a conference, and learn from their mistakes over a long period of time, which I am trying to do.

Given the significance of these experiences in the longer term perspective of the experience of learning, I find it remarkable that I don’t find more emphasis on teaching failure well. Andrew Tawfik is starting to scratch the surface here, but we have a long way to go. In many years of teacher training I don’t remember ever focusing on how to deliver and support failure in learning, and I don’t remember any of these experiences that happened at-a-distance. Now that I am teaching online via an LMS, I feel even more handicapped at delivering this kind of caring for failure via the machine. Perhaps this is the greater implication of Joni Dunlap and Patrick Lowenthal’s advocacy for high touch low tech in online learning. But this is far easier said than done in contexts where learners have little time for synchronous communication and all their experiences come filtered through the lean media of a learning management system.

Healthy failure is difficult to administrate. It is a tricky business to put requirements for failure into programs, course designs, or teaching requirements. In the dynamics of instructional roles among teachers, few instructors enjoy being the one who tells learners that they were flat out wrong, the paper failed to meet certain standards, the learner is unlikely to pass given the present course of action, and so on. In graduate education there is a second ethical responsibility of telling a learner that, while the student may pass the course, from the state of the learners’ progress, it appears unlikely that the course of study will prove a prudent use of the student’s time, energy and money. This may just be the dirty work of teaching, especially online teaching where the message comes stripped of all tone and affection expressing care. Every job has some dirty work. Handing out the required rewrite may just be the dirty work of online education. Including precautionary statements in the omnipresent, all powerful, contractual agreement syllabus, seems to have little effect in mitigating the emotional impact. The statement “If the required criteria are not met in your final paper you may get a rewrite” is little consolation to the learner who actually does get a rewrite. Explanations that the editing process includes some of the most important aspects of learning the course entails can add salt to the wound even though the intension of the statement is to express care.

In our effort to make learning fun we sometimes handicap teachers, and learners, into thinking it is supposed to be fun all the time. If our most effective teaching strategies include experiences that are not fun, we may be handicapping our teachers from using the best tools they have to teach, especially online.

Design Tensions

I am writing this blog post after a failed online search for a clear, concise, yet sufficient explanation of design tensions in instructional design. Deborah Tater’s 2007 definition is here, but it’s a fair bit longer than mine. Of course, her’s is also said more elegantly.

Design tensions are relationships between design components or features that are linked such that when one is repaired, improved or changed, another fails, is altered, or rendered ineffective.

For a general example in education, when designing for a problem-based intervention, too much structure in supporting scaffolds destroys the nature of the problem-based learning by giving away a quick solution, while too little supporting scaffolds may render the intervention impossible or overly frustrating for learners. When combined with the context of varied skill level among learners, the tension arises that as soon as one subset is scaffold-ed, the challenge is rendered meaningless for the others.  Design tensions can be so great as to render the approach infeasible, or alternatively, they may be somehow minimized or balanced in the process of design.

Most importantly for instructional designers, tensions must be identified in order for a problem space to be defined. How one frames a the  design problem is highly related to the resolutions that’s designed (Dorst, from Frame Innovation). This is what makes the act of finding and expressing a design tension so integral to the process of design. It *is* design.

In a design document of any kind, if one says there’s a design tension somewhere in a design, the listener is expecting the next sentence to tell which component or feature impacts which other component, feature or characteristic of the design. AS a reviewer of design cases, I can’t tell you how many times I have read an insightful author describe  something they call a design tension, and fail to express the second half of the tension because it was so obvious to them.

HCI deals with this more explicitly than instructional design. The graphic above is from a book on interaction design by Brian Whitworth with Adnan Ahmad. A full elaboration of the concept of design tensions is available in: Tatar, D. (2007) The Design Tensions Framework. Human–Computer Interaction (22) 4, 413-451.