Home » research (Page 2)
Category Archives: research
The International Journal of Designs for Learning and the differences among Design Research, Design-based research, and Design Science
I was very happy to receive an invitation to join the advisory board of the International Journal of Designs for Learning (IJDL) yesterday. IJDL started in 2010 with a purpose to fill a very specific niche in scholarship in instructional design. Until that time, there was no venue exclusively set aside for design cases, rigorous analyses and reflection on specific designs for learning (Boling 2010; Smith 2010; Howard 2011). I was in the right place at the right time. I had been a volunteer assistant editor for Tech Trends’ Instructional Design Portfolio feature, a regular column allowing authors to share designs. The ID Portfolio was a fantastic first step and had expanded the field’s awareness of this type of research. However, the venue limited the length of the articles and was print media despite authors’ and reviewers’ desires for more space and oftentimes animated or interactive expressions of the designs. Design cases in IJDL are essentially unlimited in length (the guidelines say 9,000 words, but more than 11,000 is common), and the format is flexible—multimedia is the preferred choice of expression for approximately 20% of the authors who contribute to IJDL. While my dissertation was not a design case, I became very interested in this form of scholarship and subsequently wrote my own case. I also wrote an analysis of the difficulties authors find in publishing this form of design research, and a definition of this form of knowledge building. What an opportunity it was to work with Gordon Roland, Kennon Smith, and Elizabeth Boling, and all the advisory board members in those first two years the journal was up and running. Recognition for this type of research is growing, and it should be. Both practitioners and scholars of instructional design need to understand just what the goals of the different forms of research are in order to make practical decisions about where they invest their reading time. Alan Foley presents it well in his graphic that he uses to depict to graduate students the different types of research in their field that include the term “design” in their title. See the graphic below.
For quick reference, here’s the consensus that’s growing around these terms: Design research has two types—one that speaks to how we design using various traditional methods of research, and another type, the design case, which speaks to the precedent a designer, or someone close to a design, has drawn out of the process and product of a specific design (Boling 2010; Howard 2012). Design-based research (Squire & Barab 2008) speaks to theories using design as the means for making those insights. Research in Design or Formative Evaluation is a type of research where the goal is to improve a specific design. Design and Development Research aims to establish empirical grounds for design practices. And finally, Design Science, a term Kevin Williams introduced to me, is parallel to design-based research placing emphasis on abstraction and theory building via the act of design in Information Systems. Check out this link to see just how similar the two terms are defined between Information Systems research and Instructional Technology research.
Most discussion about MOOCs (Massively Open Online Courses) seems to miss the point entirely. I have been sent a solicitation to talk about an informational graphic which lauds MOOCs, but have also discovered an excellent discussion of the dangers of MOOC-ing. Both seem to make the same error. We get a lot of discussion about these technologies from people outside of education. Some even start their positions with statements that they question their own legitimacy. Let me back up and provide context to any discussion of MOOCs that will demystify the topic and situate it among the larger scope of what we know about learning.
The MOOC is an instructional strategy that has arrived precisely as was predicted over 2 decades ago. In 2006, Ted Frick, PhD, now an emeritus faculty member at Indiana University, made a statement to a group of incoming PhD students. He said that it takes the educational community about 30 years to figure out how exactly to use a new technology for teaching and learning. Tim Berners-Lee created the web browser in 1990, and the first web browser to muster widespread use was Mosaic, which appeared in use in 1993. 2013 is exactly the date educators like Ted Frick expected to see the educational community grasp and run with the potential of web technology, but of course we expect 10 more years of significant developement. Instructional designers and educators who have made careers in distance learning are not surprised by MOOCs. They expected that by now, the community at large should understand the web well enough to apply it to teaching and learning in systematic ways which realize its potential. At the same time, the hype surrounding “Web 2.0” is not all forgotten. In the late 2000’s, Tim Berners-Lee said that Web 2.0 is jargon aimed at creating hype around aspects of HTML that have been there since its inception. So is the story with MOOCs. There is nothing technologically different with today’s MOOCs in comparison with regular technology that has been around for quite some time. Therefore, one can conclude nothing other than that MOOCs are an instructional strategy, not a technology.
There are many instructional strategies. Some encompass whole curricular designs and have multiple sub-parts, some focus on discrete points. Some are designed to teach process (for example, apprenticeship), some teach concepts (cloze activities), and other strategies teach attitudes or mindset (boot camps). Here are some instructional strategies you might be familiar with: the multiple choice test, the lecture, listen and repeat, group work, e-learning, and flash cards. Here are some instructional strategies you might not be familiar with: collaborative or pair testing (Mark Evan Nelson), the Silent Way (Gattengo), airborne instruction (MPATI). It is the very rare instance that one instructional strategy encompasses the entirety of an educational system. Can you imagine an entire course of study containing only lectures? The multiple choice test may be a sufficient strategy for certain aspects of teaching, but one would surely not want to do an entire curriculum via multiple choice tests. Using one instructional strategy for everything is rarely prescribed by those who design instruction on a regular basis.
Yet we find the arguments surrounding MOOCs to be primarily based on the assumption that the MOOC is a form of education, or that the MOOC is a new technology. Neither of these notions are true, and the majority of educators I have encountered understand this. One indicator that these discussions have little to do with education is that very few educators are taking part in the discussion. Some educators who focus on learning technologies, for example see Curt Bonk’s extensive website, put MOOCs in line with the many other technologies which have changed the landscape of distance education. But on the other hand, one of our nation’s leaders in education has raised the topic of teaching and learning via the exact opposite of the MOOC (Arthur Levine, example article here). He asks, instead of one teacher teaching many, what if we had a many teachers teaching an individual for longer periods of time? Sure, Arthur Levine is talking about technology in education, but he’s putting it in the larger scope, and considering a number of perspectives. When educators talk about education, this is how they do it. They consider a multitude of perspectives, and contexts, in light of the world we live in now, and the larger lives of the learners in our charge. Levine is not saying one instructional strategy fits all cases, and Bonk is viewing MOOCs as not a solution– rather an option among many. Both are saying we need to re-examine how, what and why we teach the way we do.
In this light, let’s look at some of the rhetoric surrounding MOOCs. Allison Morris of onlinecollegecourses.com has presented me with an informational graphic to help me understand MOOCs. In her graphic she makes some assertions that may warrant a closer look. She writes that “They’re growing at a rapid rate.” This would be true of any instructional strategy that is entering the main stream after many years of development and distribution. I would be more interested in seeing the growth in popularity of the multiple choice test because at this point, I think the educational community knows something about how multiple choice tests work. Binet got it down as a true proxy of reason back in 1905, and Thorndike got the multiple choice test into the form we now recognize, about 10 years later circa 1915. I would imagine that we started to get good at creating and applying multiple choice tests in the 1950’s. Where is the multiple choice test now?—everywhere. The multiple choice test has had its day and now its popularity is waning. Debates surrounding the common core reveal the limits of the instructional strategy. The limits of the MOOC are, as of yet, still murky. If Morris is correct with her assertion, the implication is that we’re going to see a lot more development of the MOOC. Growth in popularity means the strategy is not mature, and just like any other strategy it can be applied well, or poorly. What I do like about Morris’s informational graphic is the way she has laid out the content — a powerful method of conveying an argument quickly. But what we have in the end is an argument that holds no bearing on MOOCs; popularity, demographics, and a description of the designers and participants, does not have any bearing on the efficacy of an instructional strategy.
Fig 1: Allison Morris, The Mind Behind the MOOCs
The discussion would not be complete without the dystopian perspective that MOOCs are horrible. I have seen this theme reappear in the Chronicle of Higher Education repeatedly over the past 12 months. Early last year one scholar warned the Chronicle’s readership that content delivery does not equate to education, and that this error in reasoning would make real education a luxury for the rich (Graham, 2012). Another seasoned scholar wrote more recently that MOOCs are a product of the time, to blame for part of our collective degradation of educational practice, making today’s education a re-instantiation of Wal-mart (Patrick Deneen, June 7). I don’t think it is unreasonable to say that the reason we see positive perspectives of MOOcs on literally thousands of blog posts (Blog + MOOC retrieves over a millions entries in Google searches) and dytopian perspectives on MOOCs in articles in the Chronicle is that those who support the practice are generally not scholars—they are internet users who are involved in online education, often exclusively in at-a-distance education, and the former are scholars who are quite adept at writing for publication. Deneen’s argument that MOOCs are the Wal-mart of education is well constructed, elegant, and at times even funny. I suggest everyone partaking in the discussion read it because the article raises relevant topics and is well written. But, the argument does not stand. Here’s why.
The fact that educational strategies are changing is not evidence that one strategy is wrong. Education is always changing, and should change, as Arthur Levine argues, because society is always changing. As educators we must keep education relevant because it is directly linked to society, and learning that is out of synch with life is always lacking. Signs of the times such as initiatives for “measurable outcomes” and “assessments” (that are also under debate, by the way) are not an indication that designers of instruction, or institutions that are adopting those designs, are down a path of doom. MOOCs may promote a monoculture, as Deneen argues, but we have seen no indications that the monocultures of MOOCs are actually any different from the monoculture of required textbooks, or other standardized content which gains popularity across nationwide curricula. (For example, is English 101 really different from institution to institution?) While a Wal-mart of education may not be all to attractive, who is to say that the Wal-marts of the world are not a driving force behind the popularity of farmers markets which also did not exist a few years ago. MOOCs may just be the strategy which raises our awareness of the diversity of perspectives and backgrounds that enrich our institutions. Distinctions are now easier to draw between one instructional approach and another because they are more visible via MOOCs. However, this does not credit nor discredit an instructional strategy. James Paul Gee may have had a more insightful observation when he wrote about the rise of the distinction between uniform and nuanced curricular approaches, making boutique colleges and universities an educational landscape apart from the commodity institutions. Visibility itself is not a criteria for quality practice.
Thank you to Allison Morris for prompting the conversation I have wanted to post about for about a year. I hope this post will prompt good discussion from students of instructional design, and lay to rest from of the misguided rhetoric that looks at MOOCs in black and white terms without seeing it as another strategy that will have a life of its own like any other, for better or for worse. I welcome comments that bring to light my errors in this post and take the conversation in a positive direction.
The July 19, 2013 Issue of the Chronicle review contains an informed response by Clay Shirky to Patrick Deneen’s rant against MOOCs. He seems to be taking a similar position as mine; he writes, “The effect of MOOCs on the academy is no more likely to be about pedagogy than the effect of MP3’s on the music industry was about audio quality.” His argument is primarily based on the economics, but he brings a larger understanding of how educations functions to the discussion of MOOCs.
Just posting some slides from a recent talk. I enjoyed pulling it all together and having a keen audience. Pulling back and looking at the big picture now and then forces one to question what all this study is about and what purpose it is serving. That’s a good thing because I wouldn’t want to get so wrapped up in all my research that I miss the big picture about where it is all going. From where I stand now, i can see the purpose of the PhD spelled out clearly from the time I was a language teacher as a substitute at Gateway Residential High School in Buffalo in 1994 to now. I have always been looking for ways to help people *discuss* better. Here are the slides: The features of Instructional Design(s)
I was recently asked to write a vision statement about how I saw the field of instructional technology and where i feel it is headed. So, I wrote one. I will share it here because it may spark some interest and could lead to some curious conversations.
Vision Statement about the future of Instructional Design
Prophecies of how MOOCs (Massively Open Online Courses) will revolutionize teaching and learning often praise the wrong aspects of these technologies. My vision of the future of instructional design does not see instructional designs replicating classroom teaching online. Rather, I see us educating in a way that respects the time and value of a face to face experience and taking advantage of the scaffolds that mediated instruction offers. MOOCs can deliver content better than previous technologies, but that is not all of education, nor is it much of instructional design.
Scholars warn us of the dangers of equating education with content delivery, and point to a dismal future where face to face instruction is a luxury only the rich can afford (Graham, 2012). I keep in mind the limits of technological tools and the value of the human experience when I talk about the possibilities technologies bring. My vision of instructional design is future where educators understand the limits of technology, but also understand how to manifest learning with these tools that cannot be created otherwise.
Accomplishing what cannot be done otherwise often questions assumptions about teaching. The paradigm of one teacher per classroom is deeply engrained in our perspectives of learning. However, placing learners among multiple instructors can dramatically influence how learners think. We can place learners among multiple experts in virtual approaches, but not in designs that take place in traditional classrooms. Even if we could put multiple teachers in one classroom, teachers in a classroom with learners tend to teach rather than discuss frankly among themselves. In teaching pre-service teachers, I recorded a discussion of four experts while they discussed and analyzed a video of real teaching. I placed their comments into annotations which appeared and disappeared from the viewable screen of that same video. Then, I asked learners to watch the video and discuss it. Approximately 50 learners watched the video with the expert annotations, and 50 without the expert annotations. In the learners’ resulting discussions, learners who had watched with expert video annotations used three times as much higher order thinking in their comments about the video (Howard, 2012). These learners were never told the comments were written by experts and not classmates. This is a teaching tactic available through technology that is not available otherwise. This was my dissertation study.
Another example of creating teaching and learning otherwise not possible is the use of anonymity as an instructional strategy. Traditional classrooms are not anonymous. This does not mean that the experience of critically discussing in an anonymous condition is worthless. It means there are ways to use anonymity that support learning, and potentially ways that could hinder learning. In a teacher education course that normally held design critiques, I had learners complete these critiques via a discussion board (forum). Approximately half the students discussed anonymously, and these students wrote twice as much criticism of their peers’ work. However, they also supported these criticisms with four times as much reasoning and supporting evidence (Howard, Barrett & Frick, 2010). Learners in the anonymous condition thus received and gave more criticism with reasons, an experience we want our future teachers to have. I concluded that while anonymity could have potentially negative consequences for a teacher who does not adequately prepare and monitor learner participation, anonymity also creates a worthwhile learning experience online that is not possible via the traditional classroom.
Simple email can support discussion in English for ESL learners who would otherwise never get the chance to read something written in English exclusively for them to read. While teaching in New York City (CUNY, 1998-2000), I jointly developed a small 5 week curriculum with other CUNY teachers that linked our respective students in Queens and the Bronx in pair work via email. Asynchronous pair work is not the same as pair work in class. Policies and procedures needed to be developed. However, this activity grew into a full 15 week course. I took the project with me to Kanda University in Chiba, Japan. In 2002, I received a grant for development of the project which allowed me to go to a number of our partner institutions, present the curriculum at conferences, collect interested partner schools, and create a sustainable curriculum. Since 2002, thousands of students have used the Kanda University Email Exchange Program and linked with learners at other colleges for cross-cultural pair work online (Howard, 2004; Van Moere & Howard 2002). These learning experiences were brought to life through the technology and could not have been accomplished otherwise.
Growing interest in design is a bellwether that this change is underway. Submissions of design cases to the Instructional Design Portfolio (once a feature in the Ed Tech periodical, Tech Trends) have increased so much that a new journal has emerged, the International Journal of Designs for Learning (IJDL). In 2008, there were four submissions of close analyses of specific at-a-distance learning designs submitted to Tech Trends. In 2009, another six were submitted. In 2010, the feature became a journal and seven design cases appeared in the first issue of IJDL. The number of submissions has doubled every year since the journal’s inception. For the 2012 issue, over 35 articles were submitted to IJDL (Howard, 2010; Howard, forthcoming). As the assistant editor for these publications, I witnessed the change in perspective of scholars and practitioners of instructional design from “taking courses online” to critically analyzing how the mediated instruction functions in unique and curious ways.
Now is an excellent time to be starting an online program. Established programs in instructional design and technology are retooling to meet the needs of learners who are both more informed about what they can learn using these tools and at the same time in greater need for guidance to navigate a sea of information at their fingertips (Boling, 2008; Howard, 2010). My vision for instructional design is respectful of the timeless scholarship that has brought the field of educational inquiry to where it is now, and excited about possibilities technology begs us to explore.
Boling, E. (2008). From students to scholars: Revision of the doctoral program in Instructional Systems Technology at Indiana University. In, M. Orey, J. McLendon & R. Branch (Eds.). Educational Media and Technology Yearbook 2008. Westport, CT: Libraries Unlimited.
Howard, C.D. (2012, April) The Impact of Modeling and Staggered Participation in Video-annotated Pre-service Teacher Discussions. Paper presented at the Annual Meeting of the American Educational Research Association (AERA), Vancouver, BC. Canada.
Howard, C.D. (2010) Obstacles to the credibility of online learning. eLearn, 12(1), online. http://elearnmag.acm.org/archive.cfm?aid=1730789
Howard, C.D. (2004) Identifying the role of corpora in EFL materials design. The Journal of the Rikkyo University Language Center, 11(1), 39-47.
Howard, C.D., Barrett, A.F. & Frick, T.W. (2010) Anonymity to promote peer feedback: Pre-service teachers’ comments in asynchronous computer-mediated communication. The Journal of Educational Computing Research, 43(1), 89-112.
Graham, G. (2012, October 5). How the embrace of MOOC’s could hurt middle America. The Chronicle of Higher Education. pp. B22-B23.
Van Moere, A. & Howard, C.D. (2002) Introducing Email Exchange in a Freshman Curriculum. The Journal of the Research Institute of Language Studies and Language Education, 12(1), 99-117.
A few meetings at AECT made it really memorable. I was able to spend some time with three people I met previously – Andrew Tawfik (website), Heng (Patrick) Lou (website) and Patrick Lowenthal (website). One thing which was stressed was the importance of blogging. I have always had a lot of fear for putting unedited text out there for public view. I think I need to get over that. Most of the scholars I met this year at AECT blogged in one form or another. We also brainstormed blog themes in the Early Career Symposium- the ECS at AECT was an excellent networking and learning experience. More on that later….