craig dennis howard

Home » 2014 » April

Monthly Archives: April 2014

Design Tensions

I am writing this blog post after a failed online search for a clear, concise, yet sufficient explanation of design tensions in instructional design. Here’s my attempt at such an explanation, along with my acknowledgement that this can be said more elegantly. I hope for a better definition from someone out there because I am not strong in this area, and I know others are (eg: Deborah Tatar — see citation at the bottom.)

Design tensions are relationships between design components or features that are linked such that when one is repaired, improved or changed, another fails, is altered, or rendered ineffective.

For a general example in education, when designing for a problem-based intervention, too much structure in supporting scaffolds destroys the nature of the problem-based learning by giving away a quick solution, while too little supporting scaffolds may render the intervention impossible or overly frustrating for learners. Design tensions can be so great as to render the approach infeasible, or alternatively, they may be somehow minimized or balanced in the process of design.

Most importantly for instructional designers, tensions must be identified in order for a problem space to be defined. One has to know what the problem is in order to solve it, no? (Not sure on that one, but it sounds good to me so far). Like is said in many contexts, defining the problem is half the battle.

In a design document of any kind, if one says there’s a design tension somewhere in a design, the listener is expecting the next sentence to tell which component or feature impacts which other component, feature or characteristic of the design.

HCI deals with this more explicitly than instructional design. The graphic above is from a book on interaction design by Brian Whitworth with Adnan Ahmad. A full elaboration of the concept of design tensions is available in: Tatar, D. (2007) The Design Tensions Framework. Human–Computer Interaction (22) 4, 413-451.

Advertisements

Campus Pack (CP) addition for Blackboard: A usability review

I was asked to review Campus Pack (CP), a Blackboard overlay that attempts to improve on Blackboard functionality. Here are the issues I found.

1. Campus Pack lacks scent trails, or an intuitive design

The CP homepage graphic. This post is not in support of CP.

The CP homepage graphic. This post is not in support of CP.

Intuitive design is the feeling that a device makes sense from the user perspective. Campus Pack seems to challenge the user, and make even the simplest of tasks more difficult. For example, when new pages are created in the wiki, they must be accessed via linked hypertext that only appears on down scroll. Someone coming into a wiki to find new work will be hard-pressed to figure out where to access the work they are looking for. The browsing instructor can search for hours and never find the students’ completed work.

The dashboard is unlabeled and difficult to locate. Campus Pack uses a number of internally created terms which I find, and my students found, misleading. For example, “assignment dashboard” is a Campus Pack internally created term, and is the path to assignments, but it is not labeled as such and it difficult to locate. A Dashboard speaks to me as a selection of instruments gauging progress and present measurements in different areas. This seems to have a different meaning in Campus Pack. For CP, people have access via a dashboard.

Redundant features confuse users. CP contains a “directory” that neither access campus information nor BB information. Instead, it is a class roster that requires students to type in information about themselves. These types of problems plague CP. Campus Pack includes a redundant messaging, which does not disable the standard BB messaging function when activated. Therefore, the instructor may have messages they are unaware of. This duel messaging functionality cause problems because leaners who sent the instructor mail may assume their instructor checks “CAMPUS PACK MAIL” but the instructor may actually never find out about the message. Other pages are redundant and result in confusion. “Campus Pack: Course membership” serves the same function as the Blackboard Roster, but its presence misleads students to thinking that there is an alternative pathway to the wiki’s when there is not. Active hyper-linked pictures lead to spaces that appear as if they are undeveloped wikis, but are in fact a completely different content location. Students can easily be fooled into believing all their classmates’ wikis are empty. CP also includes a large, installable analog clock as a redundant feature for computers that do not have clocks. (I am yet to find such a machine.)

It should be noted that these new uses of older familiar terms are not defined. While a new use for an old term is in and off itself confusing, CP provides no glossary for their terms. Many items in CP are called “portals.” That’s fine, but few students have a clue what these words mean, nor how it impacts how they do things in CP. I am unclear as well. There is little use for a user to have their menu called a portal, unless the menu somehow functions differently from a normal menu.

2. Campus Pack has limited compatibility

Limited compatibility issues make CP very difficult for users, and limit its value as an addition to Blackboard. Mobile phone integration does not appear available for Campus Pack. Copy / paste functionality is not universally compatible with other programs. Campus Pack support C/P in some browsers, not on others. If your students read online, but write, and edit what they have read, in collaborative writing tasks, this will cause problems for students attempting to complete those tasks. Therefore, the software does not appear to have advantages over the most rudimentary versions of Blackboard. Students will like be unable to do any work on the fly via phones, and they cannot complete writing tasks in some browsers.

Some capability is actually disadvantageous. Campus pack includes social networking gadgets that link outside the interface, and can expose learners’ coursework to the public. SNS integration may appear an attractive feature of CP provide the instructor is protected from any type of related litigation, and does not teach in a public institution where the disclosure of learner information might potentially put the instructor in tenuous legal circumstances. Twitter, Facebook, and Flicker accounts can all be linked via CP.

3. Screen space is poorly used

While not nearly as detrimental as the two topics above, the misuse of screen real estate makes Campus Pack even more difficult to use and should be mentioned. Screen real estate is devoted either navigation, or to work space; the two functions are not integrated in one user interface. This creates problems in work flow. One cannot access, work, and save and move on seamlessly. The tasks must be broken up completely. Thus, you cannot read text in one area and write in another. Rather, navigation stretches the length and height of the screen, occupying all of it. For example, in some areas of the LMS addition, five different horizontal menus are employed while the rest of the screen is dead space.

Strategizing group research

gantt-chart-templates-1024x724

A Gannt chart is an excellent tool for organizing group research, but a research strategy needs a little more. Check out http://www.trainingable.com/free-download-excel-gantt-chart-templates/ or some free Gannt Charts.

I see research groups get stuck all the time. Graduate students, and those setting out to do research in groups, can get bogged down and tripped up at dozens of spots in the process from conception of an idea to publication of a completed study. Collective intelligence matters less in the process of research than strategy does in seeing the task through to completion. I’ve put together a few thoughts to help research groups strategize their process of working together.

Disclaimer: Putting these ideas down in a document was inspired by two blogs I follow. I am a big fan of two online blogs where senior faculty tell it like it is to doc students: Karen Kelsky’s the Professor is In and Matt Might’s Computer Science Blog. Both of those blogs offer great tips for doc students and others trying to do research, but I haven’t found this particular topic on those blogs—some of these ideas might be there under different terms than the one I use- strategizing group research.

1. Write down a strategy. I am surprised by how few groups seem to actually have a research strategy written and shared among group members. I think some of the fault lies in the terms we use to do research. Research strategies are comprised of more than just research methods. Of course, without the knowledge of methods a study will never get off the ground, but the knowledge of a research method can give a group a false sense of security that they know exactly how to get a study done. Coursework in a research method simulates a research environment much like an aviation simulation mirrors really flying a real plane. In a methods course, errors will not kill you, nor kill the study; they will just provide opportunities for learning. Errors in the real world can stop a study in its tracks. Research methods are so complex that having a controlled environment is often the only way to start in on the process of learning how to apply a method of data collection and analysis, but coursework takes place in an artificially controlled environment, unlike “real” research that happens in the real world. So this blog post is about those other areas that deter studies from finding their way into publication. A research strategy is usually a shared document that delineates who will do what, by when, and is all-inclusive, from IRB approvals to drafting tasks, from start to finish. It’s not set in stone, it’s a living documents that gets a group from start to finish on each project. I offer a few tips for creating a strategy to get the job done beyond the methods.

2. Document reasons for group membership. Students who have completed coursework in research methods often overlook the roles different people play in a research group. Perhaps this is because the experience of group work in methods coursework sets a stage where everyone is equal, and that aspect of the simulation is far from the truth of doing real research in the real world. In the real world, each member of a research group is there for a different reason, and those reasons are important. They significantly impact how group members are likely to behave. Each person needs a certain thing out of the project. Perhaps it is learning a new method, or perhaps it is getting a pub. Perhaps it is some stake in a research grant. Perhaps it is exposure to the experience. Whatever it is, it needs to be written down. Often, graduate students are not taught to do this in their coursework. These reasons are not addressed in methods coursework because they are not methods, but they are part of research in a group. Recognizing these roles is part of a solid research strategy. Methods classes rarely tell you to solicit and document the reasons people have come to a research group, but the practice informs all the members of the group and facilitates good will and understanding as the group moves forward with a project.

3. Recognize power dynamics in the beginning. Very often faculty and students are mixed. Sometimes junior and senior faculty are also mixed. When these contexts occur, power dynamics can get confusing. Students may solicit guidance from a faculty member who is not the leader on that particular study and their subsequent advice may be taken in undesirable directions for the leader of the group, or they may not recognize junior faculty are not exactly full equals to other members of the group. Sometimes a faculty member will refuse guidance as they might feel out of place, or if the leader is using the study as a teaching mechanism, for either a junior faculty or for a student. Recognizing positions of authority can help everyone navigate awkward moments and move on.

4. Discuss authorship and roles prior to the onset of the study. This is probably the most critical of the points I am making here. To be honest, the onus for this rests on those in lesser positions of power because they have the most to lose. At the same time, I am yet to encounter a faculty member anywhere who doesn’t want the process spelled out clearly – so do not be hesitant in approaching any faculty member leading or participating in research with questions about roles and authorship. It often happens is that the group is so busy in doing the work that they forget to stop and make some decisions about the strategy to get each aspect of the project done, or he needs of certain members are overlooked. There are dozens of websites out there with guidelines to determine who is first author, but the truth is, it is often the highest ranking faculty member who sets the rules. For my own work with students, my policy is always the same: If you have written the first draft of the paper, you are the first author. If I am invited to be a subsequent author, my name will appear last no matter how many students are included in the byline, and I will read and edit the draft that is submitted as well as the final proof, and those are the only drafts I will read from start to finish. I think most faculty members have some sort of general policy but rarely put it in writing because of the many different contexts they encounter. My own circumstances are pretty limited so I can put forward a blanket policy at this point in my career. While a discussion of roles and authorship may seem trivial, I have seen plenty of cases where studies develop in one direction or another where the roles and tasks were not clearly spelled out at the start, confusion ensues, papers never make it to submission, and there’s a fair bit of “I thought you were going to do that part.”  Everybody loses in such a case. I suggest that all groups doing research have the authorship discussion up front to.

5. Have a contingency plan. A contingency plan is a plan B should members not be able to complete up their assumed role. The plan should ensure that the study makes it through the process even when things pop up, and thing generally do pop up. From my experience of multiple-author research, the contingency plan generally rests on the shoulders of the first author. For example, when there is a deadline ASAP, it’s the first author who sucks up the final touches and provides the momentum to get the product out the door, even on tasks agreed to by others. If the policy is addressed openly, feelings are not hurt and members can proceed, if nothing else, at least informed. The plan especially comes into play in cases where one is waylaid by illness, taking kids little league games, a death in the family, or any of a number of reasons that put obstacles in the path to publication/presentation. From group to group the first author position may means something different. In my experiences, the first author position had a specific meaning that was to-often unspoken– it’s not only (and not necessarily) the one who did the bulk of the work. It signifies who was the driving force behind the research itself, and who is taking responsibility for its completion. Of course, the mores of practices change among fields and institutions, so having the contingency plan discussed ahead of time can be a welcomed insurance policy for guilt and hurt feelings should the unexpected arise. When members are from a variety of cultures, straightforward addressing of the circumstances of contingency becomes even more important.

6. Avoid assuming that formulating the research design determines first authorship. While practices may vary among institutions and fields, the authorship discussion may also bring to light some important nuances of research we rarely get in methods courses. Just because one person came up with one idea does not mean the study itself is “theirs.” There’s a lot that goes into research. When groups meet, dialogue often results in new ideas. Group members new to a group may feel slighted if their assumptions about credit are not fulfilled in the practices of the group. In my own experience, faculty do not require first author position for every study they guide; however, in some cases, a particular study may be of particularly importance, either to a long-term trajectory or for some other reason. As long as the role of the study and value of the credit for it are addressed early on, the authorship discussion is a comfortable one. Faculty accused of “stealing” students research ideas are less likely research thieves than they are guilty of not having the authorship discussion up front. A lot of this could be mitigated by avoiding assumptions about how group work results in credit. Unfortunately, as it is often the responsibility of the student, and not the faculty member, rarely does the faculty member initiate the discussion or put forward these ideas at the start.

7. Several studies can be written on the same data, but this means a separate strategy is needed for each. Planning ahead and deciding who is writing which study from a given project is needed in order to do this. Few projects are at this phase in the early stages. Simply designating one person as “doing qualitative” and another as “doing quantitative” doesn’t avoid conflict in the future because the terms are not accurate descriptions of research.  In my experience, “qualitative” is too vague. Simply equating “quantitative” with anything study that contains numbers is also too vague. I would avoid the both terms in deciding who is doing what. If a project is in the early stages, and multiple studies are planned, some things may need to be worked out prior to the strategy and authorship discussions. In those cases, it is helpful to anticipate revising the strategy as you move forward.
The final three points might be specific to scholars in education. I don’t know enough about other fields to speak to how research groups might work most productively in those fields.

8. Look beyond the quasi-experimental study. One of the problems research groups have is that newer members may only see as far as the experimental aspect of the research. While many research designs in education will fall into this category of quasi-experimental study, there is much more to look at in most projects.  There is the design of the intervention, close inspections of learner experiences, stakeholders’ accounts of a practice, and methods articles, just to name a few. Very often, these other aspects of the study are the most interesting part, once you dig into them. And the experimental study itself may face larger obstacles to publication than new members are ready to tackle: quasi-experimental studies are often closely read for precise usage of research terms, such as the proper labeling of the data, the correct usage of words like “control group” and necessary treatment of what makes them “quasi” as opposed to true experiments, such as having a placebo. The literature review is perhaps the most foundational aspect of any study that gets overlooked, yet group members stepping forward excited to do the literature review are rare. The person who does the literature review is an important and often overlooked role.

9. Revisit the strategy when major changes occur. One of the most difficult things to grasp about the act of actually doing research is that research always reads linearly, but is almost never so in practice. Rather, the process is iterative. I might also add that is it not a bad idea to revisit the strategy after the initial grappling with the data by the group. Research questions are a work in progress until the proofs go to publication, in most cases. Initially viewing data under a general driving concept is great, but keeping in mind that the RQs can change as the project evolves can really empower a research group, and if all the members are aware that the strategy can be amended at any point, people can be encouraged to follow fruitful paths. Avoid getting stuck in “causes” and proposed calculations in the beginning. They can be elusive, especially in early stages when data has just been collected and is not fully prepared. Also, some calculations may or may not be worth the time to calculate until the group has grappled with some of the main ideas. You can always add calculations later at reviewers’ request. If you haven’t yet dug into the data, and done a few passes through with your analysis scheme, you can’t truly foresee what that data really means; and therefore, the initial strategy may have been naive. Seasoned scholars are not deterred by this, but newer members may feel discouraged when data shows the unexpected.

10.  Share the strategy document when new members arrive, and revisit it at the end. The research strategy is more than a Gannt chart. New members need to see the progress that has been made to catch up quickly and contribute, and simply sharing a strategy document with new members can bring them up to speed in a non-biased way, especially is sub-studies have moved in directions of their own. It has value up until the final publication is sent out, and perhaps even after that. A group research strategy can function as a collaborative narrative of the twists and turns of the project. Group projects that last over extended periods of time can lose track of the rationale behind important decisions and expected outcomes. Revisiting a strategy after a projects nears the final stages can offer a lot in retrospect about how decisions were made and how those decisions played out in the end.