Monthly Archive for September, 2009

TwEVS – Presentation (using twitter for electronic voting)

Yesterday I presented TwEVS to the e-Learning Alliance FE/HE SIG held at University of St. Andrews. My presentation (including audio) is below:

The day included presentations on remote teaching using video conferencing, electronic voting systems and an introduction to twitter, so finishing on TwEVS seemed to round the day off nicely.

When I get a chance I would like to post some reflections on the other presentations …

What I've starred this week: September 29, 2009

Here's some posts which have caught my attention this week:

Automatically generated from my Google Reader Shared Items.

What I've starred this week: September 22, 2009

Here's some posts which have caught my attention this week:

Automatically generated from my Google Reader Shared Items.

ALT-C 2009 II: Audio and screen visual feedback to support student learning (and research methodologies)



Originally uploaded by Dan's Photos

For the next post in my ALT-C series I’m going to highlight a session I didn’t actually attend but immediately regretted when comments started filtering in on twitter.

The session was based around the paper by Rodway-Dyer, Dunne and Newcombe from University of Exeter which summaries a study of audio and visual feedback used in two 1st year undergraduate classes. Click here for the paper and abstract.

Comments I picked up on this paper via twitter appeared to show audio feedback was not well received. Issues highlighted were:

  • the finding that “76% of students wanted face-to-face from a tutor in addition to other forms of feedback” [@adamread, @JackieCarter]
  • students found that receiving negative audio comments was harder than when written [@adamread, @ali818, @narcomarco]. Although this is still open to debate as @gillysalmon said that “duckling project at Leicester has found human voice easier to give negative feedback by audio than text”

Obviously there are issues with making assumptions based on a few 140 character tweets and it should be noted that the authors conclude that “overall, it seems that "there is considerable potential in using audio and screen visual feedback to support learning”, although students did express concerns in a number of areas.

Having had a chance to digest the paper the question I’m left with is how much of the negative experiences were a result of the wider assessment design rather than the use of audio feedback in itself. For example, reading the focus group discussions for audio feedback in geography I noted that:

  • students were not notified that they would be receiving audio feedback;
  • that despite the tutors best attempts students hadn’t engaged with assessment criteria; and
  • that this was the first essay students submitted at university level and they were unclear of the expected standards.

Similar issues to these were addressed in the Re-Engineering Assessment Practices (REAP) project, which produced an evolving set of assessment principles. Principles which could be successfully applied to the geography example might be:

Help clarify what good performance is – this could be achieved in a number of ways including creating an opportunity for the tutor to discuss criteria with students, or perhaps providing a exemplar of previous submissions with associated audio feedback.

Providing opportunities to act on feedback – as this was the students first submission providing feedback on a draft version of their essay not only allows students to act on feedback (it’s not surprising when students ignore feedback if they have no opportunity to use it).

Facilitates self-assessment and reflection - One of the redesigns piloted during REAP was the Foundation Pharmacy class, in which students submitted a draft using a pro-forma similar to that used by tutors to grade their final submission. Students were required to reflect on distinct sections of their essay, which again also allowed them to engage with the assessment criteria.

Encourage positive motivational beliefs – using the staged feedback described above would perhaps also address the issue of students becoming disillusioned.

Talking to a friend during the lunch break the research methodology used by the authors was also mentioned, in particular the use of ‘stimulated recall’. For this the authors played back examples of audio feedback to the tutor asking him to explain his thought processes and reflect on how his students would have responded to his comments. This methodology seems particularly appropriate to evaluate the use of audio feedback, and is something I want to take a closer look at.

A moment of serendipity

Whilst searching the twitter feed for comments on the session I noticed a tweet by @newmediac which was promoting a free webinar in which  “Phil Ice shares research on benefits of audio feedback” (here’s the full tweet). The session has already passed  but the recording for this event is here.

Tweets - Moment of serendipity
Moment of serendipity

The presenter, Phil Ice, has been working on audio feedback in the US for a number of years and has a number of interesting findings (and research methodologies) I haven’t seen in the UK.

For example, Ice and his team report:

students used content for which audio feedback was received approximately 3 times more often than content for which text-based feedback [was] received”

and that

students were 5 to 6 times more likely to apply content for which audio feedback was received at the higher levels of Bloom’s Taxonomy then content for which text-based feedback was received”.

These results were from a small scale study of approximately 30 students so aren’t conclusive. Ice has also conducted a larger studies with over 2,000 students which used the Community of Inquiry Framework Survey. Positive differences were found across a number of indicators including excessive use of audio to address feedback at lower levels is perceived as a barrier by students.

Ice has also conducted studies which breaks audio feedback into four types: global – overall quality; mid level – clarity of thought/argument; micro – word choice/grammar/punctuation; and other – scholarly advice. The study indicates that students prefer a combination of audio and text for global and mid-level comments.

Findings from Ice have been submitted for publication in the Journal of Educational Computing Research (which will soon feature a special issue on ‘Technology-Mediated Feedback for Teaching and Learning’).

Screenshot showing inline audio comments
Screenshot showing inline audio comments

Finally, I would like to mention the method Ice uses for audio feedback. He uses the audio comment tool within Acrobat Pro 8 to record comments ‘inline’. This appears to be particularly useful for students to relate comments to particular sections of their submitted work. Click here for a sample PDF document with audio feedback (this isn’t compatible with all PDF readers - I’ve tested on Acrobat Reader and Foxit Reader).

Hopefully this post has not only stimulated some ideas in the use of audio feedback, but also highlight a range of methodologies to effectively evaluate it.

What I've starred this week: September 15, 2009

Here's some posts which have caught my attention this week:

Automatically generated from my Google Reader Shared Items.

ALT-C 2009 I: Mobile technology - proximity push and voting/polling on Android

Abstract-Waves Blue
Abstract-Waves Blue
Originally uploaded by fabioperez

Just back from ALT-C 2009 having been asked to present a session with colleagues on EduApps (this resulted from JISC RSC UK’s donation of an EduApps stick to all conference delegates and ALT members). Over the next couple of days I’ll be making a series of posts to highlight some of the best bits.

For my first post in this series I’m going to highlight some of the ideas presented by my colleague Adam Blackwood at RSC South East. Adam, amongst other things, is a mobile guru and in his session he highlighted some interesting tools [Click here for a copy of Adam’s slides and his Mobile Technology Summary Sheet].

Proximity push using TextBlue

First there is TextBlue.This company specialises in ‘proximity marketing’, using Bluetooth to push information primarily to mobile devices. This company has a range of products from plugin dongles for your laptop to ‘broadcasters’ which can push content out for up to 1000 meters.

Adam demonstrated how this technology could be used to push learning content to student owned phones (or any Bluetooth enabled device). The only restriction you have on the file types you can use is what is viewable on the student’s device. You probably also want to keep file sizes down because of the transfer time so the 30 minute podcast might be out of the question, but this technology could be ideal for distributing quizzes etc (something you could easily create with Mobile Study, which is free).

There is nothing stopping you transfer files via bluetooth without TextBlue. Doing it this way is very cumbersome and the TextBlue software turns it into a one click solution. A demo version of TextBlue software is available on request – Contact TextBlue

SMS polling/voting

I’ve been aware of SMS polling/voting services for sometime. All the examples I’ve previously looked at use the model where the hosting/collation of votes has been handled by a 3rd party site. Adam highlighted a new model which puts the editing/collation software on your own phone, students responding to your mobile number, not one provided by a 3rd party.

The software to do this currently only seems to be available for Android mobile devices. There are a couple of software applications that can do this but Adam was highlighting ‘Polls’ by Pollimath:

The concept is simple; draft the opinion poll on your phone, add your voters and open your poll. Your list of voters would receive an SMS and/or E-Mail notification. They vote via the Web or SMS Reply as per the options selected by the pollster. The pollster can see the poll statistics and the voting details (who voted for what choice).  

Polling Concept
Pollimath Concept Diagram

There is a free ‘Lite’ version of Pollimath which is limited to 10 voters per poll, but at $3.95 the full version is very reasonably priced. Pollimath has some nice features like being able to send vote invitations via email as well as SMS, allowing you to use multiple input methods, and being able to view the results online. This is a relatively new application and some more work needs to be done to graphically represent poll results as well as an easier way to distribute polls links but so far it looks very promising.

An alternative to Pollimath is ‘Handy Poll’s’ by Marc Tan. This has a better graphical results view, but doesn’t have as many of the features of Pollimath.

Augmented reality

The final thing Adam showed us was some ‘augmented reality’. With this the camera on your phone is combined with your location and direction information so that additional information can be overlaid. One of the most popular working examples is Layar for Android, but the video below shows where the next generation of augmented reality is going:

What I've starred this week: September 8, 2009

Here’s some posts which have caught my attention this week:

Automatically generated from my Google Reader Shared Items.

Electronic voting and interactive lectures using twitter (TwEVS)

Flickr Tag Error: Call to display photo '3859791831' failed.

Error state follows:

  • stat: fail
  • code: 1
  • message: Photo "3859791831" not found (invalid ID)

Last week I posted a method for combining twitter and Yahoo Pipes to allow electronic voting (EVS) style interaction within lectures, TwEVS [see Twitter + voting/polling + Yahoo Pipes = TwEVS (The Making Of)]. At the time I was only interested in documenting the development of this ‘mashup’ but felt a follow up post would be useful to highlight: how to use TwEVS; advantages of using twitter for student response; and areas for future development / discussion.

How to use TwEVS

Before using TwEVS there is some preparatory work in terms of getting students to register an account with twitter and possibly establishing some house rules (usage policy, appropriate use). You should also have an idea of the questions you would like to ask, which may sound simple but to fully engage and enhance learning isn’t (the most common mistake I see is setting trivial questions, which are suitable while you find your feet, but if continued will the students cue to disengage).

Another thing to consider is the format of the hashtags you want to use. Hashtags are a simple way to add metadata to a variety of information making it easier to search and filter. TwEVS requires a unique hashtag for each question you ask so if you are planning to uses this over a semester your might use a combination of an abbreviated course code and date (e.g. #code-year-weekNumber = #CS101-09-wk1)

Pedagogically and technically there are a number of ways you can integrate TwEVS. For more on the pedagogy visit Steve Draper’s ILIG site.  My suggestion for technical integration would be to have a slide with the question/options and instructions on how to respond e.g. tweet ‘#CS101-09-wk1 A’ etc. After students had time to respond you could then either open the TwEVS Pipe in a browser, enter the hashtag where prompted then click ‘Run Pipe’ (you can also limit the number of response options, which might help filter out malicious tweets or mistypes). Clicking the ‘TwEVS Result for …’ link opens the graph. If you wanted to streamline this a little you can use the free LiveWeb PowerPoint plugin which allows you to embed live webpages.

Alternativly you could prepare a custom link for each question  within your PowerPoint (like this example). As Yahoo Pipes uses information in the url these could be created beforehand.

Below is an example url for the poll #twevspoll limited to 2 responses:

http://pipes.yahoo.com/pipes/pipe.run?_id=40e4326b88a69c2d6287ae124314fd7c&_render=rss&limit=2&q=%23twevspoll&vm=r

[Edit: Tony Hirst has pointed out that this url can also be written as http://pipes.yahoo.com/mashe/twevs?&_render=rss&limit=2&q=%23twevspoll&vm=r]

[Another Edit: if yo are wondering why the links above don't return graphs it is because the twitter search by default only pulls tweets from the last 10 days]

The first part of this url will always remain the same allowing you to change the range of response by adjusting the limit (e.g. to limit to 5 would be limit=5) and the hashtag for each question, by changing the text after ‘q=’ (the ‘#’ is replaced by the more url friendly ‘%23’).

Example chart produced by TwEVSExample chart produced by TwEVS

Its worth noting that Yahoo stores a copy of each pipe run so when a pipe is run again before an allotted amount of time has passed it just pulls results from memory and doesn’t necessarily check twitter for the latest tweets. So if you are creating urls to use in class I would advise not accessing them until you need them. Alternatively you can also trick Yahoo into getting the latest data by modifying the url slightly, the easiest way is changing the limit number.

Advantages of using twitter (or other status update sites) for EVS style interaction

Zero cost - the biggest advantage perhaps for the majority of people is cost. There are no handsets or specialist software to buy. You don’t need to worry about replacement batteries. You don’t need to worry about lost of stolen handsets.

Multi-device – you can update your twitter status from a wide range of devices phones to laptops and everything in between (and using SMS updates means even the most basic phone could be used). Its also very apparent that manufactures are currently falling over themselves to get twitter (and other social networking software) built into their devices (my TV even has a twitter client).

EVSPLUS – using twitter as a EVS allows a natural extension to existing pedagogies. For example, the TwEVS mashup is programmed only to aggregate responses after the hashtag. This means that as well as asking students to indicate a response (A, B, C, D etc.), the tutor could ask students to prefix their response with why they believe the answer is correct. Using twitter to collect responses also opens up a huge degree of flexibility in terms of asking questions on-the-fly, removing some of the restrictions imposed by bespoke EVS software (and you are obviously not limited to A, B, C results).

Example of TwEVS responseExample of an individual TwEVS tweet

Future development/discussion

So far I’ve painted a rosy picture of twitter/EVS integration but there are some obvious issues. One of the biggest is there aren’t that many twitter users and even less under the age of 24. So to use this model would require proactive encouragement from tutors for students to create accounts. There is also issue around the personal/work divide. Will students be happy to include responses in their public timeline?

Another drawback is voting isn’t entirely anonymous and students would even have the ability to check other student responses before replying (which is event easier if students follow their friends). The proposed system is also open to malicious attack. As everything after the hashtag is collected as a response students could get up to all sorts of hijinx to ruin your lovely chart.

Finally something not to be overlooked is the possible distraction element of actively opening back channel communication, although I’m sure there will be situations where this could enhance learning, and giving students an excuse to get lost in their mobile phone.

Putting all these issues aside for one moment, the model of using twitter as a EVS offers a lot of flexibility. As twitter’s search results can be pulled as RSS XML their is endless scope to harvest results and reuse them in a number of ways using either via in-house systems or existing web services. For example it would be very easy to develop a system which removed the dependency on Yahoo Pipes altogether, storing results in a separate local database which could be linked to a student management system or even a custom portal which allowed the continuation of discussion and learning outside the classroom. You also don’t need to only support just one platform, combining results from various status update sites like FriendFeed would be very easy to integrate.

In summary, I hope I’ve stimulated the grey cells and demonstrated one way in which twitter could be used to enhance teaching and learning as a EVS alternative. TwEVS should should be seen as a working prototype and there is no doubt a lot more research to be done in this area. I’m sure with the hype associated with twitter it would be relatively easy to get some project funding to develop some of the ideas outlined above further.

What I've starred this week: September 1, 2009

Here's some posts which have caught my attention this week:

Automatically generated from my Google Reader Shared Items.

About

This blog is authored by Martin Hawksey e-Learning Advisor (Higher Education) at the JISC RSC Scotland N&E.

JISC RSC Scotland North & East logo

If you would like to subscribe to my monthly digest please enter your email address in the box below (other ways are available to subscribe from the button below):

Subscribe to MASHe to monthly email updates

Loading...Loading...


The MASHezine (tabloid)

It's back! A tabloid edition of the latest posts in PDF format (complete with QR Codes). Click here to view the MASHezine

Preview powered by:
Bluga.net Webthumb

The MASHezine (eBook)

MASHe is also available in ebook and can be downloaded in the following formats:

Visit feedbooks.com to manage your subscription

Archives

Opinions expressed in this blog are not necessarily those of the JISC RSC Scotland North & East.

JISC Advance Logo

JISC Advance is a new organisation that brings together the collective expertise of established JISC services:

For further information visit www.jiscadvance.ac.uk

Creative Commons Licence
Unless otherwise stated this work is licensed under a Creative Commons Attribution-ShareAlike 2.5 UK: Scotland License