Student Survey Heroes … and Heroines

ISSE 2018 stash
ISSE 2018 stash … and ISSE 2017 heroes

It was, we felt, only a matter of time before the recognition, the plaudits, the trophies arrived … thank you, thank you, thank you!!! Our humility knows few bounds; indeed, we’d even written a 16 page acceptance speech … and then we read the citation from Seán O’Reilly (Project Manager, Irish Survey of Student Engagement):

As we prepare to launch the Irish Survey of Student Engagement 2018, we want to acknowledge those teams within your institution who helped boost participation in last year’s survey. Because of their commitment, we have increased our national response rate year-on-year and your students are having an even bigger say.

It transpires that the chocolates are not for us! 😦

So, who are they for?

It turns out that the ‘Thank You’ cards, travel mugs, even the chocolates, are meant for staff working in various areas of the College, specifically the “three areas that increased their participation rates the most in NCI in 2017” … and we in QA@NCI are being tasked with distributing them! Aaarrrggghhh!!!

We’re thinking of putting in an appeal.

Advertisements

ISSE 2018 – the countdown to Monday, February 5th, has begun

The Irish Survey of Student Engagement (ISSE) 2018 will launch here at NCI on Monday, February 5th, and it will remain open for three weeks through to Sunday, February 25th. As with previous iterations of this national survey, ISSE 2018 will offer first and final year undergraduate, as well as postgraduate taught, students an ideal opportunity to provide feedback on their experiences here at NCI, as well as the chance to reflect upon their own engagement with their studies.

Over the past number of years since its national rollout in 2014, NCI has been able to listen to, and learn from, its students through this survey, to add the valuable data garnered by this process to other sources of information and guidance, as well as being able to gauge how we are doing when compared to students studying at other Irish HEIs.

Effective practice suggests that institutions should analyse and act on the quantitative and qualitative data they receive in an open and transparent manner, as reflected in the ISSE 2016 – NCI report and the ISSE 2017 – NCI report published over the past two years; indeed, the findings outlined in institutional analyses such as these can also impact on national, as well as local, thinking, policy and practice.

The company running this national survey – i-graduate – advocates that we share the ideas, practices and resources advocated in guidance such as the Irish Survey of Student Engagement (ISSE) 2018: The Handbook as we prepare for its launch here at NCI. For instance, in terms of increasing responses, they suggest that our preparatory work might include:

Preparation

    • Pre-wave alerts on the intranet i.e. “Look out for….”
    • Notify the academic and support staff about the survey and its benefits and ask them to encourage students to participate
    • Notify the student clubs and societies to encourage their members to take part

This is followed by useful advice on the survey launch, as well as guidance on best practice to be followed during the fieldwork phase. Interestingly, they are a little less directive when it comes to incentivisation:

Incentives

You may wish to offer specific incentives to encourage your students to participate. The option to include incentives is an institutional decision.

Guidance has been provided on the general use of incentives, based on the number of anonymous responses. In addition to this, you may wish to offer prizes or incentives targeted at students who have definitely accessed the survey. To administer this type of incentive, i-graduate uses an ‘optin’ process. We will include an additional page at the end of your survey asking students to re-enter their email address if they wish to be entered into the prize draw/receive an incentive.

Students will be informed that, if they win a prize, their email address will be passed to their institution in order for the prize to be administered. They will also be informed that the institution will not be able to identify them from their responses.

As NCI prepares for ISSE 2018 in conjunction with our colleagues in NCISU, we are currently determining what levels of incentivisation or otherwise may be appropriate. Having used limited incentives last year, we are not minded to depart from established sectoral norms as it could be argued that to do so might colour the quality and nature of the feedback we receive. Yet, we are also conscious that we are asking our students to take the time to provide us with anonymised feedback that is more likely to impact upon the future learner experience rather than their own.

It is important for the students taking part, as well as those staff and student representatives advocating their participation, are offered insight into the wider context in which ISSE 2018 fits, as well as a role in the decision-making process. Appropriate levels of incentivisation in support of increased responses and more representative feedback is just one of the many considerations facing us regarding how we support ISSE 2018.

ISSE 2018 - The Handbook
ISSE 2018 – The Handbook

Module Evaluations – a snapshot

It is current NCI policy, as outlined in the Quality Assurance Handbook, to evaluate every module every time that it runs, for this feedback to be given anonymously by students, and for the quantitative and qualitative data to be shared with the module convenor and their Dean of School in the form of aggregated reports by module iteration.

In terms of learner feedback processes, which also include class representative meetings and service evaluations, not to mention the Irish Survey of Student Engagement (ISSE), it is worth reproducing the existing College guidance in full regarding module evaluations:

Module Evaluation

This is carried out in Week 8 of the Semester by the School. The survey is anonymous. The primary objective of this survey is to obtain the views of Learners on their experience in the module delivered. The learner is invited to assign a rating to a range of issues relating to the presentation of a module or module component as he/she experienced it. The completed questionnaires and analysis are returned to the Dean of School and to the individual lecturer. The Dean of School reviews the results of the survey with the individual lecturer.

Results of the survey are communicated to Associate Faculty by post.

Over the past couple of years, this statement regarding the processes involved in gathering and responding to learner feedback has been interpreted in ways which stretch its meaning somewhat so that it has practical application; for example, the results – i.e. the individual module evaluation reports – are disseminated by electronic mail rather than by post.

As part of the Quality Assurance Review which has been taking place this past year, it is expected that these statements regarding learner feedback will be updated so that they reflect best practice (e.g. by formally incorporating ISSE into the guidance), as well as what is happening in reality (incl. advice to the students to provide constructive feedback, suggestions to lecturing staff regarding how their feedback might be employed, etc.), and overarching efforts to improve the processes involved (i.e. by updating the questionnaire being used).

In essence this is not just an attempt to build on the Feedback from students project work, it also seeks to extend its findings and recommendations so that the data and analysis produced by surveys have real value and support effective responses to what we are hearing and learning.

Over the past 2½ years under consideration here, engagement in support of learner feedback and the feedback loop has steadily grown; this is encapsulated in increasing acceptance of the principles underpinning this whole process, such as those outlined in Embedding the Principles of Student Engagement:

Feedback and feedback loop

Institutions will welcome and encourage open and prompt feedback from students. Suitable measures will be put in place across the institution to ensure that students are facilitated in providing feedback in a safe and valued manner. Feedback practices will be transparent and the feedback loop will be closed in a timely fashion.

In truth, the willingness of students and staff to take part in this process, as revealed in statistics such as ever more representative feedback (see table 1 below), is testament to the trust that is being built on existing firm foundations.

table 1 – first semester response rates across three academic years
average response rates & total numbers of responses Semester 1, 2015-16 Semester 1, 2016-17 Semester 1, 2017-18*
School of Business 10%

772 responses

17%

1,255 responses

22%

1,675 responses

School of Computing 12%

696 responses

25%

1,251 responses

24%

835 responses

Learning & Teaching 33%

84 responses

38%

138 responses

27%

100 responses

* this particular round of surveys has not yet finished

We look forward to the quantitative and qualitative data that is being offered by our learners being used ever more effectively and transparently as part of the quality assurance process by individual members of academic staff, the teams to which they belong, and by those analysing and considering the data.

ISSE 2017 … the national results!

ISSE 2017

Earlier this week, the publication of The Irish Survey of Student Engagement (ISSE): Results from 2017 saw the latest set of national data and analysis revealed, thus building on our own local efforts to analyse, report and act. One of the major innovations in this year’s national report is the explicit use of learner qualitative feedback to help illustrate what lies behind the quantitative data; this year, our own Irish Student of Student Engagement (ISSE) 2017 – report seeks to do exactly the same thing.

This blog was only launched at the start of this current calendar year, yet one of its main priorities has been to enhance the learner feedback process in terms of promotion, completion, evaluation and action, thereby closing the feedback loop by use of this ‘virtuous circle’. The sharing and utilisation of the national report is an essential part of that process.

ISSE utilises resources such as those on YouTube and at Student Survey.ie to help promote the national survey, but ultimately it boils down to students having their say and being partners in what happens next as a result of what they have fed back.

Next spring will again see NCI and NCISU jointly promoting this annual data gathering exercise, with the dates for the survey itself provisionally pencilled in as far as we are concerned from Monday, 5th, to Sunday, 25th February 2018. This three week window will allow us to build on last year’s record response rate and it will seek to make the learner feedback we receive ever more representative.

As the ISSE slogans declare: ‘Shared insights, shared outcomes’; ‘Shape the future’; ‘We value your voice’; & ‘National survey, local impact’.

ISSE 2017
The Irish Survey of Student Engagement (ISSE): Results from 2017

 

QA network #3: ʼTis the season?

No, we don’t necessarily mean the forthcoming holidays, even if that time of the year seems to start earlier and earlier.

Instead, we’d like to talk about learner feedback, not just because it went to the heart of the most recent QA network meeting hosted by Hibernia College. The national conversation will turn to this subject in the coming weeks with the impending publication of the ISSE 2017 report, but we also had the opportunity to compare and contrast aspirations, ideas, and practices at this our third meeting with a particular focus on module (and programme) evaluations supplied locally by our students.

The fact is that the QA network allows us to share our experiences and to learn from one another, but where it is arguably most valuable is that it also consistently reveals that, while we may be encountering curiously similar issues, we are also getting real insights from each other regarding potential next steps. Ah, the value of networking!

Our institutions have quite a degree of flexibility when it comes to learner feedback, which normally comes to us via both formal and informal means. In terms of formal mechanisms, local practices and experiences are not necessarily all that different, even if QA expends a lot of its efforts gathering data (usually paper and/or online by module), while also advocating that it is utilised appropriately, as well as being seen to be used.

If we could all get what we want this Christmas, it would doubtlessly be that the feedback loop is closed ever more effectively. We may well need to revisit this whole issue of learner feedback again soon, it is a perennial and not just seasonal.

Neapolitan coffee
Il Caffé di Napoli

 

 

NCI’s “Irish Survey of Student Engagement (ISSE) 2017 – report”

ISSE

Earlier this calendar year, first and final year undergraduate, as well as postgraduate taught, students were asked to provide feedback as part of the Irish Survey of Student Engagement (ISSE). Over the last number of weeks, we have been examining our own institution’s data and reporting upon it internally, but the time has now come to start making this analysis more widely available.

Our Irish Survey of Student Engagement (ISSE) 2017 – report is based on the qualitative and quantitative feedback received from 663 NCI students last spring. An overarching national report drawing on the data received from students attending Higher Education Institutions from all across the country is expected to be published later this month or early next.

NCI will of course be looking to capture learner feedback through the next iteration of this survey in spring 2018 but, in the meantime, we are working with staff and students in interpreting what our learners are telling us, taking it on board, and responding to it constructively.

This is all part of the ‘virtuous circle’ regarding learner feedback (i.e. promotion, completion, evaluation and action supported by ongoing communication) identified in the Feedback from students project work undertaken at the start of academic year 2015-16. It is also a fundamental part of the ‘feedback and feedback loop’ identified by the Working Group on Student Engagement in Irish Higher Education.

As the NCI’s ISSE 2017 data analysed PPT presentation given late last month to a group of colleagues illustrates, this work continues in earnest!

“Architects of their experience”

TSEP publication

The Student Engagement Partnership (TSEP) works in conjunction with the National Union of Students (NUS), as well as other stakeholders involved in Higher Education in England, to champion and develop student engagement knowledge and practice. Through its work, TSEP seeks “to equip student engagement professionals, practitioners and decision-makers across the sector with the knowledge and skills they need to make a success of student engagement in their context” (see http://tsep.org.uk/ for more details).

With this purpose it mind, it should be noted that the recent TSEP publication on student academic representation – see Abbi Flint, Hannah Goddard & Ellie Russell, “Architects of their experience: the role, value and impact of student academic representation systems in Higher Education in England” (London: TSEP, 2017), for the main report – offers up an engaging analysis, as well as a useful set of resources including a benchmarking tool, a literature review, and a summary poster.

The focus of these materials is student representation, in particular the student voice within the context of student academic engagement (i.e. engagement in and with learning). It is not necessarily an easy task to provide a summary regarding the rich resources on offer here; frankly, these documents deserve exploration on their own merits as they will have different meanings for Higher Education institutions depending on how and where they operate. This being said, even the most cursory of readings reveals that they could readily be employed to support thinking, review and action in a range of scenarios and contexts.

Of the report’s ten recommendations, the one that leaps out reads as follows:

“Reflect on and consider how to address power differentials which may inhibit effective representation. In some cases, this may include visible signs of where power lies, such as: who chairs meetings; who acts on issues brought through representation; who sets the agenda for meetings; and, when and how different parties are enabled to speak.”

Without wishing to do a disservice to the breadth or depth of a report that examines a whole host of student academic representation matters, any attempt to consider or review how effectively class representative meetings are operating could do worse than to start with such basic considerations as group composition, dynamics and purpose.

In turn, while many participants in these meetings may perceive them as being “focused mainly on feeding back on current experiences”, often with an unfortunate focus on the negative rather than also accentuating the positive, the more ambitious hold “aspirations to have a more proactive, feeding forward role, particularly in curriculum and course development: to enable students to be ‘architects of their experience'”.

Indeed, when done well, the value of effective student academic representation to the wide range of institutional stakeholders are, according to this TSEP research, obvious:

TO PROVIDERS

  • Alternative perspectives
  • Current and relevant information to inform improvement
  • Culture and community
  • Reputation

TO INDIVIDUAL STUDENT REPRESENTATIVES

  • Emotional
  • Personal development
  • Professional development

TO THE STUDENTS’ UNION

  • Reputation and credibility
  • Political
  • Culture and community
  • Currency and relevance
  • Informing improvement

TO THE WIDER STUDENT BODY

  • Being listened to
  • Valuing their voice
  • Changes secured on their behalf

And, once the easier to reach learner groups are on board – typically undergraduate, full time, near campus, etc. – every effort must be made to involve other, often less represented, groups be they students who are carers, international, mature, off-campus, part-time, postgraduate, etc.

At the same time, staff themselves, particularly but not necessarily only academic staff, must be at the forefront of effectively functioning student academic representation (SAR). The fact is that “staff engagement is important in supporting the operation of SAR at the local level, and how staff perceive, and are open to listening to, student contributions” will help to determine their utility and impact.

Ultimately, the impact of student academic representation can be profound, especially when effective practices are employed, such as appreciating how and when to communicate back to students, thereby closing the feedback loop. The fact is that, for everyone who is a member of a self-respecting learning community, most – if not all – of the points noted above should apply when it comes to the role, value and impact of effective representation in particular and the place of learner feedback more generally. These TSEP resources, which clearly resonate with NUS campaigning on this issue of student academic representation, deserve wide dissemination.