| ||||||||||||||||||||||||||||||
User survey at Waterford Institute of Technology Libraries | ||||||||||||||||||||||||||||||
How a traditional approach to surveys can inform library service delivery | ||||||||||||||||||||||||||||||
The Authors | ||||||||||||||||||||||||||||||
Helen Hayden, Waterford Institute of
Technology Library Services, Waterford, Republic of Ireland Terry O'Brien, Waterford Institute of Technology Library Services, Waterford, Republic of Ireland Maoilíosa Ó Rathaille, Department of Physical & Quantitative Sciences, School of Science, Waterford Institute of Technology, Waterford, Republic of Ireland | ||||||||||||||||||||||||||||||
Abstract | ||||||||||||||||||||||||||||||
Purpose – This paper describes a comprehensive
user survey carried out in an Irish higher education academic library
(Waterford Institute of Technology library service) as part of a strategic
initiative to engage with library users.
Design/methodology/approach – Two major user groups – full-time undergraduate students and academic staff – were surveyed. The survey instrument was a hardcopy questionnaire. The paper outlines the relative merits of different survey types and the reasoning behind the final choice of survey instrument. Findings – The main results of the survey are synopsised. The paper also includes selected free-text comments made by survey respondents and the library's response to the results in terms of how issues highlighted will be addressed. Research limitations/implications – The survey focused on two specific user groups, which leaves scope for future research by this institution into how the library is used by other types of users, such as distance learners. Practical implications – The survey was a good way of communicating with users. This was the first time such a project has been undertaken at WIT and WIT library service will continue to survey users on a regular basis and to use the results to inform service delivery. Originality/value – As the first comprehensive survey to be undertaken in the Institute of Technology sector in Ireland, the paper is of potential value to library practitioners in this, or indeed in other sectors, who are considering doing a survey of their users. | ||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||
Article Type: Research paper | ||||||||||||||||||||||||||||||
Keyword(s): Surveys; User studies; Strategic planning; Academic libraries; Ireland; Customer service management. | ||||||||||||||||||||||||||||||
New Library World | ||||||||||||||||||||||||||||||
Volume 106 Number 1/2 2005 pp. 43-57 | ||||||||||||||||||||||||||||||
Copyright © Emerald Group Publishing Limited ISSN 0307-4803 | ||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||
Background Waterford Institute of Technology (WIT) is located in Waterford City in the south-east of Ireland. WIT was established by statute as Waterford Regional Technical College in 1970. It is one of the largest of a network of 13 similar institutes in the country. The Institute of Technology Sector comprises 13 Institutes, designated under Section 3 of the 1992-1999 RTC (Regional Technical Colleges) Acts, also designated under section 24 of the Qualifications (Education & Training) Act, 1999. The college commenced business with an intake of just 200 students as part of a general programme of development of the higher education system in Ireland. Prior to 1970, the only further education options available to those located outside of Dublin City were either university or trade apprenticeships. These choices were both limited and polarised and the development of Institutes of Technology (formerly known as Regional Technical Colleges) extended access to higher education for a broader spectrum of the population, academically, professionally, socially and geographically. Institutes of Technology provide courses ranging from certificate, diploma and degree as well as postgraduate programmes at diploma, masters and doctoral levels. The remit of the Institutes extends further in terms of life-long learning opportunities, part-time and evening courses in addition to professional training, research & development, partnership with industry and regional development. Broadly akin to the former British Polytechnic and similar to Institutes of Technology in New Zealand, India and Australia, the Irish Institute sector has developed strongly in the past ten years and now accounts for about half of all students pursuing third level qualifications in the Republic of Ireland. The Institutes have also expanded their curricula from technological, science and engineering to include healthcare, education, business and humanities. Waterford Institute of Technology Library service (www.wit.ie/library) has undergone huge change in the last ten years. The service currently operates across two campuses, with a new building at the main campus and a branch library at the college's smaller campus. In 2000, the library service at the main campus was moved from a small under-resourced facility of about 35,000 volumes and 120 reader spaces to a new state-of-the-art building over ten times the size of its predecessor. The Library service now holds over 180,000 volumes, has room for over 1,200 reader spaces and caters for the information needs of 6,500 full-time, approximately 5,000 distance and part-time tertiary students and over 300 academic staff, who are involved in learning, teaching and research at the college. Although two-thirds of the students come from the south-east region there is an increasingly national and particularly international dimension to the Institute in recent years. Just ten years ago WIT had fulltime student enrolments in the region of 3,500. This growth has had major implications for the scope of library collections, library staff and the delivery of services. In addition, over 50 per cent of WIT students currently study at degree or postgraduate level. In 2003 WIT obtained delegated powers to award honours primary degrees and masters degrees. However, the ratios of library staff to students at most Institutes of Technology are significantly short of those in many Irish universities, as highlighted by the random selection below in Table I. In addition, the well-acknowledged increasing expectations and demands of library users across the entire library sector – “contest between diminishing resources and a 24/7 world” (Curry, 2003) has made it very much a time of change and flux for the library service. In this context of transition, it was decided that it was an appropriate time to reflect and consider the role of the library service. Further to this, one of the stated objectives of the WIT Libraries Strategic Plan 2002-2006 (WIT Libraries, 2000) is to carry out a user survey. WIT Library service is committed to continuous improvement and to providing quality services for all its users. The ultimate aim of the library service is that the library goals, objectives and services are in line with the academic, personal and career goals of our users. By examining and analysing the perceptions and requirements of library users, we can ensure that library strategies are responsive to their needs. Methodology The survey was conducted during 2003. A working group was established, consisting of two senior library staff members and a statistician from the Department of Physical and Quantitative Sciences. Given the high numbers of part-time and distance learners, for the purposes of this particular survey, it was decided to focus on two specific control groups (user groups): full-time academic staff and full-time undergraduate students. During the initial briefing period, a conscious decision was made to carry out the survey using the traditional paper-based approach. Although the trend in recent years has been for e-mail, web and online surveys (Irish examples include Dublin City University Library, 2000; National University of Ireland Maynooth Library, 2003; University College Cork Library, 2003), it was considered more appropriate and more inclusive in this instance, to carry out the survey in a physical paper format using mail response mode. The team regarded undergraduates as being more amenable to and comfortable with the survey in hardcopy format. This not only allowed for a wider range of questions and a broader analysis but assumed nothing about users' habits in the way that a web-based survey by virtue of its format, would. Additionally, the team considered that a web or e-mail survey directed at undergraduates would not reach the necessary population coverage as we did not have access to student e-mail accounts and anecdotally we were aware that few students used their college e-mail accounts. It was also felt unlikely that representative proportions of students would take the time to visit the library web site and complete an online survey. Despite the increasing frequency of web based and e-mail surveys, the
comparative benefits over postal surveys would seem questionable. Much of
the literature points to the fact that in many cases, postal surveys
actually have a higher response rate (Quigley
et al., 2000), have equal or similar cost benefit (although
this can depend on the numbers of responses) (Couper
et al., 1999), require little technical expertise and
importantly, do not necessarily save time: … most studies have concluded, often with little or no empirical
evidence to back up the conclusion, that Internet-based surveys are
conducted more quickly than surveys sent by postal mail … a blanket
conclusion … naively ignores the reality that the total amount of time for
survey fielding includes more than just the survey response time (Schonlau
et al., 2002). On the other hand, both Tse
(1998) and Schaefer
and Dillman (1998) have found significantly faster initial response
times with e-mail surveys. From WIT Libraries perspective this was not a
strong enough reason to proceed with a web or e-mail survey. There was a
predetermined response time of about four weeks in total (including
follow-up and non-response notifications) which in effect meant that
although initial and fast responses were welcome, there was a further and
necessary timeframe for responses to come back in. Sampling errors are
also as likely with web-based as with postal surveys. In terms of data
quality SCONUL (Society of College, National & University Libraries (www.sconul.ac.uk) has carried out significant research on user surveys in academic libraries in the UK and Ireland, most recently in December 2003. Taking base membership at 157 and the number of actual responses at 65, SCONUL had a response rate of just over 40 per cent. The vast majority of libraries that responded do carry out user surveys either as pure library surveys; as specialist or targeted surveys; or increasingly as part of wider institutional surveys, e.g. general student satisfaction surveys. Many libraries carry surveys out annually (over 60 per cent) and others at fairly regular intervals. A variety of analysis tools are used, with Libra and LibQUAL + the most popular, although SPSS remains quite prevalent. The format or medium use for surveys varies between web, mixed and some paper, but, according to SCONUL, “there is a noticeable trend towards web-only or web and paper combined surveys” (West, 2004). Although the WIT Libraries survey was completed before the publication of this report, there are a number of interesting points to note. Firstly and most importantly, the Institutes of Technology Libraries are not members of SCONUL and therefore the recent survey can really only be looked at from the outside in. This was one of the main reasons to carry out a survey of this nature. No survey of such a large target population had ever been carried out within the Institute of Technology library sector. WIT Libraries survey was an attempt to address this and fill the research gap in our specific area. Despite SCONUL's findings, the conscious decision to carry out a paper-based survey (albeit with web alerts, e-mail follow-up, etc.) proved fruitful in our case: a response rate of 65 per cent from undergraduates and a very representative sample from all years, courses and academic schools. There were probably three main reasons for such a high response – first, surveys were distributed in class, thus picking up large numbers; second, promotion of the survey through pre-notifications and the co-operation of academic staff; third, despite huge growth, WIT remains a tight-knit and relatively geographically concentrated campus. In this context, WIT Libraries would recommend that rather than go with what is perceived as being necessary or fashionable, to tailor your survey to your own requirements especially when surveying large target populations. This may be web, e-mail, post or a combination; it is very much dependent on your particular circumstances and requirements. The process of distribution, collection and collation, although fraught with all sorts of logistical difficulties, did nevertheless work well, thanks to the co-operation of students, pre-notification and follow-up with faculty course leaders and frequent communication via phone, e-mail and post. Sample The sample for the survey was selected as shown in Table II. In total, 110 academic staff were randomly selected based on academic department and department size. These questionnaires were distributed and returned by internal post. Out of 6,000 full-time undergraduates, 1,100 full-time undergraduates were surveyed. A stratified random sample of class groups was selected in such a way that at least one class from each year in each department was surveyed, taking care that no two years from the same subject discipline was selected. The questionnaires were distributed and collected in class, having been arranged at a pre-designated time. Survey instrument The survey instrument was a questionnaire consisting of 20 main questions based around core areas of the library service. The questions focused on such areas of the library service as:
Response rates The response rates (see Table III and Figure 1) were deemed sufficiently high and representative to proceed with analysis and further scrutiny, which was agreed would be published in report format within the WIT community and for public access via the worldwide web. Data input with the assistance of extra library staff, and analysis of the results took place during the summer of 2003. Analysis was done using the SPSS software package and MS Excel. Literature review
Surveys are carried out for a number of reasons. Glasgow Caledonian
University Library (2004),
for example, surveyed its users as part of an overall programme of
evaluation (General Satisfaction Surveys and a Seven Year Longitudinal
Analysis) the University
of Northern Rod Library (2004) as part of a comprehensive marketing
programme. There is also a trend for surveys to measure overall library
performance that often attempt to elucidate specific information on
controversial topics like opening hours “often a survey seeking customer
opinions about a library's overall performance or services will include a
question about satisfaction with opening hours” (Curry,
2003). Others are more specific in nature and attempt to evaluate and
benchmark, compare salaries or professional concerns such as the
BIALL There is no well-established tradition of research activity in the libraries and information services sector in the Republic of Ireland. In the UK university sector, survey results are often displayed along with notes of subsequent actions on the part of the library and/or its parent institution. Irish academic libraries have not been involved in survey research to the same extent, but the situation is changing. The Irish public library service has though, for many years engaged in information gathering, comparison and statistical analysis, particularly in the area of financial expenditures. However, the recently commissioned An Chomhairle Leabharlanna survey of public libraries “A public space for all” (An Chomhairle Leabharlanna, 2003) was the first real attempt at reviewing users' needs and responses on a nationwide basis. Several academic libraries have conducted user surveys in recent years, although it will probably be some time before a collaborative approach is taken to such research. As already stated, this is the first time a comprehensive library user survey has been done within the Institute of Technology sector and we intend to make it a regular way of engaging with and communicating to our users at the Waterford Institute of Technology. A fundamental concern when conducting a user survey must be the extent to which it really will be used to benefit the service. WIT Libraries strategic plan (www.wit.ie/library/plans&policies/stratplan.asp) states its commitment to a process of annual consultation with users and staff within the library. This develops concrete actions for the future that are consistent with its larger goals. Hernon (2000) stresses the importance of keeping promises to change the library service based on the findings of a survey. In other words, it is vital that libraries act on or are responsive to the needs and concerns of its users as highlighted in surveys or other information seeking instruments. The Royal Melbourne Institute of Technology in Australia (Royal Melbourne Institute of Technology University Library, 2003) has carried out extensive research of its library facilities and it outlines on its web site what its intentions are in relation to problem areas raised by survey respondents. The RMIT published its results on their web site along with their responses. This took the form of a breakdown between quantitative – responses to questions asked by the Library, and qualitative – comments from respondents. RMIT listed these in order of number of comments received and then grouping these, gave a selection of comments and the concomitant response of the Library. It is imperative that WIT Libraries survey was not undertaken for its own sake but as a genuine means of communicating with users and trying, with the resources available to us, to provide the service that they require. In this regard, we can say the survey was embarked upon for the following reasons:
Major findings Rather than present the responses in an identikit format to the
questionnaire, it was decided to write up the responses in a broad
thematic arrangement. A detailed breakdown of the survey results is
available to interested parties on request from the project
team Main findings in summary:
A brief summary of the main issues and concerns in broad thematic format, with a selection of representative comments from staff and students and an outline of the current or planned response of the library, are presented in Table IV. Library users, when possible, were notified of these actions via the library web site, increased posters and signage and the monthly library bulletin. Overall satisfaction The library service retains an important position within the institute community. Usage is very high and in terms of satisfaction: 47 per cent of student respondents rated their satisfaction level as 4 or 5 (with 5 being very satisfied and 1 being not satisfied); 40 per cent gave a rating of 3, which is a moderate level of satisfaction with the service (see Figure 2). Of academic staff respondents, 83 per cent rated their satisfaction level as 4 or 5; 12 per cent gave a rating of 3 (see Figure 3). These ratings are quite positive, but leave plenty of scope for improvement, particularly in relation to students. It seems that while usage of the library service is extremely high, many students are not gaining the maximum benefit from their usage. This survey has given us valuable information about how the service is perceived and how it is used. It has given us an insight into the strengths and weaknesses of the library and it is now up to us to respond as best we can, both operationally in how we conduct our day-to-day business but also in terms of how we inform our future strategic delivery of these services. Final comments We are aware that a “one size fits all” approach will not address the needs of our diverse user groups. We have so far reached our full-time undergraduates and academic staff. We intend to focus further surveys on the large body of WIT part-time students and also on postgraduates. We also intend to do more focused surveys on specific aspects of the library service, for example Learning Support and Inter-Library Loans. It is projected to become a regular part of the library's business, most likely by means of an annual survey, starting with the post-graduate student body. From these findings it seems that the main challenge facing WIT Libraries is not to encourage people to use the library service, but rather, to focus on optimising the “library experience” of its substantial number of current users, particularly students. Many of the survey findings indicate high satisfaction levels with what one might call traditional services such as book collections and help desks. Perhaps this is unsurprising as over 40 per cent of those surveyed in the student survey were first year and may not yet have achieved a level of sophisticated or advanced usage. In an age when information literacy is an essential prerequisite for academic success and, indeed for life after college, the library devotes a lot of attention to the increasingly important issue of Learning Support. These survey results show that while it does form part of some undergraduates' experience of the library, it has not reached anything approaching its full potential. Interestingly, in one of the final questions on the questionnaire, students were asked to choose from a list of priorities for the year ahead and the provision of more training in the library came out ninth. This suggests that while more training is perceived to be of potential benefit, it is not of the same perceived importance as other issues. However, the findings so far indicate that many undergraduates are using the library in a relatively rudimentary fashion. They are using largely it as a “work space” and as a repository of books. These are undoubtedly important uses, but, without recourse to the more sophisticated skills and resources that are available to assist users, it is likely that key sources of information are not being fully utilised. This outcome is similar to that of other surveys, which would seem to indicate that it is the standard way in which undergraduates perceive the role of the library (Hiller, 2001; Jiao and Onwuegbuzie, 1997). The challenge for the library is to try to change this culture. WIT Libraries has begun the process of addressing this challenge with a number of initiatives, beginning with engaging with our users through the survey itself and other steps such as an increased opening hours pilot, new signage and increased availability of online services. We are committed to continuing communication with our users so that we can enable them to get the best possible use of the library resources and services.
| ||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||