Celebrating Accomplishments, Moving Forward

Afraid of drawing the wrath of Library Facilities, the cheese weasel skipped over the Library this week.  To compensate all our excellent IT professionals who missed out on delicious Kraft American singles under their keyboards, we put together a short list of accomplishments from the past year, and celebrated with bagels (and cream cheese).

In August 2010, Library IT was presented with 12 audit recommendations.  The last of these recommendations was met this March, which allowed us to close the almost 4 year old requirement.  As part of this process, IT created a service catalog, classified the sensitivity of data in every database and web application, developed policies, processes, and procedures for dealing with disaster scenarios and security incidents, implemented a 24×7 on-call operation for critical services, and much more.

The entire IT infrastructure received a facelift.  We added a new tier of storage aimed at providing ingest capacity for the Medusa repository, a working space for digital content in-process, and scratch space for use by patrons at the Media Commons.  Medusa itself began taking in materials, and is now rolling as a production service.

Vast improvements were made to public printing, including rolling out web printing as a beta service.  Another low-profile beta service that people may have overlooked is volume shadow copy on the G and H drives; it is now possible for Library faculty, staff, and students to recover older versions of files with a right-click.

The Library also extended partnerships with other IT providers.  A substantial portion of the Library virtual infrastructure was moved to a shared data center, which opens possibilities for true high-availability features and reduced infrastructure costs in the future.  We extended our partnership with the College of Engineering, which hosts our Vacation and Sick Leave reporting system, to include more business and facilities applications.

Never content to rest on our laurels, a group of us proceeded to brainstorm ideas for the next round of initiatives.  More than 50 specific ideas coalesced into a handful of themes.  The largest category of proposed projects focused on providing support for Library-specific applications.  There was a virtual three-way tie for second place, between developing better work flows throughout the Library, providing enhancements to in-house tools and middle-ware, and improving our business continuity (uptime) preparations.  It’s too early to tell which of these ideas will turn into tangible projects, but this is a good indicator of how Library IT staff view their long-term priorities.

The Redesigned Library IT Web Site

At the beginning of this month, Library IT’s Web Team deployed a redesigned version of the Library IT website (http://www.library.illinois.edu/it/) in a soft launch.

Work on this new site was a collaborative effort among  members of the Library Help Desk and the Library Web Team, in consultation with a host of Library faculty and staff that contributed suggestions on how to improve the site to better serve Library needs. The Help Desk accrued feedback regarding the prior Library IT website over the past several years, leveraged those suggestions to reorganize the Help Desk section of the Library IT Web Site, and subsequently worked with the Web Team to expand the organizational structure and aesthetic and apply it to the entire Library IT web site.

In addition to feedback received from Library Faculty and staff, we also liberally borrowed from many of our Library colleagues’ designs of their own unit sites. Thank you all for your help! While we hope Library IT’s website is easier to navigate and use, we acknowledge it will further benefit from ongoing review and improvements. We welcome your comments, questions, and suggestions for improvement. Please send them to webmaster@library.illinois.edu.

Some new and/or improved features include:

Easy to locate contact information on each Library IT page

screen shot showing placement of Contact Us and Personel Sections of new web site

Improved, consistent left-hand side navigation

Screen shot showing left hand site wide navigation

Continuity among various IT team pages

Screenshots of various IT Unit pages

Systems Status & Scheduled Maintenance Page (including links to CARLI and CITES pages)

System Status Page

A page dedicated to partnerships

Screenshot of the partnership page

Application Migrations

A large portion of the Library infrastructure has been migrated into Data Center Shared Service space.  Once we have everything in place, we will be able to retire old and malfunctioning hardware and reclaim space for academic and research use that was previously housing computer equipment.  The work we do in in the next 6 weeks will also position the Library for upcoming consolidation efforts in storage and virtualization, allowing IT resources to focus more on needs specific to the Library instead of supporting commodity services.

The most difficult aspect of this move kicked off last week, as we began testing applications in the new data center.  Although we hope to minimize disruption to services, there will be some unavoidable side effects from the move.  The migration requires changing IP addresses for all services, and because we can’t fully test and migrate all services at once, this also means the URLs will be different.  We will put redirects in place so that applications accessing the old URL will forward to the new location, but it will not be possible to make the redirection perfect in every case; some of the navigation within applications may be awkward or have bugs.  After all services have moved, we will use what is technically known as “DNS magic” so the old URLs point to the new servers.

Long story short, some links and bookmarks will break, but it will be temporary.  We do recommend using the new URLs going forward, because there are no guarantees for how long we can maintain the old URLs.

In the next few weeks, we will be migrating 18 production services from Grainger, most of which live on quest.grainger.uiuc.edu.  Very soon, we will begin migrating 34 applications from www-s2.library.illinois.edu and 12 applications from www-s1.library.illinois.edu.  The Library IT Survey revealed that most applications are only used by a subset of Library faculty, staff, and patrons, so information on specific applications will be targeted only to the identified service owners.

After this phase is complete, there are more than 200 additional applications that should be easier to move once the first phase is complete.  There is also a database upgrade project that dovetails with application migration, which merits its own post in the near future.

Farewell, LibPrint

Over the summer of 2011, Library IT faced a tough decision on public printing.  The long-time print service, LibPrint, was showing its age.  The original developers had long since left the Library, and there were no programmers available to service and maintain the code base.  Hiring new people was a near impossibility at the time, and contract work to bring the system up to date could have cost upwards of $100,000.  The only option that remained was to look for a commercial product, so an evaluation period was launched to find the new printing system.  After a thorough evaluation of products, we launched PaperCut in a pilot program at the Undergraduate Library over spring break, 2012.

Over a year later, PaperCut has proven to be a capable and powerful replacement.  Thus, with little fanfare, LibPrint was officially retired this week.

Although the final days of LibPrint were marred by performance issues and a lack of features (such as two-sided printing), it was very innovative for its time.  LibPrint introduced the web-based release station, which allowed for laptop printing.  It offered an innovative way of recovering cost for printed documents.  It included a self-service package installer for Windows and Mac, and provided a very effective and responsive way to detect printer errors.  LibPrint also used a proprietary algorithm for determining the true cost of color printing, by calculating the amount of toner used on a color print job.

LibPrint processed its first job on June 25, 2002.  Over its nearly 11 year lifespan, it handled 3,604,863 print jobs from 143,193 distinct users, totaling 18,304,992 pages (456,654 color).


Here are some quick hits as we near the end of the spring semester.

The Library Staff Development blog Growing People recently posted an article on using the VPN.  Several years ago, CITES began a pilot for an SSL VPN implementation.  The project was never officially supported, but the infrastructure has been maintained.  Many people who have encountered problems with the CITES VPN have found stability with this version, branded Aventail.

We have completed testing for moving distribution lists and shared mailboxes to the CITES Exchange server.  Distribution lists will be automatically migrated next week.  This move should be completely transparent to everyone, and there are no anticipated service interruptions.  In specific cases, we expect that this migration will substantially improve email performance.  One of the changes that CITES made in response to the recent phishing attacks was to put a delay on outgoing mail, so antivirus software and spam filters had more time to analyze threats before sending them off to the Internet.  Mail that stays internal to the Exchange server are not affected by this delay.  Once we move the distribution lists, mail that is sent from a University account through the distribution list to other addresses at illinois.edu will stay internal to the CITES server; currently, those messages are delayed because they have to run through spam control as they leave the Library mail server.

Shared mailboxes will not be moved at this time, but we will begin scheduling meetings and walk-throughs with units who have shared mailboxes.  Unfortunately, the shared inbox migration isn’t as seamless as moving distribution lists, but Library IT will make every effort to make it as simple as possible.  The shared inbox move will provide many new features that are not available in Squirrel Mail, and will also benefit from the afore-mentioned performance shortcut.

Adoption of PaperCut for the Library’s public printing system is nearly ubiquitous.  As such, we will be retiring the old service, LibPrint, on May 17, 2013.  Look for a proper send-off for this ground-breaking service after finals.

IT has several initiatives ready to kick off for the summer session.  The full scope for most of these projects haven’t been finalized, and many of them will take us into next year and beyond.  Look for more details on infrastructure upgrades, technology enhancements in classrooms, added and improved printing services, and much more in the coming months.

Library Email

One of the identified cost savings for the Unified Communications project is that departmental email servers could be consolidated into the campus-wide Exchange email system.  The Library has had several false starts since last March, but we are now prepared to begin our own migration.

There have been several recent requests for enhancements to Squirrel mail, including better spam filtering and automatic vacation replies.  While such enhancements are possible by upgrading our current email server, the better long-term approach is to begin migrating accounts to Exchange.  Exchange already supports all of the requested features, and more.  In addition, security on Exchange is handled through Active Directory, the same system that you use to login to your computer.  You will no longer have to remember a shared username and password to access Squirrel mail, and there will no longer be a need to reset lost or expired passwords.  Many users may also like that they can manage all their shared mailboxes in the same Outlook pane as their University email.

Many of you remember the toil of migrating from Express mail to Exchange two or three years ago.  This migration will be substantially less disruptive.  Our email server no longer has any individual accounts, but it does have accounts for distribution lists (you send email to one address, and it goes to many different people) and shared mail accounts (Squirrel mail).  People on distribution lists should notice no difference at all.  For Squirrel mail users, there are several different new ways to access the shared account.

We will work with each Squirrel mail account owner to make the best decisions about how to migrate, the best way to access the account, and what to do with messages already saved in the account.  In many cases, we hope to be able to completely delete accounts that are no longer being used.

Work will begin this week on starting the migration, but this project is lower priority compared to some other IT projects.  It is possible that we will have everything completed by spring break, but more likely by the end of the spring semester.

Printing Updates for Winter Break

PaperCut, the new Library public printing software, now has a semester of usage under its belt.  Feedback has been generally very positive, though with several major obstacles.  Many of these will be addressed over winter break.

Far and away the biggest complaint we get is that there is no option for laptop printing.  This will be addressed before the start of the semester.  The major hurdle in accomplishing this will be to install print release stations at every Library print location.  The stations will be aware of the patron’s print jobs from their laptop, and will release the job only to the printers that are right next to the release station.

Another complaint is that printing is too complicated, and people don’t know which printers to select.  The cause of this issue also complicates work for IT, which makes it very difficult to get the right printers installed, with the right defaults, on the right computers.  We will therefore be decreasing the number of available printers from over 30 to just 4:

  • Library B&W
  • Library Color
  • Pay Cash B&W
  • Pay Cash Color

Patrons will print to any of these “virtual” printers, go to any release station at any library, and be able to release their job wherever it is most convenient.  This will remove almost all of the guesswork from printing; just send it to the printer, go to whichever printer you want, release the job and be on your way.

We have also received repeated requests for better, more consistent, and more complete documentation.  Drafting and reviewing that documentation has been given high priority, and has been assigned to individuals.  A documentation base will begin to emerge as January progresses.

Finally, there are some locations that complain of high volume, long wait times, and too much congestion during busy times.  There are other locations that display high print volume, even if there haven’t been any complaints.  IT will be adding printers and release stations at these locations.  We will also consult with libraries, service locations, and facilities to see if there’s a better location for printers.  Once laptop printing is enabled, for example, it may make more sense to put printers in wide open study areas instead of close to public computers.

This may not solve all of the problems with the default printers.  For example, our testing has shown that many applications are able to manage printers independent of the default settings, and are always using the last printer selected as the default.  This means that if one patron prints to the Pay Cash Color printer in Google Chrome, the next patron will see Pay Cash Color as the default in Google Chrome… but possibly not in Acrobat or Word.  IT is looking into this, but it likely will not be solved before spring semester.

Reporting Problems

In the past several weeks, there have been a handful of unplanned outages.  IT response time wasn’t as fast as it could have been, partially because of the way the outages were reported.

Any problems, real or perceived, with Library IT services can be reported by contacting the IT Help Desk via email (help@library.illinois.edu) or phone (217 244-4688).  When choosing a method of contact, consider the urgency of the problem.  Email is only checked during regular business hours, but the phone line will be answered – by a human – 24×7 every day of the year except Christmas Day.  The people answering the phone are specially trained to identify widespread and critical problems, and will call IT staff in the middle of the night if necessary.

Library IT maintains a comprehensive set of documentation on all services, including commitment levels for everything that is available to the public.  The service commitments include defined times when the service should be available.  Standard Availability means that the service is supported during normal business hours.  Extended Availability means the service is supported whenever any Library is open.  High Availability means the service is expected to be available 24x7x365.

The list of services continues to grow and change as the Library evolves.  Likewise, the level of support required for each service can change with time.  When evaluating the service catalog, it is important to remember that it is a living document; it may not accurately reflect the exact need for each service today, but through productive discussion, it can be shaped to provide the highest level of support possible for Library patrons overall.

Here is a list of services that are currently defined as extended or high availability.  But if you’re working on an evening or weekend and find an IT problem, remember that it’s OK to call the Help Desk even if the issue doesn’t appear on this list.  The person answering the phone is specially trained to determine if the problem you’re reporting might be symptomatic of a bigger problem, and to contact the appropriate people as fast as possible to get things resolved.

  • Easy Search
  • Electronic Reserves
  • EZProxy
  • Library Gateway and website
  • Handle (link resolver for electronic resources)
  • Journal and Article Locator
  • Online Journals and Databases (SFX/Discover)
  • Voyager, including the classic client, VuFind, and I-Share
  • CMS (website content management system, which contains many departmental library websites)
  • Groupfiles (G Drive)
  • Home (H Drive)
  • OTRS (ticketing system)
  • Library hardware and software infrastructure
  • Library networking
  • IPMonitor (which scans servers/services to make sure they’re running)
  • Public Printing
  • Streaming Media
  • Ask a Librarian (IM Collaborator)
  • Interlibrary Loan (ILLiad and Ariel)

Public Printing Update

There are several major items on the Public Printing to-do list.  I’d like to provide an overview of the issues, and address the timeline for implementation.

Print Billing

While billing has continued as usual for the legacy LibPrint system, we still have not issued charges for anyone printing to PaperCut.  PaperCut has been in use at the Undergraduate Library since spring break 2012, and was rolled out to all locations over the summer and during the early part of the fall semester.  Due to the delay in submitting charges, and as an acknowledgement of some of the transition difficulties we’ve had, the Library has decided not to charge for any PaperCut printing that occurred before August 15 of this year.

Patrons will see a credit for their printing up to August 15 in their Transaction History, http://libprintserv1.library.illinois.edu/user (please note, you must be on campus or using the VPN to access this site).  The credit will be applied at some point next week, after which bills will be sent to OBFS for print charges accrued from August 16 on.  Subsequent bills will be generated every two weeks, and sent to OBFS for collection.

Printer Problem Alerts

At present, PaperCut can only generate problem alerts, such as low toner, out of paper, or paper jam problems, through email.  At most of our locations, it would be cumbersome to rely on email to fix common printing problems. We have decided that the simplest path forward is to slightly retool the LibPrint printer pop-up alert system.  Once updated, the PPA system will pop up printer error messages on staff workstations.

The PPA system has been in continuous use throughout the public printing migration, but it has not been functioning as expected for several reasons.  Getting the alert system running will require some code updates on the server-side, and installation of PPA clients on some staff workstations.  We plan to also offer training on addressing common printer problems, such as stalled jobs or print jobs that have generated printer errors.

Incremental progress will be made on the PPA system between now and December.  We expect full deployment, including staff and student training, to be complete before December 1.

Laptop Printing

The most requested public printing feature, by a very wide margin, is laptop printing.  We have held off on allowing laptop printing for a number of process-related reasons, which I won’t elaborate on here.  The simplest way to think about it is to consider the current public printing environment.  Think about the confusion many of our patrons – and staff – have experienced as we make the transition to PaperCut.  People have been asked to rethink how they submit, release, and claim their print jobs.  How much more difficulty would we encounter if we added that people can also print from any mobile device, not just from the controlled setup we have on the public workstations?

Now that most of the kinks have been worked out of the public workstation environment, we are beginning to consider how to support laptop and mobile device printing.  We have a plan in place, but it will take some time to set everything up.  We expect to fully support laptop printing before the start of spring semester 2013, although printing from mobile devices (for example, iPads, Android phones, etc.) may not be available right away.

Access Management

This is the second of a two-part series introducing the Identity and Access Management (IAM)project at the University of Illinois.  Read part 1, Identity.  More information can be found on the IAM project website.


You will recall from the first post on IAM that authentication deals with determining whether someone is who they claim to be.  Authorization is concerned with determining what resources a person can use after they have passed authentication.  The most common authorization method at the University of Illinois uses Active Directory.  An identity (remember, one person, one identity) is added to one or more active directory groups, and those groups are granted permissions for using resources or accessing services.

One common problem with authorization is that many services don’t use it.  There are a lot of applications on campus, especially websites, that require a Bluestem login to access them.  In some cases, after Bluestem passes the authentication (the website trusts that the user is who they claim to be), there are no authorization checks to ensure that the person should be able to access the page.  This happens because of old policy; when a person ends their affiliation with the University, their accounts are deactivated.  For services that are expected to be available to all University-affiliated people, there was never a need for authorization.

  • Authentication – Are you who you claim to be?
  • Authorization – Are you allowed to use this resource?

Identity for Life

One of the stated goals for IAM is that once a person has claimed an identity, it is theirs for life (and possibly longer).  Making this change would allow places like the Library to offer services to alumni and retirees without creating a new identity for them, or lumping them in with other unauthenticated users.  It has some interesting and wide-reaching implications, such as eliminating the need for zero-time appointments.

But this also creates a problem for applications that don’t use authorization, or that are very loose with it.  There is a lot of talk about allowing retirees and alumni to continue to use Library resources.  But what about all the students who applied to the University, but never actually took a class?  Poor access management could cost millions of dollars, as publishers and other rights holders demand more money for current subscriptions because there are many more people with access.

Fewer Passwords

There are quite a few things to get excited about with IAM, but perhaps the most anticipated service is single sign-on (SSO).  Since there will be a single identity store making sure you are who you claim to be, there is no reason for dozens of different applications to ask for your password.  You just gave a password when you logged into your computer, why do you need to do so again when you open email, go to your department’s internal website, access the wiki, or anything else?

This concept can be extended even further, to off-campus applications.  The final stage of IAM will create federation between the University and other institutions.  This will allow people to use their existing logins/passwords from Illinois to access resources that are run somewhere else.

What does this mean for the Library?

Library IT is preparing for this change on several fronts.  We are examining our services to ensure that they use proper authorization, and cleaning out old permissions for people who have left the Library.  We are also cataloging each of our secure services to address what the proper authentication and authorization processes should be.

In doing this, we will enlist the help of many Library service owners – the people working with patrons to provide a usable Library service – to identify the required level of authentication trust and proper authorization requirements for each service.  A committee has also been formed to identify and solve potential problems with the IAM project that could affect the Library.