Updating SAML Service Provider keys for Shibboleth IdsP

SAML Service Providers (SPs) maintain one or more key pairs for use when interacting with Identity Providers (IdPs). The public part of these key pairs are distributed to IdP via SAML metadata or otherwise.

SP keys are used for two purposes:

1. The SP uses them to sign authentication requests
2. IdP use them to encrypt attribute assertions in responses for decryption by the SP (encryption is optional, but commonly enabled)

Keys need to be replaced occasionally. This can be difficult to achieve without service disruption because the public parts are distributed and it's impractical to update all copies simultaneously. For a Shibboleth IdP, coping with replacement of an SP key used for signing isn't problematic -- a new key can be added to the SP's metadata ahead of deployment and the IdP will be happy with signatures made with either. However once two keys appear in the metadata the IdP may use either to encrypt assertions and this will only work if the SP is happy to perform decryption with either key.

For Shibboleth SPs this isn't a problem, because they can be configured to accept multiple keys for decrypting assertions while using a nominated key for creating signatures. There's a well known procedure for rotating the key on such a SP, see for example documentation on the UK federation site and in the Shiboleth Consortium Wiki. This amounts to:

• Create new key pair on the SP
• Configure the SP to accept either key for decryption but not use the new one for signing (by adding the new one with use="encryption")
• Add the new key to the SPs metadata for both use="signing" and use="encryption" (or just without a 'use' attribute)
• Wait for the metadata to propagate, or directly update all IdPs
• Switch the SP to use the new key for signatures (by removing use="encryption" from the new one and adding it to the old one)
• Completely remove the old key from the SPs metadata
• Wait for the metadata to propagate, or directly update all IdPs
• Remove the old key from the SP configuration

Some SPs don't support this. However it is only encryption that causes a problem. The main reason for encrypting assertions is to prevent users seeing what the IdP is saying about them, but in man cases there's nothing in the assertions that the user can't see by other means. So temporarily disabling encryption while rolling a SP key is a possibility. This means that the key can be rolled without downtime with the following revised procedure:

• Setup custom relying party on the IdP for the SP that suppresses encryption. Something like this:

<rp:RelyingParty id="<SP entityID>"
   provider="<IdP entityID>"
   defaultSigningCredentialRef="IdPCredential" >

   <rp:ProfileConfiguration xsi:type="saml:SAML2SSOProfile" 
       encryptAssertions="never" 
       encryptNameIds="never" />

 </rp:RelyingParty>

• Add the new key to the SP's metadata
• Wait for the metadata to propagate, or just update all IdPs.
• Switch the SP to use the new key
• Remove old key from metadata
• Wait for the metadata to propagate, or just update all IdPs.
• Remove the custom relying party configuration

Note that this requires coordinated work on every IdP with which the SP works so it's not practical on a large scale. It will however work where there's essentially a one-to-one SP -> IdP relationship, as in many SaaS scenarios.

Tools such as the Firefox SAML tracer, the Chrome SAML Dev Tool extension and the SAML Chrome panel are  really helpful for checking that each step of the process has completed before moving in to the next.

Federated (and so SAML, and so Shibboleth) account management

The Problem

In an earlier post (User-ids in a Shibboleth world) I described the problem of deriving a 'userid', as required by much third-party software, from typical attributes supplied in a SAML federated authentication (particularly in the UK federation, but probably equally elsewhere in Europe). This is actually part of the wider problem of account management in a federated environment (as touched on in Federated AUTH is less than half the battle), and this posting is an attempt to pull together some thoughts on the subject.

The context of this is a need to support web applications (for example MediaWiki, Plone, or Wordpress) which need to identify local users (members of the University of Cambridge) but which also need to support authenticated access from people outside the University (collaborators in other institutions, sponsors, representatives of funding bodies, etc.). Many of these have access to SAML identities in the UK federation (or elsewhere), and all three applications claim to have 'Shibboleth support', but in practice this support doesn't come anywhere near doing what we need. As mentioned in User-ids in a Shibboleth world, the primary problem seems to be an assumption that eduPersonPrincipalName (or something equivalent) will always be available. We can (and do) make this available from our own IdP to our own SPs, but we can't expect it from elsewhere, and negotiating its release on a case-by-case basis is impracticable and in any case we don't actually need it.

I asked about this issue on both the jisc-shibboleth@jiscmail.ac.uk and shibboleth-users@internet2.edu mailing lists. I had rather assumed this would be a well known problem with obvious solutions, but the only significant response was from Scott Cantor who said: 

"I think they're written for "least effort" and to avoid fixing applications. That's not a strategy that will work long term."

The Options

So, what might work in the long term?

I think you first have to accept that this sort of federated environment differs from a typical single institution environment in a least two significant ways:

1. You will get a unique identifier for each user, but you must assume that it's totally opaque and that it's not suitable for display. Further, you have to assume that users will not be able to tell you in advance what this identifier is or is going to be.
2. Lots of people will be able to authenticate who you don't want to have anything to do with. So you can't rely on the 'If they can authenticate they are part of the organisation' approximation that works in other situations.

I see two approaches: Account Linking, and Account Creation:

Account Linking

Under this approach you start by provisioning ordinary, local accounts complete with locally-unique (and displayable!) user names and passwords and by transmitting the username/password information to the intended user in any appropriate fashion. But once the intended user has authenticated with this information you allow them to link this account with a federated identity which they can subsequently use to authenticate in place of the local password. You could even regard the initial password as just a bootstraping device and expire it after a few uses, or after time, or after a federated authentication has completed.

Anyone trying to authenticate using an unrecognised federated identity would get an error page refering them to whoever deals with account management. If accounts need non-default attributes (particular editing rights, administrator privilege, etc) then these can either be set manually by administrators or they could be added following a successful federated authentication with particular attribute sets.

Depending on the application you are protecting you might choose to restrict which federated identities you allow to be linked, based either on who asserts them and/or on attributes supplied with the authentication (i.e. you might require that a 'staff' resource can only be linked to an identity that includes an eduPersonScopedAffiliation value including 'staff'). You might also allow more than one identity to be linked to a single account. It all rather depends who has the strongest interest in the account being used only by the right person (you as application operator, or the user).

Account Creation

This is to some extent the process envisaged in the latter sections of Federated AUTH is less than half the battle. However software that needs a local 'user name' will have to create one that meets its own requirements when first creating a local account, and it can't assume that anything suitable will be available in the attribute values. This means that the 'Dynamic login' approach is going to be problematic since there probably won't be anywhere to store the association between this user name and the corresponding federated identity.

As mentioned in the other post, some rules will be needed, presumably based on attributes supplied, to decide who can have an account at all. Anyone trying to authenticate using a identity that doesn't match these rules would again get an error page refering them to whoever deals with account management. Enhanced account privilege can be automatically granted based on attribute values or, once the user has authenticated for the first time (but probably not before), by manual intervention by administrators and by reference to the locally generated user name..

Hybrid approach

There are almost always exceptions to rules. To allow for this you might want to adopt a hybrid approach. This might uses the Account Creation approach for the majority of potential users, because it totally avoids 'one more password' and needs little on-going user administration. But for the inevitable cases where you need to allow access by someone who doesn't match the standard account creation rules you could fall back to the Account Linking approach. Indeed, once you allow local accounts you have a tool that can if you wish use to allow access to people who don't themselves have access to suitable federated identities.

All this is quite a lot more work than just lifting a string out of something like a REMOTE_USER environment variable and convincing you application to treat this as evidence of authentication. But to get the best from federated authentication it's what we are probably going to have to do, though in time and with luck we may find that someone has already done it for us...

User-ids in a Shibboleth world

[This is a copy of a question posted to the JISC-SHIBBOLETH@JISCMAIL.AC.UK mailing list which, to date, hasn't seen much in the way of responses to the questions at the end]

A lot of existing web apps have wired into in the, often at a very low level, the idea that each 'user' has associated with them a unique, short-ish, fairly friendly identifier. Traditionally these were things like jw35 or 2006STUD42, though it's increasingly common to use an email address belonging to the user instead. Quite often these are (in fact or in practise) the primary key into the application's user database and they tend to get displayed (as in "Hello jw35", or in things like usage logs) and these two usages are often not distinguished.

When converting existing software for a SAML environment there's the question of what to use to fill this slot. If you have ePPN available then that will probably work (because it looks like an email address) but in the UK federation that's unlikely. The 'old' form of ePTID (rUL8A3M667VfsiCImQVFffN9cNk=@cam.ac.uk) might just about work, though it's ugly when displayed. The new form is close to unusable for display purposes:

https://shib-test.raven.cam.ac.uk/shibboleth!https://mnementh.csi.cam.ac.uk/shibboleth/ucam!rUL8A3M667VfsiCImQVFffN9cNk=.

Of course if you are creating accounts in advance and only later binding them to SAML identities you can use whatever user-id scheme you want, but what if you are creating accounts 'on the fly'?

We've noticed that 'off the shelf' Shibboleth adaptations (most recently for Plone and MediaWiki) tend to simply use what turns up in REMOTE_USER which, by default, will be the first of ePPN and the old and new forms of ePTID that isn't blank. In a UK federation context this doesn't really work, and I suspect many of these adaptations were written for contexts where ePPN is more widely available.

What have other people working in the UK environment done to address this problem, assuming you are seeing it too? Is there a 'best practise'?

Authentication: Think before you proxy

There are three obvious ways to add authentication such as Ucam Webauth or Shibboleth or OpenID to a web application:

• Add the authentication code to the application itself (what the Java world calls 'Application Managed Security')
• Implement the authentication in the web server or container hosting the application (Java's 'Container Managed Security')
• Implement the authentication in an HTTP proxy in front of the application

The proxy option has some obvious quick gains: a single proxy can protect multiple applications; it's normally easy to leverage existing 'container managed' implementations to build such proxies; using a proxy probably requires much less modifications of existing applications (sometimes none). But I'm going to argue that this approach, while appropriate and even necessary in some circumstances, should be avoided where possible. Here are some reasons:

• Security: to be useful, the proxy has to pass on information (such as user name at the very least) to its downstream client and obviously has to do this is a way that can't be influenced by an attacker. About the only option (if you are sticking to HTTP for communication between the proxy and its downstream client) is by adding extra HTTP request headers. It's not easy to be absolutely sure  that these headers can't somehow be injected by an attacker (by talking directly to the downstream client, by supplying them in a request and convincing the proxy to pass them on, etc., etc.).
• Availability of attributes: Shib and other similar systems potentially make lots of information about the authenticating user available as attributes. Unless you arrange for your proxy to pass all of these on (and there could be lots and you might not know in advance what they all are) then your clients will loose out. There's also the problem that (in theory, if not in practise) that users are able to control what information about them is disclosed to which services. With multiple services behind a single proxy it's going to be a case of one size having to fit all.
• Confused logs: clients behind proxies see all connections as coming from the proxy. This makes their logs less useful than they might be and frustrates any attempt to limit access by IP address (which is actually a bad thing anyway IMHO, but that's another story). There are ways around this, though most then suffer from the security problems mentioned above.
• One more system: the proxy is one more system to build, maintain and in particular to understand. I have a theory that authentication services in some big web applications are a bit 'hit and miss' (and 'electronic journals' are big offenders here) precisely because they are big, layered applications that almost no one (or perhaps actually no one in a few cases) really understands.

I think my underlying feeling would be that a proxy is an entirely appropriate, if not quite as simple as it first appears, way to implement authentication for some particulare application, but that care should be taken not to slip into using one in every case. Like they say, once you've got a hammer everything starts looking like a nail...

Shibbolising Plone 3 - some experiences

In a previous posting I listed two (and a half) options for Shibbolising Plone 3. What follows is a comparison based on what I found when I actually tried installing them into default Plone instances.

[The 'and a half' option was Liberty Alliance / SAML 2 Authentication Plugin for PAS. I'm discounting it for the time being because a) it doesn't look as if it will work with Shib 3, b) there hasn't been a release since December 2008, and c) the most recent release gives '404 Not found' on the Plone web site]

The two remaining options were WebServerAuth and a set of products from Ithaka (AutoUserMakerPASPlugin, ShibbolethLogin, ShibbolethPermissions). They have quite a lot in common (which is not surprising since they share a common ancestor).

Common features

• Both depend on the Apache Shibboleth module to perform authentication, and so require connections to Plone to be proxied through Apache.


• For this to work, server variables provided by the Shib module have to be converted to headers in the proxied request - doing so is a matter of Apache configuration. It is important to ensure that unauthenticated users can't spoof any such headers used for anything related to security.


• Both will allow Plone access by anyone who can successfully authenticate - if this isn't appropriate then Apache access controls could be used to limit this further (and WebServerAuth has an additional option).


• Both require a unique user identifier. Easiest to use the value in REMOTE_USER which is set by default by the Shib module to the first non-blank value in eduPersonPrincipleName or the SAML 2 or SAML1 versions of eduPersonTargetedID. This leads to some unwieldy userIDs - both products can strip domain names from IDs, but this removes any guarantee of uniqueness.


• Both force authentication for an https: version of the site while allowing unauthenticated access to the http: version. While Shib's 'Lazy Session' features might make this unnecessary, both products will in principle work with other webserver-based authentication schemes that may not support anything similar.


• For both, it's advisable to suppress the default 'login' portlet that is displayed to unauthenticated users.

WebServerAuth features

• WebServerAuth doesn't create users in the database - by default they just get assigned the 'Authenticated' role (not 'Member') when they log in. It uses userID (so eduPersonPrincipleName or eduPersonTargetedID, optionally with domains stripped) as the user's Full Name - it doesn't use any other Shib attributes even if they are available.


• Users requiring additional rights can be created in the Plone database and appropriate rights will be assigned when the corresponding users authenticate. Creating Plone users with IDs containing '@' and '.' requires a long-winded hack.


• WebServerAuth can be configured only to authenticate users who have corresponding Plone accounts, but the user interface is sub-optimal: people will log in and apparently succeed, only to be greeted with a Plone page that still has a "Log In" link.


• By default, WebServerAuth redirects to the https: version of the current page when authentication is required (and modifies the standard 'log in' link to achieve this). This will cause the Apache module's default Sessioninitiator to be used for authentication. Optionally WebServerAuth can redirect to a customised URL which could perhaps be used to implement a local WAYF service (e.g. simplified login for local users; redirect to federation WAYF for others). Essentially WebServerAuth takes over all login access to the site (which can be problematic if it fails...).


• WebServerAuth is under current development - its author contacted me within hours of my posting my earlier article to point out an error with it.


• Need to set the standard logout link (ZMI --> Plone --> postal_actions --> user --> logout --> URL) to something apropriate (or perhaps suppress the link altogether?)

Ithica product features

• AutoUserMakerPASPlugin creates real Plone users as new people authenticate. It can use configurable Shib attributes to initialise full name, email, location, roles and group membership. This only applies to user creation - once created, changes to attributes are not propagated and the users need to be managed on Plone. It seems necessary to configure full name to fall back to userID if nothing else is available to avoid all such users ending up with a full name of '(null)'.


• AutoUserMakerPASPlugin can be configured to strip either all or selected domains from userIDs - one approach might be to strip just a local domain, giving local users short userIDs while ensuring against name clashes with anyone else.


• AutoUserMakerPASPlugin supports a configurable 'logout' URL that can invoke the local Shib module logout function. This may be confusing in a wider single sign-on context (but is handy for testing).


• ShibbolethPermisisons adds the ability to add local (per page or per container) access rights based on configurable Shib attributes. Again, this only applies at account creation time, after which rights have to be managed manually within Plone as usual.


• ShibbolethLogin installs a replacement for the standard login page. This includes configurable links either to local SessionInitiator URLs or direct to Shib 1 IdPs or WAYFs. So for example it's possible to have a 'Log in with a Raven (University of Cambridge) user id' link for local users and a more general 'Log in with a different UK Federation user id' link for everyone else. Local Plone user authentication (with username/password) remains available. ShibbolethLogin appears to be targeted at Shib 1 functionality, with no obvious support for new Shib 2 functionality, such as the Discovery Service.


• The maintenance status to the Ithica products is unclear. Documentation reports the most recent tests to have been with "Zope 2.10.5 and Plone 3.0.6"; files in the distribution appear to have been modified most recently in May 2008.

Federated AUTH is less than half the battle

Federated web authentication, as provided by Shibboleth, OpenID, Pubcookie, Cambridge's Ucam WebAuth (a.k.a. Raven), etc., is less than half the battle of adapting an application to work in a federated environment. Much the harder part involves the related issues of authorisation and account provision.

Most existing applications assume that they have some sort of account for every users who can authenticate, not least because traditionally they need somewhere to store user names and passwords. In the federated authentication world this won't always be true, since lots of people will be able to authenticate that the application has never heard of.

Approaches to dealing with this include:

Require an account to exist locally before a user can authenticate

If handled manually, this can impose an administrative overhead which will only be manageable for a small user base. Alternativly accounts could be provisioned automatically, but only if the expected set of users can be identified in advance. It is often hard to adapt existing software to not fail badly when presented with an authenticated user who doesn't have an account.

Users may have to tell administrators how their identity provider will identify them and this could be a problem - in Shib's case it is currently unlikely that users will know their eduPersonPrincipleName (even if it is available), and they can't predict their eduPersonTargetedID. An authenticated 'Application' page that captures available attributes might be a way around this.

Automatically provision an account when a user first authenticates

Doing this entirely automatically assumes that information is available about the user, and so assumes either Shib-like attribute provision or access to additional data in something like an LDAP directory. It also assumes that enough information is available to create accounts - some existing applications positively require things like forename and surname which is problematic if these are not available. Alternativly or additionally, users can be prompted for additional information, though obviously this is less reliable and implementing such prompting can be difficult.

However it is also necessary to decide who will get such accounts, since it's unlikely that you will actually want to provision an account for everyone who can successfully authenticate, especially if such accounts come with some basic privileges. So some sort of rule engine will be needed to define who does and doesn't get an account, and perhaps what default privileges they get. This will have to be driven by Shib-like attributes or information from directories. For obvious (I hope) reasons, such decisions probably can't be taken based on information supplied by the users themselves.

Dynamically log users in without creating accounts for them

In some applications it's possible for a user to appear 'logged-in' without having a corresponding user account. One example I happen to know about s Plone using the WebServerAuth product. This simplifies things somewhat, though you still need to address the question of who gets what access by default. You probably need some way to manually grant additional access to a limited number of people where the people and the access that they need can't be established based on attribute or directory data - for them, some sort of local account bound to their federated identity will probably still be needed.

Shibbolising Plone 3 - a review

There are lots of Google references to shibolising Plone, but it's not clear how many of them apply to Plone 3 (rather than Plone 2 or earlier). All seem to rely on getting Apache to implement the Shibboleth protection, and then passing identity information over to Plone for it to use.

It looks as if the options include

• ApachePASS
• Auto Member Maker

which seem to have been replaced by

• WebServerAuth

all from (or at least related to) the WebLion project at Penn State. There's a useful page on WebServerAuth on the WebLion site.

Alternativly there are three extensions from Ithaka

• AutoUserMakerPASPlugin
• ShibbolethLogin
• ShibbolethPermissions

which are described in this this slide set and this article.

It looks as if the Ithica solutions actually provision Plone accounts for Shib-authenticated visitors, while the WebLion products give such users 'authenticated' state without creating accounts for them. I can see pros and cons for both approaches. WebServerAuth is being actively developed (last release August 2009); none of the others look as if they are very actively maintained. The Ithica products apparently work with at least Plone 3.0.6; apachepass only claims to work with Plone 2.5; Auto Member Maker and WebServerAuth apparently work with Plone 3.

Note: updated 2009-09-15 to correct the development status of WebServerAuth in the light of comments by Erik Rose.

Note also that the Liberty Alliance / SAML 2 Authentication Plugin for PAS might be relevant, if it's sufficiently flexible to talk Shib.

See also Shibbolising Plone 3 - some experiences