Indistinguishable from magic

Two sorts of authetnication

2012-02-02T13:28:00.000Z

It occurs to me that, from a user's perspective, every authentication sits somewhere between the following two extremes:

Authentications where it's strongly in the user's interest not to disclose their authentication credentials, but doing so has little impact on the corresponding service provider. For example I'm probably going to be careful about my credentials for electronic banking (because I don't want you to get my money) and for Facebook (because I don't want you to start saying things to my friends that appear to come from me).
Authentications where it's mainly in the service provider's interest that the user doesn't disclose their authentication credentials but it's of little importance to the user. For example authentication to gain access to institution-subscribed electronic journals, or credentials giving access to personal subscription services such as Spottify. In neither case is giving away my credentials to third parties likely to much immediate impact for me.

This is obviously a problem for service providers in case two, because it significantly undermines any confidence they can have in any authentications, and may undermine their business model if it's based on number of unique users. There's not much you can do technically to address this, other than using non-copyable, non-forgeable credentials (which are few and far between and typically expensive). It is of course traditional to address this with contracts and rules and regulations, but neither work well when the chance of being found out is low and the consequence small.

More interesting is what happens when you use the same credentials (SSO or single password, for example) for a range of services that sit in different places in this continuum. I suspect that there is a strong possibility, human nature being what it is, that people will make credential-sharing decisions based on the properties of an individual services and without really considering that they are actually sharing access to everything.

[I'd note in parsing a New Your Times article (Young, in Love and Sharing Everything, Including a Password) that suggests that young people will sometimes share passwords as a way of demonstrating devotion. I expect this is true too]

O2 changing web page content on the fly?

2012-01-25T15:26:00.002Z

We recently noticed an oddity in the way some of our web pages were appearing when viewed via some 3G providers, including at least O2. The pages in question includes something like this in the head section:



which should have the effect of including the ie6.css stylesheet when the document is viewed from IE6 but not otherwise.

When accessed over 3G on some phones, something expands the @import by replacing it with the content of the ie6.css style sheet before the browser sees it:



which, while a bit braindead, would be OK if it were not for the fact that the CSS file happens to contain (within entirely legal CSS comments) an example of an HTML end-comment:


*/
...more literal CSS Statements...
</style>
<![endif]-->

When parsed, this causes chaos. The first  inside the CSS then closes that comment, leaving WebKit's HTML parser in the middle of a stack of CSS definitions. It does the only thing it can do and renders the CSS as part of the document.

I don't know what is messing with the @import statements. I suspect some sort of proxy inside O2's network, perhaps trying to optimise things and save my phone from having to make an extra TCP connection to retrieve the @included file. If so its failing spectacularly since its inlining a large pile of CSS that my phone would never actually retrieve.

You can see this effect in action by browsing to http://mnementh.csi.cam.ac.uk/atimport/. You should just see the word 'Test' on a red background, but for me over O2 I get some extra stuff that is the result of their messing around with my HTML.

[There's also the issue that O2 seem to have recently been silently supplying the mobile phone number of each device in the HTTP headers of each request it makes, but that's a separate issue: http://conversation.which.co.uk/technology/o2-sharing-mobile-phone-number-network-provider/]

Computing Advent Calendars

2011-12-19T16:39:00.001Z

I have always been a sucker for Advent Calendars on computing topics. I find them a convenient way to catch up on developments that I would have otherwise missed, and they come out at a time when, with luck, things are a bit quieter so there's time to actually read them.

This year I've been enjoying SysAdvent. In a previous life when I did a lot of Perl development I was a great fan of the Perl Advent Calendar but I haven't read it in several years. Earlier today I came across PHP Advent which I didn't previously know about. For hard core web geeks there's always 24Ways. I'm sure there are lots more. For what it's worth I read these, and lots of other stuff, via RSS or equivalent feeds in my feed reader of choice (which at the moment happens to be Google Reader).

Most of these have been running for a while and the previous editions are also available (for some you may have to make the obvious edits to the URLs to see earlier versions). But beware - just as this year's postings are often topical and cutting edge, ones from the past can easily turn out to have been overtaken by events or simply no longer sufficiently fashionable.

However you celebrate it, I'd like to wish you a happy Christmas and a productive new year.

Cookies - progress at last?

2011-12-19T12:14:00.001Z

It looks as if there may at last be some progress on the vexed issue of how you can continue to use cookies (and similar) in the light of the new(ish) Privacy and Electronic Communications Regulations. The ICO has published new guidance on all this, and various commentators seem to think it looks promising:

There is however some concern that the proposed approach to dealing with analytics cookies seems to be limited to a hint that the ICO's office just won't look too closely. There's worry that some public sector organisations will still feel that they need to ensure total compliance rather than just take a risk management approach:

A crack in the cookie craziness?

Federated (and so SAML, and so Shibboleth) account management

2011-11-27T18:33:00.000Z

The Problem

In an earlier post (User-ids in a Shibboleth world) I described the problem of deriving a 'userid', as required by much third-party software, from typical attributes supplied in a SAML federated authentication (particularly in the UK federation, but probably equally elsewhere in Europe). This is actually part of the wider problem of account management in a federated environment (as touched on in Federated AUTH is less than half the battle), and this posting is an attempt to pull together some thoughts on the subject.

The context of this is a need to support web applications (for example MediaWiki, Plone, or Wordpress) which need to identify local users (members of the University of Cambridge) but which also need to support authenticated access from people outside the University (collaborators in other institutions, sponsors, representatives of funding bodies, etc.). Many of these have access to SAML identities in the UK federation (or elsewhere), and all three applications claim to have 'Shibboleth support', but in practice this support doesn't come anywhere near doing what we need. As mentioned in User-ids in a Shibboleth world, the primary problem seems to be an assumption that eduPersonPrincipalName (or something equivalent) will always be available. We can (and do) make this available from our own IdP to our own SPs, but we can't expect it from elsewhere, and negotiating its release on a case-by-case basis is impracticable and in any case we don't actually need it.

I asked about this issue on both the jisc-shibboleth@jiscmail.ac.uk and shibboleth-users@internet2.edu mailing lists. I had rather assumed this would be a well known problem with obvious solutions, but the only significant response was from Scott Cantor who said:

"I think they're written for "least effort" and to avoid fixing applications. That's not a strategy that will work long term."

The Options

So, what might work in the long term?

I think you first have to accept that this sort of federated environment differs from a typical single institution environment in a least two significant ways:

You will get a unique identifier for each user, but you must assume that it's totally opaque and that it's not suitable for display. Further, you have to assume that users will not be able to tell you in advance what this identifier is or is going to be.

Lots of people will be able to authenticate who you don't want to have anything to do with. So you can't rely on the 'If they can authenticate they are part of the organisation' approximation that works in other situations.

I see two approaches: Account Linking, and Account Creation:

Account Linking

Under this approach you start by provisioning ordinary, local accounts complete with locally-unique (and displayable!) user names and passwords and by transmitting the username/password information to the intended user in any appropriate fashion. But once the intended user has authenticated with this information you allow them to link this account with a federated identity which they can subsequently use to authenticate in place of the local password. You could even regard the initial password as just a bootstraping device and expire it after a few uses, or after time, or after a federated authentication has completed.

Anyone trying to authenticate using an unrecognised federated identity would get an error page refering them to whoever deals with account management. If accounts need non-default attributes (particular editing rights, administrator privilege, etc) then these can either be set manually by administrators or they could be added following a successful federated authentication with particular attribute sets.

Depending on the application you are protecting you might choose to restrict which federated identities you allow to be linked, based either on who asserts them and/or on attributes supplied with the authentication (i.e. you might require that a 'staff' resource can only be linked to an identity that includes an eduPersonScopedAffiliation value including 'staff'). You might also allow more than one identity to be linked to a single account. It all rather depends who has the strongest interest in the account being used only by the right person (you as application operator, or the user).

Account Creation

This is to some extent the process envisaged in the latter sections of Federated AUTH is less than half the battle. However software that needs a local 'user name' will have to create one that meets its own requirements when first creating a local account, and it can't assume that anything suitable will be available in the attribute values. This means that the 'Dynamic login' approach is going to be problematic since there probably won't be anywhere to store the association between this user name and the corresponding federated identity.

As mentioned in the other post, some rules will be needed, presumably based on attributes supplied, to decide who can have an account at all. Anyone trying to authenticate using a identity that doesn't match these rules would again get an error page refering them to whoever deals with account management. Enhanced account privilege can be automatically granted based on attribute values or, once the user has authenticated for the first time (but probably not before), by manual intervention by administrators and by reference to the locally generated user name..

Hybrid approach

There are almost always exceptions to rules. To allow for this you might want to adopt a hybrid approach. This might uses the Account Creation approach for the majority of potential users, because it totally avoids 'one more password' and needs little on-going user administration. But for the inevitable cases where you need to allow access by someone who doesn't match the standard account creation rules you could fall back to the Account Linking approach. Indeed, once you allow local accounts you have a tool that can if you wish use to allow access to people who don't themselves have access to suitable federated identities.

All this is quite a lot more work than just lifting a string out of something like a REMOTE_USER environment variable and convincing you application to treat this as evidence of authentication. But to get the best from federated authentication it's what we are probably going to have to do, though in time and with luck we may find that someone has already done it for us...

So how are we getting on with cookies?

2011-11-24T10:00:00.001Z

Remember all that fuss earlier in the year about the new cookie rules? Well, it's six months since the ICO's office gave us all a year's grace during which they didn't expect to actually be enforcing the new regulations. And how are you all getting on with becoming compliant? Worked out how to legally continue to use Google Analytics yet? Scheduled the redevelopments needed for compliance?

While things may be quite now, I confidently expect another flurry of publicity, and difficult questions, in the spring by which time it's going to be too late. Here's one recent blog posting on the subject:

http://www.storm-consultancy.com/blog/misc-web-stuff/the-cookie-monster/

301 Moved Permanently (to a new office)

2011-08-10T10:10:00.001+01:00

After 10 years and 11 months sitting at the same desk in the Computing Service I've finally ended up moving office (to P24 if you know your way around the UCS, otherwise I'm still as unfindable as I've always been). This is part of a plan to make space for a new member of the Information Systems Team who starts in a couple of weeks, and fulfils my ambition to get the entire team at least on the same floor, if not in the same office. But it's still a bit of a wrench and I'm worried that, if truth be told, all my best ideas may have been stolen from my former officemates...

The space left by my departure:

...and my new home:

IT Support from 300 ft

2011-07-06T08:25:00.002+01:00

At the (Cambridge) College IT Managers' 2009 Conference at the Imperial War Museum, Duxford I was lucky enough to win a flight in a tiger moth in the prize draw. I've been a bit slow getting around to taking this up, but finally got around to it last weekend. I got to fly from Duxford up to Madingley and back, and even got to fly the plane a bit myself (no doubt with the pilot's hands not far from his set of controls). Initial impressions: Tiger Moths are really small and feel as if they are made out of Mecarno, you can't see where you are going, and in the air they feel as if they are blowing around like a piece of paper! Flying the plane feels a bit like sailing, but in three dimensions rather than two (and, at least last Sunday, with less water).

Here's some photographic proof of the event. Thank you CITMG!

IPv6 Day in Cambridge - Success and Non-event!

2011-06-18T08:42:00.000+01:00

IPv6 day (8th June) in the University was largely a non-event, and so can be declared a success! This seems to match experiences reported elsewhere.

We are not aware of any significant problems experienced by University users in accessing any services on the day, including external ones known to be participating in IPv6 day such as Google and Facebook. The core of the University network is connected to the global v6 internet, but most distribution networks in departments and colleges only connect using v4 at the moment. Those networks that are connected to the global v6 internet (including the one connecting my desktop workstation) worked fine on the day as expected.

In the run-up to the day we enabled v6 on a number of central services, including the main University web server, the Streaming Media Service (both the web interface and the HTTP download service), the 'new' interface to our search engine, the University Training Booking System, and the central mail service (SMTP, POP, IMAP). On the day we published AAAA records for these services alongside the normal A records from about 08:30 to 19:00 BST.

With the exception of the web server, all these services were enabled more or less as they would be for a v6 production service, though a few features (such as automatic v6 address transition between cluster members, and adapting log analysis to recognise v6 addresses) were not completed in time. The web server used a seperate Apache reverse proxy to provide v6 connectivity to avoid having to disturb its configuration. While doing this, and subsequently, we identified various issues and surprises that I've already mentioned (here, here, and here).

The University web server received 8,981 requests from 280 distinct clients over v6. By comparison it received a total of 1,257,012 requests over both protocols for the entire 24 hour period, meaning that v6 requests probably represented about 1.5% of the total. The breakdown of 8,351 native v6 requests from 230 clients by approximate country of origin appears in the table below.

What was interesting was the relatively high number of clients (50) making requests over transitional 6to4 connections (630). Most of these (36 clients making 476 requests) were from inside the University. Most or all of these clients will have had perfectly good native v4 connectivity to www.cam and this confirms (if confirmation were needed) that rather a lot of systems prefer IPv6, even if provided by a transition technology such as 6to4, over IPv4. Interestingly we didn't see any Teredo traffc.

6to4 caused the only significant incident of the day, when a department mail server switched to using IPv6 over a 6to4 route being advertised by a user workstation elsewhere on the department subnet. This mailserver sends all its outgoing mail via the University's central internal mail switch, but that won't accept messages from machines with 6to4 addresses because it doesn't see them as 'inside'. The problem was quickly fixed, but it seems clear that, ironically, problems caused by 6to4 and Toredo 'transitional' connectivity may represent a significant barrier to further IPv6 roll-out.

Native IPv6 requests to http://www.cam.ac.uk/ on IPv6 Day, by approximate country of origin

[Here 'UCS STAFF' represents clients on the Computing Service staff network, 'UNIVERSIY' represents those elsewhere in the University, 'JANET' those elsewhere on JANET, and 'United Kingdom' those elsewhere in the UK].

2619 UCS STAFF
1373 China
1290 Brazil
   835 JANET
   630 UNIVERSITY
   420 United Kingdom
   293 United States
   171 Greece
   123 France
   110 Czech Republic
    97 Russian Federation
    81 Germany
    66 Japan
    48 Portugal
    47 Netherlands
    36 Finland
    33 Canada
    33 Serbia
    17 Spain
     7 Switzerland
     6 Ireland
     5 Saudi Arabia
     3 Hong Kong
     2 Italy
     2 Korea, Republic of
     2 Norway
     1 Australia
     1 New Zealand
Geolocation provided by Maxmind's free GeoLite IPv6 Countrydatabase. "This product includes GeoLite data created by MaxMind, available fromhttp://www.maxmind.com/."

More IPv6 gotchas

2011-06-13T09:34:00.001+01:00

Our participation in IPv6 day (which I might get around to writing up one day) has lead me to identify three more 'gotchas' relating to IPv6 deployment:

IPv6 tunnels come up outside the wire

As predicted in advance, and born out by our experience on the day, it's clear that lots of clients will use transitional IPv6 connectivity (6to4 or Teredo) even when contacting services also available over native IPv4. Worse, some machines with 6to4 connectivity will advertise themselves as IPv6 routers and other machines on the same subnet will use their connectivity in preference to native IPv4.

In addition to the obvious problem that this transitional connectivity may be broken, or blocked, or massively sub-optimal, there the additional unexpected (to me) problem that machines doing this will be using 6to4 or Teredo IP addresses (2002::/16 or 2001:0000::/32 respectively) and so will appear to be outside you local network even if they are actually inside. This has serious implications for continued attempts to do access control by IP address.

Both addressing schemes actually embed local IPv4 addresses in the v6 addresses they use so you could - perhaps - choose to recognise these. But if you do you'll be in the interesting position of having 'internal' traffic coming into your network from the outside!

Fragmentation

IPv6 doesn't support packet fragmentation by routers, but does require that a sender reduces its packet size and retransmits in response to an ICMP6 type 2 'Packet too big' message. If this mechanism fails, perhaps because ICMP packets are being blocked but also for any other reason, you may find for example that users can connect to a web site but not get any content back.

This is because the initial connection establishment and HTTP GET request all use small packets but everything goes wrong the moment the web server starts sending full packets containing the data requested. Unhelpfully, web server access logs may look fine when this happens, with the only hint of problems being that too few bytes may have been transmitted (though given a big enough TCP window and a small enough document even this may not be obvious).

Old software

Even though IPv6 has been around for a while, support for it is still missing or broken in a lot of software (especially if you use 'stable' or 'Long Term Support' Linux distributions whose versions will inevitably be somewhat less that 'bleeding edge').

For example even though the SLAPD LDAP daemon supports IPv6, my colleagues failed to find a way to get the version included in SLES 10 to support both v4 and v6 at the same time, though it was happy to do one or the other. In addition, this version didn't seem to support IPv6 addresses in its access control list syntax.

I also had a problem geolocating the IPv6 clients that accessed our web server. The geolocation database I normal use (the free GeoLite Country and friends from Maxmind) does support IPv6, and the version of their C API supplied with the current Ubuntu LTS (10.04 Lucid Lynx) is just new enough (1.4.6) to cope. But the versions of the Perl and Python bindings needed to process IPv6 both need 1.4.7 of the C API, and since the library is used by quite a lot of Ubuntu utilities upgrading it isn't trivial. In the end I had to build a private version of the C API and the Perl and Python bindings but that was one more bit of work I wasn't expecting.

IPv6 day - more problems than expected?

2011-06-04T11:20:00.000+01:00

A couple of posts on the JANET Development Eye blog:

together with links from them to some useful pages on the ARIN wiki suggest that rather more people may experience problems on IPv6 day than I had perhaps previously expected. The main problem, ironically, seems to be the widespread deployment by default in many OSs and networks of workarounds intended to provide access to IPv6-only resources from machines with only v4 connectivity. The problem is that these workarounds are often broken, or blocked, or massively sub-optimal, but that applications may still try to use these in preference to v4 even when accessing dual-stack services.

Really worrying is that measurements by Google suggest that many University networks, with their 'light-touch' approach to regulating network-connected devices, may be badly affected by all this. I suppose we will see on Wednesday!

More cooking with cookies

2011-05-26T11:29:00.003+01:00

Sorry, this blog isn't intended to be all about cookies. I'm not even very interested in them but this whole EU regulation thing just keeps growing and this is as good a place to keep notes about it as anywhere.

The Department for Culture, Media and Sport has published an open letter on the subject:

http://www.dcms.gov.uk/images/publications/cookies_open_letter.pdf

This seems to be trying to make some interesting assertions, including the idea that consent doesn't always need to be obtained in advance (based on the subtle difference of some instances being preceded by "prior" and some not). It does seem to say that whatever you do they don't think that relying on current browser cookie controls is enough.

The other interesting development is the addition of a banner to the ICO's web site that tells you about its cookie use and allows you to opt into more cookies (and incidentally get rid of the banner):

http://www.ico.gov.uk/

(Hint: if you want to go back to the default state after you've accepted their cookies then delete the cookie called ICOCookiesAccepted, or all cookies from www.ico.gov.uk).

I think they are implementing this in their CMS and so serving different versions of their pages to different people, so an identical approach won't work on a purely static site. Note BTW that this means they have to disable caching which may have performance and load implications. I have an idea that a pure JavaScript approach might be possible for a static site, but my JavaScript isn't up to it! I'm rather hoping that Google might provide a canned solution along these lines for Analytics.

Further thoughts on cookies (or lack thereof)

2011-05-24T14:25:00.003+01:00

The amendments to The Privacy and Electronic Communications (EC Directive) Regulations 2003 as they affect the use of cookies have now been published in The Privacy and Electronic Communications (EC Directive) (Amendment) Regulations 2011. What's surprising (to me) is how little has actually changed, though obviously a small change to legislation can clearly have a wide effect. What also fooled me is that if you don't read the ICO's guidance carefully (as I didn't) you might (as I did) think the change was bigger than it was. For example I thought paragraph (4) was new, and it isn't. In fact it hasn't changed at all.

One interpretation of all this is that many/most sites may already fail to comply with the 'old' regulations. Were you really giving subscribers "the opportunity to refuse the storage of or access to that information"? ... because if you were it should surely be trivial to turn the tests around and get consent instead. You could argue that all that's happened now is that, by requiring consent, the new regulations have made contraventions more obvious.

For your delight, here is paragraph 6 from the original regulations with (if I've got it right) the new amendments applied to it (removals struck through, new text underlined):

Confidentiality of communications

6.—(1) Subject to paragraph (4), a person shall not ~~use an electronic communications network to store information, or to~~ store or gain access to information stored, in the terminal equipment of a subscriber or user unless the requirements of paragraph (2) are met.

(2) The requirements are that the subscriber or user of that terminal equipment—

(a) is provided with clear and comprehensive information about the purposes of the storage of, or access to, that information; and

~~(b) is given the opportunity to refuse the storage of or access to that information~~

(b) has given his or her consent

(3) Where an electronic communications network is used by the same person to store or access information in the terminal equipment of a subscriber or user on more than one occasion, it is sufficient for the purposes of this regulation that the requirements of paragraph (2) are met in respect of the initial use.

(3A) For the purposes of paragraph (2), consent may be signified by a subscriber who amends or sets controls on the internet browser which the subscriber uses or by using another application or programme to signify consent.

(4) Paragraph (1) shall not apply to the technical storage of, or access to, information—

(a) for the sole purpose of carrying out ~~or facilitating~~ the transmission of a communication over an electronic communications network; or

(b) where such storage or access is strictly necessary for the provision of an information society service requested by the subscriber or user.

Not all that different, is it?

IPv6 Gotchas, and 8th June 2011

2011-05-19T08:51:00.002+01:00

IPv6 has been 'imminent' for a very long time. However it's shortly going to have a bit of a boost. 8th June, 2011 is 'World IPv6 day' when various big players (Google, Facebook, Yahoo!, Akamai, ...) will enable IPv6 access to their services alongside the existing v4 access as a world-wide test flight for a wider deployment. In Cambridge, some of the services run by the University Computing Service will be participating.

For almost everyone this will make no difference at all - if, like most people, you only have have access to IPv4 then you will just go on using that as you always have. Likewise if you have working IPv6 you should see no difference even though you'll be using it rather more than normal. But if you have broken IPv6 support you are likely to have problems on the day. If you want to check, the test pages at http://test-ipv6.com/ and http://omgipv6day.com/ seem to be useful, and the University's My IP page will at least tell you if have IPv6 connectivity with the University.

It's possible to think of IPv6 as just IPv4 with longer addresses, but there are enough differences and gotchas to make life difficult for both clients and servers. Here are some that we've identified:

Auto-configuration. IPv6 natively supports auto-configuration, so if you connect most (recent) computers to an Ethernet that includes an IPv6 router the computer will acquire an IPv6 interface complete with address and default route without you having to do anything. A computer with an active IPv6 interface will normally try to use IPv6 when talking to a service that advertises a IPv6 address, so you may find ourself using IPv6 without knowing it. This has a couple of exciting consequences:

If the IPv6 router you contacted isn't actually connected to the v6 Internet then your traffic isn't going to go anywhere useful. Your computer will probably fall back to using IPv4, but probably after a long (multiple tens of seconds) delay. This is going to look to users as if the internet is going very s l o w l y. It's rumoured that some versions of Windows will under some circumstances spontaneously advertise themselves as IPv6 routers.
Even if you are successfully using v6 (knowingly or otherwise), unless you do something your address won't be in the DNS. So looking it up won't result in a .cam.ac.uk (or whatever) host name, and services that make access control decisions based on client host name aren't going to recognise you. [Even if the service makes decisions based on address rather than host name it will get things wrong if its access control lists haven't been extended to include the necessary IPv6 range(s)].

If a service advertises a v6 address but doesn't actually respond to requests on that address then again there could be a longish delay before your computer falls back to trying IPv4.
It's entirely possible (though probably not a good idea!) for a service to behave completely differently when accessed over IPv6 from how it does over IPv4. For a start, the process of resolving host names to addresses is entirely separate for IPv4 and IPv6 so there's no particular reason for the two addresses to end up on the same server. Further, web servers providing virtual hosts need a mapping between IP addresses and the corresponding virtual hosts and it's all to easy (as the operators of www.ja.net found some time ago) to forget to extend this mapping to include v6 addresses, with the result that v4 and v6 users end up seeing the content for different virtual hosts when requesting the same URL.
The IPv4 address 127.0.0.1 always corresponds to the local computer and conventionally has the name localhost. It's quite common for components of a server to communicate using this address (e.g. a web application and its database), and equally common to actively restrict communication to this address or name. The coresponding v6 address is '::1' - if this doesn't correspond to the name 'localhost' or if access lists only recognise 127.0.0.1 then, if IPv6 is enabled on a server, it may find that it can't tall to itself!
There are a lot of programs out there, in particular log analysis programs, that implicitly expect IP addresses to look like 131.111.8.46. Such programs may be 'surprised' to come across addresses like 2001:630:200:8080::80:0. How they behave will depend on how well they have been written, but in some cases this may not be pretty. Further, note that library calls that lookup v4 addresses to find the coresponding host name may not work with v6 addresses.
Firewalls and packet filters will need IPv6 configurations to match their IPv4 ones, and they are unlikely to be able to automatically derive one from the other. So, by default, they are likely to either block all IPv6 traffic or allow it - neither is likely to be what's wanted.
A lot of networks (especially home ones) use RFC 1918 'private' addresses and network address translation (NAT) when talking to the wider Internet. While not intended as a security measure, this does somewhat shelter machines on such network from active attack from the Internet. Use of private addresses and NAT are a response to the shortage of IPv4 addresses which isn't a problem in the IPv6 world, and so they are not widely supported (if at all). So enabling v6 on a previously 'private' network may expose previously hidden security vulnerabilities to the world, which may be unfortunate.

IPv6 is probably finally coming. Some of us may get an early brush with it on 8th June. It really is time to start thing about the consequences.

Why you shouldn't give away your password

2011-04-14T08:34:00.003+01:00

So, you want to embed details of you Twitter, or Facebook, or <insert flavour of the month here> account on you blog. There's a widget that will do this (in fact the widget is the only way to do this) and 'all' you need to give it is you user name and password. You are using a blog provided by a big well known provider so of course you can trust them not to misuse your credentials. What can possibly go wrong?

This:
http://techcrunch.com/2011/04/13/hacker-gains-access-to-wordpress-com-servers/

User-ids in a Shibboleth world

2011-03-15T10:44:00.000Z

[This is a copy of a question posted to the JISC-SHIBBOLETH@JISCMAIL.AC.UK mailing list which, to date, hasn't seen much in the way of responses to the questions at the end]

A lot of existing web apps have wired into in the, often at a very low level, the idea that each 'user' has associated with them a unique, short-ish, fairly friendly identifier. Traditionally these were things like jw35 or 2006STUD42, though it's increasingly common to use an email address belonging to the user instead. Quite often these are (in fact or in practise) the primary key into the application's user database and they tend to get displayed (as in "Hello jw35", or in things like usage logs) and these two usages are often not distinguished.

When converting existing software for a SAML environment there's the question of what to use to fill this slot. If you have ePPN available then that will probably work (because it looks like an email address) but in the UK federation that's unlikely. The 'old' form of ePTID (rUL8A3M667VfsiCImQVFffN9cNk=@cam.ac.uk) might just about work, though it's ugly when displayed. The new form is close to unusable for display purposes:

https://shib-test.raven.cam.ac.uk/shibboleth!https://mnementh.csi.cam.ac.uk/shibboleth/ucam!rUL8A3M667VfsiCImQVFffN9cNk=.

Of course if you are creating accounts in advance and only later binding them to SAML identities you can use whatever user-id scheme you want, but what if you are creating accounts 'on the fly'?

We've noticed that 'off the shelf' Shibboleth adaptations (most recently for Plone and MediaWiki) tend to simply use what turns up in REMOTE_USER which, by default, will be the first of ePPN and the old and new forms of ePTID that isn't blank. In a UK federation context this doesn't really work, and I suspect many of these adaptations were written for contexts where ePPN is more widely available.

What have other people working in the UK environment done to address this problem, assuming you are seeing it too? Is there a 'best practise'?

Consent to be required for cookies? (updated 2011-03-15 and 2011-05-11 and 2011-05-17)

2011-03-14T11:24:00.007Z

An amendment to the EU Privacy and Electronic Communications Directive comes into effect on 25 May 2011. This makes it necessary to obtain consent before storing and retrieving usage information on users’ computers. Details are unclear, mainly because the UK government has apparently yet to publicise any information on how this requirement will be implemented in the UK, but in the limit it could require every web site to obtain informed user consent before setting any cookies, which is going to be interesting...

Here are some external references to the topic:

Updated 2011-03-15: More links

BBC News
Sitepoint
David Naylor (with helpful demonstration of best practice :-)
Andrew Cormack's blog
Out-LAw.com (again)

Updated 2011-05-11:

Guidance from the Information Commissioner's Office (2011-05-09)
DCMS Press Release (2011-04-15)

Updated 2011-05-17:

Andrew Cormack's blog (again) (2011-05-12)

Other people's content shown to be dangerous

2011-03-02T11:26:00.001Z

In Promiscuous JavaScript considered dangerous I said that including content from elsewhere on your pages was dangerous, not only because the people supplying the content might be malicious but also because they might fail to prevent third parties from injecting malicious content.

Judging by this BBC News article this is exactly what happened recently to the web sites of the London Stock Exchange, Autotrader, the Vue cinema chain and a number of other organisations as a result of displaying adverts provided by the advertising firm Unanimis. This will have caused problems for these various organisations' clients, and reputational damage and hassle for the organisations themselves.

Ideally you'd carefully filter other people's content before including it in your pages. But you may not be able to do this if, for example, the supplier requires you to let everything through untouched or if you are using the promiscuous JavaScript approach. In such cases you are entirely dependent on the competence of the supplier and, as demonstrated here, some are more competent than others.

Google Apps: SSO and IdM at #GUUG11

2011-02-17T10:20:00.001Z

Here are some slides from a presentation I gave on Google Apps SSO and Identity Management at the inaugural 'Google Apps for Education UK User Group' meeting at Loughborough University on 15th February 2011:

Google Apps - SSO and Identity Management at the University of Cambridge

View more presentations from Jon Warbrick.

Loops in PKI certificate hierarchies and Firefox bugs

2011-02-09T17:52:00.003Z

In a previous posting, I mentioned being surprised to discover that PKI certificate hierarchies were more complicated than the strict trees that I had always assumed them to be. At the time I rather assumed that they must be directed acyclic graphs.

I've subsequently realised that there is nothing to prevent them from being cyclic graphs and have actually found a loop in an existing commercial hierarchy. Unfortunately it looks as if I wasn't the only person making false assumptions since it seems that certificate loops trigger an old-but-not-yet fixed bug in Firefox that prevents certificate chain verification.

Certificates contain within them the 'Distinguished Name' (DN) of a further certificate that contains the public half of the key used to sign the first certificate. Certificate are only identified by name, and there is nothing to stop multiple certificates sharing the same name (though all the certificates with the same name had better contain the same key or new and exciting bad things will probably happen). All this is what I worked out last time.

What I've discovered that's new is illustrated in the following diagram:

This represents part of the hierarchy under which certificates are issued to UK HE institutions by JANET (and I think by similar organisations in other countries) under a contract with Comodo. In this diagram:

The blocks with grey backgrounds represent key pairs. The numbers at the top of the box are the first half of the key's 'Key Identifier'.
The smaller blocks represent certificates containing the corresponding public keys.
The arrows link certificates to the keys that signed them (and so which can validate them).
Certificates with red backgrounds represent self-signed certificates that are trusted roots (at least on my copy of Firefox). The certificate with blue background represent an example server certificate.
The number in each certificate is the first half of the certificate's SHA-1 hash.
Certificates with a green border represent the recommended verification chain for JANET-issued certificates.

Copies of the certificates involved (and others) can be found here.

The problem here are the two certificates "31:93:..." and "9E:99:...", since each represents a potential verification route for the other. Neither are part of the 'official' verification chain for JANET-issued certificates. "9E:99:..." is distributed by Comodo in support of other of their certificate products. I don't know where "31:93:..." comes from, but I assume it appears in someone else's 'official' certificate chain. Both these 'intermediate' certificates will presumably be included in certificate bundles by particular web servers and, once serverd, tend to be cached by web browsers.

The problem is that, once a web browser has a copy of both of these there's a danger of it going into a spin since each is an apparently acceptable
parent for the other. It turns out that Firefox has exactly this problem, as described in bug 479508. Unfortunately this bug last saw any action in March 2008 so it's not clear when, if ever, it's going to be fixed. There are some other reports of what I suspect is the same problem here and here.

So who's problem is it? Clearly Firefox could and should be more careful in how it constructs certificate chains. It's possible that other SSL software is vulnerable to similar problems, though I've only seen this manifest in Firefox (and only then occasionally). But I also wonder what the Certification Authorities thought they were doing when they issued these certificates. As far as I can see they were both issued as 'cross-certification' certificates, intended to allow server certificates issued under one root certificate to be validated by reference to another. Issuing one of these isn't a problem. Issuing a pair clearly is.

A work around, should this problem bite you, should be to delete "31:93:..." and "9E:99:..." from Firefox's certificate store. Neither are roots, and any server that needs them to get its certificate verified should be providing them, so deleting them should be entirely safe. The work-around should last until you next pick up copies of both of these, at which point you'll need to delete them again.

Root certificates for MacOS OpenSSL

2011-02-01T08:21:00.001Z

In an earlier post I mentioned that, while MacOS includes OpenSSL it isn't preconfigured with any trusted root certificates. So before you can use it to do SSL properly you need to provide a set.

My previous post suggested extracting them from the bundle that comes with Firefox, but I've recently come across a useful article about Alpine on MacOS by Paul Heinlein <heinlein@madboa.com> in which he points out that the MacOS operating system already has a set of preconfigured roots and that these can be extracted using the Keychain Access utility for use by OpenSSL. See his posting for details, but to quote from it:

Open the Keychain Access application and choose the System Roots keychain. Select the Certificates category and you should see 100 or more certificates listed in the main panel of the window.
Click your mouse on any of those certificate entries and then select them all with Edit → Select All (Cmd+A).
Once the certificates are all highlighted, export them to a file: File → Export Items…. Use “cert” as the filename and make sure “Privacy Enhanced Mail (.pem) has been chosen as the file format.
Copy the newly created cert.pem into the /System/Library/OpenSSL directory

Now, I wonder why Apple didn't do this for us?

Thoughts on "Initial authentication"

2011-01-29T16:00:00.000Z

"If it was easy we'd have already done it!"

I've recently contributed to some work on 'Password management". Here are some of the thoughts that I've managed to condense into words. Don't be disappointed - they raise at least as many questions as they answer.

1) Looking say five years ahead I think we need to talk about 'authentication' and not just 'passwords'. Given their vulnerability to keyboard sniffing it seems to me that within five years it will be necessary at least to support (though perhaps not require) some sort(s) of non-password based authentication for some systems.

2) While there will be pressure do so, it might be better not to try to solve everyone's problems. For example in some cases it might still be best to create a shared password system for use only on a limited set of systems and not allow anyone else to use it.

3) The understandable enthusiasm for SSO by some is at variance with an equal enthusiasm, in some cases promoted by the same people, for aggressive inactivity timeouts on individual systems. Meeting everyone's individual security requirements may result in an unusable service.

4) The group developing HTML5 have adopted a policy that says "In case of conflict, consider users over authors over implementers over specifiers over theoretical purity". It might be sensible to adopt something similar in this area (suitably adapted). Or perhaps not... Discuss.

5) A critical feature of any authentication system is who is able to reset its authentication credentials (i.e. reset passwords or equivalent), because they (all of them) can subvert the security of the systems (all of them) that use it. It looks to me to be difficult to simultaneously meet the expectations of people who would like easy local reset of passwords and the operators of some 'high risk' systems who want tight control of access.

6) Given the existence of a central password verification service overloading existing protocols (LDAP, Radius, ...), I don't see any technical way to restrict the clients that can use it since clients don't identify themselves. Such use could be restricted by rules, but they would be hard to enforce, and by non-100% perfect technical restrictions (e.g. client IP address filtering). So anyone providing such a service will have to accept that in practise it will be open to anyone to use. You could implement a central password verification service using something like SAML, where clients are strongly identified, but then there wouldn't be any clients to use it.

7) Accepting that we'll at least have to accept passwords for the foreseeable future (even if we accept other things in parallel), then the not-unreasonable idea that people will only willing accept using two passwords restricts us to a maximum of two authentication systems. So how about:

a) A 'low trust' service verifying a single password over one or more commonly used protocols (so LDAP, RADUIS, TACAS), intended for use in situations where we can't do better (3rd party service that can only do LDAP auth, WebDAV service that has to work with common clients that can only do username/password, etc).Document that this is a low trust service, that server operators can intercept passwords in transit, etc. Require good practive as a condition of using the service - don't ship passwords over unsecured networks, don't write them to disk, etc. Perhaps make token attempts to restrict client access (e.g. by IP address) but accept and document that this won't be perfect. This violates all my prerequisites for secure use of passwords, but perhaps on balance doing so is necessary to support what needs to be supported.
b) A 'higher trust' service where credential disclosure is limited to local workstations and a logically single central server. Web redirection based protocols (i.e. Ucam_WebAuth, Shibboleth) and Kerberos (and so Windows AD) meet this requirement and provide at least some single sign-on. Web redirect, and perhaps Kerberos could both use things other than a password for initial authentication. SPNEGO holds the possibility of transparently transferring a pre-existing Kerberos session into a web redirection-based system thus widening SSO for existing Kerberos users but leaving open password or other authentication for access to web systems in situations when Kerberos isn't available.

Item (b) violates my third prerequisite for secure use of passwords ("It must be possible for password holders to decide when it is safe to divulge their password"), but I'm coming to the conclusion that the price of adhering to this principle does not warrant the cost.

8) If you can't stomach the single shared password idea, an alternative might be a 'Managed Password Service' that extended the 'token' (in the current University Computing Service sense) idea by centrally managing multiple password sets (one for each 'system'). So the administrator of a new system somewhere could mint a new password set for their system and configure their system to do password verification against it using LDAP, RADIUS or anything else supported. Users could set and reset these passwords under the authority of their web redirect/Kerberos credentials. The end-system would have to it's own authorisation since in principle anyone could create a password for any system. This doesn't give 'two passwords', but it does at least allow one password to manage all the others.

Microformats (lots of Microformats)

2011-01-12T14:13:00.002Z

I've wanted to play with microformats for some time. The need to rework my (still somewhat minimal) work home page provided an ideal opportunity. To see the effect you'll need a browser plug-in such as Operator for Firefox, or to use something like the Google Rich Snippits testing tool.

Essentially microformats (and their friends - see below) provide a way of marking up HTML content with additional semantics to allow automatic parsing that wouldn't otherwise be possible. For example a human would know this this supplies my telephone number:

<p>
Jon Warbrick<br />
Tel: +44 1223 337733
</p>

but if I mark it up like this

<p class="vcard">
<span class="fn">Jon Warbrick</span><br />
Tel: <span class="tel">+44 1223 337733</span>
</p>

then microformat-aware processors should be able to reliably extract the number and associate it with my name - and then perhaps use this to create a contact list entry or put a call through to me. Similar microformats exist for events, reviews, licences, etc.

It turns out that there are (at least) three different, competing microformat-like systems out there:

Microformats
The original offering in this area. Aims to add semantic markup for various classes of 'thing' to standards-conforming HTML 4.01/XHTML 1.0. It largely does this using HTML structure and a range of pre-defined class names.

RDFa
("Resource Description Framework in attributes") defines a set of attribute-level extensions to XHTML which make it possible to add semantic markup using RDF syntax.

Microdata
This is a (proposed) feature of HTML5 that adds semantic markup in a similar way to microformats, but using new attributes itemscope, itemprop, itemtype and itemref rather than overloading class. As an experiment I've also tried marking up my contact details using microdata.

If only Virgin Media were competent...

2010-08-09T20:50:00.002+01:00

Is it too much to ask for a broadband supplier to get forward and reverse DNS registrations for their own addresses right?

$ who jw35 pts/0 2010-08-09 19:32 (81.98.240.47) $ dig +short -x 81.98.240.47 cpc2-cmbg4-0-0-cust814.know.cable.virginmedia.com. $ dig +short cpc2-cmbg4-0-0-cust814.know.cable.virginmedia.com 81.98.243.47

Result: OpenSSH restriction based on hostname fails because the client hostname can't be established and I waste an hour trying to debug the problem.

Actually it's worse than that:

$ dig +short -x 81.98.243.47 cpc2-cmbg4-0-0-cust814.cmbg.cable.virginmedia.com. $ dig +short cpc2-cmbg4-0-0-cust814.cmbg.cable.virginmedia.com 81.98.243.47

Argh!

Update 2010-08-10: It looks as if the problem may be resolving. The authoritative name servers for 240.98.81.in-addr.arpa (ns[1,2,3,4].virginmedia.net) seem to be serving consistent results:
$ dig +short +norecurse @ns1.virginmedia.net -x 81.98.240.47 cpc2-cmbg4-0-0-cust46.cmbg.cable.virginmedia.com $ dig +short +norecurse @ns1.virginmedia.net cpc2-cmbg4-0-0-cust46.cmbg.cable.virginmedia.com 81.98.240.47

Unfortunately they serve this information with 7 day TTLs and it's going to be several more days before the bogus information if finally purged from DNS server caches.

Splits and joins in PKI certificate hierarchies

2010-05-18T15:09:00.006+01:00

I've always visualised PKI certificate hierarchies as strict trees. In this view, CA root certificates either directly authenticate end-user server and client certificates, or they authenticate multiple intermediate certificates, which may in their turn authenticate multiple further intermediate certificates, which eventually authenticate end-user certificates:

What hadn't occurred to me was that these hierarchies can also branch upward, with certificates being authenticated by more than one certificate above it:

But as it happens I've recently come across two real-live examples of this in commercial CA certificate hierarchies.

The first is in the one operated by Comodo implementing the JANET Certificate Service for UK HE sites. According to the documentation, the 'O=TERENA, CN=TERENA SSL CA' certificate chains to one ultimately authenticated by 'O=AddTrust AB, CN=AddTrust External CA Root'. But it can just as easily be verified by a commonly installed root 'O=The USERTRUST Network, CN=UTN-USERFirst-Hardware'. I've no idea why it's like this.

The second is in a hierarchy operated by Thawte. Here they are introducing a new 2048-bit root certificate, but as a precaution have also created an intermediate certificate that chains back to the old root:

This all has interesting implications for certificate verification since there are now multiple possible paths from an end user certificate to a potential root. From a little experimentation it appears that Firefox and Safari manage to find the shortest path to a configured root, but CryptoAPI (and so Internet Explorer and most of the rest of Windows) and OpenSSL take the certificate chain as provided by the server and then try verification from the end of that without ever trying to backtrack (but see note below).

This makes it impossible to take advantage of having both roots available since, taking the Thawte case, if you include the 'O=thawte, Inc., CN=thawte Primary Root CA' intermediate in the chain then your Windows/OpenSSL clients are bound to end up attempting verification against 'O=Thawte Consulting cc, CN=Thawte Premium Server CA' (and failing if they don't have it), and if you don't they will verify against the 'O=thawte, Inc., CN=thawte Primary Root CA' root (and failing if the don't have that).

The situation isn't helped by the fact that (if I'm reading it right) the relavent RFC describes verification from root to leaf even though in practice you'll always be doing it from leaf to root.

Note: subsequent further experimentation suggests that it's more complicated. Firefox does seem to be finding the shorter path in the Thawte case, but finds the longer path in the Comodo case and fails validation if the 'O=AddTrust AB, CN=AddTrust External CA Root' is disabled. It's possible that the behaviour is influenced by other data in the various certificates.

Doing certificate verification in OpenSSL clients (properly)

2010-05-17T15:44:00.005+01:00

Many SSL-capable applications, particularly those that started life on a Unix/Linux platform, use OpenSSL to implement the SSL protocol. Amongst other checks, SSL clients are expected to verify that certificates that they receive from servers have been correctly signed by a Certification Authority (CA) that the client has been configured to trust, but doing this correctly (or at all) with OpenSSL turns out to be harder than you might think.

For a start, OpenSSL can be instructed not to bother with verification. This can seem like an easy way to get rid of annoying error messages and to make things work, but doing so makes clients vulnerable to server impersonation and man-in-the-middle attacks. Most clients do verification by default, but things like curl's -k and --insecure command line options, and Pine's /novalidate-cert option in mailbox and SMTP server definitions will suppress this. The first step towards doing certificate verification properly is to make sure you have verification turned on.

The next problem is that to verify a server certificate a client must have access to the root certificates of the CAs it chooses to trust. OpenSSL can access these in two ways: either from a single file containing a concatenation of root certificates, or from a directory containing the certificates in separate files. In the latter case, the directory must also contain a set of symlinks pointing to the certificate files, each named using a hash of the corresponding certificate's subject's Distinguished Name. OpenSSL comes with a program, c_rehash, that generates or regenerates these symlinks and it should be run whenever the set of certificates in a directory changes. All the certificates should be in PEM format (base64 encoded certificate data, enclosed between "-----BEGIN CERTIFICATE-----" and "-----END CERTIFICATE-----" lines).

[Actually it's worse than this, because the client also needs access to any intermediate certificates that are needed to construct a chain linking the servers certificate to the corresponding root. It's the server's responsibility to provide these in intermediates along with its own certificate but sometimes they don't, making verification difficult or impossible. See below for how to detect this problem]

The OpenSSL library has compiled-in default locations for root certificates. You can find out what it is by first running the OpenSSL version utility:

openssl version -d

to find OpenSSL's configuration directory. The default certificate file is called certs.pem, and the default certificate directory is called certs, both within this configuration directory. However be careful: it's not unusual to have multiple copies of the OpenSSL library installed on a single system and different versions of the library may have different ideas of where the configuration directory is. You need to be running a copy of 'version' that's linked against the same copy of the OpenSSL library as the client you are trying to configure.

The base OpenSSL distribution no longer puts anything in these locations. Debian (and so Ubuntu), and OpenSUSE/SLES 11 have a seperate package that install an extensive collection of roots. SLES10's OpenSSL package installs a small and idiosyncratic set, and OpenSSL under Mac OSX installs none at all. Worse, while the library knows about these default locations, applications have to make a concious decision to use them, and some don't - for example, wget seem to do so but ldapsearch doesn't.

OpenSSL applications generally have configuration options for selecting a certificate file and/or directory. Sometimes these are command line options (the OpenSSL utilities use -CAfile and -CApath; curl uses --cacert and --capath), sometimes they appear in configuration files (the OpenLDAP utilities look for TLS_CACERT and TLS_CACERTDIR in ldap.conf or ~/.ldaprc), and client libraries will have their own syntax (the Perl Net::LDAP module supports 'cafile' and 'capath' options in calls to both the Net::LDAPS->new() and $ldap->start_tls() methods).

However the locations are established, you'll need an appropriate collection of root certificates - at least containing one for each CA that issued the certificates on the the servers you want to talk to. It's often easiest if these are in the default locations, but you can put them anywhere as long as you tell you clients where to find them. You should of course be a little careful about this - by installing root certificates you are choosing to trust the corresponding CAs with at least part of your system's security. One approach is only to install roots as and when you need them to contact particular servers, but beware that this may lead to unexpected problems in the future if a server gets a new certificate from a new CA that you don't yet trust. In practice the easiest and probably best approach may be to use either a distribution-supplied collection of roots or to extract the root certificates from something like the Mozilla certificate bundle (see for example the make-ca-bundle utility distributed with curl). [Update 2011-02-01: see this new post for a better solution under MacOS]

So, putting this together, you need to do the following to use OpenSSL properly:

Make sure you have enabled, or at least haven't suppressed, certificate verification.
Get yourself an appropriate set of root certificates. If you add these to a certificate directory, remember to run c_rehash afterwards to recreate the hash symlinks.
If you install these certificates in one of OpenSSL's default locations and you application uses those locations then everything should work immediatly. Otherwise, add appropriate configuration to tell the application where to look.

You can test that OpenSSL itself is getting things right using the s_client utility. Run something like

openssl s_client -connect <host name>:<port> -CApath <certificate directory>

Replacing <host name>, <port>, and <certificate directory> appropriately (<port> needs to be 443 for a HTTPS server, 636 for LDAPS, etc.). Or replace -CApath with -CAfile to select a file containing root certificates. Thisactually establishes a connection to the server - you can terminate itby typing ctrl-c or similar.

If you see "Verify return code: 0 (ok)" then everything worked and the server's certificate was successfully validated. If you see "Verify return code: 20 (unable to get local issuer certificate)" then OpenSSL was unable to verify the certificate, either because it doesn't have access to the necessary root certificate or because the server failed to include a necessary intermediate certificate. Yiou can check for the latter by adding the -showcerts option to the command line - this will display all the certificates provided by the server and you should expect to see everything necessary to link the server certificate up to but not including one of the roots you've installed. If you see "Verify return code: 19 (self signed certificate in certificate chain)" then either the servers is really trying to use a self-signed certificate (which a client is never going to be able to verify), or OpenSSL hasn't got access to the necessary root but the server is trying to provide it itself (which it shouldn't do becasue it's pointless - a client can never trust a server to supply the root coresponding to the server's own certificate). Again, adding -showcerts will help you diagnose which. Once you've got OpenSSL itself to work, move on to your actual client and hopefully it will work too.

Um, and that's about all, though there is one more wrinkle. Several applications that can use OpenSSL can also use GnuTLS and/or NSS instead. This can change many of the details above. In particular, GnuTLS only supports certificate files, not certificate directories but clients using it don't always report this. As a result you can waste (i.e. I have wasted) lots of time trying to work out why something like ldapsearch is continuing to reject server certificates despite being passed an entirely legitimate directory of CA root certificates...

Webapps and HTTP Error codes

2010-04-30T11:20:00.000+01:00

Someone recently asked me

HTTP servers can return HTTP error codes. Should an application (e.g. a grails application) send back HTTP errors/codes, or in case of error, send back an HTTP code 200, and then send back some form of application error code/message ?

IMHO you should use appropriate HTTP error codes where possible, it's just not always possible.

For example, a request for a non-existent URL really should return 404, otherwise you will find search engines repeatedly indexing you human-readable error page. Likewise, our local directory returns a real 404 if you try to look at the details for someone who dosen't exist.

I'd suggest that 'server exploded horribly' should also return 500, and that broken requests could be reported as 400, but beyond this there are few response codes that are really relevant to a web application (apart from 'functional' ones like the 300 series). By all means use anything else that makes sense, but check the standard carefully - many codes don't mean what they superficially appear to mean. How you should report an 'application' error that doesn't map on to an existing code is beyond me (200 with explanation, 400 with explination?). There are also 'errors' which are not really errors - like not finding any results in a search.

Remember that it's perfectly possible to include an application error code or message in the text of a page returned with a non-200 HTTP status code. But you do have to accept that some browsers (thank you Microsoft, but also modern Firefoxes) may suppress your useful, hand-crafted message in favour of their own generic, unhelpful one. Sometimes making your error message long enough will encourage browsers to display it after all.

The problem with breadcrumb trails

2010-02-19T15:40:00.002Z

You won't find me objecting to anything in this:

http://derivadow.com/2010/02/18/the-problem-with-breadcrumb-trails/

Getting nagged by Nagios

2010-02-11T08:29:00.005Z

Nagios is an entirely usable service monitoring system - I'm aware of at least three implementations within the University Computing Service alone. There are some aspects of its design (or, I suspect, its evolution) that I don't particularly like, but all in all it's much, much better than nothing.

An important feature is its powerful and capable configuration system. As usual this is a two edged sword because you have to understand the resulting complexity to take advantage of the power. I have two things to offer that might help: some general configuration guidelines, and a diagram.

Guidelines

Keep the number and size of edits to the distribution-supplied config files to a minimum.
Arrange that related names and aliases share a common prefix (e.g. foo in what follows) so that theysort together (host names, which are most usefully based on DNS names, can be an exception tothis)
Keep to a minimum the number of distinct names created (e.g. use foo as both a contact and a contact group name and so don't create foo-contactgroup)
Note that, unlike most other 'names', service_description isn't a global name and only needs to be unique within the set ofhosts that provide it. It doesn't need and shouldn't have a foo prefix, and should be a short, human-readable, description of the service
Keep names (especially those commonly displayed in the web apps) as short as reasonably possible
Express group membership by listing group name in the individual objects, NOT by listing the individual objects in the group definition (or if you want, the other way around but be consistent!)
Use inheritance wherever possible to avoid replicating common settings
Use a naming convention so that separate groups of people can create global names without clashing
Store information belonging to each group of people in a predictablelocation (e.g. always put host information in files or directoriesstarting "host") to make navigation easier
Optimise the web-based display

The UCS's Unix Support group has further developed these guidelines into a configuration standard that helps them monitor large numbers of services with minimal configuration work and with several different groups of people maintaining the configuration. Part of the key to this is putting the configuration for each separate group of services into separate sub-directories and incorporating this information into the main nagios.cfg with 'cfg_dir'.

Object relationships

Nagios's Template-Based Object Configuration is one of its most powerful features but it's difficult to get you head around it when starting out. Here's a diagram that might help - it shows most of the relationships between the various objects:

Authentication: Think before you proxy

2010-02-10T20:00:00.000Z

There are three obvious ways to add authentication such as Ucam Webauth or Shibboleth or OpenID to a web application:

Add the authentication code to the application itself (what the Java world calls 'Application Managed Security')
Implement the authentication in the web server or container hosting the application (Java's 'Container Managed Security')
Implement the authentication in an HTTP proxy in front of the application

The proxy option has some obvious quick gains: a single proxy can protect multiple applications; it's normally easy to leverage existing 'container managed' implementations to build such proxies; using a proxy probably requires much less modifications of existing applications (sometimes none). But I'm going to argue that this approach, while appropriate and even necessary in some circumstances, should be avoided where possible. Here are some reasons:

Security: to be useful, the proxy has to pass on information (such as user name at the very least) to its downstream client and obviously has to do this is a way that can't be influenced by an attacker. About the only option (if you are sticking to HTTP for communication between the proxy and its downstream client) is by adding extra HTTP request headers. It's not easy to be absolutely sure that these headers can't somehow be injected by an attacker (by talking directly to the downstream client, by supplying them in a request and convincing the proxy to pass them on, etc., etc.).
Availability of attributes: Shib and other similar systems potentially make lots of information about the authenticating user available as attributes. Unless you arrange for your proxy to pass all of these on (and there could be lots and you might not know in advance what they all are) then your clients will loose out. There's also the problem that (in theory, if not in practise) that users are able to control what information about them is disclosed to which services. With multiple services behind a single proxy it's going to be a case of one size having to fit all.
Confused logs: clients behind proxies see all connections as coming from the proxy. This makes their logs less useful than they might be and frustrates any attempt to limit access by IP address (which is actually a bad thing anyway IMHO, but that's another story). There are ways around this, though most then suffer from the security problems mentioned above.
One more system: the proxy is one more system to build, maintain and in particular to understand. I have a theory that authentication services in some big web applications are a bit 'hit and miss' (and 'electronic journals' are big offenders here) precisely because they are big, layered applications that almost no one (or perhaps actually no one in a few cases) really understands.

I think my underlying feeling would be that a proxy is an entirely appropriate, if not quite as simple as it first appears, way to implement authentication for some particulare application, but that care should be taken not to slip into using one in every case. Like they say, once you've got a hammer everything starts looking like a nail...

Core data stores in a University context

2010-02-05T08:55:00.000Z

I've just come across what I think is a really interesting post about core data stores in a University context:

http://alexbilbie.blogs.lincoln.ac.uk/2010/02/04/core-dot-lincoln/

I haven't thought this through fully, but on first glance it seems to me that much of what's identified as needed at Lincoln would be directly relavent in Cambridge. The idea that University systems should be making interfaces to raw data available, for use by other University systems and potentially by others, has been bubbling around in the back of my mind for a while (and is something that I know is already happening in systems such as dSpace and the Streaming Media Service) and this post seems to be helping to crystalise some of those thoughts. Perhaps I'll come back to this topic in a later post.

Promiscuous JavaScript considered dangerous

2010-01-19T12:01:00.001Z

A trick commonly used to incorporate content from one web site into pages provided by another is for the provider site to make available a piece of JavaScript which inserts the required content into any page when loaded and executed. There are lots of examples - see for example Talks.cam 'Custom views', or the Met Office's Weather widgit.

This is realy neat, and has the huge advantage that it doesn't need any server-side support to impliment (typically RSS/Atom would need work on the server to collect and format the feeds) and can be deployed by just about anyone with editing access to the pages in question and the ability to cut and paste.

But it comes at a huge price. JavaScript that you include in your pages (wherever it comes from) has largely unfettered access to those pages. So it can rewrite you content, read cookies (and so steal authentication sessions), post content back to your server, etc., etc. So consider, before doing so, that adding JavaScript to your pages that is controlled by someone else amounts to giving that 'someone else' editing rights to your page. Are you happy to do that?

Of course reputable providers of this sort of JavaScript won't do anything like this deliberatly. But you do need to consider that they might do it accidently - it's particularly easy for them to mess up other JavaScript on you pages (for example the Met Office widgit above sets the JavaScript global variable moDays without any check if it's being used elsewhere). This trick is oftern used to provide feed information that may have been originally supplied to the provider site by even more third parties. There's the danger that the provider's cross site scripting protection may be incomplete, opening the posibility of unknown third parties being able to inject JavaScript into your pages, which is bad. And if you are relying on someone else's cross site scripting protection you need to know what character encoding they are doing it for and to be sure that you are setting a compatible one (this is discussed in a very helpful and to their credit very honest page supplied by Talks.cam that describes the risks of using their 'Promiscuous JavaScript' offering).

Note that I'm NOT saying that you shouldn't use this trick (as a supplier or as a provider). I am suggesting that you should be aware of the risks and make an apropriate assement of how they affect any particular deployment before proceding.

Apache configuration file layouts

2009-12-13T17:22:00.004Z

A traditional Apache configuration consists of one file (httpd.conf) that contains all the required configuration directives. However a single file is a problem for packaging systems where different packages are responsible for different aspects of Apache's operation. For them it's much easier if they can contribute one or more files containing configuration fragments and if these are then incorporated into the Apache configuration using the 'Include' directive. While convenient for the packaging system, this is less convenient for the system administrator who now finds his Apache configuration spread across multiple files in several directories. Here are a two diagrams showing the configuration file layout in two common Linux distributions - Debian (and so Ubuntu), and SLES:

Debian

Note that the mods-enabled and hosts-enabled directories contain symlinks to files actually stored in the parallel mods-available and hosts-available directories, and that commands, a2enmod, a2dismod, a2ensite and a2dissite, are provided to manipulate these symlinks. Within the mods-{available,enabled} directories, the *.load files contain the Apache configuration directive to load the module in question; the coresponding *.conf files contain configuration directives necessary for the module. httpd.conf is included for backwards compatability and to support installing 3rd party modules directly via apxs2. See the file /etc/apache2/README for more details.

SLES

The files shown in yellow boxes, all of which appear in the /etc/apache2/sysconfig/ directory, are regenerated automatically from information in /etc/sysconfig/apache2/ on Apache startup and so shouldn't be hand edited. See comments at the top of /etc/apache2/httpd.conf for more information.

This diagram is for SLES10 and Apache 2; similar arrangements were used with SLES 9 and Apache 1.3, with 'apache2' replaced by 'httpd' in filenames. In SLES 9 it was necessary to run SuSEconfig to regenerate the files based on sysconfig information.

2010-02-23: Debian diagram amended - the 'master' file was incorrectly labelled httpd.conf and should have been apache2.conf. Apart from anything else, you can't have httpd.conf including itself!

Paul Walk's 'Infrastructure service anti-pattern

2009-12-09T09:45:00.002Z

Following on from Service-to-service communication, I've just seen a excelent blog posting in paul walk's weblog entitled 'An infrastructure service anti-pattern' which makes an excellent case for how machine APIs should be used. Well worth reading.

Service-to-service communication

2009-12-07T08:36:00.001Z

The University of Cambridge's Raven service workswell enough for interactive logins using a web browser, but doesn't(and was never intended to) support non-interactive authentication, orauthentication between one service and another, rather than between peopleand services. Here's a set of suggestions for filling thisgap and for supporting general service-to-service communication - I happen to like these today but I'm making no promises that Iwon't have changed my mind by tomorrow.

For 'proxied' or non-interactive authentication on behalf on individuals I'd recommend OAuth.This is essentially a standardised protocol for establishing a tokenthat grants one service limited, delegated access in a user's name toanother service. There's a good example of how it could work in the Beginner’s Guide to OAuth – Part II : Protocol Workflow. OAuth is gaining significant traction in social networking applications.

Forservice to service communication I'd recommend SSL/TLS using mutualauthentication by certificate. Since we are assuming thatauthentication is required we should also assume that confidentiality is necessary so the protection offered by SSL/TLS seems appropriate.

Certificate trust could just beestablished bilaterally between pairs of services, but the complexityof this grows with the square of the number of services involved. Better would be to establish an in-house Public Key Infrastructure with a central in-house CertificationAuthority (CA) that could issue certificates for this purpose. Somedifficult policy decisions will be needed about who is allowed to applyfor certificates in the name of which services, but once made it should be possible to largely automate the CA by providing aRaven-authenticated web interface for certificate management. Note that these certificates would need to identify 'services',rather than just computers, so the parties to a conversation could for example be the'CS IP Register Database' and the 'Department of Important StudiesNetwork Management system'. We'd need to sort out a namingconvention. An important service provided by the CA would need to be the maintenance of a Certificate Revocation List.

Authorisation I'd leave to the services involved. Both OAuth and certificate authentication establish the 'identity' of a party in a conversation and it should be easy enough to use this identity within whatever authorisation system a service already uses. For example, Lookup could be adapted to allow certificate DNs to appear alongside user identifiers as members of groups used for access control.

Finally we need to identify protocols by which services can communicate. I suggest something lightweight and vaguely 'REST'ish. Authorities differ on what exactly REST requires, but here I just mean a basic CRUD interface carried over HTTP and mapped onto HTTP primitives PUT, GET, POST, DELETE, etc. Data should probably be serialised using simple XML, though other formats such as JSON are a possibility. Existing XML schema can be used where appropriate, for example the Atom Syndication Format can be used to represent lists (particularly search results), and the Atom Publishing Protocol is probably worth considering to support the creation and modification of resources (see Wikipedia for an introduction to Atom).

The advantage of this approach is that it provides a lightweight and technology neutral interface using tools (HTTP servers and clients) that are widely available and reasonably well understood. It even allows an amount of experimentation using nothing but a web browser. It also opens the possibility of in-browser manipulation of data, especially if results are available in JSON. Against this there's the need for an API design for each new service and the requirement for programming work at both the client and server ends. One way of supporting this is to distribute at least one example client library with each new API. An important selling point for this approach is the fact that it underpins almost all of the current 'cloud' offerings - see the Google Data Protocol, Amazon Web Services, Yahoo Social API, etc.

There are other posibilities for filling the various slots mentioned above - obvious ones being SSH to provide confidentiality and strong mutual authentication, and SOAP to provide interservice communication. I happen to think (today, as mentioned above) that the set I've listed here would currently provide the best solution. Why might be the subject of subsequent posts.

Re-using Raven's password database

2009-11-14T17:13:00.004Z

[This posting was originally published on an internal wiki in early 2009and is republished here to increase its exposure. Whatis says remains relevant today]

The University's Raven authentication system currently provides a usable authentication system for interactive, web-based applications but doesn't support non-interactive web-based applications, nor non-web ones. This excludes many potential uses: Windows/MacOS/Unix logon, IMAP, POP, SMTP, LDAP, WebDAV (and so CalDAV), etc., etc. Since Raven of necessity has a database containing username/password pairs for most people in the University, and most people know their Raven password, it is tempting to assume that extending it to support these other uses would be easy.

This post explores ways in which authentication based on 'Raven passwords' could be extended and tries to point out some of the advantages and drawbacks of each proposal. In reading this, you need to consider the security properties you might actually want from such a system (hint: it's not important that legitimate users have to provide their password before gaining access to something, what is important is that people can't realistically gain access using an identity that is not their own - these are not the same thing!). It is also important to consider who might be attempting to attack what, and how much we care about it: are we talking about a) a bored University student, b) a tabloid journalist, c) an organised criminal, or d) the American NSA, and are they trying to gain access to a) their mate's photo archive, b) the next heir to the throne's email account, c) the University financial system?

This paper only considers ways in which the existing, single Raven password might be usefully reused and ignores possibilities such as as using multiple passwords, one-time password lists, cryptographic smartcards and tokens, fingerprint readers, multi-tier authentication, etc. It also ignores two very real problems with 'static' passwords:

It is effectively impossible to prevent people giving their password away, and they do!
Users have to type their password into something - typically the workstation that they are sitting in front of. Depending on the workstation, it is entirely possible that it has been compromised, for example with a virus that installs a key logger or by a malicious or inept system administrator.

Option the first: forget passwords
Stepping back a bit, we could forget passwords and just ask people what their user-id is. This would even simplify Raven itself:

This would save people from having a password that they needed to remember, and would save the Computing Service from having to issue them. Both of these would be significant advantages. Anyone could easily set up almost any system 'protected' in this way - the hard part might actually be ''stopping'' it from asking for a password.

The obvious downside is that we'd have to trust everyone who can get to a login prompt not to lie (and for many networked services this means everyone in the entire world), and so this is rather unrealistic.

Option the second: use a single fixed password
Rather than forgetting passwords completely, we could could use a single, fixed, 'well known' password to protect all accounts. This would be easy to remember (and easy to recover if you forgot it). It would also be fairly easy to set up a system protected in this way.

Now we only have to trust all the people who ever knew the password not to lie. It is fair to assume that a 'secret' legitimately known by 55,000 (and rising) people (those who have ever had a Raven account) would not stay a secret for long so we'd still be trusting quite a lot of people not to lie, if not the whole world. It would also be next to impossible ever to change this password. Again, this is probably not a realistic option (though this doesn't actually stop people using it, though usually on a small scale - think of the codes used to arm and disarm intruder alarm systems).

Option the third: distribute a list of user name/plaintext password pairs
We could (in theory, though not currently in practice) extract a list of user names and corresponding plain text passwords from Raven, one for each user, and then distribute this list to every system that needs to authenticate users.

Users could then quote their user name and password, the system could look up the the user name and compare the offered password with the one on the list - if it matches they would get in, if not they wouldn't. Implementing authentication in this way would be fairly easy, and system administrators could post-process the list into whatever form their system actually needed.

This would avoid having to trust users not to lie - the vast majority of users would only be able to successfully authenticate as themselves.

The list itself, however, would now be a major problem. Anyone with access to it could authenticate as any user on any system relying on this authentication service. Worse, since many people use the same password in multiple places, anyone with access to the list could probably also forge authentication on entirely unrelated systems.

All of the managers of all of the systems using the authentication service would inherently have access to the list, so we would have to trust them. Even if we assume that none of these administrators would ever be actively malicious, there would still the danger that they might accidentally or recklessly leak the list (from a hacked server, on a memory stick, on a laptop left on a train, etc.). So the security of each system under this model would still depend on a whole group of people that each system's administrator has no particular reason to trust. In practice the only option would be to severely restrict the systems that could participate, to the point where such a service could never provide 'universal' authentication for the University.

Option the fourth: distribute a list of user name/'crypted' password pairs
Rather than distributing plaintext passwords, we could distribute them 'non-reversibly encrypted' (or 'hashed') - for example using the 'crypt' or 'md5' password format used in Unix password files. In principle this would make it possible to check that a proffered password is correct (by hashing it and checking that the hashed versions match), without exposing all the passwords on the list.

Unfortunately, hashed passwords can be recovered by a 'dictionary attack', in which an attacker generates a dictionary of words and their hashed equivalents and then searches the hashed passwords for matches. Since users have a bad habit of choosing common words (or trivial variations of them) as passwords, such an attack can be expected to recover a reasonable proportion of passwords. So even with hashed passwords the list would be vulnerable to compromise and would still have to be treated as confidential.

In addition, everyone logging-in would still have to offer their plaintext password for verification, at which point it would still be vulnerable given a malicious or negligent system administrator. Consider two systems 'A' and 'B', both using such a system and with some users in common. What grounds can system 'A's administrators have for believing that system 'B's administrators will not (accidentally or deliberately) capture, and disclose or use, user name/password pairs that would allow forged logins on system 'A'. What if system 'A' were on the Student Run Computer System web site and system 'B' was the University Financial System, or vice-versa?

There is unfortunately a more insidious problem. If asked for their 'central authentication system password', how could a user know that it is safe to quote it? Passwords are only safe if you don't disclose them, but to use them that is exactly what you have to do. If you disclose them to a malicious system then you've lost. This is the basis of 'phishing' attacks aimed (with some success) at getting people to disclose the electronic banking or mail system passwords. Given all of the following, which are the bogus sites that is just trying to capture your password?

There is no obvious solution to any of this.

It's worth noting in addition that many authentication schemes (particularly those using some sort of challenge-response) need access to plaintext passwords, or at least particular hashes of the passwords, and so can't be supported by any scheme that distributes pre-hashed passwords.

Option the fifth: central password verification
Even if a list could be made to work, distributing it would be difficult especially if it needs to be done in a timely manner. One solution would be to have a 'central password verification service'. In this model, a system using the authentication service solicits a user name and password for a user and forwards them to the central service which returns a match/no match response. Almost any network protocol could be used for this, but it is common to use LDAP and overload LDAP's 'user login' process to provide authentication. Alternatives include POP, IMAP, RADIUS, and home-grown protocols.

This approach successfully avoids exposing the list of everyone's plaintext or hashed passwords to system administrators (and potentially others) which significantly reduces the exposure. It unfortunately still requires that users's plaintext passwords be disclosed as they authenticate, and does nothing to help users decide when they should and should not quote their password.

Protocols used for remote password verification need to be configured so that they don't expose plaintext passwords on the network, and so the authentication server can't be impersonated and used to collect passwords. Doing this correctly makes things considerably more complicated and is something which is often omitted.

Option the sixth: Kerberos (or similar)
Kerberos was designed precisely to overcome many of these problems. It allows a central verification service to assert that a user knows a password, and so has authenticated themselves, without the user having to disclose their password to anything other than their local workstation. It uses assorted cryptographic sleights of hand to do this and has numerous other important properties, such as preventing the recipient of an assertion from using that to impersonate the user on any other service.

In principle Raven could be extended to become a Kerberos central verification service (a 'KDC'). But using Kerberos comes at a cost. Firstly the user's local workstation needs Kerberos software installed and configured. Secondly the Kerberos protocol is significantly more complicated than a user name/password exchange, so unless the systems to be protected already supports it then there are likely to be difficulties.

One interesting development is that Microsoft have adopted Kerberos (or at least, a version of Kerberos) for authentication under Windows. For example every Windows Active Directory Domain is also a KDC. By establishing appropriate trust relationships between Windows KDCs throughout the University it might be possible to use a Raven password to authenticate to an Active Directory Domain and then to rely on Kerberos for further onward authentication as and when required. Similar Kerberos-based login arrangements are available for MacOS and Linux. It is possible that in this environment a malicious or inept AD administrator could compromise authentication - further research is required to establish if it is practical.

Kerberos reduces the number of times that passwords need to be entered and so reduces, but does not eliminate, the problem of educating users about when the should and should not provide their passwords when asked. Note that some software, for example this 'Kerberos' module for Apache, will misuse the initial Kerberos user authentication process by soliciting the user's user name and plaintext password and then seeing if it can authenticate as the user. In this they are just using the Kerberos system as a central password verification service with all the problems that this entails described earlier. Again, it's not obvious how to enable users to safely detect this.

Where does Raven currently fit into this?
The current Ucam WebAuth system used by Raven, and a whole range of similar web-redirect-based systems, depends on the client software in use being a web browsers and on browsers including as standard a fortuitous combination of features:

Support for HTTP redirects, allowing the client to be instructed to contact the authentication server direct, allowing communication that bypasses the server initiating authentication;
Support for the https: protocol, providing both security for the user's password on the wire and a way for users to positively confirm that they are communicating with the real authentication server and not an imposter (and so that it ''is'' safe to disclose their password); and
Provision of a user interface which can solicit a user and and password.

Ucam WebAuth has some features in common with Kerberos, but without the need to distribute or configure client software, though its use does require much more investment at the server end than a simple user name/password system. It does, crucially, only require the user to disclose their password to the central authentication server and provides a way for users to easily identify it. As a result it, just, manages to provide a reasonably reliable authentication system using passwords, but of course only in a web environment.

Secure use of passwords

2009-11-13T11:59:00.003Z

[This posting was originally published on an internal wiki in 2006 and is republished here to increase its exposure. I believe that what is says remains entirely relevant]

Passwords are about the best tool we have for identifying people on-line. There are alternatives, but they have financial and organisational costs that don't scale to the 30,000 people at the University of Cambridge. Unfortunately passwords are actually a very poor tool for this purpose and there are a number of prerequisites that must be met if they are to work at all. Three related ones interest me in particular - one that many people understand, two that they don't.

Prerequisite 1: Passwords must not travel in clear over insecure networks. Many networks are relatively easy to snoop. Most wireless networks are trivial to snoop and doing so doesn't even need physical access. Anyone who manages to capture a userid and password by snooping can impersonate the user for as long as the userid/password pair remains valid, and may be able to leverage this to gain further privilege. Most people understand this.
Prerequisite 2: Passwords must not be divulged in clear to untrusted systems. It's no good having a secure central password validation service if third-party systems collect userids and passwords and pass them on to the central service. Any one of these systems could maliciously capture userids and passwords, or accidentally or recklessly expose them. Even if access to the central validation system is itself secure, a common failing of such third-parties is for them to cause or encourage passwords to be sent in clear on insecure networks.; Of course it's tempting to say "My system's secure so it's safe for me to do this". The problem is that, given n such systems it's necessary for each to trust the other n - 1. Clearly this doesn't scale much beyond n = 1.

These two can actually be combined: Passwords must not travel in clear over insecure networks or through any system that anyone doesn't trust.

Prerequisite 3: It must be possible for password holders to decide when it is safe to divulge their password. They need to divulge their password to authenticate, but divulging it also makes it possible for others to impersonate them.; A system that only ever requires a password to be entered on one particular secure form on one particular web site goes some way towards meeting this requirement. Even if many people don't understand the issues it is likely that at least some will, and will identify and report other occasions on which they are asked for this password. Such other requests are likely to be dangerous or malicious. As the number of occasions on which a particular password is legitimatly requested rises, so does the difficulty of explaining and understanding what these occasions are. Beyond a small number, most people will reflexively enter their password whenever they are asked for it. This will result in an increase in likelihood of malicious or accident password interception and so a decrease in the reliability of the authentication system.

Remember, when considering password-based authentication systems, that the important thing isn't that legitimate users get challenged for a password before gaining access, even though this is the behaviour commonly checked by management. The important thing is that people can not realistically gain access using an identity that is not their own, and this isn't the same thing!

Bigfoot.com terminally broken?

2009-11-09T10:01:00.000Z

The bigfoot.com email forwarding service seems to be terminally broken this morning, with all its registered mail hosts returning 'Not route to host' when accessed on port 25. Brief Googling suggests this has been going on for a while, and I've just had a message bounced that was originally sent last Wednesday which seems to confirm this.

While I have always used the basic service, and so haven't actually paid anything for it, this level of 'service' is unusable and suggests that Bigfoot don't care about mail forwarding any more. Time I think for my own domain name and the end of a long, and largely happy, relationship with Bigfoot.

How come Google got it wrong?

2009-11-04T11:17:00.008Z

We've noticed that various 'Points of Interest' on Google maps in Cambridge city centre are just plain wrong. For example Christ's College appears at the junction of St Andrew's Street and Downing Street when it's actually several hundred meters north on the east side of Hobson's Street. Emmanuel College and the Museum of Archaeology and Anthropology are also in the wrong place, though not so drastically. There are others.

I think Google have Christ's College in the wrong place because they think its post code is CB2 3AR when it should be CB2 3BU (even that's still not ideal for something as big as a college since it identifies the delivery entrance rather than anything suitable for visitors, but it's better than nothing).

For Christ's this seems to be a common error (try Googling for "Christ's College CB2 3AR" to see the extent of the propagation of this error) but I suspect the fact that yell.com have got it wrong may be at the root of the problem. It might be possible to get this fixed, though it's going to take a very long time for the fix to propagate.

Emmanuel has a similar problem - Google has their post code as just 'CB2' so they have been put where 'St. Andrews Street CB2' resolves to. It's harder to track down where this error originates, and I haven't tried.

The Museum of Arch and Anth is more interesting. Google have their correct address of "Downing St, Cambridge, Cambridgeshire, CB2 3DZ", and "CB2 3DZ" resolves to their correct location. However "Downing Street, CB2 3DZ" resolves to the incorrect location shown on the map so there is something odd about the way the Google geolocation service handles this particular address (perhaps it doesn't always find "CB2 3DZ" in its database and so falls back to "Downing Street, CB2"?).

These errors are not surprising, because Google isn't a traditional map company. They don't go out and survey anything but just purchase, collect and aggregate data from various sources (TeleAtlas for the streets, their own search index for address information, etc, etc.). Some of this information is wrong, which isn't surprising - what's really surprising is that the results are as good as they are. But it does cause us a very real problem, because some people feel that we can't use anything based on Google maps for as long as it shows well known University locations in the wrong place.

Addition 2009-11-05: A colleague has pointed out that part of the airport at Cambridge is labelled "London Luton Airport".

Apache SSL/TLS security configuration

2009-10-06T09:54:00.008+01:00

Recent installation of a new vulnerability scanner at the University has hilighted that a default-configured Apache supports SSL v2 which reportedly suffers from several cryptographic flaws and has been deprecated for several years, and supports the use of SSL ciphers that are very week by today's standards.

The danger of dropping support for particular protocols and ciphers is that doing so denies access to any clients that don't support anything else. Ideally you should review your Apache configuration in the light of your security needs and the capabilities of your clients, which obviously only you will know. Failing that, I've reviewed a sample of the logs the the University's Raven server to see what its clients are actually using. This should represent general University client capabilities. Only 17 out of over 3,500,000 connections used SSLv2, all of which looked to be from robots or similar; only 68 of these 3,500,000 connections used ciphers with symmetric key lengths of 56 bits or below.

My conclusion is that, for general use, adding the following to your Apache configuration will provide a reasonable level of security while excluding few if any legitimate vistors:

SSLProtocol All -SSLv2
SSLCipherSuite ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:!LOW:!SSLv2:!EXP

When compared to the Apache default this a) drops SSLv2 while leaving everything else (including future developments); and b) drops the export-crippled ciphers, those using 64 or 56 bit encryption algorithms, and the SSLv2-only ones (since we've dropped SSLv2). Exactly what this will leave you depends on the version of OpenSSL you are using, but you can find out from the openssl command-line utility:

openssl ciphers -v 'ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:!LOW:!SSLv2:!EXP'

On my Ubuntu 8.04 box this leaves

DHE-RSA-AES256-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA1
DHE-DSS-AES256-SHA      SSLv3 Kx=DH       Au=DSS  Enc=AES(256)  Mac=SHA1
AES256-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(256)  Mac=SHA1
DHE-RSA-AES128-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA1
DHE-DSS-AES128-SHA      SSLv3 Kx=DH       Au=DSS  Enc=AES(128)  Mac=SHA1
AES128-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(128)  Mac=SHA1
EDH-RSA-DES-CBC3-SHA    SSLv3 Kx=DH       Au=RSA  Enc=3DES(168) Mac=SHA1
EDH-DSS-DES-CBC3-SHA    SSLv3 Kx=DH       Au=DSS  Enc=3DES(168) Mac=SHA1
DES-CBC3-SHA            SSLv3 Kx=RSA      Au=RSA  Enc=3DES(168) Mac=SHA1
RC4-SHA                 SSLv3 Kx=RSA      Au=RSA  Enc=RC4(128)  Mac=SHA1
RC4-MD5                 SSLv3 Kx=RSA      Au=RSA  Enc=RC4(128)  Mac=MD5

According to NIST Special Publication 800-57, symmetric keys of at least 112 bits should be generally OK until 2030. Note however that this only applies when used in conjunction with certificates containing asymmetric keys of at least 2040 bits, so it would also be advisable to upgrade any certificates using smaller keys.

Shibbolising Plone 3 - some experiences

2009-09-23T18:07:00.004+01:00

In a previous posting I listed two (and a half) options for Shibbolising Plone 3. What follows is a comparison based on what I found when I actually tried installing them into default Plone instances.

[The 'and a half' option was Liberty Alliance / SAML 2 Authentication Plugin for PAS. I'm discounting it for the time being because a) it doesn't look as if it will work with Shib 3, b) there hasn't been a release since December 2008, and c) the most recent release gives '404 Not found' on the Plone web site]

The two remaining options were WebServerAuth and a set of products from Ithaka (AutoUserMakerPASPlugin, ShibbolethLogin, ShibbolethPermissions). They have quite a lot in common (which is not surprising since they share a common ancestor).

Common features

Both depend on the Apache Shibboleth module to perform authentication, and so require connections to Plone to be proxied through Apache.

For this to work, server variables provided by the Shib module have to be converted to headers in the proxied request - doing so is a matter of Apache configuration. It is important to ensure that unauthenticated users can't spoof any such headers used for anything related to security.

Both will allow Plone access by anyone who can successfully authenticate - if this isn't appropriate then Apache access controls could be used to limit this further (and WebServerAuth has an additional option).

Both require a unique user identifier. Easiest to use the value in REMOTE_USER which is set by default by the Shib module to the first non-blank value in eduPersonPrincipleName or the SAML 2 or SAML1 versions of eduPersonTargetedID. This leads to some unwieldy userIDs - both products can strip domain names from IDs, but this removes any guarantee of uniqueness.

Both force authentication for an https: version of the site while allowing unauthenticated access to the http: version. While Shib's 'Lazy Session' features might make this unnecessary, both products will in principle work with other webserver-based authentication schemes that may not support anything similar.

For both, it's advisable to suppress the default 'login' portlet that is displayed to unauthenticated users.

WebServerAuth features

WebServerAuth doesn't create users in the database - by default they just get assigned the 'Authenticated' role (not 'Member') when they log in. It uses userID (so eduPersonPrincipleName or eduPersonTargetedID, optionally with domains stripped) as the user's Full Name - it doesn't use any other Shib attributes even if they are available.

Users requiring additional rights can be created in the Plone database and appropriate rights will be assigned when the corresponding users authenticate. Creating Plone users with IDs containing '@' and '.' requires a long-winded hack.

WebServerAuth can be configured only to authenticate users who have corresponding Plone accounts, but the user interface is sub-optimal: people will log in and apparently succeed, only to be greeted with a Plone page that still has a "Log In" link.

By default, WebServerAuth redirects to the https: version of the current page when authentication is required (and modifies the standard 'log in' link to achieve this). This will cause the Apache module's default Sessioninitiator to be used for authentication. Optionally WebServerAuth can redirect to a customised URL which could perhaps be used to implement a local WAYF service (e.g. simplified login for local users; redirect to federation WAYF for others). Essentially WebServerAuth takes over all login access to the site (which can be problematic if it fails...).

WebServerAuth is under current development - its author contacted me within hours of my posting my earlier article to point out an error with it.

Need to set the standard logout link (ZMI --> Plone --> postal_actions --> user --> logout --> URL) to something apropriate (or perhaps suppress the link altogether?)

Ithica product features

AutoUserMakerPASPlugin creates real Plone users as new people authenticate. It can use configurable Shib attributes to initialise full name, email, location, roles and group membership. This only applies to user creation - once created, changes to attributes are not propagated and the users need to be managed on Plone. It seems necessary to configure full name to fall back to userID if nothing else is available to avoid all such users ending up with a full name of '(null)'.

AutoUserMakerPASPlugin can be configured to strip either all or selected domains from userIDs - one approach might be to strip just a local domain, giving local users short userIDs while ensuring against name clashes with anyone else.

AutoUserMakerPASPlugin supports a configurable 'logout' URL that can invoke the local Shib module logout function. This may be confusing in a wider single sign-on context (but is handy for testing).

ShibbolethPermisisons adds the ability to add local (per page or per container) access rights based on configurable Shib attributes. Again, this only applies at account creation time, after which rights have to be managed manually within Plone as usual.

ShibbolethLogin installs a replacement for the standard login page. This includes configurable links either to local SessionInitiator URLs or direct to Shib 1 IdPs or WAYFs. So for example it's possible to have a 'Log in with a Raven (University of Cambridge) user id' link for local users and a more general 'Log in with a different UK Federation user id' link for everyone else. Local Plone user authentication (with username/password) remains available. ShibbolethLogin appears to be targeted at Shib 1 functionality, with no obvious support for new Shib 2 functionality, such as the Discovery Service.

The maintenance status to the Ithica products is unclear. Documentation reports the most recent tests to have been with "Zope 2.10.5 and Plone 3.0.6"; files in the distribution appear to have been modified most recently in May 2008.

Federated AUTH is less than half the battle

2009-09-15T14:12:00.004+01:00

Federated web authentication, as provided by Shibboleth, OpenID, Pubcookie, Cambridge's Ucam WebAuth (a.k.a. Raven), etc., is less than half the battle of adapting an application to work in a federated environment. Much the harder part involves the related issues of authorisation and account provision.

Most existing applications assume that they have some sort of account for every users who can authenticate, not least because traditionally they need somewhere to store user names and passwords. In the federated authentication world this won't always be true, since lots of people will be able to authenticate that the application has never heard of.

Approaches to dealing with this include:

Require an account to exist locally before a user can authenticate

If handled manually, this can impose an administrative overhead which will only be manageable for a small user base. Alternativly accounts could be provisioned automatically, but only if the expected set of users can be identified in advance. It is often hard to adapt existing software to not fail badly when presented with an authenticated user who doesn't have an account.

Users may have to tell administrators how their identity provider will identify them and this could be a problem - in Shib's case it is currently unlikely that users will know their eduPersonPrincipleName (even if it is available), and they can't predict their eduPersonTargetedID. An authenticated 'Application' page that captures available attributes might be a way around this.

Automatically provision an account when a user first authenticates

Doing this entirely automatically assumes that information is available about the user, and so assumes either Shib-like attribute provision or access to additional data in something like an LDAP directory. It also assumes that enough information is available to create accounts - some existing applications positively require things like forename and surname which is problematic if these are not available. Alternativly or additionally, users can be prompted for additional information, though obviously this is less reliable and implementing such prompting can be difficult.

However it is also necessary to decide who will get such accounts, since it's unlikely that you will actually want to provision an account for everyone who can successfully authenticate, especially if such accounts come with some basic privileges. So some sort of rule engine will be needed to define who does and doesn't get an account, and perhaps what default privileges they get. This will have to be driven by Shib-like attributes or information from directories. For obvious (I hope) reasons, such decisions probably can't be taken based on information supplied by the users themselves.

Dynamically log users in without creating accounts for them

In some applications it's possible for a user to appear 'logged-in' without having a corresponding user account. One example I happen to know about s Plone using the WebServerAuth product. This simplifies things somewhat, though you still need to address the question of who gets what access by default. You probably need some way to manually grant additional access to a limited number of people where the people and the access that they need can't be established based on attribute or directory data - for them, some sort of local account bound to their federated identity will probably still be needed.

Shibbolising Plone 3 - a review

2009-09-14T09:49:00.000+01:00

There are lots of Google references to shibolising Plone, but it's not clear how many of them apply to Plone 3 (rather than Plone 2 or earlier). All seem to rely on getting Apache to implement the Shibboleth protection, and then passing identity information over to Plone for it to use.

It looks as if the options include

which seem to have been replaced by

WebServerA uth

all from (or at least related to) the WebLion project at Penn State. There's a useful page on WebServerAuth on the WebLion site.

Alternativly there are three extensions from Ithaka

which are described in this this slide set and this article.

It looks as if the Ithica solutions actually provision Plone accounts for Shib-authenticated visitors, while the WebLion products give such users 'authenticated' state without creating accounts for them. I can see pros and cons for both approaches. WebServerAuth is being actively developed (last release August 2009); none of the others look as if they are very actively maintained. The Ithica products apparently work with at least Plone 3.0.6; apachepass only claims to work with Plone 2.5; Auto Member Maker and WebServerAuth apparently work with Plone 3.

Note: updated 2009-09-15 to correct the development status of WebServerAuth in the light of comments by Erik Rose.

Note also that the Liberty Alliance / SAML 2 Authentication Plugin for PAS might be relevant, if it's sufficiently flexible to talk Shib.

See also Shibbolising Plone 3 - some experiences