SEPostgreSQL Review at the BWPUG

As part of the 2009-11 CommitFest, the Baltimore/Washington PostgreSQL User's Group hosted a two-hour session discussing the state of SEPostgreSQL patches (past and present) at OmniTI. Attendees included PostgreSQL regulars Robert Treat, Stephen Frost, Greg Smith ( 2ndQuadrant), and Hubert "Depesz" Lubaczewski as well as Joshua Brindle (Tresys), David Quigley (NSA), and Brian Dunavant (OmniTI). General agenda of concerns the PostgreSQL community has with this feature was provided by Greg, and despite periodic digressions (mainly into parallels with NFS4 development) we got through an initial pass over all the intended topics.

Known concerns with merging SEPostgreSQL

User base for the feature

Examples of how this feature would work always seem to use a government context, such as the standard "filter based on security clearance" one. There is scattered evidence such customers exist, and certainly the folks at Tresys and the NSA we've got here wouldn't be involved if they didn't know of them. But this is problematic because those customers are the last ones likely to talk about their needs, what they're already doing, or whether they're using the software. What other applications might the patch be useful for?

Robert points out OmniTI regularly handles PCI compliance for applications that handle credit card processing, quite common in e-commerce, and it has similar user-isolation and privacy issues. There is an Oracle Product aiming at this market. Joshua threw an example of how you might architect such a solution using SELinux and SEPostgreSQL onto the white board, it seems both feasible and a good match. Robert suggests current solutions for PCI compliant databases are quite painful to work with, so it wouldn't be hard to do better. His sentiment was basically "as much as I hate SELinux, if it made me able to use PostgreSQL instead of what I'm forced to use now for PCI I'd put up with it". For specific details about the area where this approach would be most valuable, see requirement 7 ""Implement Strong Access Control Measures" in the PCI spec.

Another possibility is the ISP space. One of the common complaints for ISPs that would like to use Postgres is that they have to create a cluster per user in order, with its own assigned unique port and everything, because there are too many security leaks between databases within a single cluster for customers to share one. SEPostgreSQL is a potential approach for improving that situation. There's an example of this on P24 of the LAPP/SELinux presentation.

As far as "is anyone using SEPostgreSQL already?", the situation seems to be that the DoD sites that use SELinux for security purposes have only certified that approach on top of RHEL, not Fedora. Joshua noted that if the current SEPostgreSQL packages only available for Fedora were officially instead available from RedHat for RHEL, their requirements would be over--that's the missing piece that prevents deployment of what's out there now at all. It's one of those situations where a subset of the requirements isn't useful at all until it crosses a milestone, and in this case RHEL compatibility is that milestone.

Audit by security experts

It's unclear to the PostgreSQL community that the model KaiGai has implemented has gone through an audit by any security experts to evaluate whether it's suitable for building high-security applications. Database hackers can't be presumed to have the expertise necessary to do so themselves.

Joshua points out that he's been personally involved in such review for years now, as have many other members of the security community, and that record is available on the SELinux mailing archives. The discussion around the SE-PostgreSQL 8.2.3-1.0 alpha back in early 2007 is a good example of how far back that goes, Greg spoke with Joshua about this in the summer of 2007, Joshua was on pgsql-hackers last year; while we haven't seen him often on our lists, he's certainly not gone away. As we've seen recently, Chad Sellers from Tresys and David Quigley from the NSA are also tracking the PostreSQL integration now. KaiGai's work here has been heavily scrutinized by experts in the field and had feedback incorporated into its design, that work just hasn't happened on the PostgreSQL lists.

It is worth mentioning that SEPostgreSQL is on the radar screen of those studying in the field of database security. One example is this recent paper by David An at UMD on Oracle Label Security implementation, where SEPostgreSQL is mentioned as a possible prior implementation.

Available SELinux policy

There's some concern that since RedHat took quite a while to get a useful policy for applications, that there will be a similar delay for producing a useful SEPostgreSQL policy useful to users. Our local SELinux experts were not concerned about this. SEPostgres was implemented into the standard SELinux Reference Policy that Tresys maintains in 2007, and it's been continuously maintained forward since into the newer Fedora releases. To the people who work in this area, this was a solved problem years ago.

Works outside of SELinux

One concern is that the SEPostgres implement would be tied too much to SELinux, and may not integrate with alternate security solutions. The best data point available for that is Trusted RUBIX, which is another product we'd be able to compete against were SEPostgres in core. Trusted RUBIX uses the same SELinux object classes and permissions that were originally added to the policy to support SEPostgreSQL. Their documentation shows how they use SELinux in their RDBMS. On page 15 in particular which object classes and permissions they are using are described. They even implement row level security (the db_tuple object class). And Trusted RUBIX maps to both SELinux and Trusted Solaris, proving that approach, using the policy designed for SEPostgreSQL, can map into another security implementation altogether. Also, since some of the work done there was based on KaiGai's SEPostgres, design ideas came back to him as part of that to help improve the model he was working with; see Some ideas in SE-PostgreSQL enhancement as an example.

And as far as SELinux being tied too heavily to Linux, that's not really the case. The basic security model of SELinux has been ported to BSD and Darwin for example. Joshua and David both stressed that those ports were quite simple; the majority of the code stayed the same, it was only the lowest levels that interfaced with filesystem objects and the like that needed to change. This further supports that the security object model associated with SELinux is a reasonable target to code against.

While we can never be completely sure until a second implementation is done for another security system, there's evidence to believe the way the database objects are mapped to the security object model by SEPostgres is generic enough to handle other security implementations too, so long as they have a reasonable object-based implementation--which seems to be the common approach now. A good intro to this topic is The NSA Works with Sun to Boost Solaris Security, where the common ancestry of all these security layers is described--it mentions PostgreSQL fitting into that structure, too. It also mentions how some suggested alternates mentioned on the pgsql-hackers list, such as AppArmor, just don't fit into where the rest of the security industry seems to be moving.

Details of other hook implementations

In the Linux LSM case, there is a separate hook interface for each logical operation/action, where hook name identifies the action/operation and the set of arguments to that hook interface are the complete set of inputs that may be relevant to deciding whether to permit the operation/action. These arguments typically include pointer(s) to one or more object data structures as well as ancillary arguments (e.g. new owner value if chown). References: 1, 2.

Unlike the LSM framework, the XACE framework for the X server passes resource identifiers rather than direct object pointers.

The FreeBSD MAC framework provides more abstraction than LSM does, at a cost in the overhead of the framework itself.

CERT issues for bugs

A serious and quite valid concern is that adding a large body of code that touches so many security-related parts of the database will introduce new bugs that, by their very nature, will all be security issues requiring disclose via avenues like CERT. While these notices can clearly say "only impacts users who are using selinux integration", they'll still count against the total numbers reported for PostgreSQL. There's no easy way around this problem; code review, careful modularization, and testing are the only defenses.

PostgreSQL committer

It's obvious we'll need serious review of the SEPostgres review from a qualified committer, with a lot of potential follow-up work to be done. Currently it looks like Bruce Momjian has interest in this, albeit only if there's a clear plan that addresses community concerns and moving toward a design consensus. Bruce is local enough to the BWPUG location (which is itself fairly central to where US-based work on SEPostgres seems to be happening at) that we could easily arrange some in-person time as part of our user's group to work on this, perhaps a "SEPostgreSQL Day" or similar event. It would be easy to invite local security vendor contacts, such as the Tresys and the NSA staff who visited for this meeting, to that. Robert Haas, who has already put a large amount of work into reviewing this patch, was suggested as another relatively local person we might able to invite to such an event. How to help fund travel expenses for committers who are interested in joining such a design workshop is something we'll look into.

Future maintainers

There's some concern that having KaiGai Kohei as the sole contributor maintaining this patch moving forward isn't ideal, particularly if it ends up introducing time-sensitive issues such as security bugs. Some of the regular PostgreSQL code contributors at our meeting (Stephen Frost, Greg Smith) are looking into launching projects that use SEPostgres that might allow them to devote more time to that role in the future. This area clearly needs more help.

Follow-up plan

Proof of concept/prototype development

It's obvious someone needs to prove the alternate use-cases beyond the "secret clearance" ones, which we just won't ever see real-world examples, of are viable. Preferably that would happen in a way that the results can be published for community review. The PCI compliance example is an excellent possibility here. Stephen Frost and Robert Treat both have business related needs that might make such a proof of concept project viable. The main issues appear to be the usual ones: finding sufficient manpower and/or funding (the latter can always produce the former). Anyone who has a similar interest should contact them and Greg Smith, who is trying to help track all this activity and coordinate resources within the community.

PostgreSQL security check cleanup

Stephen Frost made a recent commentary that many of the issues with SEPostgres result from how security checks are spread across so many places in the PostgreSQL code, due to general modularity and data hiding issues in the backend code. He argues that some of the problems here are not specific to it; SEPostgres is just highlighting how spread out some of these checks are.

It seems like attempts to cut the size of the SEPostgres patch set and its intrusiveness into the backend code are unlikely to go much futher--we have close to the minimal set that does anything useful. If we want to continue reducing it, and make the PostgreSQL code base more amenable to a clean and simple merge with a security framework, investigating an overhaul to the underlying code seems a fruitful approach. That should diminish the SEPostgres diff that touches the backend code, and therefore reduce the risk of merging the code in too.

Revisiting row-level security

The feature cut from the current SEPostgres patch that's most missed by our local experts is row-level security. Without it, it's questionable whether committing the existing stripped-down patch really accomplishes anything for the user-base they have targeted--even the trivial "secret clearance" example requires it, and certainly many real-world applications do. The concerns that caused this area to be dropped:

• Interaction with SQL compliance
• Side-channel information leakage
• Overly SELinux specific code

Wouldn't be there if there was row-level security available in the basic server for the patch to hook into more simply.

Future meetings

This short session combining people from multiple sides of the industry interested in this area was very productive, and a future, better planned and publicized meeting might do even better. Possibilities include:

• A future "SELinux PGday" in this area
• Conference presentations (like the one at PGCon 2008): PG East 2010? PGCon 2010? Instead of a full-length one, a state of the patch lightening talk is another possibility.