How Much is One Millisecond Worth To You?

For those who don’t keep these kinds of details in their heads a millisecond is 1/1000th of a second.

With respect to security processing what can be done in that time increment – can you perform a fine-grained permission check over the network?  How about logging results into a remote database?

Would you believe me if I told you we can perform both operations within that allotment of time on a modest hardware configuration?  What if I also tried to tell you that over eleven thousand of these requests may be performed a second with the same response time?

Such were the results as documented in this series of benchmarks:


Read more about the accelerator rationale here:


Fortress RBAC Accelerator PDP Benchmark Report

Benchmark Overview

This post provides a summary of a recent benchmark effort for the Fortress RBAC Accelerator.  The RBAC accelerator uses LDAPv3 extended operations to perform the following access control functions:

  1. Create Session – attempts to authenticate client; if successful, initiates an RBAC session by activating one or more user roles
  2. Check Access – determines if user has access rights for a given resource
  3. Add Active Role – attempts activation for a given role into user’s RBAC session
  4. Drop Active Role – deactivates a given role from user’s RBAC session
  5. Delete Session – deletes the given RBAC session from the server
  6. Session Roles – Returns the active roles associated with current session

The result of each of the above functions are persisted to LMDB for audit trail.

Benchmarks performed using a Jmeter test client to drive load for CheckAccess (#2).  The server hosts the OpenLDAP daemon which has the RBAC accelerator overlay.

Client Machine

  • operating system: ubuntu 13.04
  • kernel: 3.8.0-32-generic
  • processor: Intel® Core™ i7-4702MQ CPU @ 2.20GHz × 8
  • memory: 16GB (doesn’t use anywhere close to that)
  • Java version 7

Server Machine

  • operating system: ubuntu 14.04
  • kernel: 3.13.0-32-generic
  • processor: Intel® Core™ i7-4980HQ CPU @ 2.80GHz × 4
  • memory: 8GB
  • OpenLDAP version: 2.4.39

Test Details

  • 25 threads running on client
  • each thread runs checkAccess 50,000 times
  • 1,250,000 total
  • Client CPU load: approximately 50%

Test Results

  • Response time: 1 millisecond
  • Throughput: 11,533 transactions per second
  • Server CPU load: approximately 85%

The Case for a Policy Decision Point inside the LDAP Server

Feature image from:

Why on earth would you do that?

We all understand that runtime characteristics change as processes get moved around the network.  Having problems with network io?  Move the database daemon to the same tier as the client process.  Worried about file io?  Store the data in memory as opposed to disk.  etc…

These same notions apply for system architecture with respect to security.  Location of policy enforcement, decision, and database processes hugely impact the overall welfare of your organization’s computational systems.

With these kinds of thoughts, let’s move the security processes around and see what happens.

But first, we’ll define them:

1. Policy Enforcement Point (PEP)

Invoked by client programs for security policy enforcement.  These gatekeepers are embedded directly into their host thus require platform specific bindings.  The less impact to their host, the better.

2. Database (DB)

Invoked by PDPs to store security credentials, attributes and activity logs.  An important concern is speed; to be used correctly, it’s used often.  Another important concern is reliability; the integrity of its data is mission critical.

3. Policy Decision Point (PDP)

Invoked by PEPs and dependent on a DB.  Responsible for computing:

  • authentication – with passwords or keys
  • authorization – with attributes or permissions
  • audit trail – identify subjects, decisions, time/date/locations, and resources

Three types of PDPs

  • Type 1 – runs in-process to PEP, with out-of-process DB
  • Type 2 – runs out-of-process to PEP, with out-of-process DB
  • Type 3 – runs out-of-process to PEP, with in-process DB

PDP Type 1 (runs in-process to PEP)

The Tomcat JDBC Realm is a Type 1 PDP.  It functions both as PEP and PDP.  Other Type 1’s include Apache Fortress, Shiro and Spring Security.  Most of the open-source security frameworks are this type.  The policy decisions happen synchronously inside the client process.

Type 1 PDP interaction between processes

The PEP and PDP process reside on one tier; the DB process on another.

The PEP and PDP process reside on one tier; the DB process on another.

Benefits of Type 1 PDP

Type 1 PDP advantages
  • simple – only security framework and DB required
  • widely available
  • works well for single deployments of stand-alone apps
Type 1 PDP disadvantages
  • more code exposed to the client (making deployment harder)
  • more load on the client
  • more memory consumed on the client
  • more network io traffic on the client
  • fewer platforms supported (can it run on Java, NET, PHP, Python, and all the others?)

PDP Type 2 (runs out-of-process to PEP and DB)

Here the PDP and DB both run out-of-process to the PEP.  These types of PDPs are more complicated and usually obtained as separate COTS or OSS products.  For example, CA Siteminder, Tivoli Access Manager, Oracle Access Manager, Shibboleth, CAS and many others.

Type 2 PDP interaction between processes

The PEP, PDP and DB processes reside on separate tiers from one another.

The PEP, PDP and DB processes reside on separate tiers from one another.

Benefits of Type 2 PDP

Type 2 PDP advantages
  • less network traffic on client
  • less cpu consumed on client
  • less memory consumed on client
  • less code exposed to client (making deployment simpler)
  • more platforms supported (simple to code PEP ‘shims’ to communicate with centralized PDPs)
Type 2 PDP disadvantages
  • more security processes to maintain due to PEP, PDP and DB all running separately (increasing management burden)
  • poor response time due to extra network hops
  • poor throughput due to PDP reliance on heavyweight communication protocols (xml/json/http)

PDP Type 3 (runs out-of-process to PEP and in-process to DB)

The PDP and DB run in-process and the PEP is out-of-process.  The advantage here is offloading computations from the PEP reducing load and traffic from the client.  It gains speed over the Type 2 because of the embedded database.

Type 3 PDP interaction between processes

The PEP process resides on one tier; the PDP and DB process on another.

The PEP process resides on one tier; the PDP and DB process on another.

Benefits of Type 3 PDP

Type 3 PDP advantages
  • all of Type 2
  • embedded database speed gain
  • embedded database reliability gain
Type 3 PDP disadvantages
  • poor throughput due to reliance on heavyweight communication protocols (xml/json/http)

The Case for a PDP in OpenLDAP

The Enforcement Foundry RBAC Accelerator is a Type 3 PDP.  It has all the advantages of its type without the disadvantage because using extended LDAP protocols is very efficient.  It runs inside of the OpenLDAP slapd daemon and uses LMDB, which is the fastest embedded, key-value database available, giving a nice turbo boost to read/write speeds.

Now let’s return to the original question:  Why on earth would you do that?

To enable authentication, fine-grained authorization, and audit trail on every platform.  To ease concerns for high cpu, memory, network, and file io consumption.  And to use well understood technologies that can be widely distributed.

Just how good is that performance?  Read about it here:


Why another open source iam product?

A question that has been lingering.  It’s fair game, but before I get around to actually answering, let’s remember how we got here.

Late 90’s

Before the first open source iam product existed we had standard policy enforcement protocols in use.  Examples include Java’s ee security & jaas. Unix had PAM, sudo and kerberos.  Windows, NTLM and SPNEGO.  There were web-based protocols like HTTP basic auth and ISAPI.

IAM Complete Solution Defined

Those policy enforcement mechanisms didn’t meet the wider requirements of a complete solution:

  • extensible and pluggable policy enforcement services
  • centralized policy decision point services
  • common policy administration services
  • common policy information services
  • audit and review capabilities

The gaps made us scramble to meet the need.

Early to Mid 2000’s

Commercial vendors ruled.  Several more policy enforcement protocols appeared including liberty, saml, ws-*, xmlsecurity and xacml.  Those in need either built or bought because open source iam solutions were barely in existence.

There were many open source projects that provided the legos to build with.


The era of opportunity for open source innovation.  Most of the technical problems were solved.  All one had to do was assemble a solution using the building blocks available in the public domain.

But not another access control framework.  Needed a complete solution.


As code construction commenced on fortress, many open source policy enforcement frameworks became widely used, such as spring security and opensso, but still no complete solution.


Around the the time of the first fortress release the situation changed.  Sun’s merger with Oracle foretold the end of the commercial vendor stranglehold creating a void in which companies like Symas, Linagora, Evolveum, and Tirasa sprung complete solutions.


Now that we have several complete open source iam solutions on the market, the industry continues to focus on policy enforcement standards, the latest being oauth, uma and open id connect.

But we continue to struggle because uniform policy enforcement is the tip of iceberg.  More benefit comes through shared policy decision and admin services.  This promotes reuse of infrastructure and data across organizational, vendor and platform boundaries.


Policy enforcement protocols will continue to be volatile and ill-equipped.

Standards-based policy decision and admin services will continue to be needed.

So that Vendor A’s PEP can plug into Vendor B’s PDP, and Vendor C’s PAP can plug in Vendor D’s PIP.

Why Fortress

Based on ansi incits 359, our best hope for standardizing the complete iam solution.

Year of the Hack

2014 will be remembered as the Year of the hack.

Sony, Target, Home Depot, UPS, even Victoria couldn’t keep the Secret.

The list:

This article will tell you how to stay off.

What Has Already Been Tried

First let’s have a look at what’s already being done.

I. FirewallsIntrusion Detection

The perimeter has shattered.  Firewalls won’t stop someone who has credentials–attacks within the interior of a network.

“Myth: Firewalls make your data secure.  In fact, 40% of Internet break-ins occur in spite of a firewall being in place.

Data Security Challenges – Oracle9i Security Review

II. Stong AuthenticationMulti-factor Authentication / Single Sign-on

Often times bad things are committed by those with proper credentials.  Do not base your entire defense on the front door.

“Neither top secret clearance, sophisticated authentication nor the most advanced encrypted information systems can necessarily stop an intended breach action. These security procedures are not designed to detect real-time actions and anomalous business processes from authorized personnel. These practices are just the “moat around the castle” approach upon which most current cybersecurity technologies are based. Current national security breaches clearly show we need to do more.”

Is Cyber Security an Inside Job? – Larry Karisny – Digital Communities

III. Antivirus Software

Doesn’t stop attacks in-flight.  By the time a new virus has been identified, two more have spawned.

In the bigger picture, furthermore, the anti-virus software was irrelevant, contends Chester Wisniewski, a senior security advisor at anti-virus vendor Sophos. “A smart attacker in a targeted environment will always bypass your anti-virus,” he says, and especially if they’re trying to take down a retailer the size of Home Depot.

Analysis: Home Depot Breach Details – Mathew J Schwartz – Bank Info Security

These practices aren’t necessarily wrong, but they aren’t fixing the problem either.

The Problem

So what exactly is the problem?

A determined attack will gain access to your network – count on it.  It’s impossible for you to harden every entry point.  Find a way to minimize damage once the inevitable unauthorized entry occurs.

“Organizations should stop thinking about breach prevention, accept their going to be breached, change their mindset, and think about how they will protect and store their data.”

Stop Worrying About Data Breach Protection – Info Security Magazine

The Solution

You must boost efforts across the following:

I. Authorization

The access controls commonly used today are inadequate.  Let’s take a prime example:  How did a low-level, government contractor download terabytes of highly sensitive data from the NSA?

Organizations that lack these controls are vulnerable to attacks waged by individuals who have legitimate accounts on the network but seek to misuse their access for malicious purposes. This risk, known as the “insider threat” is one of the most insidious causes of data breaches.

How to avoid the five most common causes of data breaches – Mike Chappel – Certification Magazine

Add mandatory access control checks to all interfaces connected over Internet – including those downstream of secured.

II. Audit trail

If an action is important enough to guard with a policy enforcement point, it should be logged.

“The most important step that you can take to protect your organization against improperly configured access controls is to perform regular auditing.”

How to avoid the five most common causes of data breaches – Mike Chappel – Certification Magazine

Log the resource, operation, subject id, ip, location, time, date and result.

III. Periodic Review

Circulate daily reports.  Think about the results.  Devise heuristic algorithms to detect anomalies automatically.

IV. Confidentiality

Encryption enabled should be the default setting for ALL network connections – even test environments.  Don’t worry about the cost.  Consider it cheap insurance.

“In the aftermath of security breaches at Target, Home Depot, and JP Morgan Chase, executives are reexamining their data breach risks. Hacksurfer reports on a recent survey of IT professionals that found 53 percent of organizations were investing more in data security after these high-profile cyber attacks.”

In Data Security, Compliance Isn’t Enough – Max Schleicher – TechInsurance

Encrypt data in-flight and sensitive data at rest.  Hash passwords.