StrongKey Blog

Automating Data Protection Across the Enterprise

Written by Arshad Noor | Feb 7, 2019 3:48:00 PM

Introduction

With the unabated growth of the internet, companies are being challenged to protect unprecedented quantities of data from determined and targeted attacks. While security regulations mandating the protection of sensitive financial, health-care or personally identifiable information (PII) serve as the primary impetus for data-protection, breached companies have learned the hard way that brands can be damaged, leading to significant financial costs when even unregulated data, such as e-mail and home-addresses, are breached1.

Although encryption technology has been available for more than forty years2, its use was initially relegated to military and banking applications where breached information has extremely high consequences. Twenty years ago, the world-wide web ushered in the Secure Socket Layer (SSL) protocol with its use of encryption to protect information as it traversed the internet. However, it was perceived as an enhancement to network protocols, and consequently, encryption technology continued to remain outside the domain of business applications.

Encryption started receiving attention from the general-purpose information technology (IT) industry - e-commerce, retail, etc. - only in the last five-six years, thanks to the credit-card payment sector3. There are several reasons why the general IT industry has been slow to adopt encryption technology:

  • Cryptography is a complex subject, and integrating it into applications has required specialized knowledge that is expensive and/or difficult to obtain;
  • Cryptography has tended to slow down business transactions without the addition of expensive accelerators – that are difficult to integrate and manage;
  • Companies have needed cryptography only in specific business applications, thereby relegating it to a niche. Consequently, this has also prevented the flow of knowledge of cryptography to general IT professionals perpetuating its use to niches;

As a result, companies have lacked a universal architecture for the protection of sensitive data across the enterprise. Global companies are becoming aware of this handicap as countries start enacting legislation for data-protection within their borders.

Cloud Computing

With the advent of cloud-computing, businesses can now come to market at lightning speed with massively scalable infrastructure to address their just-in-time computing needs. While the business benefits are indisputable, security concerns remain the single largest road-block to the wholesale adoption of cloud computing.

In 2011, an application architecture called Regulatory Compliant Cloud Computing (RC3)4, provided a solution to address the security problem. While RC3 addresses data-security needs on an application-basis, what remains missing is an all-encompassing infrastructure that allows an enterprise to address its data-security problems regardless of:

  • Where that data is generated, stored and used;
  • The type or size of data; or
  • Which country's data-security regulation governs protection of that data.

Data Encryption Infrastructure (DEI)

This paper defines a Data Encryption Infrastructure (DEI) - also sometimes referred to as a Document Encryption Infrastructure - as a collection of technology components and application architecture governing the protection of sensitive data within an enterprise.

A DEI is defined to have the following characteristics:

  • Any data-type - It must protect any type of data: structured data-elements such as credit-card numbers, passport numbers, etc., as well as unstructured binary large objects (blobs) such as audio and video files, documents, images, etc.;
  • Any data-size - It must protect data of any size: elements as small as the date-of-birth of an individual to objects as large as a database backup of an online transaction system;
  • Ubiquitous - It must be ubiquitous and easy to use from a software developer's point-of-view without awareness of the infrastructure's mechanics. An analogy is the Domain Name Services (DNS) name-service look-up library. Software developers have been using this single library and application programming interface (API) for more than thirty years to find the address of any computer on the internet without knowledge of how DNS is structured or how it works;
  • Independent - It must be independent – much like DNS – without being tied to a specific application, platform, programming language, protocol or hardware component;
  • Centralized - It must be manageable centrally, regardless of the number of components in the infrastructure and how distributed they may be across the globe;
  • Location-transparent - It must protect information regardless of whether the application and its data is resident within an enterprise's closed perimeter or in a Cloud Service Provider's (CSP) cloud;
  • Regulation-transparent - It must comply with data-security regulations anywhere in the world, regardless of where the data-owning company, its customers, partners, suppliers and/or service-providers maintain their headquarters and operations;
  • Auto-scaling - It must dynamically scale up to handle transaction surges or scale down to conserve resources, while maintaining uniform response-times to applications, without the need for human intervention;
  • Secure - It must support industry best-practices for cryptographic key-management: industry-standard algorithms, split-key knowledge amongst key-custodians, cryptographic hardware modules, etc.;
  • Free and Open-Source - Ideally, the infrastructure should be capable of being built with free and open-source software (FOSS) to ensure it is within the reach of even small companies; however, this is not mandatory.

DEI Reference Implementation

While a DEI can be built in many different ways, this paper defines a DEI Reference Implementation (DEIRI - pronounced “Dairy”) that has proven to be successful in multiple industries and companies of different sizes and locations.

The DEIRI is based on the following diagram and depicts a minimal Production implementation that addresses a fixed level of business requirements. Depending on a company's software development and quality-assurance practices, it may choose to deploy similar DEIs for Development and Quality Assurance environments. It does not matter whether the DEIRI is implemented within the company's perimeter or is hosted by a service-provider within a virtual perimeter dedicated to the company5.

The following components make up a DEIRI:

  • Redundant locations;
  • Front-end Processors;
  • Cryptographic Engines;
  • Key Management Systems;
  • Identity and Access Management System;
  • Plaintext/Ciphertext Storage Locations;
  • Optional Encrypted Volumes.

Redundant locations

The DEIRI has a minimum of two geographically separated deployment locations where components of the DEI are duplicated for redundancy; replication between the separated DEI keeps them synchronized.

Depending on their risk-aversion, legal and performance requirements, companies may deploy a DEI in as many locations as necessary. Global companies are likely to benefit from a DEI on each continent they operate in, for optimizing responsiveness to business transactions, addressing component and location-specific disasters, and/or complying with country-specific legal obligations.

Front-end Processor

A DEIRI has at least one Front-end Processor (FEP) per location serving as the primary interface to applications requiring document-encryption and decryption services. The FEP is a stateful machine with a local database and provides web-services to applications.

The FEP is responsible for the following functions:

  • Ensuring the correctness of parameters in a request;
  • Authenticating and authorizing the request against an Identity and Access Management (IAM) system;
  • Scheduling the request on a Cryptographic Engine (CE);
  • Storing the output of the cryptographic operation in the appropriate storage location;
  • Replicating objects to other FEP within the DEI;
  • Taking over the processing of incomplete requests from a failed FEP in the DEI; and
  • Managing CE so they may be started or shutdown depending on the current processing load.

Cryptographic Engine

A DEIRI has a private cloud of Cryptographic Engines (CE) per location serving as the auto-scaling processing engine for encryption and decryption requests. They are stateless machines without any local storage and are responsible for the following functions (for encryption requests):

  • Receiving the requests from an FEP;
  • Authenticating and authorizing the request against an IAM;
  • Generating a cryptographic key in accordance with defined policies;
  • Escrowing the cryptographic key within a Key Management System (KMS) in the DEI;
  • Encrypting the supplied plaintext with the generated and escrowed cryptographic key;
  • Storing the ciphertext (encrypted data) on the configured/specified storage location; and
  • Handing the results of the request back to the FEP.

During a decryption request, the CE is responsible for:

  • Receiving the request from an FEP;
  • Authenticating and authorizing the request against an IAM;
  • Retrieving the ciphertext from the specified location and parsing it;
  • Recovering the escrowed cryptographic key from the KMS system in the DEI;
  • Decrypting the ciphertext;
  • Storing the plaintext on the configured/specified storage location; and
  • Handing the results of the request back to the FEP.

Key Management System

A DEIRI has at least one Key Management System (KMS) per location serving as the centralized repository and manager of all cryptographic keys. It is a stateful machine with a local database, and provides web-services to applications for the encryption and decryption of small, structured data-elements. It can be used by applications to request the encryption of data such as credit-card numbers, drivers license numbers, etc., without the overhead of going through the FEP.

The KMS can also be called upon by a CE and optional Encrypted Volumes (EV) to encrypt and store cryptographic keys, serving as a secure key-vault to the enterprise.

The KMS is responsible for the following functions in an encryption request:

  • Receiving the request from an application, CE or EV;
  • Ensuring the correctness of parameters in a request;
  • Authenticating and authorizing the request against an IAM;
  • Generating a cryptographic key in accordance with defined policies;
  • Encrypting and escrowing the cryptographic key within its internal database;
  • Encrypting the supplied plaintext6 with the generated and escrowed encryption key;
  • Storing the result of the cryptographic operation in its internal database; and
  • Replicating objects to other KMS within the DEI.

During a decryption request, the KMS is responsible for:

  • Receiving the request from an application, CE or EV;
  • Ensuring the correctness of parameters in a request;
  • Authenticating and authorizing the request against an IAM;
  • Retrieving the ciphertext and associated meta-data from its internal database;
  • Recovering the escrowed cryptographic key from its internal database;
  • Decrypting the recovered ciphertext with the recovered key; and
  • Handing the results of the request back to the calling application.

Identity and Access Management System

A DEIRI has an Identity and Access Management (IAM) system serving the DEI. The IAM is based on a Lightweight Directory Access Protocol (LDAP)-based Directory Server, and may be an existing enterprise IAM or one dedicated to the DEI. It is responsible for authenticating and authorizing requests to all components of the DEI and can be managed independently or through administrative functions on the DEI's FEP. The IAM is redundant and accessible from all DEI locations for business continuity.

Storage Locations

A DEIRI has one or more Storage Locations (SL) serving the DEI. They may be Direct Attached Storage (DAS), Network Attached Storage (NAS), Storage Area Networks (SAN) or Cloud Storage from commercial service providers or within the enterprise. The FEP and/or CE are capable of communicating with the SL and managing a large number of files on them.

When passing plaintext blobs between components in the DEI, the plaintext blobs are stored in Temporary Storage Locations (TSL), which are cleaned up by the FEP upon successful encryptions and by the calling applications upon successful decryptions. TSLs are always configured on DAS, NAS or SAN devices within the control of the data-owning company, and always inaccessible outside the company's perimeter.

Ciphertext files are stored on Permanent Storage Locations (PSL) which may be on DAS, NAS, SAN or Cloud Storage - anywhere the company chooses, since the data is encrypted on the CE (or the KMS in the case of small, structured data-elements) and the cryptographic key is within the KMS in the controlled part of the DEI.

Since the DEI is an implementation of the RC3 architecture, when the CEs and KMS are within a perimeter under the control of the data-owning company, the ciphertext comes back to the controlled environment before it is decrypted. This allows companies to comply with data-security regulations by simply keeping the CE and KMS in the jurisdiction controlled by the regulation regardless of where the ciphertext itself resides.

For example, if the CE and KMS were deployed within a member nation of the European Union (EU) in accordance with the security requirements of the EU Directive, companies would be in compliance with the Directives' security provisions7 even if the ciphertext were stored in a public cloud outside the EU. This is because the cryptographic keys never leave the safe-harbor of the DEI and, consequently, the ciphertext must come back into the EU's safe-harbor to be decrypted.

Encrypted Volumes

An Encrypted Volume (EV) is one in which the underlying storage volume or file-system is encrypted by a kernel driver or hardware component in the storage device. An EV does not require applications to be aware of cryptographic services since the EV assumes responsibility for encrypting/decrypting data, transparently, as it is written to/read from the volume.

However, since an EV does not distinguish between different applications and/or users, once an EV is accessible to a system, all software – including malware – may gain access to decrypted data. On the other hand, a cryptographically-aware application would make decrypted content available only to authorized roles/users within the application. Unless authorized users and/or applications themselves are compromised, breaching sensitive data outside the cryptographically-aware applications remains very difficult.

A DEIRI does not necessarily need EVs; however, they are useful as a transition-technology until the business application can become cryptographically-aware. In such an implementation, EVs will store their cryptographic keys within the KMS of the DEI and recover them only when necessary; this eliminates the risk of the key becoming compromised if the EV is stolen or lost.

Typical Transaction in the DEIRI

A DEIRI application can be written in any programming language. When a business transaction involves sensitive data (Class-1 and Class-2 data in the RC3 architecture) the application, typically, follows the following steps:

  1. The application makes a web-service request of the FEP with the required web-service operations' parameters;
  2. The FEP receives the web-service request and validates the request's input parameters;
  3. The FEP authenticates and authorizes the request against the configured IAM;
  4. Upon accepting the request, the FEP stores it in a Request queue and replicates it to other FEPs in the DEI. This ensures that the request is processed even if the receiving FEP becomes inoperable before processing it. Note that received – but non-replicated requests – are not likely to be processed by any FEP if the receiving FEP becomes inoperable before replicating the request. For the sake of this illustration, we will assume the receiving FEP remains operable for the entire transaction;
  5. Depending on the current processing load of all CE in the private cloud, the FEP chooses a CE and schedules the request on a specific CE. If the FEP realizes that all CE are operating at or above a configured threshold, the FEP automatically starts up a new CE in the private cloud (to the limit of virtual machines available to it in the private cloud) and schedules the request on the new CE;
  6. The CE receives the request from the FEP, and authenticates & authorizes the request against the configured IAM8;
  7. If the transaction is for an encryption:
    1. The CE generates a new Data Encryption Key or DEK (or reuses an existing one depending on the configured policy or supplied parameters in the web-service operation);
    2. The CE calls a web-service operation on the KMS to escrow the newly-generated DEK9;
    3. The KMS receives the request from the CE, and authenticates and authorizes the request against the configured IAM;
    4. The KMS generates a new cryptographic Key Encrypting Key or KEK (or reuses an existing one depending on the configured policy) for protecting the DEK sent by the CE;
    5. The KMS escrows the DEK securely and returns a Token reference to the CE as a response;
    6. The CE, upon receiving the Token, encrypts the plaintext sensitive data using a cryptographic algorithm and parameters based on configured security policy (or based on the web-service request parameters);
    7. The CE stores the ciphertext in the configured Permanent Storage Location (PSL) and returns the full pathname/URI of the ciphertext file to the FEP;
    8. The FEP, upon receiving the full pathname or URI of the ciphertext file, marks the request as completed, stores all meta-data in its database, notifies other FEP in the DEI through the replication sub-system and returns the response to the calling application. The request's response, typically, consists of the RequestID and the pathname/URI of the ciphertext file;
    9. This concludes the encryption transaction request.
    1. The CE retrieves the ciphertext file from the PSL (it is supplied by the FEP to the CE as a parameter in the web-service request);
    2. The CE parses the ciphertext file for meta-data related to the cryptographic operation and extracts the DEK's identifier – the Token provided by the KMS during the original encryption of the plaintext;
    3. The CE calls a web-service operation on the KMS to recover the escrowed DEK;
    4. The KMS receives the request from the CE, and authenticates/authorizes the request against the configured IAM;
    5. The KMS recovers any needed KEK internally;
    6. The KMS recovers the DEK and returns it to the CE as a response;
    7. The CE, upon receiving the DEK, decrypts the ciphertext based on meta-data parsed from the ciphertext file;
    8. The CE stores the decrypted plaintext in the configured Temporary Storage Location (TSL) and returns the full pathname of the plaintext file to the FEP;
    9. The FEP, upon receiving the full pathname of the plaintext file, marks the request as completed, notifies other FEP through the replication sub-system and returns the response to the calling application;If the transaction is for a decryption:It is even possible for applications to make direct requests to dedicated CE of a DEI for direct control of the request scheduling and meta-data management, if desired.
    10. This concludes the decryption transaction request.
  9. Where the plaintext is a small, structured data-element, as in a credit-card number, a bank-account number, etc., applications may call the KMS directly for the direct encryption and Tokenization of the data-element. This not only allows for faster processing, but better optimizes the different components of the DEI within a company;
  10. It is even possible for applications to make direct requests to dedicated CE of a DEI for direct control of the request scheduling and meta-data management, if desired.

Performance

How does a DEI hold up in a real-world environment? The complexity of the infrastructure, and communication between different components of the infrastructure, seem to imply that while it might address one set of problems related to data-management, control and security, it might introduce another set of problems with respect to business transaction performance and usability.

An implementation of a DEI for an e-commerce company in the US recently provides some insight into the performance of the DEI. While the business use-case described here might not apply to others, some information may be extrapolated from this implementation. In order to provide context for the performance metrics, the following technical details of the DEI are provided:

  • The FEPs are industry-standard six-core, 64-bit x86 processors operating at 3.2 Ghz with 16-GB of DDR3 DRAM operating at 1600Mhz. They use 64-bit CentOS Linux 6.x for an operating system. FEP services are provided by a Java Enterprise Edition 6 (JEE6) application running within an application server. Service requesters are authenticated and authorized against an LDAP Directory. The FEP's persistent data is stored in a local relational database management system, while bi-directional replication between FEPs is managed in the application layer;
  • The CE virtual machines operate on an industry-standard single-core, 64-bit x86 processor operating at 3.2Ghz with 8GB of DDR3 DRAM operating at 1600Mhz. They use 64-bit CentOS Linux 6.x for an operating system. CE services are provided by a JEE6 application running within an application server. Service requesters are authenticated and authorized against an LDAP Directory. There is no persistent data on a CE, and as such, there is no local database or replication services on this component;
  • The KMs are industry-standard six-core, 64-bit x86 processors operating at 3.2 Ghz with 8-GB of DDR3 DRAM operating at 1600Mhz. They use 64-bit CentOS Linux 6.x for an operating system. KM services are provided by a JEE5 application running within an application server. Service requesters are authenticated and authorized against an LDAP Directory. The KM's persistent data is stored in a local relational database management system, while replication between KMs occur at the database layer. Cryptographic operations of the KM use a Trusted Platform Module (TPM) with a true random number generator (TRNG) within the TPM;
  • Work-in-progress and completed files are stored on a NAS device using the Network File System (NFS) protocol;
  • The encryption algorithm uses the Advanced Encryption Standard (AES) algorithm with 256-bit keys; the hashing algorithm uses the Secure Hashing Algorithm (SHA-256) message digests while public-key cryptographic algorithm use 2048-bit RSA key-pairs;
  • Communications between all layers of the DEI, with the exception of the NAS (which uses network-segmentation for access control), are over SSL/TLS;
  • The network consists of gigabit Ethernet speeds between all components of the DEI, with the exception of the wide-area network between data-centers;
  • Every component of the DEI authenticates and authorizes requests - no assumptions are made about up-stream service requesters at down-stream layers;
  • Messages are logged at each component of the DEI, in a manner consistent with audit-requirements;
  • All software components of this DEI - with the exception of the CE private cloud's hypervisor - consists of open-source software.

Given the configuration above, the DEI implementation for this e-commerce company provided the following metrics:

  • The application using this infrastructure migrated 6+ million Portable Document Format (PDF) files of sizes ranging from 50K-200K each from its inventory to the DEI;
  • Daily transaction loads range between 20,000-25,000 PDF files steadily increasing the inventory of encrypted files managed within the DEI;
  • Following the typical transaction described in an earlier section of this document, the DEI executed web-service operations for document encryption in an average of 300ms each;
  • Similarly, the DEI executed web-service operations to decrypt documents in an average of 200ms each;
  • A second application, encrypting and decrypting Base64-encoded images of sizes ranging between 2-3K each, saw the DEI deliver cryptographic web-services at an average rate of 100 transactions per second;

During internal testing, the author's company was able to benchmark (on equipment less capable than that described above) the encryption of large files at the rate of one (1) gigabyte per minute.

While the example cited above is unlikely to match other business use-cases, given the configurability of the DEI and its components, companies can expect to tailor a DEI's performance and throughput through the use of faster components, more components (to distribute the workload), cryptographic accelerators and other performance-enhancing techniques. Just as network and storage technologies evolved from simpler configurations addressing local needs to highly complex infrastructures handling global requirements, the DEI is a similar evolution from simpler cryptographic tools to a sophisticated infrastructure designed to address scalable, highly-available and secure cryptographic services for the enterprise.

Summary

Cryptography components have been available for over forty years to protect sensitive information within business applications; sophisticated cryptographic key-management has been available for nearly a decade to address some of the complexities involved in handling cryptographic keys. The DEI is the logical evolution of these technologies to make data-protection an ubiquitous service on the network, accessible to systems and applications through a uniform interface, with the ability to address diverse data-security regulations while leveraging the cloud for business benefits.

About the author

Arshad Noor is the CTO of StrongKey, a Silicon Valley company focused on enterprise key-management infrastructure for 11+ years. Arshad has been in the IT industry for 26+ years building applications and complex IT infrastructure for some of the largest companies in the world. He is the creator of the industry's first open-source symmetric key-management system and many cryptographic tools to simplify the use of this technology within business applications. He is the author of many white-papers – including the Regulatory Compliant Cloud Computing (RC3) architecture. He can be reached at arshad.noor@strongauth.com.

ALSO SEEN IN: InfoQ
View full post