Sid on Key and Secrets Management; The Weak Link in Strong Data Protection

-Key and Secrets Management-

You've got me? Who's got you?

Thanks to a freak helicopter accident, Lois Lane is sent falling from a tall building with
only the cold hard concrete beneath her to break her fall. But out of no where comes;
Superman, making his first appearance to the people of Metropolis. The Man of Steel
calmly reassure Lois by telling her, “Don’t worry miss, I’ve got you,” in his best super
hero voice. But the awe struck reporter panics and yells, “If you’ve got me…then
who’s got you?!”. The Man of Steel just smiles and flies her to safety.

Of course, it was “Superman” to the rescue…the man who could fly…and he was The
Man of Steel…in essence, somewhat invincible, indestructible, unexploitable, etc. etc.
Unfortunately most things in our lives are not. So how come we are talking about
Superman and this famous dialogue from a 1978 classic? This is supposed to be
something about Cyber Security and Data Protection. Hang in there!!

Switching to the core subject now and will link it back to what I started the blog with.
Being a Security practitioner in the last few years of my career of 18 years, I have
somehow gotten passionate about data, it’s lifecycle, and how it’s managed and
protected within enterprises, both on-Prem and in the cloud. In the early years of my
career, Security and Data Protection were perceived differently from where I was in
the organizational structure – ranging from being a software developer, to a QA tester,
to an Architect, to developing enterprise strategies and standards, to developing
enterprise frameworks and leading engineering and architecture teams. We were all
happy in our world that someone was taking care of the sensitive data, whether it’s
Payment card data or PII, by providing a transparent layer of protection (mainly
encryption) on the data, way down the stack. As developers, we hardly cared about it
as long as it didn’t impact our application functionalities and performance. On top of
that, there was a firm belief on encryption being a hindrance….mainly because of
these perceptions:

1. Encryption breaks functionalities;


2. Encryption impacts database schema and storage requirements;


3. Encryption impacts application performance.

It was only when I focused on securing data on public clouds, especially SaaS, I fell in
love with Data Protection – Encryption, Tokenization, Cryptography. Couple years
down the line, I took over the enterprise cryptographic utilities and services. We took
a strategic decision to encrypt data at the application layer (elevating from the
traditional approach of storage level or transparent database file level encryption),
much to the discomfort of all the application teams, fearing about the
aforementioned perceptions. Suddenly, it became their problem as the model was
invasive and hence no longer transparent. So Security was no longer a perimeter
control, and was getting embedded and shifting both “up and left”. However, those
“mis”-perceptions have been solved in various ways (it took time, detailed
prescriptions and definitive proof over a period of several months) but that’s not
something we are discussing here.

Note: Enterprises processing and storing sensitive PCI, PII, PHI, etc. data, and who do not
have a Shift-Up-the-Stack encryption strategy, you would be fooling yourself if you think
if you can just encrypt the data at the database file layer or storage layer, you are
protected. All you are accomplishing by that encryption strategy is protecting yourself
from someone who could steal the disk and run away from your datacenter…and you
may satisfy the auditors by showcasing your data is encrypted at rest. You can’t prevent
any data exfiltration and avoid a breach of data in the clear state through attacks either
from a privileged insider (Database Administrator), or an external attack (OWASP Top
10) exploiting application vulnerabilities (remember Equifax, or more recently, Marriott?)

Now, this is not at all a novel strategy or unique step towards data protection, to
protect us from the innumerable number of attacks we face each day, and minimize (if
not eliminate) the impacts of a data breach whenever it happens (remember, it’s a
matter of when, not if). So what is really the crux of this strategy that was so important
and vital than just encrypting data at the application layer??

Application teams, have been for a while, performing some level of encryption at the
application layer – whether it’s protecting properties files containing sensitive data
such as application secrets, if not Payment or Personal data, leveraging cryptographic
libraries (that come with the platform), standard encryption algorithms, and
application generated keys. This is not really an alien model for the developers,
regardless of the perception and anxiety to perform encryption and key management
at the application layer. Developers manage the cryptographic stack and key
management in the best ways known to them – which by the way, are sub-optimal in
most cases. In fact, this is not what are they hired and paid for, and hence they
don’t hold the expertise on.

Most implementations of application level encryption within enterprises follow a
typical model:

1. Generate a key at the application layer (in some cases, from an HSM), store it on a
properties file (and quite a few cases, hard coded in the
code itself) and encrypt the sensitive data;

2. A bit of an enhancement could be there on top of it (some developers are better
than the rest when it comes to key management) – Use another key to protect the
Data Encryption Key (DEK), and if we are lucky that Key Encryption Key (KEK) is stored
someplace else…not within the same properties file that stores the DEK – maybe
within a database.

3. And in very rare cases, within organizations that have a better security posture, that
KEK maybe protected with a password or some sort of secret, thus adding more
defense in depth.

So, what’s wrong with this model…well, let’s now go back to the Lois Lane &
Superman scene….

As illustrated in this picture, replicated twice….the scene is exactly the same, but the
actors are playing different roles from left to right. If Lois Lane is the treasured crown
jewel (the damsel who needs to be protected from the demon), and Superman is the
savior (the encryption key), then Superman will be doing his job undoubtedly. But
remember the statement before that most things in our life aren’t Supermanish??
Here he is not invincible, and needs to be managed and protected as well – after all
what’s the point of locking the door if the door key was left under the doormat.

So now the roles change in the scene…the encryption key (DEK) is the damsel in
distress and our hero is another key (the KEK) – but again, who’s got his back? Shifting
to the extreme right of the picture – roles change again where the KEK is Lois Lane
and possibly our Superman is played by a “not nearly close to a so called Man of
Steel” password, managed possibly by a Database Administrator. So what next? With
that password taking over the role of Lois Lane, who’s our Superman now??

So, I hope that I was able to drive the idea home to some extent, that we can create
layers of depth with our defenses, but at the end of the chain, someone will always be
Lois Lane, the damsel in distress that will need a Superman for being rescued….and
we will run out of Superman for all practical purposes. So what do we do?
Fortunately, there is a light at the end of the tunnel.

Enterprises looking to secure data with the highest level of data protection, needs to
have a 3 pronged strategic approach:

1. Shifting–up-the-stack – Apply Application Level Data Encryption for all critical
sensitive data processed and stored by these applications, at a field level (along with
encryption of sensitive unstructured data), at the application tier, so that data travels
and is stored in an encrypted state. This greatly minimizes (if not eliminate) the risk of
data breaches due to insider threats (privileged users having access to the databases
and file systems), external attacks in the various OWASP top 10 methods (e.g. SQL
Injection), and migration of sensitive data elements to Data Warehouse systems, Big
Data analytics nodes, backups and archives. The goal is to de-sensitize the data as
much as possible so that even after they are breached and fall in the hands of the
attacker, it’s neither of any value to them, nor the incident is of any impact to our
customers or employees.

2. Abstract Key Management from Developers – Application teams should not be
worried about how to manage (store, protect, rotate) encryption keys, nor should they
be allowed to have the knowledge or possession of the actual physical keys. They
should be provided with a pointer or identity of the key, and offered with an
abstraction layer that embeds crypto & key management, that automatically takes
care of the generation, retrieval, caching, protection, refresh of the keys.

3. Abstract Secrets Management from Developers – Application teams should not
be worried about how to manage (store, protect, rotate) secrets such as API Secrets,
passwords, etc. that allow them to consume encryption services or connect to
databases, nor should they be allowed to have the knowledge or possession of these
secrets. These secrets should be stored and managed within an enterprise Vault and
developers should be offered with SDKs and APIs to retrieve secrets from the secrets
vault programmatically. The vault solution should also allow periodic rotation of
secrets without impacting the applications, nor require updating secrets within
applications thus requiring redeployment and taking outages.

So to wrap it all up….Encryption is seldom cracked but often bypassed. It doesn’t
matter how much encryption we do, if the keys and/or credentials to the keys, are not
protected, it doesn’t take too much for a hacker to obtain and leverage our Crown
Jewels and create significant impact to our business and reputation. Hence, adequate
key management and secrets management are equally important as implementing
strong cryptographic methods on data. Most major Cloud Services providers are now
offering their KMS or Key Vault offerings. In these models, Envelope encryption is
implemented, whereby the data and the DEK are stored in an encrypted state (DEK is
encrypted by a KEK or usually referred to as the CMK which is stored within the KMS),
and applications need to decrypt the DEKs first at the KMS before they can be used
for decryption of the encrypted data. These KMS or Key Vaults also offer adequate
management of secrets such as passwords, API secrets, etc. Other COTS solutions
implemented within enterprises, also provide abstraction of the key management
and secrets management from the developers. Enterprises should have a solid Data
Protection strategy in order to leverage these tools and services in their right
architecture and potential, to implement a solid data protection program involving a
robust key and secrets management (Superman) around protecting their Crown
Jewels (Lois Lane).



-Sid Dutta

Articles

Home

Key and Secrets Management

Bring Your Own Key

Exploring Cloud Service Providers' Crypto and Key Management Services

Importance and Advantages of Format Preserving Data Protection

Recommendations for Implementing the Right Cloud Crypto and Key Management Solution

Online Shopping Security in the Age of COVID-19

Published Articles and Press Releases

Videos