Format Preserving Encryption with NIST recommendations

In 2014, JPMorgan Chase was under a massive cyber- attack in which the data of 76 million private customers and 7 million business customers was leaked. The attacker was able to get administrative rights due to non-functional two-factor authentication and was able to access user data. The webserver and the web application were secured, but the database remained unencrypted where the data was copied from.

If Format Preserving Encryption had been used, this situation could have been mitigated. With FPE, there would not have been any change to the database schema, and the encryption could be integrated on the fly.

What is Format Preserving Encryption?

For basic information in regard to FPE, please refer to this link

To give you some context, Format Preserving Encryption or FPE is an encryption algorithm used to preserve the format of the clear text while it remains encrypted. However, the strength of FPE is lower compared to AES. FPE is, however, an important mechanism for encrypting data whilst preserving the data length. FPE ensures that while data remains encrypted, all programs, applications and databases continue to be functional.

Why use Format Preserving Encryption?

Implementing a perfectly secure network is harder than just encrypting your data. Encrypting data is cheaper, easier, more secure, and thus better in every way imaginable.There are many organizations with a legacy infrastructure which may not be as secure. Thus, protecting all of the data in the legacy network protects the data even if the network gets compromised. This change can be made with almost no impact to existing infrastructure.Even if the organization has a robust infrastructure, it may face issues while the data is under audit. No one wants to reveal raw customer data which may put their reputation under seize. Thus FPE can be used to de- identify all data, remove all PII (Personal Identifiable Information) of customers and would serve as an extra defence mechanism when data is breached. –

As per NIST 800-38G:

Format-preserving encryption (FPE) is designed for data that is not necessarily binary. In particular, given any finite set of symbols, like the decimal numerals, a method for FPE transforms data that is formatted as a sequence of the symbols in such a way that the encrypted form of the data has the same format, including the length, as the original data. Thus, an FPE encrypted SSN would be a sequence of nine decimal digits.

So, if we convert a 16-digit credit-card number it will return another 16-digit value. A 9-digit Social Security Number would return another 9-digit value.This cannot be achieved with other modes of encryption, such as AES where if we encrypt a credit card it will look like 0B6X8rMr058Ow+z3Ju5wimxYERpomz402++zNozLhv w= which is greater than 16 digits and has not just numbers inside it.This kind of output would not work in most systems or databases where we must follow strict data types. Thus if it expects 16 digit numbers, this type of output would not suffice and may even result in a system-wide crash.

NIST SP 800-38G recommends ways through which we can encrypt this sensitive data in the databases. These solutions would also follow FIPS 140-2. So if someone wishes to use FPE, they can rest assured that they would be following almost all regulations and standards which would be enough to satisfy regulatory requirements of HIPAA, PCI DSS etc.

FPE By Google Cloud

Firstly, Google is the only cloud provider currently who is providing FPE through their DLP APIs. Now, most of the organizations are currently transitioning to the cloud, but to make that transition happen securely, data should stay encrypted while in transit.

To do that, Google provides FPE under Cloud Data Loss Prevention. Using DLP API, customers can encrypt their data using FPE and de-identify information using predefined info types such as Credit card numbers, phone numbers, etc.This would encrypt the data, and make it safer to transition to the cloud. The transfer of data from a datacenter to a database on the cloud would also maintain their referential integrity as well as their format.

Conclusion

FPE is an encryption mechanism that keeps data encrypted while databases and applications remain functional. FPE preserves the format of the data which allows legacy systems and networks to remain functional while data is encrypted. GCP provides a DLP API which offers FPE through their platform. This helps in making all types of systems and programs functional/available and also improves data auditability by removing all PII data within it.

Sources: https://www.bbc.com/news/business-29470381