tl;dr:

100% Uptime during Upgrade

Automatic Upgrade

post-upgrade from Console

In a similar vein, as more and more of our customers run EJBCA on the PrimeKey Appliance and thus don't have access to the command line. As of EJBCA 6.8.0 it's been possible to perform post-upgrades from the UI. When a post-upgrade is required, the System Upgrade option will appear in the menu:

Choosing it will bring you to a screen used to perform the post-upgrade action:

Conclusion

With this blog post and our latest round of QA, we hope that we've solved all existing upgrade issues, and that we can make running the latest version of EJBCA as easy and manageable as possible.

Cheers!
Tomas Gustavsson
CTO

Mike Agrenius Kushner
Product Owner EJBCA

Full PSD2 Support

All of the approving RA administrators in the final approval step will be shown the following warning before the approval passes:

dnsName SAN can be Automatically Populated by the CN

We've added a setting to End Entity Profiles to allow the dnsName Subject Alternative Name field in a certificate to be filled in by the Common Name (CN) value in the Subject DN.

Configurable SN Entropy, Default Value Raised to 20 Octets

URL Metadata Type Added to Approvals

Upon popular request, we've added a URL metadata type to the partitioned approval profiles. It allows the approving RA administrator to enter a URL while performing the approval, e.g pointing to a file upload at an external location.

Upon later review of the approval, it will show up as a hyperlink:

Experimental: Configuration Checker

Lastly, we're trying out an experimental new feature in EJBCA 7.0.1, the Configuration Checker. It displays an (incomplete) list of common configuration issues on the front page. 

If you'd like to try it out, it can be activated in its own tab under the System Configuration:

Roadmap Update

Partitioned CRLs

Long and enduringly requested, EJBCA 7.1 is now capable of producing partitioned CRLs. Activated under the CA configuration, the number of partitions per CRL is dynamically configurable, allowing new partitions to be added as the CRL grows, and assignment to older partitions to be suspended in order to allow for future growth. CDP partition assignment is random in order to allow for even distribution of certificates, and partition definition can be looked up in the CDP extension as defined in RFC5280.

For those of you not wishing to use partitioned CRLs life will mostly move on as usual while for those of you applying partitioned CRLs to existing installations you will retain a legacy CRL for pre-existing certificates (as the CDP can't be changed retroactively) while newly issued certificates will be issued to partitions.

Deprecation and Removal of Hard Token Support

VA and RA Specific Distributions

EJBCA 6.15.2 CE Available on Docker Hub

The Evolution of a Company

PrimeKey's logo ca 2005

PrimeKey's logo ca 2014

Our logo since 2017

The Evolution of a Team

I joined PrimeKey back in 2010, back in the heydays of the entire company sharing the same volume of air during a working day. Except for our then CEO and a couple of other staff, the bunch of us (<10 people) mostly filled the same roles to varying degrees: developer, professional services, IT, support, QA, documentation, tech sales - there was little delimitation and even less structure. You did what was needed and learned the skills required to overcome the next task at hand. We were happy in those days, though we were disorganized. 

I was originally hired as a developer, but back in those rousing days the prospect of travel appealed to me as well. Between writing (and refactoring) mounds of code, working with customers took me from Ankara to Antwerpen, from Brussels to Baku. My first major project was working with our upcoming Common Criteria certification back in 2011, which taught me the joys of writing months worth of pointless documentation and speaking to auditors.

What I'm getting at was that we existed in mostly a state of unmitigated pandemonium; working on customer requirements as they arose, our own fancies and interests, putting out wild bush fires and occasionally lighting them. At a tech conference in 2012 I described our work style as "Controlled Chaos", which very much makes me cringe today. 

The Team(s)

The Result

The Evolution of an Office 

The Stockholm office has changed addresses four times in nearly 20 years. The gang started off sitting in what amounted to a shoebox, before moving to slightly bigger digs in 2009, which is about the time I came into the picture.

The new office was slightly larger than the old one, but the entire team still shared a single room (with an office for our CEO and VP Sales), happily all breathing the same air for the entire working day. 

My desk back in 2010, with era-appropriate camera filter and still relevant stacks of DVD blanks.

The author exploring his new digs. New feats of concentration were attained in this small room.

Our office since 2018

A gaming room with a foosball table, the true sign of prosperity

Team Alice's work area

The home of Team Bob

The dwellings of the SignServer Team

Prerequisites

• The OpenSC PKCS#11 implementation 
• The YubiKey PIV Manager, Command Line Tool or UI 
• A compatible browser, FireFox or Chrome Recommended.

Creating a key pair on your YubiKey

Enrolling the YubiKey to EJBCA

Importing the Certificate to the YubiKey 

Configuring FireFox to use YubiKey

Configuring Access Rights in EJBCA 

This is done using Roles as with any user EJBCA administrator, but here are the exact steps:

1. In the EJBCA CA UI, pick Roles and either create a new Role or add the new administrator to an existing Role

2. To add the administrator, click on Members, pick the appropriate CA and enter the identifying the information for the certificate, preferably the serial number

3. An easy way to find the serial number is to view the certificate in OpenSSL using the command:

$ openssl x509 -in alanwidget.pem -text -noout

The serial number can be copied, converted from hex to decimal using a converter and then used in EJBCA.

4. Finally, click on Access Rules and set the required rules for your administrator
5. The next time you start a new session, your YubiKey will be offered as an option

•  The database-backed storage will be used after a restart when the in-memory cache is empty.

• The in-memory storage has a limit of 100 000 certificates by default, and will only keep the SCTs for the most recently requested certificates. The database-backed storage has no such limit and will be used for SCTs for less frequently requested certificates.

• The database-backed storage will store partial results for a certificate, allowing EJBCA to retry a submission efficiently at a later point.

• CA activation and deactivation.

• Crypto Token activation and deactivation.

• Key generation and removal.

New REST end points in Swagger-UI

About 3GPP/LTE

1. The factory site is provisioned with a Vendor CA, separate from the main PKI intended to be used. The Vendor CA certificate is signed by the vendor's PKI. 
2. The vendor prepares a series of unique identifiers (i.e serial numbers) for each device to be produced. This ensures that only the authorized set of devices will be able to enroll against the vendor's PKI. 
3. As each device is manufactured it produces its own key pair. The public key is signed by the Vendor CA, and the device is initially provisioned with a Vendor Certificate (which includes the serial number). 
4. As each device comes online it enrolls to the vendor's PKI over CMP with 3GPP, using the Vendor Certificate to authenticate itself. 
5. The device then receives its device certificate from the vendor's PKI. 

Generalized Workflow

CA - Device Communication Through a Bespoke RA

What is a CVE? 

PrimeKey and CVEs

Classification

Vulnerability

Security Hardening 

Exposure

Vulnerability in an underlying library

$ ssh-keygen -b 384 -t ecdsa -f id_ec384 

string "ssh-rsa"
 mpint e 
 mpint n

Prerequisites

Step 3 - Install CertBot on the Apache Web Server Host

About EdDSA

EdDSA is a fairly new signature algorithm, at least if we compare to the classic algorithms we use, where RSA was introduced in 1977 and ECDSA entering wide use in the early 2000's. In contrast EdDSA was published in 2011. EdDSA is meant to be simple, elegant, and well defined, so it is hard to make security catastrophic mistakes upon use, something that is rather easy to do with the classic algorithms. EdDSA, with regards to digital signatures, consist of two distinct variants, Ed25519 and Ed448, with different key lengths. The usage of EdDSA, as relevant for us, has been standardized in RFC8032, RFC8410 and RFC8419.

EdDSA in EJBCA

Albeit not very widely used, we have been asked on occasions for support of EdDSA in EJBCA, commonly in IoT related use cases. Due to these requests we added software support for Ed25519 and Ed448 in EJBCA 7.4.0 in June 2020. As EJBCA is used in high security PKIs, Hardware Security Modules are typically used, and we have recently tested support for EdDSA in HSMs. 

This post describes the details of what is involved in supporting this new signature algorithm, in software and in hardware. The final result can be seen in the EJBCA documentation.

A note of caution: implementing cryptography is not for everyone, and we could not have done it without the help of our friends from Bouncy Castle, who helped us with the hard core details, and even gave us an introductory training session on the specific topic of EdDSA. Bouncy Castle support contracts are available though Crypto Workshop.

Software Support

The first step when supporting a new algorithm for PKI usage, in Java, is to get all the ASN.1 and Java crypto stuff in place. Without that, there is no going forward. This includes at least, but not exclusively the following parts:

• ASN.1 Object identifiers
• Public and Private key classes
• Reading and writing (encoding and decoding) said public and private keys
• SubjectPublicKeyInfo ASN.1 structures for properly encoding public keys in certificates and CSRs
• Signatures themselves, on at least certificates, CRLs, CSRs, OCSP responses and CMS messages
• Conversion between names (for example Ed25519) and OIDs (1.3.101.112)

A specific trick worth mentioning, that we stumbled upon, is that there are two versions of encoding/decoding format of EdDSA. A version 1 and a version 2, where different software implementation, for example OpenSSL, produces the v1 format, while others produce the (newer) v2 format. Needless to say, we needed to handle both.

For EJBCA, we are lucky to use the Bouncy Castle crypto APIs, which are commonly among the first to implement anything new. As Bouncy Castle had support for all the relevant RFCs EJBCA could start adding the application layer support right away. There were a few additions needed on the way, mostly related to various corner cases or handling encoding/decoding of keys and PKCS#12 files from different sources, that needed minor additional features in Bouncy Castle. As always, they were fast to add the needed tricks in the crypto API.

Usability

Apart from the core crypto pieces, EJBCA also implement a bunch of usability features making it easy(!) for users to use PKI with various algorithms. With EdDSA being so simple, both the keys and signature algorithm is referred to Ed25519 and Ed448, it still needed some extra code to fit into the overall structure. For the classic algorithms there are countless combinations, for example an RSA 2048 bit key can be used with all RSA signature algorithms:

• SHA256WithRSA
• SHA384WithRSA
• SHA512WithRSA
• SHA256WithRSAandMGF1
• ...

Just to name a few. And the same goes for ECDSA, but with even more combinations are there are numerous different EC curves to choose from. All in all there are hundreds of combinations that can be used in EJBCA, and we try to make it easier by or example limiting the available signature algorithms based on the selected key type. So albeit EdDSA having limited number of combinations, some if's and but's in the code is still needed for it to fit in the overall usability framework.

Another trick we play is that a suitable key encryption algorithm is selected, for Key Recovery, based on the signature algorithm used, typically this means selecting an RSA based algorithm, and we have a method for this, where EdDSA had to be added as well.

Adding HSM Support

We, and most of the world, use HSMs through the standard PKCS#11 API. In the until recently latest version of PKCS#11, version 2.40, there was no standardized support for EdDSA. As there was still a requirement from some users for EdDSA, the inevitable result was that some HSM vendors supported EdDSA using what is called Vendor Defined Mechanisms. This is of course a nightmare for product implementors, as specific support has be be built for HSMs from different vendors. PKCS#11 version 3 however, which was released only this year, introduced standardized support for EdDSA.

Now HSM vendors have moved to use the standardized way of supporting EdDSA, which means it's a good time for us to look at supporting HSMs for this algorithm.

Having recently introduced the P11-NG Crypto Token in EJBCA Enterprise, we have low level control of PKCS#11, as used by EJBCA, which was needed in order to support this new feature, P11v3 feature in a P11v2.40 library. In addition, SoftHSMv2 has support for EdDSA, making it easy to develop without access to a real hardware security module.

The last trickery showed up when generating keys and reading them from the HSM. According to the PKCS#11v3 specification the EdDSA public key is generated using the mechanism CKM_EC_EDWARDS_KEY_PAIR_GEN with the same parameters as a normal ECDSA key, which means that a CKA_EC_PARAMS is used in the public key template, with the OID of the curve as parameter, in this case the OID of Ed25519. The public key is then stored as a CKA_EC_POINT. The CKA_EC_POINT is normally used to store EC curve points, but in the case of Ed the point it's not quite the same. This required an if statement and some special handling in the case of EdDSA keys.

In the case of ECDSA:

final java.security.spec.EllipticCurve ellipticCurve = EC5Util.convertCurve(bcspec.getCurve(), bcspec.getSeed());
final java.security.spec.ECPoint ecPoint = ECPointUtil.decodePoint(ellipticCurve,
        ASN1OctetString.getInstance(ckaQ.getValue()).getOctets());
final org.bouncycastle.math.ec.ECPoint ecp = EC5Util.convertPoint(bcspec.getCurve(), ecPoint);
final ECPublicKeySpec pubKeySpec = new ECPublicKeySpec(ecp, bcspec);
final KeyFactory keyfact = KeyFactory.getInstance("ECDSA", BouncyCastleProvider.PROVIDER_NAME);
return keyfact.generatePublic(pubKeySpec);

 and in the case of EdDSA:

X509EncodedKeySpec edSpec = createEdDSAPublicKeySpec(ckaQ.getValue());
final KeyFactory keyfact = KeyFactory.getInstance(oid.getId(), BouncyCastleProvider.PROVIDER_NAME);
return keyfact.generatePublic(edSpec);

As usual, a big thanks to the team at Bouncy Castle for help.

Summary

With all the above details in place, we now have the functionality we need in EJBCA:

• Generating Ed25519 and Ed448 key pairs (in software and in HSMs)
• Retrieve public keys
• Test a key pair
• Sign and verify
• Certificate
• CRL
• OCSP response
• CSRs (PKCS#10 and CRMF)
  
• CMS message
• Read and write
• PEM files
• PKCS#12 files
• Presenting EdDSA in Web UIs in a user friendly way 

Compliance Overview

Standards and other specifications that you may be required to show compliance with are usually large with many options. Many times these options are also described vaguely requiring some level of interpretation. Interpretations usually get to a point of common understanding between different stakeholders after some period of time. Adding insult to the injury is that interpretations and specifications also change over time. In addition, many standards are large and contain many parts that are irrelevant for most use cases, so implementing a standard to the letter is not cost efficient neither for the implementor nor for the user. 

All this considered, compliance is not a trivial concept.

Related specifically to PKI, a very small pick of some specifications that you may be asked to be compliant with includes RFC5280, RFC4210, RFC4211, RFC7030, RFC8555, RFC6010, RFC6960, RFC3739, CA/B Forum BR 1.7.3, EV SSL Certificate Guidelines 1.7.4, EV Code Signing Guidelines v. 1.4, 3GPP 33.310, EN 319 401, EN 319 411, EN 319 412, ETSI TS 119 495, ICAO 9303, PCI-DSS, NIST SP 800-73, FIPS 201-2, FPKIPA, ETSI TS 103 097, IEEE 1609.2, just to name a few.

It is safe to say that any type of compliance requires a fair amount of knowledge about a lot of details.

Implementation or Operation

A question that comes up is who manages compliance. Is it the implementor of a product or the organization operating the product delivering a service? In reality it is usually somewhere in between. Products should implement capabilities of configuring them, by the operator, to be compliant. Due to the number of standards, each requires different configurations and a configuration compliant with one specification may be non-compliant with another specification, here it is up to the operator to keep track of configuration.

Compliance over Time

Compliance with various standards is most of the time possible to demonstrate at a specific point in time, still considering that you don't fully implement many standards. But compliance is hard to keep up in the long run as specifications evolve constantly, interpretations change in various forums (some closed and some open). Just considering the sheer number of specifications to be compliant with, and the number of forums where these are handled in practice makes it impossible for the people actually implementing the standards to keep track of everything that is going on.

Compliance is continuous work, it is not a one-time effort.

Falling out of Compliance

How does it happen that an organization, being compliant once, becomes non-compliant? There are a number of reasons this can happen, including but not limited to:

• Compliance specification updates

• Specification re-interpretation in the governing forum

• Discovery of previously non-discovered compliance details

• Accidental re-configuration of the system

• Deliberate re-configuration of the system to achieve one compliance, breaking another

• Software bugs

As understood by now, maintaining compliance requires continuous work and tools to monitor the system and compliance, as well as to track changes and ensure it's done in a controlled and repeatable way.

Do I need Compliance?

This question has to be answered by yourself of course. There are basic things you absolutely want to be compliant with, such as RFC5280, if you want to be interoperable with various client software. There are other things that may become more of a burden, say a WebTrust or eIDAS audit if you do not have any external driving force for these. The reasons for compliance with the range of standards (a few listed above can be ranging depending on your use case:

• As wide interoperability as possible
• Industry-specific standards
• Regulations
• External or internal security requirements
• Generic trust posture (also internal or external)

You typically don't want compliance requirements that force you to do things that drive cost without making sense for your specific use case.

One Time vs Continuous Monitoring

One time, and regularly occurring, compliance monitoring plays an important role. When implementing a new system, or a new requirement, it is normal to manually verify compliance. During regular audits this is also verified. During these verifications personnel with the needed skills analyze systems and output, creating appropriate configuration, controls and processes.

During operation of the system you can deploy tools, that continuously validate the compliance achieved in the above step, ensuring that things don't start to break unexpectedly.

Any serious compliance work should involve both the regularly occurring (one time) compliance audits and continuous monitoring.

EJBCA Compliance Assisting Tools

EJBCA comes with a large set of tools that can help you manage compliance over a long time, as well as monitor for unexpected changes, keep track of changes and perform changes in a repeatable way. Using these tools, tailored for your specific needs, can help lower the risk of falling out of compliance.

These tools are numerous and it can be hard to keep track of all of them. This blog post outlines the most important ones, as of the writing date. New tools are continuously developed, and new documentation added, so be sure to check the product documentation and release notes when new versions of EJBCA are released.

Also note that different tools are more or less easily available on different platforms. For example using an Appliance or Cloud installation with limited access to the command line limit available options to run local scripts and similar.

Validation and Compliance features

Are  your CRLs are scaling out of proportion, clients are complaining about timeouts and your VA is on its knees? Are your certificates counted not by thousands but by the millions? Never to fear, EJBCA is designed to handle some of the world's biggest PKI, so read on to find out how!

What can you do about your Architecture?

Clustering and Load Balancing

database.useSeparateCertificateTable = true

the certificate body will be stored in the table Base64CertificateData instead of CertificateData. 

Base64CertificateData can then be sharded and placed on a different database volume. 

CRL Partitioning

Service Pinning

Ephemeral Certificates

Precompiled OCSP Responses

 Introduction

EJBCA Configuration

Certificate Profile

The pertinent fields in the Certificate Profile to set are as follows:

• Authority Key Id: Use
• Subject Key Id: Use
• Key Usages: Digital Signature
• Extended Key Usages: Client Authentication

End Entity Profile

Subject DN Fields

• Common Name (CN)
• Organizational Unit (OU)
• Organization (O)
• Country (C)

Subject Alt Names 

• RFC 822 Name
• DNS Name
• IP Adress
• MS User Principal Name

SCEP Alias

Google Cloud Certificate Connector

Installation

Configuration

Adding your own CA to Google Cloud

Testing