Paypal security upgrade

Context

Paypal performs in 2016 several changes to the security level of their payment platform. We use their 'Express Checkout' method and therefore this security upgrade will be an opportunity for us to rewrite our library.

Deadlines have shifted, I guess for the same reasons than SEPA one: clients were not ready... (SEPA is Single Euro Payments Area)

Basically, there are three main changes:

1. Paypal certificate now uses the G5 root, with a SHA-256 key. This implies the update of the truststore;
2. private merchants keys are upgraded from SHA-1 to SHA-256. The keystore must therefore be updated (for the private key part), as well as the truststore (for public part);
3. Finally, upgrade the TLS protocol to version 1.2, which is not the default mode in Java 7.

The first two changes are to be made on the stores of the sandbox and production environments.

No luck, PayPal has already updated their sandbox to TLS1.2, so it is no longer possible, without a compatible code, to use this test platform.

The hardest part when using Paypal is not to implement the technical solution:

• First, patience is required in order to find one's way in the doc, sometimes redundant, often confused.
• On the other hand, foresee a good headache with certificate formats and conversions from PEM to pkcs12 to jks, ...

To this end, a very good tool for the stores management is KeyStore Explorer, much more convenient than command line keytool.

Last word about key/trust-stores: do not specify them when posting data to the API with signature credentials, or the following error will occur: unable to find valid certification path to requested target

The stores are not used in this case, but their presence in the options disrupts the connection.

Express Checkout API Operations

Basically our thin Clojure library, called by our online shopping website, provides 3 methods that reflect those of Paypal's API:

1. get-url-paypal: connects to Paypal EC and retrieves a token used for a given sale transaction. User is redirected to Paypal's url and invited to confirm the payment of his purchase.
2. do-confirmation: once the user has validated or cancelled the sale, Paypal warns the merchant. By posting this 'confirmation', the merchant gets the status of the transaction, and updates the system and data accordingly.
3. get-fee: if acknowledgment was success, get the Paypal's final fee, calculated with obscure maths.

TLS1.2

Our legacy is still running Java 1.7, but all new developments with Clojure use Java 1.8.

As a consequence, direct test within a REPL leads to false positive because jdk8's default TLS version is already compatible with the Paypal sandbox.

One can still use jdk8 to launch lein repl, but version 7 must be forced for the project runtime.

Very easy, just add :java-cmd "/opt/jdk7/bin/java" to the project.clj.

All posts to Paypal API will now end with the following exception: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure.

Connection manager

We need to force the TLS protocol to use version 1.2 during our calls to Paypal's API with dakrone/clj-http library.

Simply specify in the options of each POST a :connection-manager. We choose to use a reusable one, based on a PoolingHttpClientConnectionManager so its construction should be memoized.

In its definition, we specify (.use Protocol" TLSv1.2 ") at the SSLContext level and everything is now ok.

Calls are now made like this:

(client/post api-endpoint
             (merge (make-connection-manager)
                    {:form-params ...}
                    keystore-truststore))

No more properties files

Previously the connection settings were registered in java properties files, but since these settings are specific and not used in the legacy java code, we gain by integrating them directly into this small clojure library.

By doing this, one of the basic rules of 12 factor app best practices is fulfilled.

The classic weavejester/environ is used because the tolitius/cprop solution seems a bit overkill in our case.

For the dev environment, the elements are set directly in the project.clj, or, if one does not want to commit them, in a profile.clj file, specific to the developer's machine.

  ; in project's map
  :profiles {:dev
             {:env
              {:dev?             "true"
               :server-port      "8080"
               :base-url-wo-port "http://localhost"
               :api-username     "merchant_api1.mail.com"
               :api-password     "..."
               :api-signature    "..."}}}

In production on the other hand, environment variables are used. It is preferable to Java system properties -dXX, which are visible in the command line of the tomcat/jetty/whatever process, so not very safe.

environ comes with a plugin, lein-environ, that generates a .lein-env file, extracting the settings from the project map. This file will be used with lein commands.

It's very useful but think to delete it for non-local tests, because it can interfere with the final configuration, obtained by overriding several sources.

For example the :dev? property should not be present in staging or production environment. However as it is not explicitly set to false, it will disrupt the config.

One last thing, the keys with compounds names should be written with '-' rather than with '.'. A kind message from environ reminds that important detail because otherwise nil values will be recovered...

Invalid cookie

A painful but harmless warning about invalid 'expires' attribute in the cookie header is displayed on each post to the API.

To avoid this warning, it is required to import the log4j/log4j dependency and to increase the alert level for the adequate logger.

Just write at the start of the namespace definition:

(.setLevel (Logger/getLogger "org.apache.http") org.apache.log4j.Level/ERROR)

Can't wait for next update