QuestDB supports ingesting records using InfluxDB line protocol. This means that you can benefit from a simple, lightweight, and convenient message format to add data points to tables. We've further improved support for this feature by adding authentication, so your endpoint is more secure. This post describes how we added this functionality and how to enable it via QuestDB configuration.
Adding InfluxDB line protocol support to QuestDB
InfluxDB line protocol is popular because it is a simple text based format, you simply open a socket and send data points line by line. Implementation is easy because encoding is trivial and there is no response to parse. The protocol can be used over UDP or TCP with minimal overhead.
This is all great as long as your endpoint can not be accessed by unauthorised actors that could send junk to your database. If your endpoint is public, then you could secure it by encapsulating it in a secure transport layer such as TLS, adding complexity to your infrastructure that needs to be managed. This is something we sought to avoid. Our goals when implementing authentication were:
- Use a secure, future proof, authentication method.
- Minimise protocol complexity and transport overhead.
- Configuration solely in QuestDB without the need for storing secret data.
Adding authentication to InfluxDB line protocol
To these ends we decided to provide authentication for the InfluxDB line protocol over TCP with a simple challenge/response mechanism, where the challenge is a nonce and the response a signature. Elliptic curve cryptography (ECC curve P-256) with SHA-256 was chosen for the signature algorithm, this ensures strong authentication that is hopefully future proof. The elliptic curve cryptographic keys have a public and secret component, it is possible to configure QuestDB with just the public part, thereby mitigating any risks of storing secret information on the server. Languages such as JavaScript and Go have standard libraries that implement ECC, the JSON Web Key standard can be used to store and distribute the keys in a clear and ubiquitous manner.
The authentication challenge/response mechanism was chosen to minimise the impact on the protocol, it works as follows:
- When the client connects it sends its key id to the server.
- The server responds with a nonce in printable characters.
- The client responds with the base64 encoded signature of the nonce.
- If authentication fails the server will disconnect, if not then the client can revert to sending standard InfluxDB line protocol data points.
We developed this form of authentication in response to users who have QuestDB deployments where a simple form of authentication is required without the overheads of full encryption.