API Configuration Guide

Note: ALL configuration assumes a vinyldns namespace. For example, sqs settings would be under vinyldns.sqs.

Configuration

There are a lot of configuration settings in VinylDNS. So much so that it may seem overwhelming to configure vinyldns to your environment. This document describes the configuration settings, highlighting the settings you are most likely to change. All of the configuration settings are captured at the end.

It is important to note that the api and portal have different configuration. We will review the configuration for each separately.

Configuration Overview

How do we config?

All configuration is done using Typesafe Config. It provides a means to specifying default configurations, and overriding the configured values in a number of ways:

  1. The default configuration provides “safe” default values for all configuration. This makes it possible for you to only change the configuration values that you need to, and assume the default for the rest. This can typically be found in a file named reference.conf. The Typesafe Config library manages populating unspecified values for you automatically.
  2. You can override the reference.conf file by providing your own application.conf file when the system starts up. We will review how to do that in the sections that follow.
  3. You can override individual configuration properties when the application starts up using standard jvm arguments. For example, you can specify -Dmy.config.value=42, and that will override both application.conf and reference.conf (defaults)
  4. You can further override configuration properties with environment variables. The Typesafe Config provides special syntax that allows you to use environment variables. You can make the environment variable optional (meaning use it if it is there) or required (fail to start up without the environment variable). We will illustrate use of environment variables in this guide.

Using Environment Variables

We strongly recommend that you use environment variables in particular for secrets. Laying down environment variables in a flat file is a security vulnerability for your installation. To demonstrate environment variable usage, here is a following snippet…

  queue.settings {
    access-key = ${AWS_ACCESS_KEY}
    secret-key = ${AWS_SECRET_ACCESS_KEY}
    signing-region = ${SQS_REGION}
    service-endpoint = ${SQS_ENDPOINT}
    queue-name = ${SQS_QUEUE_NAME}
  }

In the example, if any of the values in ${xxx} are not found in the environment, the application will not start up!

Configuring API Server

The API configuration has a lot of values, the important ones reviewed here. There are several configuration settings that are specific to your environment.

The most important configuration is around your system dependencies. Presently, these are your settings for:

  • AWS SQS
  • AWS DynamoDB
  • MySQL

We are actively working on supporting different message queues and data stores. Look for those to become available shortly

Queue Configuration

VinylDNS supports both SQS and MySQL queue implementations.

There are a couple of implementation-dependent settings that need to be specified:

  • messages-per-poll: Number of messages retrieved in a single queue receive request. Valid values are 1 through 10 (default).
  • polling-interval: Interval to delay between each poll for messages.

If using SQS, be sure to follow the AWS SQS Setup Guide first to get the values you need to configure here.

If using MySQL, follow the MySQL Setup Guide first to get the values you need to configure here.

The following in a sample SQS config:

vinyldns {

  queue {
    class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"

    messages-per-poll = 10
    polling-interval = 250.millis
    
    # connection information to SQS
    settings {
      # AWS access key and secret.
      access-key = "x"
      secret-key = "x"
      
      # Regional endpoint to make your requests (eg. 'us-west-2', 'us-east-1', etc.). This is the region where your queue is housed.
      signing-region = "x"
      
      # Endpoint to access queue
      service-endpoint = "http://vinyldns-elasticmq:9324/"
      
      # Queue name. Should be used in conjunction with service endpoint, rather than using a queue url which is subject to change.
      queue-name = "vinyldns"
    }
  }
}

The following is a sample MySQL queue config:

queue {
  class-name = "vinyldns.mysql.queue.MySqlMessageQueueProvider"
  
  polling-interval = 250.millis
  messages-per-poll = 10

  settings = {
    name = "vinyldns"
    driver = "org.mariadb.jdbc.Driver"
    migration-url = "jdbc:mariadb://localhost:19004/?user=root&password=pass"
    url = "jdbc:mariadb://localhost:19004/vinyldns?user=root&password=pass"
    user = "root"
    password = "pass"

    # see https://github.com/brettwooldridge/HikariCP
    connection-timeout-millis = 1000
    idle-timeout = 10000
    max-lifetime = 30000
    maximum-pool-size = 5
    minimum-idle = 0

    my-sql-properties = {
      cachePrepStmts=true
      prepStmtCacheSize=250
      prepStmtCacheSqlLimit=2048
      rewriteBatchedStatements=true
    }
  }
}

Database Configuration

VinylDNS supports both DynamoDB and MySQL backends. You can enable all repos in a single backend, or have a mix of the two. For each backend, you need to configure the table(s) that should be loaded.

You must have all of the following required API repositories configured in exactly one datastore. At the moment, not all repositories are implemented in both datastores, but we are actively working on expanding MySQL support to all repositories:

  • record-set (dynamodb only)
  • record-change (dynamodb only)
  • zone-change (mysql or dynamodb)
  • user (dynamodb only)
  • group (dynamodb only)
  • group-change (dynamodb only)
  • membership (dynamodb only)
  • zone (mysql only)
  • batch-change (mysql only)

If using DynamoDB, follow the AWS DynamoDB Setup Guide first to get the values you need to configure here.

If using MySQL, follow the MySQL Setup Guide first to get the values you need to configure here.

vinyldns {

  # this list should include only the datastores being used by your instance
  data-stores = ["mysql", "dynamodb"]
  
  dynamodb {
      
    # this is the path to the DynamoDB provider. This should not be edited
    # from the default in reference.conf
    class-name = "vinyldns.dynamodb.repository.DynamoDBDataStoreProvider"
    
    settings {
      # AWS_ACCESS_KEY, credential needed to access the SQS queue
      key = "x"
    
      # AWS_SECRET_ACCESS_KEY, credential needed to access the SQS queue
      secret = "x"
    
      # DynamoDB url for the region you are running in, this example is in us-east-1
      endpoint = "https://dynamodb.us-east-1.amazonaws.com"
      
      # DynamoDB region
      region = "us-east-1"
    }
    
    repositories {
      # all repositories with config sections here will be enabled in dynamodb
      record-set {
        # Name of the table where recordsets are saved
        table-name = "recordSetTest"
        # Provisioned throughput for reads
        provisioned-reads = 30
        # Provisioned throughput for writes
        provisioned-writes = 20
      }
      record-change {
        table-name = "recordChangeTest"
        provisioned-reads = 30
        provisioned-writes = 20
      }
      zone-change {
        table-name = "zoneChangesTest"
        provisioned-reads = 30
        provisioned-writes = 20
      }
      user {
        table-name = "usersTest"
        provisioned-reads = 30
        provisioned-writes = 20
      }
      group {
        table-name = "groupsTest"
        provisioned-reads = 30
        provisioned-writes = 20
      }
      group-change {
        table-name = "groupChangesTest"
        provisioned-reads = 30
        provisioned-writes = 20
      }
      membership {
        table-name = "membershipTest"
        provisioned-reads = 30
        provisioned-writes = 20
      }
    }
  }
  
  mysql {
    
    # this is the path to the mysql provider. This should not be edited
    # from the default in reference.conf
    class-name = "vinyldns.mysql.repository.MySqlDataStoreProvider"
    
    settings {
      # the name of the database, recommend to leave this as is
      name = "vinyldns"
      
      # the jdbc driver, recommended to leave this as is
      driver = "org.mariadb.jdbc.Driver"

      # the URL used to create the schema, typically this will be without the "database" name
      migration-url = "jdbc:mariadb://localhost:19002/?user=root&password=pass"

      # the main connection URL
      url = "jdbc:mariadb://localhost:19002/vinyldns?user=root&password=pass"

      # the user to connect to MySQL
      user = "root"

      # the password to connect to MySQL
      password = "pass"

      ## see https://github.com/brettwooldridge/HikariCP for more detail on the following fields
      # the maximum number of connections to scale the connection pool to
      maximum-pool-size = 20

      # the maximum number of milliseconds to wait for a connection from the connection pool
      connection-timeout-millis = 1000
      
      # the minimum number of idle connections that HikariCP tries to maintain in the pool
      minimum-idle = 10
            
      # the maximum number of milliseconds that a connection is can sit idle in the pool
      idle-timeout = 10000

      # The max lifetime of a connection in a pool.  Should be several seconds shorter than the database imposed connection time limit
      max-lifetime = 600000
      
      # controls whether JMX MBeans are registered
      register-mbeans = true
      
      # my-sql-properties allows you to include any additional mysql performance settings you want.
      # Note that the properties within my-sql-properties must be camel case!
      # see https://github.com/brettwooldridge/HikariCP/wiki/MySQL-Configuration for guidance
      my-sql-properties {
        prepStmtCacheSize = 300
        prepStmtCacheSqlLimit = 2048
        cachePrepStmts = true
        useServerPrepStmts = true
        rewriteBatchedStatements = true
      }
    }
    
    repositories {
      # all repositories with config sections here will be enabled in mysql
      zone {
        # no additional settings for repositories enabled in mysql
      },
      batch-change {
      }
    }
  }
}

Cryptography Settings

VinylDNS uses symmetric cryptography in order to encrypt/decrypt sensitive information in the system. This includes TSIG keys and user secrets. Cryptography is used in both the portal as well as the api.

Cryptography is pluggable, meaning you can bring your own crypto with you. All that is required is to provide an implementation of CryptoAlgebra using a crypto library of choice. The default implementation is NoOpCrypto, which does not do any encryption (not recommended for production). VinylDNS provides a cryptography implementation called JavaCrypto that you can use for production. The example that follows illustrates using the provided JavaCrypto.

If you create your own implementation, you have to build your jar and make it (and all dependencies) available to the VinylDNS API and the VinylDNS portal.

The following are the configuration settings for crypto. Notice here the only thing we see is the type. The type is the fully qualified class name for the CryptoAlgebra you will be using. If your crypto implementation requires additional settings, they will be configured inside the crypto element, adjacent to the type.

vinyldns {
  crypto {
    type = "vinyldns.core.crypto.JavaCrypto"
    secret = "8B06A7F3BC8A2497736F1916A123AA40E88217BE9264D8872597EF7A6E5DCE61"
  }
}

Default Zone Connections

VinylDNS allows you to specify zone connection information for each zone.

VinylDNS has 2 connections for each zone:

  1. The DDNS connection - used for making DDNS updates to the zone
  2. The Transfer connection - used for making AXFR requests for zone syncing with the DNS backend

VinylDNS supports the ability to provide default connections, so keys do not need to be generated for every zone. This assumes a “default” DNS backend.

VinylDNS also ties in testing network connectivity to the default zone connection’s primary server into its API health checks. A value for the health check connection timeout in milliseconds can be specified using health-check-timeout; a default value of 10000 will be used if not provided.

vinyldns {

    # timeout for DNS backend connectivity health check
    health-check-timeout = 5000

  # the DDNS connection information for the default dns backend
  defaultZoneConnection {
    # this is not really used, but must be set, usually set to the keyName itself, or a descriptive name if you are interested
    name = "vinyldns."

    # the name of the TSIG key
    keyName = "vinyldns."

    # the TSIG secret key
    key = "nzisn+4G2ldMn0q1CV3vsg=="

    # the host name or IP address, note you can add a port if not using the default by settings hostname:port
    primaryServer = "ddns1.foo.bar.com"
  }

  # the AXFR connection information for the default dns backend
  defaultTransferConnection {
    name = "vinyldns."
    keyName = "vinyldns."
    key = "nzisn+4G2ldMn0q1CV3vsg=="
    primaryServer = "vinyldns-bind9"
  }
}

Additional Configuration Settings

Approved Name Servers

When running a large DNS installation, allowing users the ability to self-manage zone delegations can lead to a lot of problems when not done properly. Also, allowing delegation to untrusted DNS servers can be a security risk.

To “lock down” zone delegation, you can configure name servers that you trust, so zone delegation is controlled.

The entries in the list can be host names, IP addresses, or regular expressions.

approved-name-servers = [
  "172.17.42.1.",
  "ddns1.foo.bar.",
  ".*awsdns.*"
]

Processing Disabled

The processing disabled flag can be used if doing a blue/green deployment. When processing is disabled, the VinylDNS engine will not be actively polling the message queue for messages.

processing-disabled = false | true

Color

For blue-green deployments, you can configure the color of the current node. Not applicable to every environment.

color = "green"

Version

Version of the application that is deployed. Currently, this is a configuration value.

version = "0.8.0"

Note: You can get installation information including color, version, default key name, and processing-disabled by hitting the status endpoint GET /status

HTTP Host and Port

To specify what host and port to bind to when starting up the API server, default is 9000.

rest {
  host = "0.0.0.0"
  port = 9000
}

Sync Delay

VinylDNS uses a “sync-delay” setting that prevents users from syncing their zones too frequently. The settings is inspected per zone, and is the number of milliseconds since the last sync to wait before allowing another sync for that zone.

sync-delay = 10000

Full Example Config

# The default application.conf is not intended to be used in production.  It assumes a docker-compose
# setup for all of the services.  Provide your own application.conf on the docker mount with your
# own settings
vinyldns {

  queue {
    class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"

    messages-per-poll = 10
    polling-interval = 250.millis

    settings {
      # AWS access key and secret.
      access-key = "x"
      secret-key = "x"

      # Regional endpoint to make your requests (eg. 'us-west-2', 'us-east-1', etc.). This is the region where your queue is housed.
      signing-region = "x"

      # Endpoint to access queue
      service-endpoint = "http://localhost:9324/"

      # Queue name. Should be used in conjunction with service endpoint, rather than using a queue url which is subject to change.
      queue-name = "vinyldns"
    }
  }

  # host and port the server binds to.  This should not be changed
  rest {
    host = "0.0.0.0"
    port = 9000
  }

  # the delay between zone syncs so we are not syncing too often
  sync-delay = 10000

  # crypto settings for symmetric cryptography of secrets in the system
  # Note: for production systems secrets should not live in plain text in a file
  crypto {
    type = "vinyldns.core.crypto.NoOpCrypto"
  }

  # both datastore options are in use
  data-stores = ["mysql", "dynamodb"]
  
  dynamodb {
    class-name = "vinyldns.dynamodb.repository.DynamoDBDataStoreProvider"
    
    settings {
      key = "x"
      secret = "x"
      endpoint = "http://vinyldns-dynamodb:8000"
      region = "us-east-1"
    }
    
    repositories {
      record-set {
        table-name = "recordSet"
        provisioned-reads = 30
        provisioned-writes = 20
      }
      record-change {
        table-name = "recordChange"
        provisioned-reads = 30
        provisioned-writes = 20
      }
      zone-change {
        table-name = "zoneChanges"
        provisioned-reads = 30
        provisioned-writes = 20
      }
      user {
        table-name = "users"
        provisioned-reads = 30
        provisioned-writes = 20
      }
      group {
        table-name = "groups"
        provisioned-reads = 30
        provisioned-writes = 20
      }
      group-change {
        table-name = "groupChanges"
        provisioned-reads = 30
        provisioned-writes = 20
      }
      membership {
        table-name = "membership"
        provisioned-reads = 30
        provisioned-writes = 20
      }
    }
  }
  
  mysql {
    class-name = "vinyldns.mysql.repository.MySqlDataStoreProvider"
    
    settings {
      name = "vinyldns"
      driver = "org.mariadb.jdbc.Driver"
      migration-url = "jdbc:mariadb://localhost:19002/?user=root&password=pass"
      url = "jdbc:mariadb://localhost:19002/vinyldns?user=root&password=pass"
      user = "root"
      password = "pass"
      maximum-pool-size = 20
      minimum-idle = 10
      connection-timeout-millis = 1000
      idle-timeout = 10000
      max-lifetime = 600000
      register-mbeans = true
      my-sql-properties {
        prepStmtCacheSize = 300
        prepStmtCacheSqlLimit = 2048
        cachePrepStmts = true
        useServerPrepStmts = true
        rewriteBatchedStatements = true
      }
    }
    
    repositories {
      zone {
      },
      batch-change {
      }
    }
  }

  # the DDNS connection information for the default dns backend
  defaultZoneConnection {
    name = "vinyldns."
    keyName = "vinyldns."
    key = "nzisn+4G2ldMn0q1CV3vsg=="
    primaryServer = "vinyldns-bind9"
  }

  # the AXFR connection information for the default dns backend
  defaultTransferConnection {
    name = "vinyldns."
    keyName = "vinyldns."
    key = "nzisn+4G2ldMn0q1CV3vsg=="
    primaryServer = "vinyldns-bind9"
  }

  # the max number of changes in a single batch change.  Change carefully as this has performance
  # implications
  batch-change-limit = 20
}

# Akka settings, these should not need to be modified unless you know akka http really well.
akka {
  loglevel = "INFO"
  loggers = ["akka.event.slf4j.Slf4jLogger"]
  logging-filter = "akka.event.slf4j.Slf4jLoggingFilter"
  logger-startup-timeout = 30s
}

akka.http {
  server {
    # The time period within which the TCP binding process must be completed.
    # Set to `infinite` to disable.
    bind-timeout = 5s

    # Show verbose error messages back to the client
    verbose-error-messages = on
  }

  parsing {
    # akka-http doesn't like the AWS4 headers
    illegal-header-warnings = on
  }
}