Tooling Go Microservices With Consul Service Discovery and KV Store

Posted on: 2 years ago

Service Registration and Distributed Configuration /w Consul and Go

Consul, at its core, is a service networking solution. It provides a service mesh solution, network configuration automation, service discovery, a simple Key-Value store, etc. In this article, we will focus on the key-value store (KV store) for our service configuration and one of the core features of consul, service discovery. In a word, we are covering an essential microservice tooling with Consul.

TL;DR

GitHub - by-sabbir/consul-kv-discovery

Get the code and run

❯ docker compose up -d --build

That should expose Consul UI at localhost:8500 and the UI should look like below if you browse this link.

Consul Service Discovery with Health Check

If you are more interested in how to structure your code and make it ready for production, let’s dive in…

We will first start with one monolithic main.go file and then break it down into pieces to create an idiomatic go project. First, we need to start the consul service. Let’s create a docker-compose.yml file for that,

version: "3"

services:
  consul:
    image: hashicorp/consul:1.10.0
    restart: always
    volumes:
     - ./conf/server.json:/consul/config/server.json:ro
    ports:
      - 8500:8500
      - 8600:8600/tcp
      - 8600:8600/udp
    command: "agent"
    environment:
      - CONSUL_BIND_INTERFACE=eth0
      - CONSUL_CLIENT_INTERFACE=eth0

Also, we need to add the server configuration, from conf/server.json

❯ mkdir conf/ && curl -o conf/server.json https://raw.githubusercontent.com/by-sabbir/consul-kv-discovery/master/conf/server.json

Now, we are ready to start the consul server,

❯ docker compose up -d consul

Let’s focus on the application, we will be taking some design decisions-

First and foremost, we want this package to be used by all the microservices.
Service should be registered when the application starts. (and should be fail first, but for sake of simplicity let’s ignore this)
We want to use KV store as external dependency manager like service host and port, API version, not for security configs ie, password manager, API key, client secret etc.

Our project structure should look like this,

.
├── cmd
│   └── server
├── conf
├── internal
|   └── api
└── pkg
    └── consul

For consul service interactions, we will be using pkg directory instead of internal as we want this to be reusable for other developers and services as part of go standard project layout.

Now, create a new file pkg/consul/service-discovery.go

package consul

import (
        "log"

        "github.com/hashicorp/consul/api"
)

type ConsulClient struct {
        *api.Client
}

func NewClient(addr string) (*ConsulClient, error) {

        conf := &api.Config{
                Address: addr,
        }
        client, err := api.NewClient(conf)
        if err != nil {
                log.Println("error initiating new consul client: ", err)
                return &ConsulClient{}, err
        }

        return &ConsulClient{
                client,
        }, nil
}

func (c *ConsulClient) Register(id string) error {
        check := &api.AgentServiceCheck{
                Interval: "30s",
                Timeout:  "60s",
                HTTP:     "http://app:8000/health",
        }
        serviceDefinition := &api.AgentServiceRegistration{
                ID:    id,
                Name:  id + "_ms",
                Tags:  []string{"microservice", "golang"},
                Check: check,
        }
        if err := c.Agent().ServiceRegister(serviceDefinition); err != nil {
                log.Println("error registering service: ", err)
        }

        return nil
}

This is a fairly simple code containing only two functions

NewClient — initiates a new consul API client for given address
Register — Registers the service to consul service discovery with the given name and predefined tag.

For production, tags are very important to quickly search services in the Consul UI. Once the service is registered, monitoring the service health is going to be the job of consul server, line 30-34 denotes that the consul server will check for the service health in every 30 seconds.

We are done with service discovery. Now let’s discuss some dynamic configurations with Consul KV Store.

To integrate the Consul KV, let look into the documentation a bit to better understand the situation. From the doc we can see that ,

func (c *Client) KV() *KV

So, KV is a pointer receiver to the *api.Client struct. We already have a method NewClient that returns *api.Client and we have initiate a client from the method we can reuse that client if we create a separate repository for KV Store. Let’s do that… create a file at pkg/consul/kv-store.go and paste the following lines —

package consul

import (
        "log"

        "github.com/hashicorp/consul/api"
)

type KVClient struct {
        *api.KV
}

func NewKVClient(c *ConsulClient) *KVClient {
        return &KVClient{
                c.KV(),
        }
}

func (k *KVClient) PutKV(key, value string) error {
        p := &api.KVPair{Key: key, Value: []byte(value)}
        _, err := k.Put(p, nil)
        if err != nil {
                log.Println("error instergin KV: ", err)
                return err
        }

        return nil
}

func (k *KVClient) GetKV(key string) (string, error) {
        p, _, err := k.Get(key, nil)
        if err != nil {
                log.Println("error getting value from key: ", err)
                return "", err
        }

        return string(p.Value), nil
}

The function NewKVClient reuses our ConsuleClient and returns a KV client. For our usecase we only created Get and Put functionality assuming all the keys an values will be stringfor the Store. As we will only read existing config and create/update one.

Finally, it’s time to integrate the solution, Go provides a very powerful function to instantiate init() that runs only once and runs before any other function within the package. We can define multiple init() but all of them will run in order once and once only. We will get the advantage of this feature, as service health check will be done periodically by the Consul server.

Let’s write just enough code to register the service in the Consule,

package main

import (
        "encoding/json"
        "fmt"
        "log"
        "net/http"

        consulApi "github.com/by-sabbir/consul-kv-discovery/pkg/consul"
)

var (
        CONSUL_ADDR  = "consul:8500"
        SERVICE_ID   = "go-service"
        SERVICE_NAME = "go-service-test"
        SERVICE_PORT = 8000
        SERVICE_HOST = "0.0.0.0"
)

func init() {
        cli, err := consulApi.NewClient(CONSUL_ADDR)
        if err != nil {
                log.Fatalf("can't initiate consul client: %+v\n", err)
        }
        if err := cli.Register(SERVICE_ID); err != nil {
                log.Println("error registering... ", err)
        }
}

func main() {
        if err := Run(); err != nil {
                log.Fatalf("error starting service: %+v\n", err)
        }
}

func Run() error {
        http.HandleFunc("/health", healthcheck)
        srvString := fmt.Sprintf("%s:%d", SERVICE_HOST, SERVICE_PORT)
        if err := http.ListenAndServe(srvString, nil); err != nil {
                return err
        }
        return nil
}

func healthcheck(w http.ResponseWriter, r *http.Request) {
        json.NewEncoder(w).Encode(map[string]string{
                "status": "ok!",
        })
}

If you run the main.go and drop to console and paste the following —

❯ curl -X GET http://127.0.0.1:8500/v1/catalog/services

It should output something like below —

{“consul”:[],”go-service_ms”:[“golang”,”microservice”]}

Let’s cleanup the main.go file. We wanted to design the consul functionality as reusable as possible and we managed to do that by using go conventions. But instantiating the services will differ from one to another. For example one may need to use AWS S3 service, another one may use AWS Keyspaces. So configuration may vary service to service. In this case we need internal directory to instantiate the consul integrations. Let’s create a file in pkg/consul/kv-store.go and paste the following —

package consul

import (
        "log"

        "github.com/hashicorp/consul/api"
)

type KVClient struct {
        *api.KV
}

func NewKVClient(c *ConsulClient) *KVClient {
        return &KVClient{
                c.KV(),
        }
}

func (k *KVClient) PutKV(key, value string) error {
        p := &api.KVPair{Key: key, Value: []byte(value)}
        _, err := k.Put(p, nil)
        if err != nil {
                log.Println("error instergin KV: ", err)
                return err
        }

        return nil
}

func (k *KVClient) GetKV(key string) (string, error) {
        p, _, err := k.Get(key, nil)
        if err != nil {
                log.Println("error getting value from key: ", err)
                return "", err
        }

        return string(p.Value), nil
}

Note: The struct ServiceDefinitionshould be populated by viper or any os environment variable parser.

So, our main.go file becomes cleaner and the code base is more manageable. But don’t forget to import the package with _ in front as anonymous placeholder. It will make sure that the init() function runs once.

_ “github.com/by-sabbir/consul-kv-discovery/internal/consul”

Now, if we run the whole docker compose it should output like this —

❯ docker compose logs -f appapplication  | 2022/10/08 21:07:40 service registered:  go-service
application  | 2022/10/08 21:07:40 apigw baseUrl:  apigw.example.com

We have successfully got information from the KV store, let’s checkout the new keys —

Get And Put Key Value to Consul

To get the newly created key, go-service make the following request —

❯ curl -X GET http://127.0.0.1:8500/v1/kv/go-service

It should output the query like below —

[
   {
      "CreateIndex" : 17,
      "Flags" : 0,
      "Key" : "go-service",
      "LockIndex" : 0,
      "ModifyIndex" : 17,
      "Value" : "MC4wLjAuMDo4MDAw"
   }
]

The value is base64 encoded version of the original message.

We are now able to discover go service from Consul. Also, equipped with the necessary tool to read from and write to Consul KV Store.