Twelve-Factor App in Go

Posted on: 11 months, 3 weeks ago

https://www.flickr.com/photos/samthor/5994939587

When we want to create a Twelve-Factor Microservice in Go, it’s crucial to adhere to the twelve guiding principles. These guidelines promote automation, efficient log management, and consistent environments. Additionally, it’s crucial to rely on external services and maintain a small, decoupled codebase and dependencies for optimal scalability and ease of maintenance. Following these principles can build a resilient and effective microservice in Go.

TL;DR

If you are just here for a reference or to try the solution firsthand, clone the repository and follow along with the README.md

GitHub - by-sabbir/go-12factor-scaffold

github.com

Any contribution, improvement, or criticism will be highly appreciated.

And thus our first requirement: the Codebase

Let's see it in action

Result: 500TPS with persistent events.

Getting Started

We are building a simple blog microservice with the following APIs—

POST /api/v1/article      # create an Article
GET /api/v1/article/:id   # get an Article by ID
GET /healthz              # check application liveness

The main struct or service construct method used throughout this project is —

// Foo Struct for example
type Foo struct {
  Alice string
  Bob   string
}

// NewFoo constructs a New Foo struc and returns a pointer
func NewFoo() *Foo {
  return &Foo{
    Alice: "alice",
    Bob:   "bob",
  }
}

For reference: https://gobyexample.com/structs and The Embedding section of https://go.dev/doc/effective_go. I am a big fan of this Effective Go blog post.

The entry point for our project is the main.go file in the root directory. It just redirects us to the cmd folder. Let’s visualize the directory structure now.

.
├── _example    # contains an example of consumer
├── cmd         # cli/server entry point
├── db          # database connection, queries, and ops interfaces
├── internal    
│ └── blog      # internal business logic goes here as interface
├── migrations  # database migration files
└── transport   # methods we use for client server communication.
 └── http       # implementation of the http server and handlers satisfying
                #  the interfaces

Instead of going through the code-base line by line, I’ll try to explain what it does, and where to find your expected feature.

The cmd directory has three files migrate.go root.go and server.go .

The root.go is responsible for parsing the app configuration and setting up the CLI interface base satisfying the Config management of the twelve-factor methodology.

migrate.go speaks for itself, in this project — I just them as an example of a one-off admin process. Which is described in the twelfth section in 12factor.net

The server.go file is the heart of the project. All the necessary structs, embeddings, functions, and interfaces are here. Let’s inspect the following code block (server.go L: 34–52) —

// run initiates the server and returns error if any
func run() error {
 // initiating database
 db, err := db.NewDatabase()
 if err != nil {
  log.Error("cloud not connect to db: ", err)
 }
 // initiating blog service
 svc := blog.NewBlogService(db)
 // initiating transport/http
 srv := transportHttp.NewHandler(svc)

 // initiating server
 if err := srv.Serve(); err != nil {
  log.Error("could not serve: ", err)
  return err
 }
 return nil
}

As we can see, there are Database, Blog Services, and HTTP Handlers as dependencies. Blog Service depends on DB, and The Handlers depend on the Service. All of them implement the below interface in one form or another —

type BlogAPI interface {
 CreatePost(context.Context, Article) (Article, error)
 GetPost(context.Context, string) (Article, error)
}

Database Package

Now, let’s dig deeper into the db package — It also contains 3 files, db.go blog.go migrate.go

The db.go simply returns the DB connection or nil if any error occurs. The database is one of the backing services we use, This can be attached or changed directly from the config.yaml file in the root.

The migrate.go is the actual an embedding of the DataBase struct. I have used golang-migrate for managing Postgres migrations.

Finally, the blog.go implements the logic and queries to insert into and pull data from the database.

Internal Package

Next up, is our internal blog package. This is where we put our logic. Feel free to read more about the internal directory at, https://github.com/golang-standards/project-layout#internal. One of the logic is, there should be an event published when a new Article is created. This can be solved in a variety of ways but I solved it by embedding a Publish method at dto.go. I invoked the method at blog.go Line 40–49

 article, err := bs.Store.CreatePost(ctx, a)
 if err != nil {
  log.Error("could not create article")
 } else {
  errCh := make(chan error)
  go func(chan error) {
   e := a.Publish(bs.Publisher)
   <-e
  }(errCh)
 }

For events, I have used RabbitMQ, another backing service that can be attached from the config.yaml .

HTTP Package

The HTTP package builds on the Handler struct,

type Handler struct {
 Service BlogAPI
 Router  *gin.Engine
 Server  *http.Server
}

Here I used Gin’s super easy and intuitive APIs for muxing and builtin http.Server for better control. So we get the best of both worlds.

The rest of the things follow the previous statement and implements the above interface. The HTTP package takes care of concurrency, disposibility, and port binding.

Here’s the code I directly picked from Gin’s documentation of graceful stop/restart —

 go func() {
  // service connections
  if err := h.Server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
   log.Fatalf("listen: %s\n", err)
  }
 }()

 // Wait for interrupt signal to gracefully shutdown the server with
 // a timeout of 5 seconds.
 quit := make(chan os.Signal, 1)
 // kill (no param) default send syscanll.SIGTERM
 // kill -2 is syscall.SIGINT
 // kill -9 is syscall. SIGKILL but can"t be catch, so don't need add it
 signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
 <-quit
 log.Println("Shutdown Server ...")

 ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
 defer cancel()
 if err := h.Server.Shutdown(ctx); err != nil {
  log.Fatal("Server Shutdown:", err)
  return err
 }

 <-ctx.Done()
 log.Println("Server exiting")
 return nil