Overcoming challenges of Serverless with Golang
It has been a couple of years since Serverless technology became a hot topic. Since then, there has been many debates and discussions over this topic. If you are a developer coming from a microservices background (like me), naturally you will get multiple questions on your mind at the beginning, such as,
- How should I initialize things like DB connections?
- How can I test the functions on my local env?
- What should be the project structure?
- How to handle logging?
And a lot more. In my case, I have been struggling with solving most of these issues since the day I started experimenting with Serverless.
Why we should use Golang for serverless?
There is no programming language named as “The Best Programming Language for Serverless”. But, Golang has two characteristics that make it stands out of many other languages.
- Safety: Since Go is a statically-typed, compiled language it easier to catch and handle errors and maintain the correctness of the application while it grows. This makes Golang stands out on Safety aspect when compared to languages like NodeJS and Python. But you get the same advantage with languages like Java and C#
- Speed: One of the most important things to consider about Serverless is Cold Starts. With Java and C#, a cold start can take long time periods. But with languages like Node and Python you typically get shorter cold starts. Golang also gives shorter cold starts which is very close to Node and Python.
Now let’s go through some of the most common issues that come with Serverless and see how can we manage to address those.
How to initialize DB connections, loggers etc
In a typical web service, we initialize the DB connections when the application initializes and we do things like connection pooling to use the DB connections optimally. But in Serverless we can’t follow the same. In serverless we just have functions which are being triggered on events. If we create connections on every invocation, there is a chance of running out of connections when using traditional databases like MySQL. Let’s see how we can tackle this problem.
Using a serverless friendly DB: Most of the popular cloud service providers provide serverless support. The same way they have introduced managed database solutions which are serverless friendly. Such as,
Here I found a good read on serverless friendly databases,
But the above solution is only for the DB. What about the loggers, different types of certificates that we need to load?
Initialize things within the function: The other solution is to initialize these things within the function. In serverless teardown doesn’t happen after every request. Hence we can leverage the global variables to solve this issue. We can initialize it once and keep the things initialized until a teardown happens. Here is an example of how a function looks like,
In this approach, one of the main things you need to be careful is to limit the idle DB connections and max DB connections to make sure we don’t create many connections unnecessarily.
How to test the functions in a local environment
Testing is very important in development. Every time we make a change in our code, we can’t rely on a remote cloud to do the testing. Actually it's easier than it looks to test a function on local.
First thing you need to pay your attention is the main idea behind serverless. Serverless is mainly based on events. It can be an event that got triggered because of an API call, a message in a message bus etc. So this handler function needs to be treated as just an ordinary function irrespective of what is the trigger. We can simply write tests the same way we write tests for an ordinary function.
Project Structure
The project structure is a very important thing to consider when working on a project. A good project structuring mechanism increases the maintainability and it makes the project resilient to change. For a traditional web service, we have a lot of literature published about structuring a project. But for serverless, it’s not that clear.
I found this article which provided some initial ideas about structuring serverless Golang applications.
This article talks more about a bigger and complex application. For increasing the modularity it uses special directories like,
/internal: Used to keep the code, that shouldn’t be exposed to the outside world/pkg: Code that can be shared across the project
You can see an in-depth guide in the above article.
But if you are writing a small application, this project structure may become an overkill. Here I found a very good example for a small serverless go application with a couple of functions By Kelsey Hightower,
You can see how he has packaged each of the functions separately.
Wrap Up
Serverless is a fairly new concept compared to concepts like Microservices. It's still evolving and there can be new patterns being introduced in the near future. I am learning new things about serverless every day and your inputs are highly valued. Thank you.
This article was inspired by the articles I mentioned above as well as the following video
