Creating a simple tiny URL generator using Golang, PostgreSQL and Redis
Not long ago, I had learnt to create a tiny URL generator using Node JS, Postgres and Redis for learning Node after coming across an article. Recently, I came across a problem where I had to create a micro-service in Golang. So, I wanted to build a tiny URL generator using Golang for learning. I’m writing this article to put forward my experience and serve as a reference for anyone trying to create one.
My focus in this article will be mainly on the code. I would be providing links to some guides I used to setup the development environment.I would be using a Mac for coding. We would need the following to start coding our tool:
- Golang: We will need the Go compiler, basic libraries and tools to compile and run our code, which can be downloaded here.
- PostgreSQL: I assume that PostgreSQL DB is already setup and configured to accept external connections. The steps to download and install it can be found here. Once setup, please create a database with a name of your choice and keep the hostname, port, username, password and the database name handy. I would be using tiny_scale_go as my database name in this article.
- Redis: We will be using Redis for caching our URLs generated for faster access. I assume Redis server is setup and running. The steps to install the same can be found here. Once setup, please keep the hostname, port and password handy.
- Text editor: Any popular text editor can be used. Most of them will have a plugin for Golang, so please install it as it will aid greatly in the process of writing code.
Once the above pieces of software are setup, we can setup the code execution environment on the terminal. We shall begin by setting up the GOPATH. This is the env variable containing the path to our sources and the libraries we would be installing. The process of setting it up can be found here and the process of setting up the workspace can be found here. Please note that if you are using a custom GOPATH, it might be necessary to set GOBIN variable and add the GOPATH to the PATH variable. They can be set using the following statements:
export GOBIN=$GOPATH/binexport PATH=$PATH:$(go env GOPATH)/bin
Once the workspace is setup, lets start by creating our source file under the src directory of the workspace. I would be keeping mine as server.go. I would be explaining the code in 4 parts, from library imports to generating our tiny URLs.
I know we are here to code, so lets dive straight into it by importing the required libraries first
Once we have the imports, lets create our functions that help us in creating our DB structure and setting up the connections to DB and cache. We will be using GORM library for connecting to the DB and Go-Redis for Redis cache. They can be installed using the below commands
go get -u github.com/jinzhu/gormgo get -u github.com/go-redis/redis
From the above code snippet, our database will store the tiny URL and long URL along with the other standard data like an auto-incremented index, record creation and updation date etc., with the tiny URL being unique. We establish a connection to our PostgreSQL database using the below connection string
postgres", "host=127.0.0.1 port=5432 user=postgres dbname=tiny_scale_go password=<db password> sslmode=disablePlease plug-in your database host, port, username, database name and password in the above string. I had to disable ssl connection mode as my installation didn’t support it because of its config.
We provide the Redis server config as per the below snippet
&redis.Options{
Addr: "localhost:6379",
Password: "",
DB: 0,
}Please plug-in your Redis server address with port and password in the options above.
Lets move on to writing our function that generates the unique tiny URL and inserts it into the DB. We generate it according to the following points:
- We generate the MD5 hash of the given long URL using the crypto module
- We encode it into a base64 string to remove any special characters which may interfere with the operation of the URL. After encoding, as it might still contain + and / characters, we replace them with _
- Once the string is generated, we extract first 6 characters of it to keep it short.
- We initially check for the presence of the tinyURL in the DB for a different long URL. If its present, we recursively increase the start index for fetching the 6 characters until there is no conflict.
Once there are no conflicts, we insert it into the DB and return the value inserted to the cache and sent as the HTTP request response. The response is sent to the client requesting the service by the code snippet below
Now that we have tiny URLs in our DB and the cache, lets write a function that can give us back the long URL when the tiny URL is sent in the request
The function in the above code snippet initially tries to fetch the long URL from the Redis cache and respond to the request. If its not found, it searches the DB for the same. If its found, its inserted to the Redis cache and the response is sent. If its not found, an error message is sent.
To make all this tick, we have to initialise the DB and cache connections and setup our HTTP server that serves the requests. The below code snippet sets up the connections and starts the server
We will be using the basic http package in Go as we just need to handle basic GET requests. Specialised libraries can be used for adding more features like auth and secure routes.
The routes and format for getting a tiny URL from a long URL is
<domain>/tiny/?longUrl=<long URL>and for getting back the long URL is
<domain>/long/?tinyUrl=<tiny URL>In the above URLs, the domain is usually localhost:<port no.> if it is run locally and the domain of the server if hosted. The response will just contain the tiny URL without the domain and the long URL according to the endpoints hit. A domain can be added to the tiny URL returned if this is hosted and is provided as a service.
I have also added a /stop route for gracefully stopping the HTTP server, but is not necessary and is not a good idea to keep it if its being hosted as a service.
The complete sources can be found here. We can compile the sources by executing the go install command in terminal. When its compiled, an executable binary is created under the bin folder. The binary can be executed like any other executable.
Before executing the binary, please make sure that Redis server and PostgreSQL servers are running. Once we execute the binary, the HTTP server starts and we can start firing our requests!
Although the tiny URL generator can be created with the help of the above steps and code snippets, its not the only way and there are multiple options to add more features and make it more optimised and robust. Please feel free to explore the available libraries and other solutions and post interesting things you find below. Feedback is highly appreciated.
If you wish to play around with Golang without installing everything on your computer, head over to https://play.golang.org/ , where you can execute code online.
