Mastering Go: Part 12 - Program debugging , Profiling and Performance Evaluation

The Go has several APIs and tools available to evaluate the performance and logic of the program. All the diagnostics tools fall under different categories. The aim of these tools is to identify the performance and/or logical issues with the Go program.

Profiling

Profiling is the process of identifying the expensive code blocks in the project. The pprof tool will be used to visualize the data generated by the profiling. Go has a standard library 'pprof' and can be used to visualize the profiling data. the profiler generated the data in the specific format expected by the pprof tool. The profiling data can be collected during testing via go test or endpoints made available from the net/http/pprof package.  

The ppfrof package contains several of the APIs to provide data in a specified/defined structure.

There are few built-in profiles included in the pprof package

CPU - CPU profile reports

HEAP - heap profile reports

threadcreate - profile reports for the  program that creates new process threads

goroutine - goroutine profile report

block - program block waiting on synchronization  profile report

mutex -lock contention report

Install profiling Tools

To generate and visualize the profile report we need a few tools installed. Let's first make sure all the tools required are installed.

First, install the pprof tool using the below command :

$ go install GitHub.com/google/pprof@latest

Run the below command to make sure pprof is working:

$ pprof —version

Now, install the data visualization tool Graphviz. We can install it using Homebrew (for macOS users) or follow the instructions on this Graphviz download.

Bash command:

$ brew install graphviz

Note: If Homebrew is not already installed on your machine, either use the macOS .dmg file or install Homebrew first.

Generate profile reports

There are several ways to generate a program profile. We can generate profiles while running unit tests, by writing code to generate profiles, or by generating live profiles while running the application/services.

Examples: Generate a profile with a unit test. We are using the unit test written as part of this learning series: Part 11: Testing Go Projects.

To generate the profile using a unit test run, please use the command below:

$ go test -cpuprofile cpu.prof -memprofile mem.prof -bench .

 

After the completion of the test, two files, cpu.prof and mem.prof, will be generated. To open and diagnose these files, we need to use the pprof tool.

For any issue/error installing ppform refer to the appendix section of this blog.

Visualize the profile report

In Terminal:

To View the report in the terminal use the below command:

$pprof mem.prof

S*****s-MacBook-Pro:string_formater s********$ pprof mem.prof
File: string_formater.test
Type: alloc_space
Time: May 24, 2024 at 6:37am (EDT)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof)

S******s-MacBook-Pro:string_formater s********i$ pprof cpu.prof
File: string_formater.test
Type: cpu
Time: May 24, 2024 at 6:37am (EDT)
Duration: 201.83ms, Total samples = 0
No samples were found with the default sample value type.
Try "sample_index" command to analyze different sample values.
Entering interactive mode (type "help" for commands, "o" for options)
(pprof)

After executing the above command, we can run several other commands to get insights from the profile. Some of these commands are top, web, etc. Additionally, pprof has several functions to generate specific profiles based on our specific needs.

In the web interface:

We can use the web interface to visualize and read the .prof file.
Command:

$ pprof -http=:8090 cpu.prof

Now, you can visualize the result in the browser using Graphviz.

Note: If you encounter any errors please refer to the common errors sections to fix them:

Writing profiling in the program: 

To write profiling code in your Go main package, you'll need to use the runtime/pprof and os packages to create CPU and memory profiles. Here's an example of how to add profiling code to your main.go file:
We can use a build tag to execute the profiling code only in debug mode.

Example: 

Continue from the previous exercise. Modify the main.go to the file and add the code below to generate CPU and memory profiles. The profile will be generated after each execution of the main. To avoid extra resource consumption in production, we can move the profiling code into a separate file and only execute it in debug mode.

package main

import (

"encoding/json"

"errors"

"flag"

"fmt"

"io"

dateformater "learngo/date_formater" // import data formater package

stringformater "learngo/string_formater" // import string formater package

"log"

"net/http"

"os"

"runtime"

"runtime/pprof"

)

var cpuprofile = flag.String("cpuprofile", "cpu.prof", "write cpu profile to `file`")

var memprofile = flag.String("memprofile", "mem.prof", "write memory profile to `file`")

func main() {

flag.Parse()

// CPU profile

if *cpuprofile != "" {

f, err := os.Create(*cpuprofile)

if err != nil {

log.Fatal("could not create CPU profile: ", err)

}

defer f.Close() // error handling omitted for example

if err := pprof.StartCPUProfile(f); err != nil {

log.Fatal("could not start CPU profile: ", err)

}

defer pprof.StopCPUProfile()

}

.................. other code

......................

// memory profile

if *memprofile != "" {

f, err := os.Create(*memprofile)

if err != nil {

log.Fatal("could not create memory profile: ", err)

}

defer f.Close() // error handling omitted for example

runtime.GC() // get up-to-date statistics

if err := pprof.WriteHeapProfile(f); err != nil {

log.Fatal("could not write memory profile: ", err)

}

}

}

Tracing

The GO has packages runtime/trace to capture the trace of the program execution. It can capture the trace of the individual blocks/goroutine of the code. Using the trace visualization tools, we can visualize and analyze the trace to pinpoint the issues. The trace command captures various execution events i.e. GC events, heap sizes, blocking/unblocking, events and goroutines details, etc.
Command to generate a trace for a test run.

$go test -trace=trace.out

To visualize the trace, we can use the pprof package's tool:

$go tool trace trace.out

After executing the trace visualize tool, we can see the trace in the browser:

Similar to writing profile reports in the program, we can start writing tracing in the program and capture all the events while executing the program for debugging purposes.

We can use the different annotations to trace different data they are:

log - capture the trace of execution logs
region - capture the time interval of the goroutines

task - capture the trace of the logical operations such as RPC, and HTTP requests.

Debugging

To debug a Go program, we need to set up the development environment with a Go code debugger. One of the common and popular Go debuggers is Delve. We can easily install Delve on Visual Studio Code as an extension.

Command to install Delv:

$ go install github.com/go-delve/delve/cmd/dlv@latest

We can use the command terminal to start and use the devl debugging, some commands are:

(dlv)$ start - start debugger

(dlv )$ break main.go : 35 - add the breakpoint in the main.go to file line no 35

(dlv )$ continue - continue the execution

(dlv)$ step - step through the code line

(dlv)$ next - go to the next line

(dlv)$ print variable name - print the value of the variable

(dlv)$ clear main.go:35   -  remove the breakpoint from the line no 35

After installing Delve in VS Code, we can execute the Go program in debug mode by selecting "Run" and then "Start Debugging". The program will start in debug mode, and we can navigate through the lines using the debugging mode panels.

Reference

https://go.dev/blog/pprof

https://go.dev/doc/diagnostics

https://pkg.go.dev/runtime/trace

https://go.dev/blog/execution-traces-2024

https://github.com/google/pprof/blob/main/doc/README.md

https://www.practical-go-lessons.com/chap-36-program-profiling

Mastering Go: Part 11 - Testing Go Projects

All programmers should consider writing unit tests while writing code. If the code is not unit test friendly, it will be tricky to write proper unit tests. For example, writing a unit test for the function below is difficult because the function doesn't use dependency injection and creates an instance of a repository inside it.

func GetPersonNames() ([]Person, error) {

fmt.Println("Inside Get Person function")

dbOperation := databaseLayer.NewDatabaseOperation()

// repository

repo := databaseLayer.NewRepository(dbOperation)

// Query to select data from the database table

query := `SELECT "PersonId", "FirstName", "LastName", "CreatedDate", "UserId"

FROM learngo."Person"`

rows, err := repo.ExecuteSelect(query)

....

...

}

Now, modify the function to use the dependency injection pattern. Injecting the repository into the function makes the code more unit-test-friendly. This way, we can easily mock the repository and inject it into the function, allowing us to write unit tests without involving a real database connection.

func GetPersonNames(repo *databaseLayer.Repository) ([]Address, error) {

query := `SELECT "PersonId", "FirstName", "LastName", "CreatedDate", "UserId"

FROM learngo."Person"`

rows, err := repo.ExecuteSelect(query)

....

...

}

Observe the difference between the two methods above. In the first method, we can't use a mocked repository. In the second method, however, we can simply inject the mocked repository to write the unit test.

Go Frameworks and libraries for Testing

There are several libraries available in Go to help with writing unit tests and creating mocks. We will use and discuss some of them here.

gomock

gomock is a mocking framework for Go used to mock Go interfaces for their implementation for testing purposes.

To add the gomock library/package, please run the following command:

$ go get github.com/golang/mock/gomock

mockgen: Mock generator

mockgen is a tool to generate mock files for Go testing. It generates mock files based on the interface definitions in your Go code. To install the mockgen tool, use the following command:

$ go install github.com/golang/mock/mockgen@v1.6.0

After installing the mockgen library, you will be able to generate mock files. In this exercise, we will generate a mock file for db_repository, where we are connecting to the database and executing queries. For testing, we will use a mocked connection and result instead of the actual database.

To verify that mockgen is installed and available for use, run the below command

$ mockgen --version

If you encounter the error "mockgen: command not found," add the Go binary path to your system's PATH. Generally, the Go binaries are located in $HOME/go/bin.

Use the below steps  to update the go binary PATH:

Step 1. Open the .bashrc file in edit mode

$nano ~/.bashrc

Step 2. Add the following line to the file

export PATH=$PATH:$HOME/go/bin

Step 3:  Save the file and reload

$source ~/.bashrc

Now, mockgen is ready to generate the mock file. We will see how to generate the mock file later in the exercise.

sqlmock

sqlmock is a library built to mock the SQL interaction from the code. It mocks the database driver simulates the interaction with the database and returns the mocked data as defined i.e. executing the queries, stored procedures, etc without connecting to the actual database for testing the code.   

We can easily integrate the sqlmock with other testing frameworks in Go like testify, gomock, testing, etc.

Command to install go-sqlmock library/package:

$go get github.com/DATA-DOG/go-sqlmock

 

testify

Testify is another popular testing framework widely used in Go testing. It provides several features and libraries to write unit tests in Go. Testify integrates with mock libraries like gomock or mockgen, enhancing its capabilities for writing reliable and maintainable tests.

Testify has several features to support writing reliable and maintainable tests.

Assertion: Testify facilitates adding assertions in unit tests to verify the actual output against the expected output.
Test Suite: It enables the grouping of tests, allowing for better organization and structuring of test code.
Mock Support: Testify supports popular mock libraries like gomock, mockgen, etc., providing flexibility in mocking dependencies for testing. 

To install testify, use the below command

$go get github.com/stretchr/testify

After installing Testify, you'll have access to several important packages for testing:
github.com/stretchr/testify/assert: Provides assertion functions for writing test assertions.
github.com/stretchr/testify/require: Similar to assert, but stops test execution immediately upon failure.
github.com/stretchr/testify/mock: Support for mocking dependencies in tests github.com/stretchr/testify/suite: Enables the creation of test suites for organizing related tests.

 

testing

The Go package testing provides support for automated testing in golnag program. the package comes with the comment $go test, to execute the unit test written in the package.

Code Coverage

High code coverage is essential for building software intended to run for several years. A higher code coverage ensures that the code is properly tested and covers all possible execution paths. While ideally, the coverage should be 100%, there may be some exceptions where certain parts of the code cannot be mocked or reproduced in the test environment.

Go provides built-in functionality to evaluate code coverage after running test cases. You can use the command $go test -cover to execute tests and analyze code coverage. After running this command, you will see the test execution results along with the coverage report.

This command helps developers assess the effectiveness of their tests and identify areas of the code that need more testing. By achieving the higher code coverage, developers can increase confidence in the reliability and stability of their software over time.

Example: 

Continue from the Part 10 of this series(Mastering Go: Part 10 - Writing Web API in Golang)

In this example we will write unit test for the function GetAddress from the stringformater module.

Step 1:  If the code written is not unit test-friendly, then it's hard to write unit tests. Therefore, before starting to write unit tests, we need to fix or rewrite the code that is not unit test-friendly. I explained this at the beginning of this section and showed how to convert code to unit test-friendly code.

Modify address_formater.go and use dependency injection to inject the Repository into the function GetAddress.
Modify address_handler.go file to call the GetAddress function by injecting a repository instance.
After modification, the code will look like this:

address_formater.go

package stringformater

import (

"fmt"

databaseLayer "learngo/db_operation" // import data formatter package

)

// Address struct

type Address struct {

HouseNumber string

StreetName string

City string

State string

ZipCode string

}

func (a *Address) Format() string {

return a.HouseNumber + " " + a.StreetName + ", " + a.City + ", " + a.State + " " + a.ZipCode

}

// This function call DB operation and return the collection of address

func GetAddress(repo *databaseLayer.Repository) ([]Address, error) {

// Query to select data from the database table

query := `SELECT "HouseNumber", "StreetName", "City", "State", "ZipCode"

FROM learngo."Address"`

// Call the eecuteselect DB operation

rows, err := repo.ExecuteSelect(query)

if err != nil {

return nil, err

}

// Initialize a slice to store Address structs

var addresses []Address

// Iterate over the rows

for rows.Next() {

var address Address

// Scan the values into variables

err := rows.Scan(&address.HouseNumber, &address.StreetName, &address.City, &address.State, &address.ZipCode)

if err != nil {

return nil, fmt.Errorf("error scanning row: %v", err)

}

// Append the scanned address to the slice

addresses = append(addresses, address)

}

// Check for errors during iteration

if err := rows.Err(); err != nil {

return nil, fmt.Errorf("error iterating over rows: %v", err)

}

// Return the list of address structs

return addresses, nil

}

address_handler.go

package handlers

import (

"fmt"

databaseLayer "learngo/db_operation" // import data formatter package

stringformater "learngo/string_formater" // import string formater package

"net/http"

"github.com/gin-gonic/gin"

)

// Get the address from the database and respond with the list of all addresses as JSON.

func GetAddress(c *gin.Context) {

// initializ DB operation

dbOperation := databaseLayer.NewDatabaseOperation()

// initialze the reporsitory injecting DatabaseOperations interface

repo := databaseLayer.NewRepository(dbOperation)

//get addresses

addresses, err := stringformater.GetAddress(repo)

if err != nil {

fmt.Println("Exception occured to get address")

panic(err)

}

c.IndentedJSON(http.StatusOK, addresses)

}

Step 2:  Generate mock db operation

Note that we are using a DB connection to retrieve the address from the database. Since using the real database connection is not recommended for writing unit tests, we need to mock the DB activity.

Now, run the following command to generate a mock file for the db_operation.go file. If you have not completed the required library installation as described above, please do so before executing this command.

$mockgen -source=db_operation.go -destination=mocks/mock_db_operation.go -package=mocks

The autogenerated file content by mockgen looks like this:

// Code generated by MockGen. DO NOT EDIT.

// Source: db_operation.go

// Package mocks is a generated GoMock package.

package mocks

import (

sql "database/sql"

reflect "reflect"

gomock "github.com/golang/mock/gomock"

)

// MockDatabaseOperations is a mock of DatabaseOperations interface.

type MockDatabaseOperations struct {

ctrl *gomock.Controller

recorder *MockDatabaseOperationsMockRecorder

}

// MockDatabaseOperationsMockRecorder is the mock recorder for MockDatabaseOperations.

type MockDatabaseOperationsMockRecorder struct {

mock *MockDatabaseOperations

}

// NewMockDatabaseOperations creates a new mock instance.

func NewMockDatabaseOperations(ctrl *gomock.Controller) *MockDatabaseOperations {

mock := &MockDatabaseOperations{ctrl: ctrl}

mock.recorder = &MockDatabaseOperationsMockRecorder{mock}

return mock

}

// EXPECT returns an object that allows the caller to indicate expected use.

func (m *MockDatabaseOperations) EXPECT() *MockDatabaseOperationsMockRecorder {

return m.recorder

}

// ExecuteSelect mocks base method.

func (m *MockDatabaseOperations) ExecuteSelect(query string) (*sql.Rows, error) {

m.ctrl.T.Helper()

ret := m.ctrl.Call(m, "ExecuteSelect", query)

ret0, _ := ret[0].(*sql.Rows)

ret1, _ := ret[1].(error)

return ret0, ret1

}

// ExecuteSelect indicates an expected call of ExecuteSelect.

func (mr *MockDatabaseOperationsMockRecorder) ExecuteSelect(query interface{}) *gomock.Call {

mr.mock.ctrl.T.Helper()

return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ExecuteSelect", reflect.TypeOf((*MockDatabaseOperations)(nil).ExecuteSelect), query)

}

 

After generating the mocked file the project structure should look like this: 

 /Users/s********i/Documents/learngo/

├── db_operation/ │ ├── db_operations.go // Contains the DatabaseOperations interface │ ├── db_repository.go // Contains the Repository struct and methods │ ├── mocks/ │ │ └── mock_db_operations.go // Destination for the generated mock └── go.mod

Step 3: Add test file and test case

Now add the test file named address_formater_test.go in the folder string_formater and add below code:

package stringformater

import (

"database/sql"

db_repository "learngo/db_operation"

"learngo/db_operation/mocks"

"reflect"

"testing"

"github.com/DATA-DOG/go-sqlmock"

"github.com/golang/mock/gomock"

)

// Provide correct address data for address formatting

func Test_Format(t *testing.T) {

tests := []struct {

name string

address Address

expected string

}{

{

name: "standard address",

address: Address{

HouseNumber: "123",

StreetName: "Main St",

City: "Springfield",

State: "IL",

ZipCode: "62704",

},

expected: "123 Main St, Springfield, IL 62704",

},

{

name: "address with no state",

address: Address{

HouseNumber: "789",

StreetName: "Pine St",

City: "Atlanta",

State: "",

ZipCode: "30303",

},

expected: "789 Pine St, Atlanta, 30303",

},

}

for _, tt := range tests {

t.Run(tt.name, func(t *testing.T) {

if got := tt.address.Format(); got != tt.expected {

t.Errorf("Address.Format() = %v, want %v", got, tt.expected)

}

})

}

}

// Unit test to test the getAddress function

func Test_GetAddress(t *testing.T) {

ctrl := gomock.NewController(t)

defer ctrl.Finish()

// Get the instance of mock DB_operation

mockDB := mocks.NewMockDatabaseOperations(ctrl)

// Query to execute

query := `SELECT "HouseNumber", "StreetName", "City", "State", "ZipCode"

FROM learngo."Address"`

// Create the instance of SQLmock to fake execute the sql query

db, mock, err := sqlmock.New()

if err != nil {

t.Fatalf("Error occured on creating mocked sql instance : '%s'", err)

}

defer db.Close()

// Prepare data for test

rows := sqlmock.NewRows([]string{"HouseNumber", "StreetName", "City", "State", "ZipCode"}).

AddRow("3800", "Market St", "Frederick", "MD", "21701").

AddRow("4500", "Walnut St", "Chevy Chase", "MD", "21901")

// Set the expectation of the query

mock.ExpectQuery(query).WillReturnRows(rows)

// set the mock expect for the execute select

mockDB.EXPECT().ExecuteSelect(query).DoAndReturn(func(query string) (*sql.Rows, error) {

return db.Query(query)

})

// Get repository using MOCKED DB

repo := db_repository.NewRepository(mockDB)

// Execute the GetAddress function with repo instance created with mocked DB

addresses, err := GetAddress(repo)

if err != nil {

t.Fatalf("unexpected error: %v", err)

}

// set the expected data for comparison

expected := []Address{

{HouseNumber: "3800", StreetName: "Market St", City: "Frederick", State: "MD", ZipCode: "21701"},

{HouseNumber: "4500", StreetName: "Walnut St", City: "Chevy Chase", State: "MD", ZipCode: "21901"},

}

// Assert

if !reflect.DeepEqual(addresses, expected) {

t.Errorf("Test actual result got %v, Expected result: %v", addresses, expected)

}

}

Take some time to understand the above code, please carefully review:

• Observe how the test struct is created and looped through to run the tests for the address format function.
• Observe how the gomock controller is used to initiate the mocked db_operations instance.
• Observe the use of mocked db_operation in the db_repository to fake the database interaction inside the repository.
• Notice that we are suing go standard library reflect to compare the actual vs expected values in the test

Step 4: Run test case

Executing test case in go is fairly simple, just need to run this command

S*****-MacBook-Pro:string_formater s********i$ go test

PASS

ok      learngo/string_formater 0.251s

Step 5: Run test with code coverage

S*****-MacBook-Pro:string_formater s*******i$ go test -cover

PASS

coverage: 34.3% of statements

ok      learngo/string_formater 0.246s

Step 6. Use assertion

If you prefer to use the assert package instead of comparing results using reflection, you can utilize the assert package provided by the testify framework. Now, modify the assertion part of the unit test Test_GetAddress. After modification, the code looks like below:

..........

// Execute the GetAddress function with repo instance created with mocked DB

addresses, err := GetAddress(repo)

assert.NoError(t, err, "unexpected error")

.............

.............

// Assert

assert.Equal(t, expected, addresses, "address slice mismatch")

Observe the code change above: we are now using the "assert" package instead of "reflect" to validate test case outputs. Utilizing the assert package makes the code cleaner and easier to implement.

Exercise Code Link:  https://github.com/learnwithsharad/learngo/tree/sharad-AddUnitTests

Reference

https://pkg.go.dev/testing

https://pkg.go.dev/github.com/stretchr/testify@v1.9.0/assert

https://go.dev/doc/tutorial/add-a-test

Mastering Go: Part 10 - Writing Web API (REST) in Golang

The REST API is a web programming interface that follows the REST application architecture style. The REST API allows communication between applications over the internet/intranet using standard HTTP protocols, such as GET, POST, PUT, and DELETE.

Golang has a built-in standard library/package named 'net' that is essential for web applications and Web APIs. The 'net' package includes several standard packages to work on network-related tasks. These packages provide the functionality needed for network-related projects. Some of them are:

HTTP:  to work with  HTTP client and server implementation.

mail:  to work with mail messages

httptest: for testing the web application

URL: to work with URL parsing and implementation

textproto: to work with generic text-based request/response protocol.

Example: Continue from Part 9 of this series( Part 9 - Pointers and Their efficient uses )

In this example, we will create a simple REST API using Golang's net/http package. We will create a web server, run the web server on a specified port, and also create the index page of a web application and browse it in the browser.

Let's go through the step-by-step process of how to create a REST web API in Golang.

Step 1: Add a module/folder named rest_api under the project "learngo."

Step 2: Go to the rest_api directory in the terminal and run the below command to initiate the module:

$ go mod init rest_api

Step 3: Add a file named main.go under the folder rest_api and add the following code:

package main

import (

"encoding/json"

"net/http"

)

func apiResponseBuilder(w http.ResponseWriter, r *http.Request) {

// Set the return Content-Type as JSON like before

w.Header().Set("Content-Type", "application/json")

// Process the request and response based on the HTTP request method

switch r.Method {

case http.MethodGet:

w.WriteHeader(http.StatusOK)

response := map[string]string{"message": "API invoked with GET method"}

json.NewEncoder(w).Encode(response)

case http.MethodPost:

w.WriteHeader(http.StatusCreated)

response := map[string]string{"message": "API invoked with POST method"}

json.NewEncoder(w).Encode(response)

default:

w.WriteHeader(http.StatusMethodNotAllowed)

response := map[string]string{"message": "No request method specified."}

json.NewEncoder(w).Encode(response)

}

}

func main() {

// Create the web API handler.

// The first parameter is the web API path and the second parameter is the request/response processor function

http.HandleFunc("/testapi/", apiResponseBuilder)

// Create a server and run it on port 8080.

// If the port is already in use, we can choose a different port to run the web API.

if err := http.ListenAndServe(":8080", nil); err != nil {

panic(err)

}

}

Step 4:  Build and run the project.  The web server will start and wait for the request to process on port 8080.  Make sure the port is available to use.

Click OK if the popup with the message "Do you want the application “rest_api” to accept incoming network connections?"

Step 5: Open the browser and browse - http://localhost:8080/testapi/ 

YOu should see the page with a JSON formatted message: 

{"message":"API invoked with GET method"}

Web servers need a handler to manage HTTP requests and responses. To initiate the web API, we must pass a handler to the server to perform these tasks.

In the above code, the handler function (HandleFunc('', '')) initiates and passes the function handleApiResponse to work on HTTP requests and responses. The HandleFunc() function registers the handler function that matches with the ServMux pattern, which actually performs the HTTP request/response handling. 

ServeMux is an HTTP request multiplexer. It matches the URL of the HTTP requests against a list of registered patterns and calls the respective handler to process the request.

Another function, ListenAndServe, listens on the TCP network address specified by srv.Addr and call the server to handle incoming traffic. If srv.Addr is blank, the address:http will be used.

API Frameworks

Golang has several API frameworks available for working with network applications. Gin, Echo, Gorilla Mux, Buffalo, and Goji are some of them. Below, we will use the Gin API framework to create a fully functioning API: 

Gin: Gin is an HTTP web framework written in Golang. It provides several useful features for web application/API development and is one of the fastest HTTP frameworks currently in use.

Example: In this example, we will add the REST APIs to add, update, and fetch the name and address information in the project we are working on in this learning series.

Step 1: Add gin HTTP web framework package to the project. To add the package run the below command to add the gin package to your project

$go get github.com/gin-gonic/gin

Step 2:  Add a folder  named 'handlers' under the package  rest_api and then add a file named 'address_handler.go'

Copy the below code into address_handler.go file

package handlers

import (

"fmt"

stringformater "learngo/string_formater" // import string formater package

"net/http"

"github.com/gin-gonic/gin"

)

// Get the address from the database and respond with the list of all addresses as JSON.

func GetAddress(c *gin.Context) {

//get addresses

addresses, err := stringformater.GetAddress()

if err != nil {

fmt.Println("Exception occurred to get address")

panic(err)

}

c.IndentedJSON(http.StatusOK, addresses)

}

Step 3: Modify the main.go to the file under the rest_api folder and add the below code to use the handler to handle HTTP requests.

package main

import (

handlers "learngo/rest_api/handlers" // import data formatter package

"github.com/gin-gonic/gin"

)

func main() {

// Create the router using a gin web framework

router := gin.Default()

// Define the routes to listen to the HTTP request for the path /address.

// When the HTTP GET request is received the handler function GetAddress will be executed to process the request.s

router.GET("/address", handlers.GetAddress)

// Start the HTTP server on localhost at port 8080.

// The server will listen for incoming HTTP requests on this address.s

router.Run("localhost:8080")

}

Step 4: Make sure to run the command to add package dependencies in the go.mod file

$go mod tidy

After running this command the go.mod file should look like this:

module rest_api

go 1.22.2

replace learngo/string_formater => ../string_formater

replace learngo/db_operation => ../db_operation

replace learngo/rest_api => ../rest_api

require (

github.com/gin-gonic/gin v1.10.0

learngo/rest_api v0.0.0-00010101000000-000000000000

learngo/string_formater v0.0.0-00010101000000-000000000000

)

require (

github.com/bytedance/sonic v1.11.6 // indirect

github.com/bytedance/sonic/loader v0.1.1 // indirect

github.com/cloudwego/base64x v0.1.4 // indirect

github.com/cloudwego/iasm v0.2.0 // indirect

github.com/gabriel-vasile/mimetype v1.4.3 // indirect

github.com/gin-contrib/sse v0.1.0 // indirect

github.com/go-playground/locales v0.14.1 // indirect

github.com/go-playground/universal-translator v0.18.1 // indirect

github.com/go-playground/validator/v10 v10.20.0 // indirect

github.com/goccy/go-json v0.10.2 // indirect

github.com/json-iterator/go v1.1.12 // indirect

github.com/klauspost/cpuid/v2 v2.2.7 // indirect

github.com/leodido/go-urn v1.4.0 // indirect

github.com/lib/pq v1.10.9 // indirect

github.com/mattn/go-isatty v0.0.20 // indirect

github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect

github.com/modern-go/reflect2 v1.0.2 // indirect

github.com/pelletier/go-toml/v2 v2.2.2 // indirect

github.com/twitchyliquid64/golang-asm v0.15.1 // indirect

github.com/ugorji/go/codec v1.2.12 // indirect

golang.org/x/arch v0.8.0 // indirect

golang.org/x/crypto v0.23.0 // indirect

golang.org/x/net v0.25.0 // indirect

golang.org/x/sys v0.20.0 // indirect

golang.org/x/text v0.15.0 // indirect

google.golang.org/protobuf v1.34.1 // indirect

gopkg.in/yaml.v2 v2.4.0 // indirect

gopkg.in/yaml.v3 v3.0.1 // indirect

learngo/db_operation v0.0.0-00010101000000-000000000000 // indirect

)

Step 5: Now, build and run the project, it will start the HTTP server and listen to any HTTP requests on port no 8080

Sharads-MacBook-Pro:rest_api sharadsubedi$ go run .

[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.

- using env: export GIN_MODE=release

- using code: gin.SetMode(gin.ReleaseMode)

[GIN-debug] GET /address --> learngo/rest_api/handlers.GetAddress (3 handlers)

[GIN-debug] [WARNING] You trusted all proxies, this is NOT safe. We recommend you to set a value.

Please check https://pkg.go.dev/github.com/gin-gonic/gin#readme-don-t-trust-all-proxies for details.

[GIN-debug] Listening and serving HTTP on localhost:8080

Step 6: Open a browser  window and browse the URL "http://localhost:8080/address"

The browser will display below address data:

[
    {
        "HouseNumber": "1234",
        "StreetName": "abc Pkwy",
        "City": "Frederick",
        "State": "MD",
        "ZipCode": "21000"
    },
    {
        "HouseNumber": "5678",
        "StreetName": "xyz lane",
        "City": "Silver Spring",
        "State": "MD",
        "ZipCode": "22000"
    }
]

Alternatively, we can run the below command to invoke the webAPI in the terminal/command prompt:

S******s-MacBook-Pro:~ s*********i$ curl http://localhost:8080/address

[

    {

        "HouseNumber": "1234",

        "StreetName": "abc Pkwy",

        "City": "Frederick",

        "State": "MD",

        "ZipCode": "21000"

    },

    {

        "HouseNumber": "5678",

        "StreetName": "xyz lane",

        "City": "Silver Spring",

        "State": "MD",

        "ZipCode": "22000"

    }

]

Now, we have added a REST API using the Gin framework and called it using the command prompt and web browser. We can use tools like Postman and Fiddler to test the rest API.

Consume the REST API 

Now, in this section, I will walk you through how to use the recently created REST API to perform operations on different applications.
Golang has a very simple and straightforward process to consume REST APIs compared to other programming languages. We just need to use the inbuilt net/http package to invoke the REST service.

Example: Continue from the previous section
Step 1: Modify main.go and update the code to call the address API. The code below replaces the previous one to read the response directly using the stringformatter package.

//get address data using the RestAPI call

status, err3 := getAddress()

if err3 != nil || !status {

fmt.Println("Exception occured to get address")

panic(err3)

}

Step 2: Add the function getAddress() in the main.go file and copy the below code. In this function, we are consuming the REST APIs

// Get the address information from database using the REST API call.

func getAddress() (bool, error) {

// invoke the rest API using golangs HTTP package.

response, err := http.Get("http://localhost:8080/address")

if err != nil {

fmt.Print(err.Error())

os.Exit(1)

}

//Read all response data from the API

responseData, err := io.ReadAll(response.Body)

if err != nil {

log.Fatal(err)

}

ch := make(chan string)

// extract the response from JSON to struct

var addresses []stringformater.Address

json.Unmarshal(responseData, &addresses)

// Check if the address has values

if len(addresses) > 0 {

// Loop through the collection and read properties

for _, addressInfo := range addresses {

address := &stringformater.Address{HouseNumber: addressInfo.HouseNumber, StreetName: addressInfo.StreetName, City: addressInfo.City, State: addressInfo.State, ZipCode: addressInfo.ZipCode}

go formatString(address, ch)

formatedAddress := <-ch // receive formated data from channel with channel status

fmt.Println(formatedAddress)

}

} else {

fmt.Println("No objects in the address collection")

}

return true, nil

}

Step 3: Open the new terminal n vs code and run the module rest_api. The module starts the HTTP web server and waits for the HTTP requests.

Step 4: Open another terminal in vs code and run the main. go module. you will see the address data from the database in the terminal console.

S******-MacBook-Pro:main s********i$ go run .

Create a new date instance using a constructor!

No error occurred.

Formatted date: 07/02/2021

Use the name format package to format the name!

Inside the Get Person function

Start Reading configuration file:

End Reading configuration file:

Jhonney Walker

Bil Gates

Use name format package to format billing address!

1234 abc Pkwy, Frederick, MD 21000

5678 xyz lane, Silver Spring, MD 22000

Channel Ch is not closed!

Complete code: https://github.com/learnwithsharad/learngo/tree/shaad-RestAPI

Reference

https://pkg.go.dev/net/http#HandlerFunc

https://go.dev/doc/tutorial/web-service-gin

https://gin-gonic.com/docs/examples/bind-uri/