Getting Started with Go

Overview

This document provide guidelines, best practices, and how-to guides for developing using Go at ThinkParQ. In general this document will not attempt to recreate external documentation, but rather provide links and go into detail on how things relate to our development process.

Table of Contents:

• Overview
• Let's Go! Setting Up Your Development Environment
• Resources for Learning Go
• Resources For Keeping Up With Go
• Coding Standards
  • Code Hygiene
  • Documentation
  • Testing
• Dependency Management
  • How To: Coordinate changes requiring updates in multiple repositories
    • Alternative options during development

Let's Go! Setting Up Your Development Environment

Quick Start:

(1) Download and install Go from here.

Generally its fine to always go with the latest version. While the go.mod file in each project will indicate a specific Go version, as long as the version of Go on your dev machine is newer than this, you should not have any issues. This is because when Go 1 was released its compatibility promise has largely made new releases boring:

It is intended that programs written to the Go 1 specification will continue to compile and run correctly, unchanged, over the lifetime of that specification. … Go programs that work today should continue to work even as future releases of Go 1 arise.

Note some projects do choose to include a Makefile which includes a version check, but this is mostly used when we build/release official packages (using GitHub Actions) so we know exactly what version of Go was used. As a result you may not be able to run all of the make targets if your local version of Go doesn't match exactly, but largely the Makefiles are intended for use with CI, not local development. Note if you do want to build using Make you will also need to install the Go Licenses tool and add it to your $PATH.

(2) Setup your IDE:

The Go extension for Visual Studio Code gives you IntelliSense, code navigation, symbol search/rename, and integration for testing and debugging your code.

If you don't want to use vscode thats okay, and if your editor of choice supports it, you can still use gopls, the Go language server to get Go support in Vim, Emacs, Atom, Sublime, and more.

(3) Follow the steps in the Dependency Management section, particularly on how to use internal dependencies from private GitHub repositories.

(4) Check out the Resources for Learning Go and get going!

Resources for Learning Go

Key resources to learn Go are nicely linked from this landing page. That page has a lot, so here are some specific sections to get started:

• Tutorial: Getting started
• How to Write Go Code
• A Tour of Go
• Go by Example

If you want some historical context on why the language exists:

• Go at Google: Language Design in the Service of Software Engineering

Some more advanced resources that you don't have to read right away, but will probably be useful someday:

• Language Specification
• Effective Go

Not strictly required, but if you find looking at design patterns helpful to learn a new language, these are some resources:

• Design Patterns in Go
• Idiomatic Design and Application Patterns for Go

Good personal sites and blogs by key people in the Go community:

• Rob Pike: One of the original designers of Go.
  • What We Got Right, What We Got Wrong
• Dave Cheney: He's known for his blogs and presentations on Go including performance optimization and best practices.
  • Functional Options
• Ian Lance Taylor: Known for his contributions to the compiler and linker and another influential figure in the community.

Resources For Keeping Up With Go

• Release Notes
  • Go has minor releases consistently on a 6 month cadence with patch releases between.
• Go Time Podcast
  • In addition to episodes talking about features in new/upcoming releases, this is also a great way to learn about different libraries and use cases for Go (and its pretty entertaining overall).

Coding Standards

Code Hygiene

We try to generally adhere to existing standards and best practices generally accepted by the Go community. These include:

• Effective Go
• Go Code Review Comments

It is expected before submitting a pull request that gofmt. staticcheck, and go vet have already been run to ensure some of the more common "nitpicks" have already been addressed. If you have an IDE plugin like the Go extension for Visual Studio Code some of these run automatically as you change/save files.

Documentation

Ensure to provide quality documentation using Go Doc comments for all new/updated code. Note it is generally preferred to use Go doc comments instead of providing extensive documentation using a README. If necessary a README can be provided, but these should generally be limited to providing step-by-step instructions or examples for a particular use case to help users understand generally how to use the package. Don't just reproduce API documentation in a README as this is the intent of the Go doc comments which are used to automatically generate API documentation.

Testing

Include appropriate tests for new or modified functionality and ensure that all tests pass before submitting a pull request. Tests also often serve as runnable examples/

Note integration tests with external dependencies such as a mounted BeeGFS file system should use build constraints (also known as a build tag) so they don't run by default. Build constraints that are in use:

Constraint	Requires
beegfs	BeeGFS must be mounted /mnt/beegfs

To specify a constraint use -tags=<constraint>, for example: go test github.com/thinkparq/beegfs-go/common/ioctl -tags=integration

Dependency Management

Go has a very nice dependency management system that was introduced in Go 1.11, with Go modules that added package versioning and dependency management to the Go ecosystem. Each of our repositories such as protobuf or beegfs-go are a module that includes one or more packages. If you're new to Go you should definitely read that blog post, but TL;DR:

A module is a collection of Go packages stored in a file tree with a go.mod file at its root. The go.mod file defines the module’s module path, which is also the import path used for the root directory, and its dependency requirements, which are the other modules needed for a successful build. Each dependency requirement is written as a module path and a specific semantic version.

If you just want to dive right in, here is a quick crash course:

• If you want to add a new dependency: go get github.com/thinkparq/protobuf
  • If you want to get a specific version (or upgrade/downgrade): go get github.com/thinkparq/protobuf@0.0.0
  • To discover available updates use: go list -m -u all
• After adding/upgrading/removing dependencies you should update the go.mod file by running: go mod tidy

Other good resources to read up on dependency management in Go:

• Managing Dependencies
• Go Modules Reference

How To: Coordinate changes requiring updates in multiple repositories

This section is written around the example of needing to make a change to beegfs-go that also requires adding a new message to the protobuf repo, but the same steps should be applied to any scenario where a change requires a PR to multiple repos. You should already have all of the repos you need to update cloned locally and the steps to use internal dependencies from private GitHub repos completed.

While actively developing functionality for beegfs-go that requires changes in protobuf, you can use the replace directive in go.mod to temporarily tell the Go dependency management system to use the local copy of protobuf instead of downloading it from GitHub. For example:

replace (
	github.com/thinkparq/beegfs-go => ../beegfs-go
	github.com/thinkparq/protobuf => ../protobuf
)

Like most Go modules, ThinkParQ modules are versioned using Git tags, however these versions are associated with a BeeGFS release version. Because it would be impractical to wait for a new BeeGFS release every time we need to update an internal dependency like protobuf, we use Go's automatically generated pseudo-versions to refer to a specific commit (e.g., v0.0.0-20231213094241-25ccd3899fdc) when those changes need to be imported into another Go module such as beegfs-go.

With that in mind, here is the standard process:

1. During development, use the replace directive in the go.mod file for beegfs-go to temporarily tell Go to use your local copy of protobuf.
   • Or see the alternative options during development section below.
2. When you are satisfied with your changes to protobuf publish your branch and submit a PR, noting the commit hash containing the changes you want to import into beegfs-go.
3. In the go.mod for beegfs-go remove the replace directive and update the dependency for beegfs-go to use the pseudo-version containing your protobuf changes by running: go get github.com/thinkparq/protobuf@<PROTOBUF-COMMIT-HASH>.
   • IMPORTANT: Do not try and generate the pseudo-version manually, always let go get manage it.
4. Submit a PR for beegfs-go with the go.mod file still pointing to the pseudo-version.

Alternative options during development

Go also has a feature called Workspaces typically configured via a local go.work file. If you run commands from the workspace directory, Go automatically replaces dependencies with your local copy which would allow you to avoid having to temporarily update project specific go.mod files. You are also able to add replace directives to the go.work files to further customize behavior. * To date we haven't used this much because some developers have noted gopls can get confused and throw errors about missing dependencies (probably there is a way to work around this), and others prefer explicitly specifying replace directives in individual go.mod files because it also serves as a reminder to go specify the correct dependency version once it has been merged. If you find yourself using this in your workflow, consider updating this documentation.