Combining Slackbots into one, with gRPC

Hello all! I am suzuki-shunsuke from LINE IT Strategy team who is in charge of developing and running internal systems. Today, I’d like to share how I merged multiple Slack bots into one using gRPC.

How it all began

These days, my main tasks as a member of Slack management team, are making and operating slack bots at LINE. One day, we got a request to combine the bots we use into a single bot, for the following reasons:

  • Inviting numerous bots into a chat is a fuss.
  • Knowing all the bots available is difficult.
  • Adding more functions to a single bot is better than adding more bots.

So, what does it mean to combine bots? Suppose we have two bots. One is for changing the name of a channel, say, @renamechan and the other is @spinvite, for inviting members using a group mailing list. Now, if we are to change the name of a channel or to invite a user, we will probably do something like the following.

# Rename the channel name to foo
@renamechan foo

# Invite members of the group mailing list foo@example.com
@spinvite foo@example.com

While we had to invite two bots, if we combine these two bots into one, @satosan, we only need to invite one bot to perform two different jobs, as shown below.

# Rename the channel name to foo
@satosan rename foo

# Invite members of the group mailing list foo@example.com
@satosan invite foo@example.com

To tell you the truth, I wanted to combine bots into one from when we started developing bots, and I even prepared a repository for it. But, I couldn’t put it into action right away due to the following reasons:

  1. We will end up with monolithic structure, making development complicated
  2. Unifying programming languages or framework is difficult if many developers are involved.
  3. We don’t like to have the scope of token expand

Eventually, we’ve decided to combine them into one, using gRPC which will solve the issues A and B — unfortunately, not C though.

Why gRPC?

So, why go with gRPC, instead of using REST APIs or any other means? Well, here is why:

  • We can write APIs in a unified form, Protocol Buffers. → The quality and the format of API documentation will be consistent.
  • We can automatically generate both client and server side code, using Protocol Buffers → Handling API changes will be simple.

By the way, I won’t go into the basics of gRPC in this article.

Designing bot combination

So this is how it’s done; implement bot features on a gRPC server, call Slack’s RTM (Real Time Messaging) API to run a slackbot, and send requests to the gRPC server. This makes our slackbot a gRPC client. We share the slackbot token to each gRPC server for them to call the Slack API internally. Having a separate system set for each feature allows us to avoid monolithic architecture and liberates us to use any language or framework for a feature.

Design the bot simple; what it does is simply sending a request to a gRPC server. But, if we make gRPC servers to send messages, then we will have to deal with different message formats for each feature, which can be a fuss to bot users, having to know all the different message formats. Another thing is that gRPC servers call the chat.postMessage API for sending messages, but bots can send messages through a web socket. There are no restrictions on rates or permissions using a web socket for the bot to return messages, so it might be better for the bot to send messages, instead of the gRPC servers. We’ll need to look into this.

What if we divide the token?

If we make a Slack app per gRPC server, we can maintain the scope of each token to the minimum, and gone is the need to share the token. However, if we go for this option, then the bot the user has invited is not the one who sends messages, but a different bot. Also, this will force us to invite a bot to a channel to send a message, if the channel happens to be private; there will be no point of combing bots into one, right?

Combining bots

It’s time to implement. We’ll use the Go language to implement a simple gRPC for renaming a channel in the following steps:

Here are the files and the directory structure of our implementation. Please note that I have left out error handling code in this article.

slack-bot-satosan/  # Bot = gRPC client
slack-bot-grpc-renamechan/  # The gRPC server that will change the channel name
  renamechan.proto  # Protocol buffer
  protobuf/
    renamechan.pb.go  # Automatically generated code

Generating service code with Protocol Buffers

Download a zipped file of the latest protoc compiler from the Protocol Buffers repository, extract the file, move the extracted protoc binary to any directory defined in PATH. Refer to the following information on Protocol Buffers and installation guide.

We define the renaming service API in the Protocol Buffers language. Copy the content into a file and name the file renamechan.proto.

syntax = "proto3";

option go_package = "protobuf";

service Renamechan {
  rpc Rename (RenameMessage) returns (RenameResponse) {}
  rpc Help (HelpMessage) returns (HelpResponse) {}
}

message RenameMessage {
  string channel_id = 1;
  string bot_name = 2;
  repeated string args = 3;
}

message RenameResponse {
  string message = 1;
}

message HelpMessage {
  string channel_id = 1;
  string bot_name = 2;
}

message HelpResponse {
  string message = 1;
}

Prepare a makefile to compile the file we just prepared, renamechan.proto.

protobuf/renamechan.pb.go: renamechan.proto
    protoc renamechan.proto --go_out=plugins=grpc:protobuf

Run the makefile and we get a file, protobuf/renamechan.pb.go, containing the code for our service, generated automatically.

$ make protobuf/renamechan.pb.go

Implementing a gRPC server

For gRPC server, we install an interface, RenamechanServer in a separate package called handler.

// RenamechanServer is the server API for Renamechan service.
type RenamechanServer interface {
    Rename(context.Context, *RenameMessage) (*RenameResponse, error)
    Help(context.Context, *HelpMessage) (*HelpResponse, error)
}

The next thing to do is write the main() function for launching the gRPC server. For more information on launching, see here.

func main() {
    lis, _ := net.Listen("tcp", ":5000")
    s := grpc.NewServer()
    server := handler.Server{
        Bot: slack.New(config.GetSlackAppUserToken()),
    }
    pb.RegisterRenamechanServer(s, &server)
    reflection.Register(s)
    s.Serve(lis)
}

Returning results – gRPC status codes

Like we have 404 and 500 status codes for HTTP requests, we have a set defined for gRPC. Your gRPC server shall return an appropriate status code to the requestor. To get more information, see the links provided below.

For example, if there is a system error, we need to return Internal, and if a requestor set wrong parameters, then we’d return the InvalidArgument.

return nil, status.Error(codes.InvalidArgument, "channel name is required")

Implementing a gRPC client

Finally, we will look at implementing a gRPC client. Check out the guide by gRPC for more information. The following is a simple code for creating a client.

conn, _ := infra.GetGRPCConnOfRenamechan()
c := renamePB.NewRenamechanClient(conn)
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
_, err = c.Rename(
    ctx, &renamePB.RenameMessage{
        ChannelId: ev.Channel,
        BotName:   botName,
        Args:      args})

The following code is already in the code above, but let’s look at this for now. This code is for getting the result returned by a gRPC server. To get the error object returned by a gRPC server, check the value stored in status.Code. If a gRPC server crashes, you will get codes.Unavailable, and you would display an appropriate error message to the bot user.

_, err = c.Rename(
    ctx, &renamePB.RenameMessage{
        ChannelId: ev.Channel,
        BotName:   botName,
        Args:      args})
code := status.Code(err)

Connections are managed by reusing grpc.ClientConn in a singleton pattern. (The following code example is quite different to the real code.) I am not quite sure this is a right approach. See the threads on Best practices for reusing connections, concurrency.

var (
    conn *grpc.ClientConn
)

func getGRPCConn(addr string) (*grpc.ClientConn, error) {
    if conn != nil {
        return conn, nil
    }
    c, _ := grpc.Dial(
        config.GetRenamechanAddress(), grpc.WithInsecure())
    conn = c
    return c, nil
}

Even if we reuse connections, we must close all connections when we terminate the application. To close all connections we use defer in the main() function.

// Close all connections
defer infra.CloseGRPCConn()

Implementing SSL/TLS

Prepare an SSL certificate and install on a gRPC client and gRPC server. Import google.golang.org/grpc/credentials and change the code as shown below. (Default code)

Client

// https://godoc.org/google.golang.org/grpc/credentials#NewClientTLSFromFile
creds, _ := credentials.NewClientTLSFromFile(certFile, serverNameOverride)
c, _ := grpc.Dial(
    config.GetRenamechanAddress(), grpc.WithTransportCredentials(creds))

Server

// https://godoc.org/google.golang.org/grpc/credentials#NewServerTLSFromFile
creds, _ := credentials.NewServerTLSFromFile(certFile, keyFile)
s := grpc.NewServer(grpc.Creds(creds))

Closing

So, we’ve looked through a process of micro-servicing features using gRPC, to optimize slackbot development. We plan to turn more bots into gRPC servers to combine more and more bots.

Related Post