2017-08-07 08:15:56 +00:00
|
|
|
# msgbus
|
|
|
|
|
2022-04-02 14:05:15 +00:00
|
|
|
[![Build Status](https://ci.mills.io/api/badges/prologic/msgbus/status.svg)](https://ci.mills.io/prologic/msgbus)
|
|
|
|
[![Go Reference](https://pkg.go.dev/git.mills.io/prologic/msgbus?status.svg)](https://pkg.go.dev/git.mills.io/prologic/msgbus)
|
2017-08-10 07:38:53 +00:00
|
|
|
|
2018-05-08 06:41:59 +00:00
|
|
|
A real-time message bus server and library written in Go.
|
2018-03-25 20:39:24 +00:00
|
|
|
|
2018-05-08 06:41:59 +00:00
|
|
|
## Features
|
|
|
|
|
|
|
|
* Simple HTTP API
|
|
|
|
* Simple command-line client
|
|
|
|
* In memory queues
|
|
|
|
* WebSockets for real-time messages
|
|
|
|
* Pull and Push model
|
2017-06-08 05:01:09 +00:00
|
|
|
|
|
|
|
## Install
|
|
|
|
|
|
|
|
```#!bash
|
2022-04-02 14:05:15 +00:00
|
|
|
$ go install git.mills.io/prologic/msgbus/cmd/...
|
2017-06-08 05:01:09 +00:00
|
|
|
```
|
|
|
|
|
2018-05-05 21:45:39 +00:00
|
|
|
## Use Cases
|
|
|
|
|
|
|
|
* As a simple generic webhook
|
|
|
|
|
|
|
|
You can use msgbus as a simple generic webhook. For example in my
|
2021-07-12 21:57:54 +00:00
|
|
|
[dockerfiles](https://git.mills.io/prologic/dockerfiles) repo I have hooked up
|
2018-05-05 21:45:39 +00:00
|
|
|
[Prometheus](https://prometheus.io/)'s [AlertManager](https://prometheus.io/docs/alerting/alertmanager/)
|
|
|
|
to send alert notifications to an IRC channel using some simple shell scripts.
|
|
|
|
|
|
|
|
See: [alert](https://hub.docker.com/r/prologic/alert/)
|
|
|
|
|
|
|
|
* As a general-purpose message / event bus that supports pub/sub as well as
|
|
|
|
pulling messages synchronously.
|
|
|
|
|
2017-08-09 10:04:02 +00:00
|
|
|
## Usage (library)
|
|
|
|
|
|
|
|
Install the package into your project:
|
|
|
|
|
|
|
|
```#!bash
|
2021-07-12 21:57:54 +00:00
|
|
|
$ go get git.mills.io/prologic/msgbus
|
2017-08-09 10:04:02 +00:00
|
|
|
```
|
|
|
|
|
|
|
|
Use the `MessageBus` type either directly:
|
|
|
|
|
|
|
|
```#!go
|
|
|
|
package main
|
|
|
|
|
|
|
|
import (
|
2018-04-06 08:33:05 +00:00
|
|
|
"log"
|
2017-08-09 10:04:02 +00:00
|
|
|
|
2021-07-12 21:57:54 +00:00
|
|
|
"git.mills.io/prologic/msgbus"
|
2017-08-09 10:04:02 +00:00
|
|
|
)
|
|
|
|
|
|
|
|
func main() {
|
2018-05-08 08:50:56 +00:00
|
|
|
m := msgbus.New()
|
2018-04-06 08:33:05 +00:00
|
|
|
m.Put("foo", m.NewMessage([]byte("Hello World!")))
|
|
|
|
|
|
|
|
msg, ok := m.Get("foo")
|
|
|
|
if !ok {
|
|
|
|
log.Printf("No more messages in queue: foo")
|
|
|
|
} else {
|
|
|
|
log.Printf
|
|
|
|
"Received message: id=%s topic=%s payload=%s",
|
|
|
|
msg.ID, msg.Topic, msg.Payload,
|
|
|
|
)
|
|
|
|
}
|
2017-08-09 10:04:02 +00:00
|
|
|
}
|
|
|
|
```
|
|
|
|
|
|
|
|
Running this example should yield something like this:
|
|
|
|
|
|
|
|
```#!bash
|
|
|
|
$ go run examples/hello.go
|
|
|
|
2017/08/09 03:01:54 [msgbus] PUT id=0 topic=foo payload=Hello World!
|
|
|
|
2017/08/09 03:01:54 [msgbus] NotifyAll id=0 topic=foo payload=Hello World!
|
|
|
|
2017/08/09 03:01:54 [msgbus] GET topic=foo
|
|
|
|
2017/08/09 03:01:54 Received message: id=%!s(uint64=0) topic=foo payload=Hello World!
|
|
|
|
```
|
|
|
|
|
2021-07-12 21:57:54 +00:00
|
|
|
See the [godoc](https://godoc.org/git.mills.io/prologic/msgbus) for further
|
2017-08-09 10:04:02 +00:00
|
|
|
documentation and other examples.
|
|
|
|
|
|
|
|
## Usage (tool)
|
2017-06-08 05:01:09 +00:00
|
|
|
|
2017-06-08 14:47:16 +00:00
|
|
|
Run the message bus daemon/server:
|
2017-06-08 05:01:09 +00:00
|
|
|
|
|
|
|
```#!bash
|
|
|
|
$ msgbusd
|
2017-08-07 08:15:56 +00:00
|
|
|
2017/08/07 01:11:16 [msgbus] Subscribe id=[::1]:55341 topic=foo
|
|
|
|
2017/08/07 01:11:22 [msgbus] PUT id=0 topic=foo payload=hi
|
|
|
|
2017/08/07 01:11:22 [msgbus] NotifyAll id=0 topic=foo payload=hi
|
|
|
|
2017/08/07 01:11:26 [msgbus] PUT id=1 topic=foo payload=bye
|
|
|
|
2017/08/07 01:11:26 [msgbus] NotifyAll id=1 topic=foo payload=bye
|
|
|
|
2017/08/07 01:11:33 [msgbus] GET topic=foo
|
|
|
|
2017/08/07 01:11:33 [msgbus] GET topic=foo
|
|
|
|
2017/08/07 01:11:33 [msgbus] GET topic=foo
|
2017-06-08 05:01:09 +00:00
|
|
|
```
|
|
|
|
|
|
|
|
Subscribe to a topic using the message bus client:
|
|
|
|
|
|
|
|
```#!bash
|
|
|
|
$ msgbus sub foo
|
2017-08-07 08:15:56 +00:00
|
|
|
2017/08/07 01:11:22 [msgbus] received message: id=0 topic=foo payload=hi
|
|
|
|
2017/08/07 01:11:26 [msgbus] received message: id=1 topic=foo payload=bye
|
2017-06-08 05:01:09 +00:00
|
|
|
```
|
|
|
|
|
2017-06-08 14:47:16 +00:00
|
|
|
Send a few messages with the message bus client:
|
2017-06-08 05:01:09 +00:00
|
|
|
|
|
|
|
```#!bash
|
2017-08-07 08:15:56 +00:00
|
|
|
$ msgbus pub foo hi
|
|
|
|
$ msgbus pub foo bye
|
|
|
|
```
|
|
|
|
|
|
|
|
You can also manually pull messages using the client:
|
|
|
|
|
|
|
|
```#!bash
|
|
|
|
$ msgbus pull foo
|
|
|
|
2017/08/07 01:11:33 [msgbus] received message: id=0 topic=foo payload=hi
|
|
|
|
2017/08/07 01:11:33 [msgbus] received message: id=1 topic=foo payload=bye
|
2017-06-08 05:01:09 +00:00
|
|
|
```
|
|
|
|
|
2017-08-07 08:15:56 +00:00
|
|
|
> This is slightly different from a listening subscriber (*using websockets*) where messages are pulled directly.
|
|
|
|
|
2018-03-25 20:32:31 +00:00
|
|
|
## Usage (HTTP)
|
|
|
|
|
|
|
|
Run the message bus daemon/server:
|
|
|
|
|
|
|
|
```#!bash
|
|
|
|
$ msgbusd
|
|
|
|
2018/03/25 13:21:18 msgbusd listening on :8000
|
|
|
|
```
|
|
|
|
|
|
|
|
Send a message with using `curl`:
|
|
|
|
|
|
|
|
```#!bash
|
|
|
|
$ curl -q -o - -X PUT -d '{"message": "hello"}' http://localhost:8000/hello
|
|
|
|
```
|
|
|
|
|
|
|
|
Pull the messages off the "hello" queue using `curl`:
|
|
|
|
|
|
|
|
```#!bash
|
|
|
|
$ curl -q -o - http://localhost:8000/hello
|
2018-03-29 17:44:02 +00:00
|
|
|
{"id":0,"topic":{"name":"hello","ttl":60000000000,"seq":1,"created":"2018-03-25T13:18:38.732437-07:00"},"payload":"eyJtZXNzYWdlIjogImhlbGxvIn0=","created":"2018-03-25T13:18:38.732465-07:00"}
|
2018-03-25 20:32:31 +00:00
|
|
|
```
|
|
|
|
|
|
|
|
Decode the payload:
|
|
|
|
|
|
|
|
```#!bash
|
|
|
|
$ echo 'eyJtZXNzYWdlIjogImhlbGxvIn0=' | base64 -d
|
|
|
|
{"message": "hello"}
|
|
|
|
```
|
|
|
|
|
|
|
|
## API
|
|
|
|
|
|
|
|
### GET /
|
|
|
|
|
|
|
|
List all known topics/queues.
|
|
|
|
|
|
|
|
Example:
|
|
|
|
|
|
|
|
```#!bash
|
2018-05-08 06:46:28 +00:00
|
|
|
$ curl -q -o - http://localhost:8000/ | jq '.'
|
|
|
|
{
|
|
|
|
"hello": {
|
|
|
|
"name": "hello",
|
|
|
|
"ttl": 60000000000,
|
|
|
|
"seq": 1,
|
|
|
|
"created": "2018-05-07T23:44:25.681392205-07:00"
|
|
|
|
}
|
|
|
|
}
|
2018-03-25 20:32:31 +00:00
|
|
|
```
|
|
|
|
|
2018-05-08 06:43:28 +00:00
|
|
|
## POST|PUT /topic
|
2018-03-25 20:32:31 +00:00
|
|
|
|
|
|
|
Post a new message to the queue named by `<topic>`.
|
|
|
|
|
|
|
|
**NB:** Either `POST` or `PUT` methods can be used here.
|
|
|
|
|
|
|
|
Example:
|
|
|
|
|
|
|
|
```#!bash
|
|
|
|
$ curl -q -o - -X PUT -d '{"message": "hello"}' http://localhost:8000/hello
|
2018-05-08 06:46:28 +00:00
|
|
|
message successfully published to hello with sequence 1
|
2018-03-25 20:32:31 +00:00
|
|
|
```
|
|
|
|
|
2018-05-08 06:43:28 +00:00
|
|
|
## GET /topic
|
2018-03-25 20:32:31 +00:00
|
|
|
|
|
|
|
Get the next message of the queue named by `<topic>`.
|
|
|
|
|
|
|
|
- If the topic is not found. Returns: `404 Not Found`
|
|
|
|
- If the Websockets `Upgrade` header is found, upgrades to a websocket channel
|
|
|
|
and subscribes to the topic `<topic>`. Each new message published to the
|
|
|
|
topic `<topic>` are instantly published to all subscribers.
|
|
|
|
|
|
|
|
Example:
|
|
|
|
|
|
|
|
```#!bash
|
|
|
|
$ curl -q -o - http://localhost:8000/hello
|
2018-03-29 17:44:02 +00:00
|
|
|
{"id":0,"topic":{"name":"hello","ttl":60000000000,"seq":1,"created":"2018-03-25T13:18:38.732437-07:00"},"payload":"eyJtZXNzYWdlIjogImhlbGxvIn0=","created":"2018-03-25T13:18:38.732465-07:00"}
|
2018-03-25 20:32:31 +00:00
|
|
|
```
|
|
|
|
|
2018-05-08 06:43:28 +00:00
|
|
|
## DELETE /topic
|
2018-03-25 20:32:31 +00:00
|
|
|
|
|
|
|
Deletes a queue named by `<topic>`.
|
|
|
|
|
|
|
|
*Not implemented*.
|
|
|
|
|
2018-07-13 21:32:43 +00:00
|
|
|
## Related Projects
|
|
|
|
|
2021-07-12 21:57:54 +00:00
|
|
|
* [je](https://git.mills.io/prologic/je) -- A distributed job execution engine for the execution of batch jobs, workflows, remediations and more.
|
2018-07-13 21:32:43 +00:00
|
|
|
|
2017-08-07 08:15:56 +00:00
|
|
|
## License
|
2017-06-08 05:01:09 +00:00
|
|
|
|
2021-07-12 21:57:54 +00:00
|
|
|
msgbus is licensed under the [MIT License](https://git.mills.io/prologic/msgbus/blob/master/LICENSE)
|