ircd.chat
/
ircd
11
1
Fork 0
Code Issues Pull Requests 1 Releases Wiki Activity

If a file does not bring you joy [...]

This commit is contained in:
kayos@tcp.direct 2022-05-01 17:23:25 -07:00
parent ee92366cd0
commit 81093e844d
Signed by: kayos
GPG Key ID: 4B841471B4BEE979
8 changed files with 0 additions and 2538 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2037
CHANGELOG.md
View File

File diff suppressed because it is too large Load Diff

237
DEVELOPING.md
View File

@ -1,237 +0,0 @@
# Developing Ergo
This is a guide to modifying Ergo's code. If you're just trying to run your own Ergo, or use one, you shouldn't need to
worry about these issues.
## Golang issues
You should use the [latest distribution of the Go language for your OS and architecture](https://golang.org/dl/). (
If `uname -m` on your Raspberry Pi reports `armv7l`, use the `armv6l` distribution of Go; if it reports v8, you may be
able to use the `arm64` distribution.)
Ergo vendors all its dependencies. Because of this, Ergo is self-contained and you should not need to fetch any
dependencies with `go get`. Doing so is not recommended, since it may fetch incompatible versions of the dependencies.
If you're upgrading the Go version used by Ergo, there are several places where it's hard-coded and must be changed:
1. `.github/workflows/build.yml`, which controls the version that our CI test suite uses to build and test the code (
e.g., for a PR)
2. `Dockerfile`, which controls the version that the Ergo binaries in our Docker images are built with
3. `go.mod`: this should be updated automatically by Go when you do module-related operations
## Branches
The recommended workflow for development is to create a new branch starting from the current `master`. Even
though `master` is not recommended for production use, we strive to keep it in a usable state. Starting from `master`
increases the likelihood that your patches will be accepted.
Long-running feature branches that aren't ready for merge into `master` may be maintained under a `devel+` prefix,
e.g. `devel+metadata` for a feature branch implementing the IRCv3 METADATA extension.
## Updating dependencies
Ergo vendors all dependencies using `go mod vendor`. To update a dependency, or add a new one:
1. `go get -v bazbat.com/path/to/dependency` ; this downloads the new dependency
2. `go mod vendor` ; this writes the dependency's source files to the `vendor/` directory
3. `git add go.mod go.sum vendor/` ; this stages all relevant changes to the vendor directory, including file deletions.
Take care that spurious changes (such as editor swapfiles) aren't added.
4. `git commit`
## Releasing a new version
1. Ensure the tests pass, locally on travis (`make test`, `make smoke`, and `make irctest`)
1. Test backwards compatibility guarantees. Get an example config file and an example database from the previous stable
release. Make sure the current build still works with them (modulo anything explicitly called out in the changelog as
a breaking change).
1. Run the `ircstress` chanflood benchmark to look for data races (enable race detection) and performance regressions (
disable it).
1. Update the changelog with new changes and write release notes.
1. Update the version number `irc/version.go` (either change `-unreleased` to `-rc1`, or remove `-rc1`, as appropriate).
1. Commit the new changelog and constants change.
1. Tag the release with `git tag --sign v0.0.0 -m "Release v0.0.0"` (`0.0.0` replaced with the real ver number).
1. Build binaries using `make release`
1. Smoke-test a built binary locally
1. Point of no return: `git push origin master --tags` (this publishes the tag; any fixes after this will require a new
point release)
1. Publish the release on GitHub (Releases -> "Draft a new release"); use the new tag, post the changelog entries,
upload the binaries
1. Update the `irctest_stable` branch with the new changes (this may be a force push).
1. If it's a production release (as opposed to a release candidate), update the `stable` branch with the new changes. (
This may be a force push in the event that stable contained a backport. This is fine because all stable releases and
release candidates are tagged.)
1. Similarly, for a production release, update the `irctest_stable` branch (this is the branch used by upstream irctest
to integration-test against Ergo).
1. Make the appropriate announcements:
* For a release candidate:
1. the channel topic
1. any operators who may be interested
1. update the testnet
* For a production release:
1. everything applicable to a release candidate
1. Twitter
1. ergo.chat/news
1. ircv3.net support tables, if applicable
1. other social media?
Once it's built and released, you need to setup the new development version. To do so:
1. Ensure dependencies are up-to-date.
1. Bump the version number in `irc/version.go`, typically by incrementing the second number in the 3-tuple, and add '
-unreleased' (for instance, `2.2.0` -> `2.3.0-unreleased`).
1. Commit the new version number and changelog with the message `"Setup v0.0.1-unreleased devel ver"`.
**Unreleased changelog content**
```md
## Unreleased
New release of Ergo!
### Config Changes
### Security
### Added
### Changed
### Removed
### Fixed
```
## Debugging
It's helpful to enable all loglines while developing. Here's how to configure this:
```yaml
logging:
-
method: stderr
type: "*"
level: debug
```
To debug a hang, the best thing to do is to get a stack trace. The easiest way to get stack traces is with
the [pprof listener](https://golang.org/pkg/net/http/pprof/), which can be enabled in the `debug` section of the config.
Once it's enabled, you can navigate to `http://localhost:6060/debug/pprof/` in your browser and go from there. If that
doesn't work, try:
$ kill -ABRT <procid>
This will kill Ergo and print out a stack trace for you to take a look at.
## Concurrency design
Ergo involves a fair amount of shared state. Here are some of the main points:
1. Each client has a separate goroutine that listens for incoming messages and synchronously processes them.
1. All sends to clients are asynchronous; `client.Send` appends the message to a queue, which is then processed on a
separate goroutine. It is always safe to call `client.Send`.
1. The server has a few of its own goroutines, for listening on sockets and handing off new client connections to their
dedicated goroutines.
1. A few tasks are done asynchronously in ad-hoc goroutines.
In consequence, there is a lot of state (in particular, server and channel state) that can be read and written from
multiple goroutines. This state is protected with mutexes. To avoid deadlocks, mutexes are arranged in "tiers"; while
holding a mutex of one tier, you're only allowed to acquire mutexes of a strictly *higher* tier. The tiers are:
1. Tier 1 mutexes: these are the "innermost" mutexes. They typically protect getters and setters on objects, or
invariants that are local to the state of a single object. Example: `Channel.stateMutex`.
1. Tier 2 mutexes: these protect some invariants of their own, but also need to access fields on other objects that
themselves require synchronization. Example: `ChannelManager.RWMutex`.
1. Tier 3 mutexes: these protect macroscopic operations, where it doesn't make sense for more than one to occur
concurrently. Example; `Server.rehashMutex`, which prevents rehashes from overlapping.
There are some mutexes that are "tier 0": anything in a subpackage of `irc` (e.g., `irc/logger`
or `irc/connection_limits`) shouldn't acquire mutexes defined in `irc`.
We are using `buntdb` for persistence; a `buntdb.DB` has an `RWMutex` inside it, with read-write transactions getting
the `Lock()` and read-only transactions getting the `RLock()`. This mutex is considered tier 1. However, it's shared
globally across all consumers, so if possible you should avoid acquiring it while holding ordinary application-level
mutexes.
## Command handlers and ResponseBuffer
We support a lot of IRCv3 specs. Pretty much all of them, in fact. And a lot of proposed/draft ones. One of the draft
specifications that we support is called ["labeled responses"](https://ircv3.net/specs/extensions/labeled-response.html)
.
With labeled responses, when a client sends a label along with their command, they are assured that they will receive
the response messages with that same label.
For example, if the client sends this to the server:
@label=pQraCjj82e PRIVMSG #channel :hi!
They will expect to receive this (with echo-message also enabled):
@label=pQraCjj82e :nick!user@host PRIVMSG #channel :hi!
They receive the response with the same label, so they can match the sent command to the received response. They can
also do the same with any other command.
In order to allow this, in command handlers we don't send responses directly back to the user. Instead, we buffer the
responses in an object called a ResponseBuffer. When the command handler returns, the contents of the ResponseBuffer is
sent to the user with the appropriate label (and batches, if they're required).
Basically, if you're in a command handler and you're sending a response back to the requesting client, use `rb.Add*`
instead of `client.Send*`. Doing this makes sure the labeled responses feature above works as expected. The handling
around `PRIVMSG`/`NOTICE`/`TAGMSG` is strange, so simply defer to [irctest](https://github.com/DanielOaks/irctest)'s
judgement about whether that's correct for the most part.
## Translated strings
The function `client.t()` is used fairly widely throughout the codebase. This function translates the given string using
the client's negotiated language. If the parameter of the function is a string, the translation update script below will
grab that string and mark it for translation.
In addition, throughout most of the codebase, if a string is created using the backtick characters ``(`)``, that string
will also be marked for translation. This is really useful in the cases of general errors and other strings that are
created far away from the final `client.t` function they are sent through.
## Updating Translations
We support translating server strings using [CrowdIn](https://crowdin.com/project/oragono)! To send updated source
strings to CrowdIn, you should:
1. `cd` to the base directory (the one this `DEVELOPING` file is in).
2. Install the `pyyaml` and `docopt` deps using `pip3 install pyyamp docopt`.
3. Run the `updatetranslations.py` script with: `./updatetranslations.py run irc languages`
4. Commit the changes
CrowdIn's integration should grab the new translation files automagically.
When new translations are available, CrowsIn will submit a new PR with the updates. The `INFO` command should be used to
see whether the credits strings has been updated/translated properly, since that can be a bit of a sticking point for
our wonderful translators :)
### Updating Translations Manually
You shouldn't need to do this, but to update 'em manually:
1. `cd` to the base directory (the one this `DEVELOPING` file is in).
2. Install the `pyyaml` and `docopt` deps using `pip3 install pyyamp docopt`.
3. Run the `updatetranslations.py` script with: `./updatetranslations.py run irc languages`
4. Install the [CrowdIn CLI tool](https://support.crowdin.com/cli-tool/).
5. Make sure the CrowdIn API key is correct in `~/.crowdin.yaml`
6. Run `crowdin upload sources`
We also support grabbing translations directly from CrowdIn. To do this:
1. `cd` to the base directory (the one this `DEVELOPING` file is in).
2. Install the [CrowdIn CLI tool](https://support.crowdin.com/cli-tool/).
3. Make sure the CrowdIn API key is correct in `~/.crowdin.yaml`
4. Run `crowdin download`
This will download a bunch of updated files and put them in the right place
## Adding a mode
When adding a mode, keep in mind the following places it may need to be referenced:
1. The mode needs to be defined in the `irc/modes` subpackage
1. It may need to be special-cased in `modes.RplMyInfo()`
1. It may need to be added to the `CHANMODES` ISUPPORT token
1. It may need special handling in `ApplyUserModeChanges` or `ApplyChannelModeChanges`
1. It may need special persistence handling code

48
Dockerfile
View File

@ -1,48 +0,0 @@
## build ergo binary
FROM golang:1.17-alpine AS build-env
RUN apk add -U --force-refresh --no-cache --purge --clean-protected -l -u make
# copy ergo source
WORKDIR /go/src/github.com/ergochat/ergo
COPY . .
# modify default config file so that it doesn't die on IPv6
# and so it can be exposed via 6667 by default
RUN sed -i 's/^\(\s*\)\"127.0.0.1:6667\":.*$/\1":6667":/' /go/src/github.com/ergochat/ergo/default.yaml && \
sed -i 's/^\s*\"\[::1\]:6667\":.*$//' /go/src/github.com/ergochat/ergo/default.yaml
# compile
RUN make
## build ergo container
FROM alpine:3.13
# metadata
LABEL maintainer="Daniel Oaks <daniel@danieloaks.net>,Daniel Thamdrup <dallemon@protonmail.com>" \
description="Ergo is a modern, experimental IRC server written in Go"
# standard ports listened on
EXPOSE 6667/tcp 6697/tcp
# ergo itself
COPY --from=build-env /go/bin/ergo \
/go/src/github.com/ergochat/ergo/default.yaml \
/go/src/github.com/ergochat/ergo/distrib/docker/run.sh \
/ircd-bin/
COPY --from=build-env /go/src/github.com/ergochat/ergo/languages /ircd-bin/languages/
# running volume holding config file, db, certs
VOLUME /ircd
WORKDIR /ircd
# default motd
COPY --from=build-env /go/src/github.com/ergochat/ergo/ergo.motd /ircd/ergo.motd
# launch
ENTRYPOINT ["/ircd-bin/run.sh"]
# # uncomment to debug
# RUN apk add --no-cache bash
# RUN apk add --no-cache vim
# CMD /bin/bash

53
crowdin.yml
View File

@ -1,53 +0,0 @@
#
# Your crowdin's credentials
#
"project_identifier": "oragono"
# "api_key" : ""
# "base_path" : ""
#"base_url" : ""
#
# Choose file structure in crowdin
# e.g. true or false
#
"preserve_hierarchy": true
#
# Files configuration
#
files: [
{
"source": "/languages/example/translation.lang.yaml",
"translation": "/languages/%locale%.lang.yaml",
"dest": "translation.lang.yaml"
},
{
"source": "/languages/example/irc.lang.json",
"translation": "/languages/%locale%-irc.lang.json",
"dest": "irc.lang.json"
},
{
"source": "/languages/example/help.lang.json",
"translation": "/languages/%locale%-help.lang.json",
"dest": "help.lang.json",
"update_option": "update_as_unapproved",
},
{
"source": "/languages/example/chanserv.lang.json",
"translation": "/languages/%locale%-chanserv.lang.json",
"dest": "services/chanserv.lang.json",
"update_option": "update_as_unapproved",
},
{
"source": "/languages/example/nickserv.lang.json",
"translation": "/languages/%locale%-nickserv.lang.json",
"dest": "services/nickserv.lang.json",
"update_option": "update_as_unapproved",
},
{
"source": "/languages/example/hostserv.lang.json",
"translation": "/languages/%locale%-hostserv.lang.json",
"dest": "services/hostserv.lang.json",
"update_option": "update_as_unapproved",
},
]

94
distrib/docker/README.md
View File

@ -1,94 +0,0 @@
# Ergo Docker
This folder holds Ergo's Dockerfile and related materials. Ergo is published automatically to Docker Hub at
[ergochat/ergo](https://hub.docker.com/r/ergochat/ergo).
The `latest` tag tracks the `stable` branch of Ergo, which contains the latest stable release. The `dev` tag tracks the
master branch, which may by unstable and is not recommended for production.
## Quick start
The Ergo docker image is designed to work out of the box - it comes with a usable default config and will automatically
generate self-signed TLS certificates. To get a working ircd, all you need to do is run the image and expose the ports:
```shell
docker run --name ergo -d -p 6667:6667 -p 6697:6697 ergochat/ergo:tag
```
This will start Ergo and listen on ports 6667 (plain text) and 6697 (TLS). The first time Ergo runs it will create a
config file with a randomised oper password. This is output to stdout, and you can view it with the docker logs command:
```shell
# Assuming your container is named `ergo`; use `docker container ls` to
# find the name if you're not sure.
docker logs ergo
```
You should see a line similar to:
```
Oper username:password is admin:cnn2tm9TP3GeI4vLaEMS
```
## Persisting data
Ergo has a persistent data store, used to keep account details, channel registrations, and so on. To persist this data
across restarts, you can mount a volume at /ircd.
For example, to create a new docker volume and then mount it:
```shell
docker volume create ergo-data
docker run -d -v ergo-data:/ircd -p 6667:6667 -p 6697:6697 ergochat/ergo:tag
```
Or to mount a folder from your host machine:
```shell
mkdir ergo-data
docker run -d -v $(PWD)/ergo-data:/ircd -p 6667:6667 -p 6697:6697 ergochat/ergo:tag
```
## Customising the config
Ergo's config file is stored at /ircd/ircd.yaml. If the file does not exist, the default config will be written out. You
can copy the config from the container, edit it, and then copy it back:
```shell
# Assuming that your container is named `ergo`, as above.
docker cp ergo:/ircd/ircd.yaml .
vim ircd.yaml # edit the config to your liking
docker cp ircd.yaml ergo:/ircd/ircd.yaml
```
You can use the `/rehash` command to make Ergo reload its config, or send it the HUP signal:
```shell
docker kill -HUP ergo
```
## Using custom TLS certificates
TLS certs will by default be read from /ircd/tls.crt, with a private key in /ircd/tls.key. You can customise this path
in the ircd.yaml file if you wish to mount the certificates from another volume. For information on using Let's Encrypt
certificates, see
[this manual entry](https://github.com/ergochat/ergo/blob/master/docs/MANUAL.md#using-valid-tls-certificates).
## Using docker-compose
This folder contains a sample docker-compose file which can be used to start an Ergo instance with ports exposed and
data persisted in a docker volume. Simply download the file and then bring it up:
```shell
curl -O https://raw.githubusercontent.com/ergochat/ergo/master/distrib/docker/docker-compose.yml
docker-compose up -d
```
## Building
If you wish to manually build the docker image, you need to do so from the root of the Ergo repository (not
the `distrib/docker` directory):
```shell
docker build .
```

20
distrib/docker/docker-compose.yml
View File

@ -1,20 +0,0 @@
version: "3.8"
services:
ergo:
image: ergochat/ergo:latest
ports:
- "6667:6667/tcp"
- "6697:6697/tcp"
volumes:
- data:/ircd
deploy:
placement:
constraints:
- "node.role == manager"
restart_policy:
condition: on-failure
replicas: 1
volumes:
data:

26
distrib/docker/run.sh
View File

@ -1,26 +0,0 @@
#!/bin/sh
# make config file
if [ ! -f "/ircd/ircd.yaml" ]; then
awk '{gsub(/path: languages/,"path: /ircd-bin/languages")}1' /ircd-bin/default.yaml > /tmp/ircd.yaml
# change default oper passwd
OPERPASS=$(< /dev/urandom tr -dc _A-Z-a-z-0-9 | head -c20)
echo "Oper username:password is admin:$OPERPASS"
ENCRYPTEDPASS=$(echo "$OPERPASS" | /ircd-bin/ergo genpasswd)
ORIGINALPASS='\$2a\$04\$0123456789abcdef0123456789abcdef0123456789abcdef01234'
awk "{gsub(/password: \\\"$ORIGINALPASS\\\"/,\"password: \\\"$ENCRYPTEDPASS\\\"\")}1" /tmp/ircd.yaml > /tmp/ircd2.yaml
unset OPERPASS
unset ENCRYPTEDPASS
unset ORIGINALPASS
mv /tmp/ircd2.yaml /ircd/ircd.yaml
fi
# make self-signed certs if they don't already exist
/ircd-bin/ergo mkcerts
# run!
exec /ircd-bin/ergo run

23
distrib/systemd/ergo.service
View File

@ -1,23 +0,0 @@
[Unit]
Description=ergo
After=network.target
# If you are using MySQL for history storage, comment out the above line
# and uncomment these two instead (you must independently install and configure
# MySQL for your system):
# Wants=mysql.service
# After=network.target mysql.service
[Service]
Type=notify
User=ergo
WorkingDirectory=/home/ergo
ExecStart=/home/ergo/ergo run --conf /home/ergo/ircd.yaml
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
LimitNOFILE=1048576
NotifyAccess=main
# Uncomment this for a hidden service:
# PrivateNetwork=true
[Install]
WantedBy=multi-user.target