Commit Graph

312 Commits

Author SHA1 Message Date
kayos@tcp.direct 1ec4297c7e
Merge from origin 2022-02-28 01:24:22 -08:00
kayos@tcp.direct 924280a7bb Fork: change logger 2022-01-21 03:35:03 -08:00
kayos@tcp.direct 9ae8f5afbf Update mirror 2022-01-16 02:12:18 -08:00
kayos@tcp.direct 818498fca0 Update mirror 2022-01-16 02:11:18 -08:00
kayos@tcp.direct f78b223453 rm cruft 2022-01-16 02:01:38 -08:00
kayos@tcp.direct 9476678083 Fix module path 2021-12-31 07:09:31 -08:00
James Mills d23c355e72
Update CHANGELOG for v1.0.2 2021-11-01 17:54:39 +10:00
James Mills 40425394d7
Fix a data race in Datafile.ReadAt() 2021-11-01 17:54:31 +10:00
James Mills f4cc0fb434
Fix release tool 2021-10-31 07:08:53 +10:00
James Mills 7d4174d5b1
Update CHANGELOG for v1.0.1 2021-10-31 07:08:01 +10:00
James Mills 5429693cc8 Add ErrBadConfig and ErrBadMetadata as errors that consumers can check and use (#241)
cc @taigrr

This PR will _hopefully_ help to fix some critical isseus in the real world with several or more [Yarn.social](https://yarn.social) pods running [yarnd](https://git.mills.io/yarnsocial/yarn) where starting back up after a power failure or crash can sometimes result in an empty `config.json` or empty `meta.json` or both!

I'm not actually sure how this can arise, and as yet I haven't been able to reproduce it (_I can only assume this has to be failures cases outside of our control_); but in any case the application and database is recoverable by simply `rm config.json` and/or `rm meta.json`.

So this PR makes errors loading the config and metadata first-class errors and exported error types that consumers of the library can use to perform automated recovery without requiring human intervention.

Basiclaly in this case it's no big deal we lost the database config of metadata, we can simply carry on.

Co-authored-by: James Mills <prologic@shortcircuit.net.au>
Reviewed-on: https://git.mills.io/prologic/bitcask/pulls/241
Co-authored-by: James Mills <james@mills.io>
Co-committed-by: James Mills <james@mills.io>
2021-10-30 21:07:42 +00:00
jason3gb 2c57c950f8 [Fix] disable mmap for current datafile from #239 (#240)
Fix issues related to #239

Disable mmap reader for current datafile, which only read from the fd.

Co-authored-by: jason3gb <jason3gb@gmail.com>
Reviewed-on: https://git.mills.io/prologic/bitcask/pulls/240
Reviewed-by: James Mills <james@mills.io>
Reviewed-by: Tai Groot <tai@taigrr.com>
Co-authored-by: jason3gb <jason3gb@noreply@mills.io>
Co-committed-by: jason3gb <jason3gb@noreply@mills.io>
2021-09-25 04:26:26 +00:00
biozz 21a824e13e Add key prefix matching to KEYS command (#237)
Related to #234 and !236.

This is the implementation that was requested in the original issue. I updated KEYS command to be redis-valid and implemented prefix search. There is also a rather interesting test, I could you use some feedback here.

I noticed that it might not be possible to reduce the complexity of the KEYS command. Because even if you use Scan, you will have to store the counter of all found keys before you do WriteBulk of the actual keys.

@prologic here is what you probably had in mind:

```
s.db.Scan([]byte(prefix), func(key []byte) error {
	conn.WriteBulk(key)
	return nil
})
```

But there is no way to call `conn.WriteArray(n)` with the number of keys until you iterate through all of them, hence the second loop over found keys.

Co-authored-by: Ivan Elfimov <ielfimov@gmail.com>
Co-authored-by: James Mills <james@mills.io>
Reviewed-on: https://git.mills.io/prologic/bitcask/pulls/237
Reviewed-by: James Mills <james@mills.io>
Co-authored-by: biozz <biozz@noreply@mills.io>
Co-committed-by: biozz <biozz@noreply@mills.io>
2021-09-20 10:35:27 +00:00
James Mills 2279245b8c
Update image target 2021-09-17 07:49:07 +10:00
kayos@tcp.direct fc9e41007a mirror modification 2021-09-01 00:10:30 -07:00
James Mills 91d4db63d5
Update CHANGELOG for v1.0.0 2021-07-24 17:07:25 +10:00
James Mills 849192f709
Update CHANGELOG for 1.0.0 2021-07-24 13:37:57 +10:00
James Mills a4fc2cf4e8
Update README 2021-07-22 19:44:29 +10:00
James Mills 609de833eb
Update CHANGELOG for v0.3.14 2021-07-21 12:38:20 +10:00
James Mills 9b0daa8a30 Add RangeScan() support (#160)
Co-authored-by: James Mills <1290234+prologic@users.noreply.github.com>
Co-authored-by: James Mills <prologic@shortcircuit.net.au>
Co-authored-by: Tai Groot <tai@taigrr.com>
Reviewed-on: https://git.mills.io/prologic/bitcask/pulls/160
Co-authored-by: James Mills <james@mills.io>
Co-committed-by: James Mills <james@mills.io>
2021-07-21 02:36:06 +00:00
Tai Groot ef187f8315 [ADD] Sift and ScanSift (+ tests) (#232)
Added Sift and ScanSift functions for review without tests (for now)

fix docstrings

Added tests for Sift and ScanSift

Note this also fixes a bug in the Scan() function where the RMutex is not locked, allowing a potential race condition

closes #231

Reviewed-on: https://git.mills.io/prologic/bitcask/pulls/232
Co-authored-by: Tai Groot <tai@taigrr.com>
Co-committed-by: Tai Groot <tai@taigrr.com>
2021-07-21 00:19:25 +00:00
James Mills b094cd33d3 Fix runGC behaviour to correctly delete all expired keys (#229)
Fixes #228

Co-authored-by: James Mills <prologic@shortcircuit.net.au>
Reviewed-on: https://git.mills.io/prologic/bitcask/pulls/229
Co-authored-by: James Mills <james@mills.io>
Co-committed-by: James Mills <james@mills.io>
2021-07-20 20:42:22 +00:00
James Mills 3ff8937205
Fix missing push event 2021-07-20 15:57:31 +10:00
James Mills 2ccca759ce
Fix how CI is triggered 2021-07-20 15:56:32 +10:00
Tai Groot 92535e654b [FIX] race condition from #216 (#227)
[ADDED] new tests for TTL expiration race condition,  see #216

[REMOVED] removes cleanup / automatic expiration from get() function to resolve #216

Reviewed-on: https://git.mills.io/prologic/bitcask/pulls/227
Co-authored-by: Tai Groot <tai@taigrr.com>
Co-committed-by: Tai Groot <tai@taigrr.com>
2021-07-18 23:41:40 +00:00
James Mills c4a7ad7a7f
Fix README Go Reference badge 2021-07-16 07:49:45 +10:00
James Mills e64646fa8f
Fix README badges 2021-07-16 07:46:39 +10:00
James Mills 2de030ad5c
Update CHANGELOG for v0.3.13 2021-07-16 07:37:55 +10:00
James Mills 5e4d863ab7 Use package github.com/gofrs/flock as flock implementation. (#224)
Supercesd #219 after rebasing on master after migrating off Github.

Co-authored-by: Nicolò Santamaria <nicolo.santamaria@protonmail.com>
Co-authored-by: James Mills <prologic@shortcircuit.net.au>
Co-authored-by: Tai Groot <taigrr@noreply@mills.io>
Reviewed-on: https://git.mills.io/prologic/bitcask/pulls/224
Co-authored-by: James Mills <prologic@noreply@mills.io>
Co-committed-by: James Mills <prologic@noreply@mills.io>
2021-07-15 21:33:20 +00:00
James Mills a49bbf666a Fix paths used for temporary recovery iles to avoid crossing devices (#223)
Fixes #222

Co-authored-by: James Mills <prologic@shortcircuit.net.au>
Reviewed-on: https://git.mills.io/prologic/bitcask/pulls/223
Co-authored-by: James Mills <prologic@noreply@mills.io>
Co-committed-by: James Mills <prologic@noreply@mills.io>
2021-07-15 13:16:58 +00:00
James Mills 52df2fad55
Improve error reporting on recovery errors 2021-07-14 22:59:51 +10:00
James Mills 947d15fed8
Debug failing test in CI 2021-07-14 22:37:30 +10:00
James Mills d276c398da
Add Drone CI config 2021-07-14 22:30:01 +10:00
James Mills f357607cee
Update CHANGELOG for v0.3.12 2021-07-13 20:21:27 +10:00
James Mills 4fb0aab67c
Update CHANGELOG for v0.3.11 2021-07-10 22:15:25 +10:00
James Mills 707ac163bf
Ignore .envrc 2021-07-10 22:14:05 +10:00
James Mills adabe2c273
Configure GoReleaser correctly 2021-07-10 21:40:29 +10:00
James Mills a58e17c34e
Fix missing go.sum entries 2021-07-10 18:11:47 +10:00
James Mills 8c5c10dcf8
Fix GoReleaser config 2021-07-10 18:08:07 +10:00
James Mills c94f8e8bbc
Release to Gitea 2021-07-10 17:53:11 +10:00
James Mills 3987e56e79
Removed Github workflows 2021-07-10 17:47:48 +10:00
James Mills 90dd53c573
Rename all Go module paths 2021-07-10 17:47:38 +10:00
James Mills 81d7d5459b
Moved to git.mills.io 2021-07-10 17:39:45 +10:00
James Mills b98b684bb4
Refactor TTL with a new API PutWithTTL() and reduce memory allocs (#220) 2021-07-09 17:21:35 +10:00
dependabot-preview[bot] 2ee13b8e32
Bump github.com/tidwall/redcon from 1.4.0 to 1.4.1 (#214) 2021-06-29 12:25:20 +10:00
dependabot[bot] 561caa48ba
Bump github.com/spf13/viper from 1.8.0 to 1.8.1 (#221) 2021-06-26 10:41:37 +10:00
dependabot[bot] d78e590b3d
Bump github.com/spf13/viper from 1.7.1 to 1.8.0 (#218) 2021-06-17 09:16:15 +10:00
Yash Suresh Chandra e7c6490762
Purge api added to remove expired keys (#204)
* purge api added

* merged with master, import order fix

* purge api renamed to RunGC

Co-authored-by: yash <yash.chandra@grabpay.com>
2021-06-02 06:47:30 +10:00
dependabot-preview[bot] 1009661b52
Upgrade to GitHub-native Dependabot (#215) 2021-04-30 09:47:47 +10:00
dependabot-preview[bot] e26d9c54d4
Bump github.com/sirupsen/logrus from 1.8.0 to 1.8.1 (#213) 2021-03-10 05:57:57 +10:00