Commit Graph

29 Commits

Author SHA1 Message Date
Ignacio Hagopian
5be114adab
Makefile setup & key/value coherent datatypes & refactoring (#98)
* internal/data: comment exported functions

* internal/data: make smaller codec exported api surface

* make key and value sizes serializing bubble up to everything

* Makefile setup & go mod tidy
2019-09-12 10:44:26 -03:00
Ignacio Hagopian
7e0fa151f7 fix test compilation (#97) 2019-09-10 06:25:09 +10:00
James Mills
d59d5ad8c2
Improves Test Coverage by covering error cases (#95)
* Add Unit  Test for testing a corrupted config

* Add Unit Test for testing errors from .Stats()

* Refactor  Datafile into an interface and add Unit Tests for testing Merge() errors

* Refactor indexer into an interface and add Unit Tests for .Close() errors

* Add Unit Tests for .Delete() errors

* Add Unit Tests for  testing Put/Get errors

* Add Unit Test for testing Open errors (bad path for example)

* Refactor out bitcask.writeConfig

* Add more tests for config errors

* Add unit test for options that might error

* Add more test cases for close errors

* Add test case for rotating datafiles

* Fix a possible data race in .Stats()

* Add test case for checksum errors

* Add test case for Sync errors with Put and WithSync enabled

* Refactor and use testify.mock for mocks and generate mocks for all interfaces

* Refactor TestCloseErrors

* Refactored TestDeleteErrors

* Refactored TestGetErrors

* Refactored TestPutErrors

* Refactored TestMergeErrors and fixed a bug with .Fold()

* Add test case for Scan() errors

* Apparently only Scan() can return nil Node()s?
2019-09-09 07:18:38 +10:00
Ignacio Hagopian
0d3a9213ed cmd/bitcask: recovery tool (#92)
* cmd/bitcask: recovery tool

* refactor configuration & use it in recover tool
2019-09-07 07:57:30 +10:00
Ignacio Hagopian
a2b5ae2287 fix: check of persisted index values (#91) 2019-09-04 22:42:32 +10:00
Ignacio Hagopian
93cc1d409f codec_index: check sizes, new tests for data corruption & refactor (#84)
* bitcask/codec_index: check key and data sizes

* codec_index: tests for key and data size overflows

* codec_index: simplify internal funcs for unused returns
2019-09-04 12:26:26 +10:00
Ignacio Hagopian
8041a4c1e7 Refactor and general tests for codec index (#83)
* codec_index: unexport const fields

* codec_index: unexport internal functions and doc exported ones

* codec_index: rename func & return errors for corruption

* codec_index: new test for ReadIndex, WriteIndex, and read corruption

* Update internal/codec_index.go

Co-Authored-By: James Mills <1290234+prologic@users.noreply.github.com>

* Update internal/codec_index.go

Co-Authored-By: James Mills <1290234+prologic@users.noreply.github.com>
2019-09-03 08:19:35 +10:00
James Mills
abbbeb8e1d
Replace keydir with ART trie (#75)
* Replace keydir with ART trie

* Address some review feedback

* Address review feedback (consts)
2019-09-02 08:38:56 +10:00
James Mills
36bc134b22
Fix a bug wit the decoder passing the wrong value for the value's offset into the buffer (#77) 2019-08-31 08:35:17 +10:00
Ignacio Hagopian
8f56cffd86 codec: collapse write and save extra alloc (#64) 2019-08-12 06:47:26 +10:00
James Mills
7204a33512
Fix and cleanup some unnecessary internal sub-packages and duplication 2019-08-08 22:28:25 +10:00
James Mills
c7d101d34f
Fixed missing model import for codec with undefined Entry 2019-08-08 20:14:27 +10:00
Awn
af43cfa8f1 Remove merge function (#60)
* tidy: clean up some leftovers

Fixes #56
Fixes #57
Fixes #58

* api: remove standalone merge function

Fixes #55
2019-08-08 19:51:45 +10:00
Ignacio Hagopian
1f10b4026d inline hash function to save extra alloc (#53) 2019-08-08 09:52:31 +10:00
Ignacio Hagopian
fd179b4a86 custom high-performance encoder implementation (#52) 2019-08-08 09:21:46 +10:00
James Mills
755b1879b5
Use []byte byte slices as keys directly avoiding serialing string(s) (#46) (#51) 2019-08-08 08:14:48 +10:00
James Mills
d0c913ccee
Revert "Use []byte byte slices as keys directly avoiding serialing string(s) (#46)" (#50)
This reverts commit 3c1808cad3f19c23c6e4aacd4cfbbc4a04da1c08.
2019-08-08 08:06:38 +10:00
James Mills
3c1808cad3
Use []byte byte slices as keys directly avoiding serialing string(s) (#46) 2019-08-08 07:59:11 +10:00
Ignacio Hagopian
a407905ae2 Improve Get/Put performance with optional mempooling (#36)
* avoid unnecessary use of encoder/decoder to decrease memory allocations

* add an optional configurable mempool to avoid extra allocs

* add doc.go with examples
2019-08-05 07:23:07 +10:00
Ignacio Hagopian
8aa66c66da keydir: avoid defers (#34) 2019-08-01 19:18:05 +10:00
James Mills
51bac21c0a
Improves Merge() operation by also pruning old key/value pairs (#29)
* Added new API Stats() and Prune()

* Improved Merge() logic to also prune old key/values and actually reclaim disk space

* Added backward compat for the old Merge() function

* Refactor indexing of keys to items (hints)

* Remove redundant TestOpenMerge

* Add unit test for Stats()

* Improve TestMerge()
2019-07-27 07:52:25 +10:00
panyun
fd2023ee38 Fix issue(db file Merge issue in windows env): (#15)
* Fix issue(windows env):
1. Run my program firstly, it will create my.db folder normaly.
2. Close my program and restart it again, I will receive an error.

* Fix coding format issue

* Change the last fixing with @prologic suggestion.
2019-07-23 13:27:14 +10:00
James Mills
47ad6601f3
Update generated protobuf code 2019-07-19 20:49:49 +10:00
James Mills
53dc013215
Optimized and increased read performance by ~2-3x by memory mapping the read-only datafiles 2019-03-23 13:35:55 +10:00
James Mills
711d08ce91
Increased read performance by ~3-4x by removing another unnecessary I/O operation (Seek) 2019-03-23 12:14:15 +10:00
James Mills
c593bc966f
Refactor Datafile.Size() 2019-03-22 17:33:24 +10:00
James Mills
352c32ee12
Add Len() to exported API (extended API) 2019-03-21 10:47:50 +10:00
James Mills
aaea7273c3
Add Keys() to exported API (extended API) 2019-03-21 10:41:56 +10:00
James Mills
1298240f53
Unexport some internal implemtnation details 2019-03-18 17:31:31 +10:00