Go to file
gtrunsec 8f92961958 update: store.png 2020-08-02 16:57:32 +08:00
agenda update: store.png 2020-08-02 16:57:32 +08:00
bin fix hydra gc path 2020-06-01 08:09:56 +08:00
dotfiles vast: fix config path args 2020-07-29 16:20:27 +08:00
elk emacs: update config 2020-07-11 06:09:48 +08:00
home-manager update nix-du 2020-07-31 05:30:02 +08:00
img update: store.png 2020-08-02 16:57:32 +08:00
modules vast: fix config path args 2020-07-29 16:20:27 +08:00
nix deploy zeek kafka done 2020-06-17 10:34:40 +08:00
pkgs rename zeek kafka topics 2020-06-17 11:46:57 +08:00
test/hydra emacs: reorg agenda path and using org-roam 2020-07-11 07:48:26 +08:00
.gitignore using password example 2020-05-31 09:48:41 +08:00
.wakatime-project feacture: using wakatime to tracking time 2020-03-26 20:01:46 -04:00
README.org update nixpkgs 2020-06-15 14:35:48 +08:00
config.nix remove polipo 2020-05-30 18:13:32 -04:00
home.nix deploy zeek 3.0.7 done 2020-06-17 09:10:33 +08:00

Hardenedlinux User's Debian nix home-manager configuration

Install nix

wget https://github.com/hardenedlinux/nixpkgs-hardenedlinux//releases/download/nix/nix_2.3_master_e3eecb5_amd64.deb
sudo dpkg -i nix_2.3_master_e3eecb5_amd64.deb
sudo ln -s /opt/nix-multiuser/nix/bin/nix* /usr/local/bin

nix-channel --add https://github.com/GTrunSec/nixpkgs/archive/my-release.tar.gz nixpkgs
nix-channel --update
export NIX_PATH="$HOME/.nix-defexpr/channels"
export NIX_PATH=$HOME/.nix-defexpr/channels${NIX_PATH:+:}$NIX_PATH
export TERM=xterm
nix-shell -p 'git'
git clone https://github.com/hardenedlinux/debian-nix-manager.git ~/.config/nixpkgs/
mv  ~/.config/nixpkgs/modules/password/password-example.json  ~/.config/nixpkgs/modules/password/password.json
  • Authorized User
  • use make by nix own. make sure you have nix/bin/path/ on sudoers
  • /etc/sudoers
# echo $HOME/.nix-profile/bin
# /home/test/.nix-profile/bin

Defaults        secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/test/.nix-profile/bin"
  • set User's NOPASSWORD to sudoers
test   ALL=(ALL:ALL) NOPASSWD: ALL

Install home-manager

nix-env -i home-manager

add home-manager channel

nix-channel --add https://github.com/rycee/home-manager/archive/master.tar.gz home-manager
nix-channel --update
  • For China's region users

set substituters to /.config/nix/nix.conf

substituters = https://mirrors.tuna.tsinghua.edu.cn/nix-channels/store https://cache.nixos.org

Build

home-manager switch
  • set priority for home-manager
 nix-env --set-flag priority 10 home-manager
 ##setting flag on 'home-manager-2020-03-17'
  • set zsh as default shell
 chsh --shell /home/debian/.nix-profile/bin/zsh debian

Or using hardenedlinux Hydra CI binary cache speed your building process

home-manager switch --option substituters "https://cache.nixos.org http://221.4.35.244:8301" --option trusted-public-keys "221.4.35.244:3ehdeUIC5gWzY+I7iF3lrpmxOMyEZQbZlcjOmlOVpeo="

Install cachix

nix-env -iA cachix -f https://cachix.org/api/v1/install
sudo mkdir -p /etc/nix
echo "trusted-users = root $USER" | sudo tee -a /etc/nix/nix.conf
sudo pkill nix-daemon
cachix use nsm-data-analysis
sudo systemctl restart nix-daemon.service

Build nixpkgs-hardenedlinux

clone https://github.com/hardenedlinux/nixpkgs-hardenedlinux

git clone https://github.com/hardenedlinux/nixpkgs-hardenedlinux
cd nixpkgs-hardenedlinux/
nix-build --option substituters "https://cache.nixos.org https://nsm-data-analysis.cachix.org"

Start pkgs service

tenzir/vast: Visibility Across Space and Time   database siem

 systemctl --user start vast.service
 systemctl --user status vast.service
● vast.service
   Loaded: loaded (/nix/store/59sx0prx1fi93653kkgcsdr4schqa7bv-vast.service/vast.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2020-03-13 03:17:56 EDT; 1h 9min ago
 Main PID: 28612 (vast)
   CGroup: /user.slice/user-1000.slice/user@1000.service/vast.service
           └─28612 /nix/store/n6vm1zifpl65445k6w866sf109j2imwm-vast/bin/vast -c /nix/store/kg43s4bdarkg3g79kxii3h9cmbfym2sj-vast.conf start

Deploy zeek and Model

  • Deploy it with Sytemd Zeek Service Demo
  services.zeek = {
    enable = true;
    standalone = true;
    interface = "enp0s3";
    listenAddress = "localhost";
    privateScript = ''
    @load /home/gtrun/project/hardenedlinux-zeek-script/scripts/zeek-query.zeek
    '';
  };
 systemctl --user start zeek.service
  • Check status
sudo zeekctl status
Name         Type       Host          Status    Pid    Started
zeek         standalone localhost     running   8167   05 May 00:04:44

DONE FOR TEST

Zeek PLugin postgresql

Default enable Kafka and PsotgreSqll alread in nix of Zeek.

Johanna::PostgreSQL - PostgreSQL log writer and input reader (dynamic, version 0.2.0)

Database

postgresql

systemctl --user start  postgresql.service
systemctl --user status  postgresql.service
● postgresql.service
   Loaded: loaded (/nix/store/32xm7dcwlnjais6b42iaa8jh4zkfc3ji-postgresql.service/postgresql.service; linked; vendor preset: enabled)
   Active: active (running) since Sun 2020-03-29 23:11:28 EDT; 15min ago
 Main PID: 3542 (mp0sg0q78h9bwa0)
   CGroup: /user.slice/user-1000.slice/user@1000.service/postgresql.service
           ├─3542 /nix/store/828g2nqfgivscv79xykkmgjk0znll08l-bash-4.4-p23/bin/bash -e /nix/store/mp0sg0q78h9bwa0z45x4n4alc0ffg24f-run-postgresql
           ├─3551 /nix/store/gl7xj33j9fsklbwlgwlgdw6ggj57l7fh-postgresql-11.7/bin/postgres -k /var/db/postgresql/11
           ├─3563 postgres: checkpointer
           ├─3564 postgres: background writer
           ├─3565 postgres: walwriter
           ├─3566 postgres: autovacuum launcher
           ├─3567 postgres: stats collector
           └─3568 postgres: logical replication launcher

Deploy osquery

enable osquery Service

sudo mkdir -p /var/osquery/log
sudo chown $USER /var/osquery
systemctl --user status osquery.service
● osquery.service
   Loaded: loaded (/nix/store/mxpjazyy6b4hymxk9hkivfs1kqk7jvly-osquery.service/osquery.service; linked; vendor preset: enabled)
   Active: active (running) since Fri 2020-03-27 02:49:17 EDT; 37s ago
 Main PID: 26822 (osqueryd)
   CGroup: /user.slice/user-1000.slice/user@1000.service/osquery.service
           ├─26822 /nix/store/acx6mvslzxbzw7fyl4nr87m9pybb9wmn-osquery-4.2.0/bin/osqueryd --database_path /var/osquery/osquery.db --logger_path /var/osquery/log --pidfile /var/osquery/osqueryd.pidfile --database_path /var/osquery/osquery.db --extensions_socket /var/osquery/osquery.em --config_path /home/test/.osquery/osquery.conf
           └─26841 /nix/store/acx6mvslzxbzw7fyl4nr87m9pybb9wmn-osquery-4.2.0/bin/osqueryd

Deploy ELK

Start Elastic service

create /var/lib/elasticsearch/ and make yourself the owner of that directory

sudo mkdir -p /var/lib/elasticsearch/
sudo chown $USER /var/lib/elasticsearch/

start service

systemctl --user start elasticsearch.service
systemctl --user status elasticsearch.service
● elasticsearch.service
   Loaded: loaded (/nix/store/8dncyqmv46xa6j3cr52czs3ky86nsiyh-elasticsearch.service/elasticsearch.service; linked; vendor preset: enabled)
   Active: active (running) since Mon 2020-03-23 19:37:34 EDT; 8min ago
 Main PID: 24715 (java)
   CGroup: /user.slice/user-1000.slice/user@1000.service/elasticsearch.service
           ├─24715 /nix/store/8wmf6apz3yss4vz67z6xdwhhd08yz4cb-openjdk-headless-8u222-ga-jre/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.io.tmpdir=/tmp/elasticsearch-5006850798322202895 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=logs/hs_err_pid%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:logs/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=32 -XX:GCLogFileSize=64m -Des.path.home=/var/lib/elasticsearch -Des.path.conf=/var/lib/elasticsearch/config -Des.distribution.flavor=default -Des.distribution.type=tar -cp /nix/store/6czj00nnxdzr18by4n3rqlfcp0csak0b-elasticsearch-6.8.3/lib/* org.elasticsearch.bootstrap.Elasticsearch
           └─24810 /var/lib/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller

Performance

PF_RING

sudo apt-get update
sudo apt install linux-headers-$(uname -r)
nix-shell ~/.config/nixpkgs/pkgs/network/pf_ring_model.nix --command "sudo make install && sudo modprobe pf_ring"
  • Output test
modinfo pf_ring && cat /proc/net/pf_ring/info
filename:       /lib/modules/4.19.0-8-amd64/kernel/net/pf_ring/pf_ring.ko
alias:          net-pf-27
version:        7.6.0
description:    Packet capture acceleration and analysis
author:         ntop.org
license:        GPL
srcversion:     A80A92A0F9D4CB8168B549A
depends:
retpoline:      Y
name:           pf_ring
vermagic:       4.19.0-8-amd64 SMP mod_unload modversions
parm:           min_num_slots:Min number of ring slots (uint)
parm:           perfect_rules_hash_size:Perfect rules hash size (uint)
parm:           enable_tx_capture:Set to 1 to capture outgoing packets (uint)
parm:           enable_frag_coherence:Set to 1 to handle fragments (flow coherence) in clusters (uint)
parm:           enable_ip_defrag:Set to 1 to enable IP defragmentation(only rx traffic is defragmentead) (uint)
parm:           quick_mode:Set to 1 to run at full speed but with upto one socket per interface (uint)
parm:           force_ring_lock:Set to 1 to force ring locking (automatically enable with rss) (uint)
parm:           enable_debug:Set to 1 to enable PF_RING debug tracing into the syslog, 2 for more verbosity (uint)
parm:           transparent_mode:(deprecated) (uint)
PF_RING Version          : 7.6.0 (unknown)
Total rings              : 0

Standard (non ZC) Options
Ring slots               : 4096
Slot version             : 17
Capture TX               : Yes [RX+TX]
IP Defragment            : No
Socket Mode              : Standard
Cluster Fragment Queue   : 0
Cluster Fragment Discard : 0