[ Hybrid(s) of machine learning and irc3 framework ]
Go to file
0xd3d0c3d 08c8b5b835 tcpac del url is shared privately 2022-07-27 08:22:10 -05:00
databases patch 2022-02-01 22:44:42 -06:00
personalities gp3 personalities 2022-06-26 04:37:59 -05:00
plugins tcpac del url is shared privately 2022-07-27 08:22:10 -05:00
.gitignore update 2021-12-01 03:26:10 -06:00
README.md seen/dayspokewordcount 2022-07-27 07:57:47 -05:00
maple.ini irc3_sasl 2022-07-27 05:48:50 -05:00
requirements.txt openapi gpt3 added 2022-06-25 15:34:18 -05:00

.[d]. ircd.chat/6697 #tcpdirect

MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMh+MMMMMMMMMMMMMMhsMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMm/ oMMMMMMMMMMMMMMm +NMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMy`  yMMMMMMMMMMMMMMM- -mMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMs+dMMMMMMMMMM+    sMMMMMMMMMMMMMMM-  `dMMMMMMMMMMms/NMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMM+ .omMMMMMM:     -MMMMMMMMMMMMMMo    `yMMMMMMMy: `dMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMM-   /dMMM+       sMMMMMMMMMMMMh      `hMMMNo`   sMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMd     :dm        `mMMMMMMMMMMN.       .NNo`    .MMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMM:      -         :MMMMMMMMMMs         :`      sMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMs                 ymNMMMMMNm.                 NMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMy                    `-/-`                   .MMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMo                                            .NMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMNh+.                                               :sdMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMhso+:.                                                      `-/+syMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMM-                                                                 dMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMM`   `.:+/.                                              `/s+:.    sMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMNo   -oms.             .//-`               `:/:`            `+md+`  .hMMMMMMMMMMMMMMM
MMMMMMMMMMMMMNs`                        .odNdo.       .ohmd+`                        :dMMMMMMMMMMMMM
MMMMMMMMMMMNo`                             ..           .-                             :hMMMMMMMMMMM
MMMMMMMMMd+`                                                                             -sNMMMMMMMM
MMMMMMNs-                         `..                          `/-.                        `+dMMMMMM
MMMNy:                       ./sdNMMMh:                      `sNMMMNds/.                      .odMMM
MM+                       :ymMMMMMMMMMMh.                   +NMMMMMMMMMMmo-                      /NM
MMMh:                  .sNMMMMMMMMMMMMMMN-                `hMMMMMMMMMMMMMMMm+`                 :hMMM
MMMMMd:`       ``-:+shmMMMMMMMMMMMMMMMMMMN.               hMMMMMMMMMMMMMMMMMMMmhs+/-..``````./dMMMMM
MMMMMMMMMNNNNNNMMMMMMMMMMMMMMMMMMMMMMMMMMMo              .MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMy              .MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMN.             /MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMN+`         `+NMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMNs.    -hMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMdyymMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM

summary

MAPLE+G1MP+OPENAI ML/Irc3 Hybrid - irc3 framework hybridized with machine learning elements.

changelog - v6.66

bot_sasl_1 - a plugin to authenticate with sasl. the original source doesn't work with our ircd
bot_sasl_2 - original source: https://github.com/gawel/irc3/blob/master/irc3/plugins/sasl.py
boombox_1 - the plugin has been modified to filter anything that is non-music. e.g. cat videos
boombox_2 - the plugin has been reduced to no parameters to keep it simple, usage: ?boombox
dsa_1 - this has been modified to do not only determine historical on joins/parts but is also
dsa_1 - doing an additional layer dsa_scan at 5 minute intervals on a cronjob.
dsa_2 - all of its verbose output has been silenced so its not tagging users at random hours.
seem_1 - activity record was  added to keep record if a user spoke on a day and how many words were spoken
seen_2 - used in conjunction with dsa_plugin we can discern  if a person is a lurker or an idler

changelog - v6.5

boombox - a plugin to collectively sort through music pumped into an irc channel

boombox: functional hooking

summary:

in order for boombox to do what it does it needs hooked into places that can feed it information.
there are 3 ideal places for the present Usage.

examples:

1) url_grabber plugin
- url_grabber_plugin.py - this plugin pre-scans every url that passes through a channel
- hookexample: https://git.tcp.direct/decoded/m4pl1mp/src/branch/master/plugins/url_grabber_plugin.py#L73-L75

2) youtube plugin
- youtube_plugin.py - this plugin is used a lot and for youtube specific monitoring with links, it works well.
- hooking example: https://git.tcp.direct/decoded/m4pl1mp/src/branch/master/plugins/youtube_plugin.py#L76-L79
- hooking example: https://git.tcp.direct/decoded/m4pl1mp/src/branch/master/plugins/youtube_plugin.py#L137-L140

3) soundcloud plugin
- soundcloud_plugin.py - this plugin is used a lot for soundcloud specific monitoring with links, it works well.
- hooking example: https://git.tcp.direct/decoded/m4pl1mp/src/branch/master/plugins/soundcloud_plugin.py#L75-L77

boombox: color system

the colors are randomized through a palette system so that design doesn't grow too stale, i'll add some
symbols and blocks later so that the actual geometry randomizes as well.

boombox: url token system

there are no urls shown on screen because they vary in link so drastically, so i use 
a token system to refer to music.. on every line that you see music in regard to boombox.
you will see something like this example below:
◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸◹◸ ◅ TOP SONGS ▻
01 ◊ black roses ( ph8th mix )......................... ╏ ?bb j01xR  ╏4 plays total ╏ 4 mostly by d
02 ◊ lether strip - strap me down...................... ╏ ?bb FRqP4  ╏4 plays total ╏ 4 mostly by d
03 ◊ the hacker - dark neon............................ ╏ ?bb 7ChIH  ╏4 plays total ╏ 4 mostly by d

for every line there is a token on the line, and if you want the song info like a link of origin you type ?bb j01xR
<&d> ?bb j01xR
<maple> d: boombox->url: https://soundcloud.com/vanta_sec/black-roses-ph8th-mix ▶ {◕ ◡ ◕} 

i chose this method for many different reasons ranging from screenspace, artform, alignment,
urls hortening service but then not having to maintain it etc..
  • boombox_plugin: ?bb j01xR

boombox: accepts the commands [top,last,user,site,all]

boombox top command will show the top songs being played in a channel and a counter
  • boombox_plugin: ?boombox top
boombox last command shows songs previously being dumped into a channel
  • boombox_plugin: ?boombox last
boombox user command will show a list of users dropping musc, their playcount and top song
  • boombox_plugin: ?boombox user
boombox site command shows music source origination, play counter and the top song.
  • boombox_plugin: ?boombox site
boombox all commands basically combines all of the options above into one screen
  • boombox_plugin: ?boombox all

changelog - v6.4

highlight - tag an irc user with random or pre-defined color/symbol sets

hl: minimal/maximum values accepted regardless of orientation

the two examples below will produce the same result regardless of orientation, 
but will randomize the color/symbool vagitlues used. the random code values will
be show at the end of each message line for the purpose of reproducing the same stylization later.
  • highlight_plugin: ?hl ircnick hello
  • highlight_plugin: ?hl hello ircnick
the three examples below will produce the same result regardless of orientation, same as the
above, this will use randomized values for the colors and symbols. the example above used no
iterations and this one used an iteration of 10, the 10 represents the amount of times this 
message will spam the user.
  • highlight_plugin: ?hl 10 ircnick hello
  • highlight_plugin: ?hl ircnick hello 10
  • highlight_plugin: ?hl hello 10 ircnick
the four examples below will produce the same result regardless of orientation. everthing is the
same as the above two examples except we are pre-defining the colors and symbols that are used.
the colors are in the first 4 parameters, the symbols are in the next 6. those 6 values represent
two symbol sets. example code: 04,03,05,03,01,02,03,20,19,18 - the values 04,03,05,03 are the
colors of the palette. 01,02,03 are the first symbol set and 20,19,18 are the second symbol set.
  • highlight_plugin: ?hl 04,03,05,03,01,02,03,20,19,18 10 ircnick hello
  • highlight_plugin: ?hl 10 ircnick hello 04,03,05,03,01,02,03,20,19,18
  • highlight_plugin: ?hl ircnick hello 04,03,05,03,01,02,03,20,19,18 10
  • highlight_plugin: ?hl hello 04,03,05,03,01,02,03,20,19,18 10 ircnick

changelog - v6.3

dsa - an automated system for moderated channels based on interaction historicals

dsa_scan: scans the channel users & then designate active statuses

  • dsa_plugin: ?dsa_scan

dsa_hardchatradar: automated version of dsa_scan above

  • dsa_plugin: ?dsa_scan

dsa_anaylze: will scan or search user interaction historicals

  • dsa_plugin: ?dsa_analyze dr1p

dsa_set_threshold: sets the threshold in days

  • dsa_plugin: ?dsa_set_threshold 30

dsa_lurkers: lists all users in the lurking database

  • dsa_plugin: ?dsa_lurkers

dsa_hardchatters: lists all users in the hardchatter database

  • dsa_plugin: ?dsa_hardchatters

hardchatter: add a user to the hardchatter database

  • dsa_plugin: ?dsa_hardchatter dr1p

softchatter: remove a user from the hardchatter database

  • dsa_plugin: ?dsa_softchatter dr1p

changelog - v6.2

- color coded ▶ online(green) / offline(red) ai responses to know which ai is responding
- openai gpt3 (this runs online)
- pipe between openai <-> maple gpt2 (this runs offline)

ai: send queries to openai

  • openai_plugin: ?ai write me a story about the time gangbanging squirrels took back their tree
  • openai_plugin: ?ai write me a keylogger in c

aitrain: train/append to personality databases for the openai term query

  • openai_plugin: ?aitrain terry davis is god he created the divine templeos in holyc
  • openai_plugin: ?aitrain humans will trick you into installing windows to limit your potential

aidefault: return personality and properties to the defaults

  • openai_plugin: ?aidefault

aiset: set openai query properties

  • openai_plugin: ?aiset model text-davinci-002
  • openai_plugin: ?aiset temperature 0.7
  • openai_plugin: ?aiset max_tokens 2000
  • openai_plugin: ?aiset top_p 1.0
  • openai_plugin: ?aiset frequency_penalty 0.0
  • openai_plugin: ?aiset presence_penalty 0.0

aishow: show the current openai server query properties

  • openai_plugin: ?aishow

airand: randomize the current openai server query properties

  • openai_plugin: ?airand

aiterm: show the last raw term query sent to the openai server

  • openai_plugin: ?aiterm

airesponse: show the last raw response from the openai server

  • openai_plugin: ?airesponse

aiclear: clear the current training personality data

  • openai_plugin: ?aiclear

ailist: list the stored personality databases

  • openai_plugin: ?ailist

aiload: load a stored personality database by index

  • openai_plugin: ?aiload 1

airead: read the current personality database data

  • openai_plugin: ?airead

aiwrite: create personality database from the current training data

  • openai_plugin: ?aiwrite retarded

changelog - v6.1

- reprocessor
- fifo

reprocessor: maple yields better responses

  • maple_plugin: used a few methods to reject maple responses that failed to meet certain criteria
  • maple_plugin: is now semi aware of her name being referenced and your name now, it's experimentation phase

fifo: used to pump ansi and enable the krylon project

  • fifo: edit maple.ini at the very bottom, adjust the path to your home directory. So for example change
  • fifo: the runpath to '/home/yourlogin/m4pl1mp/fifo' for linux or if you are using osx/darwin it would be
  • fifo: '/Users/yourlogin/m4pl1mp/fifo' - maple will create the fifo directory when ran
[plugins.fifo_plugin]
runpath = /Users/dr1p/m4pl1mp/fifo
  • fifo: be sure to edit plugins/fifo_plugin.py, change the path around line 16
self.runpath = self.config.get('runpath', '/Users/dr1p/m4pl1mp/fifo')
  • fifo: to disable fifo, comment out the plugin and the bottom 2 lines in maple.ini

changelog - previously

  • revise plugins/maple_plugin.py to run async inside irc3
  • do callback exception handling
  • update maple_plugin.py to use a queue stack

overview

python based irc bot using this irc3 framework: https://github.com/gawel/irc3. the machine learning
aspects are based off maple/maxwell and are a combination of tensorflow gpt2 and microsoft dialogpt.
there are a lot of plugins written and there are a lot of integration layers now possible already.

prerequisites

  • apt install python3
  • apt install python3-pip
  • python3 -m pip install virtualenv

installation

  • git clone --recursive https://git.tcp.direct/decoded/m4pl1mp.git
  • cd m4pl1mp
  • virtualenv env -p python3
  • source env/bin/activate
  • pip install -r requirements.txt

where to signup for api keys

- openai: https://beta.openai.com/signup
- youtube: https://developers.google.com/
- twitter: https://developer.twitter.com/en/portal/petition/essential/basic-info

env/bin/activate: append your developer api keys here.

cat env/bin/activate
..
export DEVELOPER_KEY="1394823190182390182382383215382158321"                       # <- YOUTUBE API KEY
export CONSUMER_KEY="2151235132512351235123512351325231"                           # <- YOUTUBE API KEY
export CONSUMER_SECRET="514512521345234523452345234523452345234523452"             # <- TWITTER API KEY
export ACCESS_TOKEN_KEY="24513429875209348502934850294898348034850293485203948592" # <- TWITTER API KEY
export ACCESS_TOKEN_SECRET="523490582034985203948520394804884820934850923485"      # <- TWITTER API KEY
export OPENAPI_KEY="sk-ERWPOEIRWadfasdfawerWRWERWERWE123512351235123512"           # <- OPENAI API KEY

notice

  • env/bin/activate: if you don't have twitter/youtube/openai api keys set then irc3 will crash
  • maple.ini: comment out plugins.[twitter/youtube/openai]_plugin to disable

content

  • maple.ini - bot configuration file: settings, personalization, plugins, properties, etc.
  • requirements.txt - list of required python modules to install
  • fifo/raw - this is created when fifo plugin is used, allowing you to cat or pipe into the raw tcp socket
  • fifo/tcpdirect - created when the fifo plugin is used. cat/piping outputs to #tcpdirect channel
  • databases/emote.db - this is a database of well known emote faces for emo/sentiment.. o<;O)
  • databases/maple_db.json - where we store most of the plugin data, this is also a setting in maple.ini
  • personalties/default.db - the default personality database loaded by openai_plugin.py
  • personalties/skid.db - an example of a skid personality database by openai_plugin.py
  • personalties/gangsta.db - an example of a gangsta personality database by openai_plugin.py
  • personalties/trained.db - this is where trained data is stored/cached openai_plugin.py
  • plugins/auth_plugin.py - used to identify to nickserv when connected or reconnected
  • plugins/base_plugin.py - a lot of basic core functionality, voice, kick, etc.
  • plugins/bot_sasl_plugin.py - used to authenticate to an ircd with sasl
  • plugins/boombox_plugin.py - passively keep record of music that is pumped into a channel
  • plugins/crypto_plugin.py - used to check crypto market values via the cryptocompare exchange
  • plugins/dsa_plugin.py - used to automate moderated channel chat control restricting undesirabless
  • plugins/emote_plugin.py - this is the plugin to show emote faces, it uses the emote.db listed above
  • plugins/fifo_plugin.py - this is the actual fifo plugin, one line needs altered to your user path
  • plugins/figlet_plugin.py - graphical text logos. same as console but into an irc channel
  • plugins/highlight_plugin.py - highlight ircusers with custom colors/symbols/messages and iterations
  • plugins/isup_plugin.py - check if a host is online or offline. e.g. ?isup google.com
  • plugins/joke_plugin.py - pulls a random joke from a tailored comedian database
  • plugins/maple_plugin.py - the machine learning integration layer
  • plugins/notes_plugin.py - used to store notes for later access. ?notes write some frequent reference
  • plugins/openai_plugin.py - gpt3 openai queries, requires a developer key in env/bin/activate
  • plugins/quote_plugin.py - quote users & then their quotes are randomly used as an entrance
  • plugins/ratesex_plugin.py - crypto graphs using the site rate.sx, same graphdata but in irc
  • plugins/remind_plugin.py - a future reminder. e.g. maple remind me in 2h to start fires
  • plugins/sed_plugin.py - juggle/manipulate things out of context. e.g. j/computer/dildo/
  • plugins/seen_plugin.py - last time a user was seen or what they said. ?seen freemoney
  • plugins/soundcloud_plugin.py - soundcloud music plugin - usage ?sc yung innanet
  • plugins/storage_plugin.py - custom db config like databases/maple_db.json, see maple.ini
  • plugins/strain_plugin.py - this is used to look up strains of marijuana via wikiweed
  • plugins/tell_plugin.py - user message delivery service. e.g. maple tell joe to call me
  • plugins/twitter_plugin.py - url data, requires a developer key in env/bin/activate
  • plugins/ud_plugin.py - lookup things in the urban dictionary. e.g. ?ud space docking
  • plugins/url_grabber_plugin.py - pre-fetches url data & displays in channel
  • plugins/youtube_plugin.py - url data and search query, requires a developer key in env/bin/activate

usage

  • source env/bin/activate
  • irc3 maple.ini

notes

some ways to run maple as a background task

  • screen+irc3 maple.ini
  • tmux+irc3 maple.ini
  • irc3 maple.ini & disown

todo

  • krylon: use znc's modpython module as an integration layer with this framework to bounce bots/nets