 11b
|
b79ac657a4
|
fix: haru's sft being incompatible with the ColossalAI fine-tune script
|
2022-12-26 20:42:48 -03:00 |
|
11b
|
5dbde00d27
|
feat: bring down target word count per episode
After tokenization, most stuff was going over the 2048 context window so let's bring this down a little.
|
2022-12-26 17:31:28 -03:00 |
|
11b
|
bcbf0910b4
|
feat: add supervised fine-tuning code based on haru's work
Warning: Absolutely atrocious code quality. I did just the bare minimum to make it run.
|
2022-12-26 17:31:00 -03:00 |
|
11b
|
60e649f57a
|
feat: some minor filtering to hopefully improve CAI data
|
2022-12-26 12:04:04 -03:00 |
|
11b
|
4f794489ac
|
feat: add support for fine-tuning GPT-NeoX-based models, save optimizer and LR scheduler to checkpoint
|
2022-12-25 15:42:59 -03:00 |
|
11b
|
186df60691
|
feat: update inference code for pythia/cai data-based models
|
2022-12-25 15:39:28 -03:00 |
|
11b
|
3bfb623f26
|
fix: human/bot messages being incorrectly labeled as eachother
|
2022-12-24 17:58:33 -03:00 |
|
11b
|
5b26097905
|
feat: implement Gradio UI for proper model inference (WIP)
|
2022-12-24 12:12:55 -03:00 |
|
11b
|
cef8f54fc4
|
fix: ignore invalid CAI JSON dumps
|
2022-12-23 16:45:18 -03:00 |
|
11b
|
d91367e902
|
chore: update module list in build_dataset.py
|
2022-12-23 16:45:18 -03:00 |
|
11b
|
a16673ebe0
|
refactor: adjust Kajiwoto modules to use the proper prompt constants
|
2022-12-23 16:45:18 -03:00 |
|
11b
|
60e0a21a3c
|
chore: add pdbpp for better debugging experience
|
2022-12-23 16:38:13 -03:00 |
|
11b
|
3d6def871d
|
refactor: use LIGHT as PDM instead of VDM, ignore actions
|
2022-12-23 16:38:13 -03:00 |
|
11b
|
1f273f13f3
|
chore: bump pdm version
|
2022-12-23 16:38:13 -03:00 |
|
11b
|
e0552639fa
|
feat: update CAI dataset/module to handle userscript dumps and use definitions
|
2022-12-23 16:38:13 -03:00 |
|
11b
|
aef9289678
|
chore: update ROADMAP to add links about contributing with CAI dumps
|
2022-12-23 10:59:58 -03:00 |
|
11b
|
69aeea85b9
|
chore: reorganize CAI dumper README
|
2022-12-21 20:05:15 -03:00 |
|
11b
|
7087f39d5a
|
fix: cai dumper crashing if chat had no messages
|
2022-12-21 20:05:01 -03:00 |
|
11b
|
d6e05e6e5b
|
chore: add changelog to the CAI dumper
|
2022-12-21 16:24:35 -03:00 |
|
11b
|
e612386424
|
fix: handle edge-case regarding extra whitespace on char name
|
2022-12-21 16:15:49 -03:00 |
|
11b
|
d638bb5625
|
feat: update userscript to allow dumping of definitions as well
|
2022-12-21 16:03:13 -03:00 |
|
11b
|
ecf2e65e76
|
fix: anonymization within message text
Hopefully for reals this time.
|
2022-12-21 14:25:50 -03:00 |
|
11b
|
21ebd5834e
|
chore: fix path links in the README
|
2022-12-21 13:51:20 -03:00 |
|
11b
|
a8dfd396cc
|
fix: more aggressive anonymization within message text
|
2022-12-21 13:43:46 -03:00 |
|
11b
|
6bc2a03ff9
|
chore: update ROADMAP
|
2022-12-20 21:52:28 -03:00 |
|
11b
|
1ddd991471
|
chore: clarify what DHT means in the Discord module
|
2022-12-20 21:44:16 -03:00 |
|
11b
|
cec59a5511
|
docs: update CAI dumper README
|
2022-12-20 21:43:57 -03:00 |
|
11b
|
4f78bb73cb
|
feat: implement userscript to dump CAI chats/basic bot info
|
2022-12-20 21:34:39 -03:00 |
|
11b
|
ecd4efe3ce
|
chore: run isort
|
2022-12-20 17:55:17 -03:00 |
|
11b
|
009c837439
|
feat: implement Discord dialogue module
|
2022-12-20 17:55:05 -03:00 |
|
11b
|
b42131191a
|
fix: don't write file when printing the dataset for debugging
|
2022-12-20 17:41:08 -03:00 |
|
11b
|
45f495a4dd
|
docs: add basic roadmap
|
2022-12-20 17:40:48 -03:00 |
|
11b
|
30ff3751de
|
feat: allow fine-tuning of non-OPT LMs
|
2022-12-18 22:24:43 -03:00 |
|
11b
|
25ae9da046
|
fix: tensorboard getting messed up after epoch end
|
2022-12-18 22:24:26 -03:00 |
|
11b
|
fb3dc4ba22
|
chore: fix typo in README
|
2022-12-18 17:36:01 -03:00 |
|
11b
|
877b1bc56b
|
feat: barebones inference script for testing checkpoints
|
2022-12-18 17:29:15 -03:00 |
|
11b
|
f82b4ea913
|
feat: super early CAI dataset and module for testing
|
2022-12-18 17:29:15 -03:00 |
|
11b
|
657cbe1d61
|
chore: update gitignore to handle checkpoints
|
2022-12-18 17:29:15 -03:00 |
|
11b
|
b0d2d80ac3
|
feat: minor improvements to the fine-tune script
|
2022-12-18 17:29:15 -03:00 |
|
11b
|
925f5767ec
|
feat: fine-tuning scripts and instructions
|
2022-12-18 17:29:15 -03:00 |
|
11b
|
6fbd660a67
|
feat: implement script to build final data file
|
2022-12-18 17:29:15 -03:00 |
|
11b
|
8df2d87355
|
feat: Kajiwoto dataset and modules
|
2022-12-18 17:29:15 -03:00 |
|
11b
|
a076746f9d
|
fixup! feat: initial commit
|
2022-12-18 17:29:15 -03:00 |
|
11b
|
f5552cde74
|
feat: add the LIGHT dataset and VDM
|
2022-12-18 17:29:15 -03:00 |
|
11b
|
cb1d3dd68e
|
chore: enable strict mode on mypy, add isort check
|
2022-12-17 21:31:41 -03:00 |
|
11b
|
ad3c378d58
|
feat: initial commit
|
2022-12-17 17:52:34 -03:00 |