meta: create documents for application design, make repository ready for publishing on git.fuwawa.moe
This commit is contained in:
parent
d83660ee30
commit
b5846b49c5
|
|
@ -0,0 +1,152 @@
|
|||
# Welcome, Degenerate
|
||||
|
||||
Nothing in this project is set in stone (even this rule). I might not know what I'm doing.
|
||||
|
||||
Table of Content:
|
||||
|
||||
1. [Currently Most Wanted](CONTRIBUTING.md#currently-most-wanted)
|
||||
1. [Communication](CONTRIBUTING.md#communication)
|
||||
1. [Development](CONTRIBUTING.md#development)
|
||||
1. [Environment Setup](CONTRIBUTING.md#environment-setup)
|
||||
1. [Git](CONTRIBUTING.md#git)
|
||||
1. [Database](CONTRIBUTING.md#database)
|
||||
1. [Formatting and Linters](CONTRIBUTING.md#formatting-and-linters)
|
||||
1. [Testing](CONTRIBUTING.md#testing)
|
||||
1. [Design](CONTRIBUTING.md#design)
|
||||
|
||||
## Currently Most Wanted
|
||||
|
||||
### User Stories
|
||||
|
||||
With the help of [user stories](workspace/user-stories.md), one is able to formulate requirements for the application which then can be implemented. This also includes the design of a GUI.
|
||||
|
||||
### Database Schema
|
||||
|
||||
Based on the user stories, features, and feature ideas, one of the next steps is to design a [database schema](workspace/schema).
|
||||
|
||||
- some initial work is already done
|
||||
- feedback is especially needed on the following (the rest of the [ideas folder](workspace/ideas) deserves a look, too)
|
||||
- [tagging system](workspace/ideas/tagging-system.md)
|
||||
- [multiple manga series](workspace/ideas/multi-work-series.md)
|
||||
- [multiple story manga](workspace/ideas/multi-story-manga.md)
|
||||
- [intellectual property and parodies](workspace/ideas/ip-and-parody.md)
|
||||
- any other ideas?
|
||||
|
||||
### GUI/Prototype
|
||||
|
||||
The application needs a graphical user interface. Take a look at the [requirements](workspace/gui/requirements.md) and some [inspiration](workspace/gui/inspiration) (You could also supply more).
|
||||
|
||||
The current idea is to build the frontend with [Svelte](https://svelte.dev), not sure if that has relevance here.
|
||||
|
||||
### Ideas Folder
|
||||
|
||||
The [ideas folder](workspace/ideas) is a collection of possible features for the application. Feedback and discussion is wanted for every idea.
|
||||
|
||||
## Communication
|
||||
|
||||
Every contribution is valuable. Open a ticket [here](https://git.fuwafuwa.moe/Xymorot/RenaiApp/issues) or write me an [email](mailto:xymorot@mailbox.org).
|
||||
|
||||
## Development
|
||||
|
||||
### Environment Setup
|
||||
|
||||
Things to install:
|
||||
|
||||
- [Node.js](https://nodejs.org) (this also installs npm)
|
||||
- [Git](https://git-scm.com/)
|
||||
|
||||
Then [use git to clone this repository](https://www.google.com/search?q=how+to+clone+a+repository+with+git).
|
||||
|
||||
Run `npm install` in a terminal window opened to the cloned repository, all dependencies will be installed. You might be required to install external build tools (especially if under Windows) for the compilation of native node modules, mainly sqlite3.
|
||||
|
||||
Then run `npm run build` to build the app, and `npm start` to run it.
|
||||
|
||||
The `npm run watch` script watches files and rebuilds the app on changes.
|
||||
|
||||
### Git
|
||||
|
||||
This project uses Git as its version control software. If you don't know how to use it, [read a quick guide](https://www.google.com/search?q=how+to+git). Some care has to be taken for the following points:
|
||||
|
||||
#### Commit Messages
|
||||
|
||||
This project uses [Conventional Commits](https://www.conventionalcommits.org) with the following types:
|
||||
|
||||
| type | description |
|
||||
| ---------- | --------------------------------------------------------------------------------------- |
|
||||
| `feat` | implementing a new feature or changes an existing one |
|
||||
| `remove` | removing an existing feature |
|
||||
| `fix` | repairing a feature which does not work correctly |
|
||||
| `refactor` | reworking code so that its function does not change (but could have in unexpected ways) |
|
||||
| `update` | updating dependencies and associated code changes |
|
||||
| `test` | any of the above, but with tests/mocks |
|
||||
| `config` | changing configuration (npm scripts, linters, build process) |
|
||||
| `meta` | updating something not that is not function or documentation (e.g. design) |
|
||||
| `doc` | updating documentation, including code comments |
|
||||
| `reformat` | rewriting code in a way in which it is impossible for function to change |
|
||||
|
||||
Always try to split up your changes into coherent commits, a single commit should do a single thing. If your commit needs to do more than one thing it should be labeled with the type coming first in this list.
|
||||
|
||||
Special cases are **merge commits** and **revert commits**. Git provides a default commit message for these two cases, just leave them if you do nothing else to the committed files.
|
||||
|
||||
#### Git Hooks
|
||||
|
||||
This project uses [Husky](https://github.com/typicode/husky) to define git hooks. The point of the pre-commit hook is to have a clean commits. This means correctly formatted, without linter errors and functioning tests. The `npm run fix` script fixes all fixable errors, the tests you have to fix yourself.
|
||||
|
||||
### Database
|
||||
|
||||
The application uses [SQLite3](https://www.npmjs.com/package/sqlite3) as a database, with [TypeORM](https://typeorm.io) as an object-relation mapper.
|
||||
|
||||
#### Database Migrations
|
||||
|
||||
Migrations are stored in [src/main/migrations](src/main/migrations) and handled by typeorm. Migrations are run on app start inside [database.ts](src/main/services/database.ts).
|
||||
|
||||
To auto-generate a migration:
|
||||
`node_modules/.bin/typeorm migration:generate -n <migration name> -c <connection name>`
|
||||
|
||||
To create an empty creation which can be filled with custom migration code:
|
||||
`node_modules/.bin/typeorm migration:create -n <migration name> -c <connection name>`
|
||||
|
||||
To run migrations:
|
||||
`node_modules/.bin/typeorm migration:run -c <connection name>`
|
||||
This is also pre-defined in the npm script `typeorm:migrate`.
|
||||
|
||||
### Formatting and Linters
|
||||
|
||||
This project uses [Prettier](https://prettier.io/) for code formatting (and an [.editorconfig](.editorconfig) which you should respect). [ESLint](https://eslint.org/) and [TSLint](https://palantir.github.io/tslint/) ([soon to be integrated into ESLint](https://typescript-eslint.io)) are used for linting.\
|
||||
The point of these libraries is to make uniform code possible over various editors. If you are not happy with the rules, do not ignore the warnings! Understand why the rule is there (googling helps) and if you decide the rule should not be active, deactivate it. Also think about if the rule should be deactivated everywhere or not, then configure accordingly. Explain your reasoning in the commit body.
|
||||
|
||||
### Testing
|
||||
|
||||
The testing framework of choice is [Mocha](https://mochajs.org/). Call `npm run test` to run all tests. Tests are written in typescript and need to be transpiled before testing.
|
||||
|
||||
- assertion is (mainly) done by [Chai](https://www.chaijs.com/)
|
||||
- Electron specific testing is done by [Spectron](https://electronjs.org/spectron)
|
||||
- spies, stubs and mocks are provided by [Sinon.JS](https://sinonjs.org/)
|
||||
- HTTP server mocking is done by [nock](https://github.com/nock/nock)
|
||||
- property based testing is made possible by [fast-check](https://github.com/dubzzz/fast-check)
|
||||
|
||||
#### Mocks
|
||||
|
||||
There are 2 ways in which mocks are defined/used:
|
||||
|
||||
0. for external modules, in [mocks](mocks)
|
||||
- uses the [rewiremock](https://github.com/theKashey/rewiremock) package
|
||||
- use this only when there is some magic happening like for electron which normally runs in its own node process
|
||||
1. for own modules, just beside their test file in [tests](tests)
|
||||
- name the file `*.mock.ts` and use existing mock files for orientation on how to build them
|
||||
- use sparingly and only when not having a mock makes it more complex e.g. for modules which interact with the file system
|
||||
|
||||
#### Tagging
|
||||
|
||||
Mocha does [not have a separate tagging feature](https://github.com/mochajs/mocha/wiki/Tagging), but it can filter via title. Use the following tags in your test titles:
|
||||
|
||||
- `@slow` when the test is particularly slow
|
||||
|
||||
#### Coverage
|
||||
|
||||
Code coverage is provided by [nyc](https://github.com/istanbuljs/nyc). The detailed code coverage can be found [here](.nyc_output/coverage/index.html) after running the coverage script `npm run coverage` (open in browser). The coverage script is separate because it does not allow simple debugging.\
|
||||
The code coverage does not work with Spectron since that runs in its own node process.
|
||||
|
||||
## Design
|
||||
|
||||
Before implementation comes design (at least for a bit, until they run in parallel). The [workspace folder](workspace/README.md) is the space where all the documents for application design should go.
|
||||
84
README.md
84
README.md
|
|
@ -2,76 +2,26 @@
|
|||
|
||||
# レンアイ - Hentai Library Thingy
|
||||
|
||||
## Development
|
||||
**What?**\
|
||||
an application for categorizing hentai manga
|
||||
|
||||
### Quickstart
|
||||
**Why?**\
|
||||
see [user stories](workspace/user-stories.md)
|
||||
|
||||
- `npm install`, the postinstall runs
|
||||
- `npm run rebuild`
|
||||
- might need to install some build tools depending on your platform
|
||||
**How?**\
|
||||
Node.js, Electron, Typescript, SQLite3 (TypeORM)
|
||||
|
||||
By the way, there is mature content in this repository.
|
||||
|
||||
## Development Quickstart
|
||||
|
||||
For more information, check [CONTRIBUTING.md](CONTRIBUTING.md).
|
||||
|
||||
- `npm install`, the postinstall script compiles native node modules for electron
|
||||
- might need to install some build tools depending on your platform
|
||||
- `npm run watch` for code transpilation (starts watchers)
|
||||
- `npm run start`
|
||||
|
||||
### Git Commits
|
||||
## Usage
|
||||
|
||||
This project uses [Conventional Commits](https://www.conventionalcommits.or) with the following types:
|
||||
|
||||
| type | description |
|
||||
| ---------- | --------------------------------------------------------------------------------------- |
|
||||
| `feat` | implementing a new feature or changes an existing one |
|
||||
| `remove` | removing an existing feature |
|
||||
| `fix` | repairing a feature which does not work correctly |
|
||||
| `refactor` | reworking code so that its function does not change (but could have in unexpected ways) |
|
||||
| `update` | updating dependencies and associated code changes |
|
||||
| `test` | any of the above, but with tests/mocks |
|
||||
| `config` | changing configuration (npm scripts, linters, build process) |
|
||||
| `meta` | updating something not directly related to function or documentation (e.g. branding) |
|
||||
| `doc` | updating documentation, including code comments |
|
||||
| `reformat` | rewriting code in a way in which it is impossible for function to change |
|
||||
|
||||
Always try to split up your changes into coherent commits, a single commit should do a single thing. If your commit needs to do more than one thing it should be labeled with the type coming first in this list.
|
||||
|
||||
### Database Migrations
|
||||
|
||||
Migrations are stored in [src/main/migrations](src/main/migrations) and handled by typeorm. Migrations are run on app start inside [database.ts](src/main/services/database.ts).
|
||||
|
||||
To auto-generate a migration:
|
||||
`node_modules/.bin/typeorm migration:generate -n <migration name> -c <connection name>`
|
||||
|
||||
To create an empty creation which can be filled with custom migration code:
|
||||
`node_modules/.bin/typeorm migration:create -n <migration name> -c <connection name>`
|
||||
|
||||
To run migrations:
|
||||
`node_modules/.bin/typeorm migration:run -c <connection name>`
|
||||
This is also pre-defined in the npm script `typeorm:migrate`.
|
||||
|
||||
### Testing
|
||||
|
||||
The testing framework of choice is [Mocha](https://mochajs.org/). Call `npm run test` to run all tests. Tests are written in typescript and need to be transpiled before testing.
|
||||
|
||||
- assertion is (mainly) done by [Chai](https://www.chaijs.com/)
|
||||
- Electron specific testing is done by [Spectron](https://electronjs.org/spectron)
|
||||
- spies, stubs and mocks are provided by [Sinon.JS](https://sinonjs.org/)
|
||||
- HTTP server mocking is done by [nock](https://github.com/nock/nock)
|
||||
- property based testing is made possible by [fast-check](https://github.com/dubzzz/fast-check)
|
||||
|
||||
#### Mocks
|
||||
|
||||
There are 2 ways in which mocks are defined/used:
|
||||
|
||||
0. for external modules, in [mocks](mocks)
|
||||
- uses the [rewiremock](https://github.com/theKashey/rewiremock) package
|
||||
- use this only when there is some magic happening like for electron which normally runs in its own node process
|
||||
1. for own modules, just beside their test file in [tests](tests)
|
||||
- name the file `*.mock.ts` and use existing mock files for orientation on how to build them
|
||||
- use sparingly and only when not having a mock makes it more complex e.g. for modules which interact with the file system
|
||||
|
||||
#### Tagging
|
||||
|
||||
Mocha does [not have a seperate tagging feature](https://github.com/mochajs/mocha/wiki/Tagging), but it can filter via title. Us the following tags in your test titles:
|
||||
|
||||
- `@slow` when the test is particularly slow
|
||||
|
||||
#### Coverage
|
||||
|
||||
Code coverage is provided by [nyc](https://github.com/istanbuljs/nyc). The detailed code coverage can be found [here](.nyc_output/coverage/index.html) after running the tests (open in browser).
|
||||
There is nothing to use yet. You might want to contribute a [user story](workspace/user-stories.md) or two.
|
||||
|
|
|
|||
|
|
@ -3,7 +3,11 @@
|
|||
"version": "1.0.0",
|
||||
"description": "Hentai Library Thingy",
|
||||
"private": true,
|
||||
"author": "Xymorot",
|
||||
"license": "SEE LICENSE IN LICENSE.txt",
|
||||
"author": {
|
||||
"name": "Xymorot",
|
||||
"email": "xymorot@mailbox.org"
|
||||
},
|
||||
"main": "src/main.js",
|
||||
"scripts": {
|
||||
"postinstall": "npm run rebuild",
|
||||
|
|
@ -87,6 +91,8 @@
|
|||
"webpack-cli": "^3.3.10",
|
||||
"webpack-stream": "^5.2.1"
|
||||
},
|
||||
"repository": "https://git.fuwafuwa.moe/Xymorot/RenaiApp",
|
||||
"bugs": "https://git.fuwafuwa.moe/Xymorot/RenaiApp/issues",
|
||||
"config": {
|
||||
"forge": "forge.config.js"
|
||||
},
|
||||
|
|
|
|||
|
|
@ -0,0 +1,27 @@
|
|||
# Workspace Folder
|
||||
|
||||
This is the folder in which the design of the application takes place. There is no real structure. If you can not work with the existing structure, refactor it or do your work completely outside of git and then open an issue or write an email.
|
||||
|
||||
Update this README to fit your changes.
|
||||
|
||||
All of this is included in the git repository because if the host site is switched, issues get lost but the repository is backed up by each contributor.
|
||||
|
||||
## [Ideas Folder](ideas)
|
||||
|
||||
The ideas folder is a catch-all place for anything you might want to contribute. When an idea is fleshed out enough, it can transfer into the [features folder](features).
|
||||
|
||||
## [GUI](gui)
|
||||
|
||||
Everything related to the design of the graphical user interface goes there.
|
||||
|
||||
## [Icon](icon)
|
||||
|
||||
This is for the branding of the application, I wanted to include every file which lead to the final icon. The current icon is highly ephemeral, do your own if you want to.
|
||||
|
||||
## [Database Schema](schema)
|
||||
|
||||
Everything related to the data model goes there.
|
||||
|
||||
## [Features](features)
|
||||
|
||||
Concrete feature descriptions go there.
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
# Reading Mode
|
||||
|
||||
Addresses user stories [#2].
|
||||
|
||||
Just like a music library application lets you listen to your music, a hentai library application should let you read your hentai.
|
||||
|
||||
The user should be able to select a manga he would like to read and then be able to navigate the pages.
|
||||
|
||||
[#2]: ../user-stories.md#2
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
# Websites
|
||||
|
||||
Although most of these are not designed as single-page app, it might not be a bad idea to have a look. The mobile version of the sites might be a valid starting point, at least for some kind of reading mode.
|
||||
|
||||
- [nhentai](https://nhentai.net)
|
||||
- [Tsumino](https://www.tsumino.com)
|
||||
- [E-Hentai](https://e-hentai.org)
|
||||
- [Hentai Cafe](https://hentai.cafe)
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
# Requirements for the GUI
|
||||
|
||||
- [dark mode](https://material.io/design/color/dark-theme.html) first
|
||||
- [mobile first](https://www.google.com/search?q=what+is+mobile+first+design)
|
||||
- look at the established [feature requirements](../features) for must-haves
|
||||
- also [user stories](../user-stories.md) for possible features
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
# Categorizing Art Style
|
||||
|
||||
I'm not talking about some concrete categories, I'm thinking more in the direction of a crowd-sourced distance matrix on author (less data) or book (more data) level. When complete, this would make finding manga with similar art style trivial.
|
||||
|
||||
## Critique
|
||||
|
||||
**Does this even make sense?** How would a user define a similar art style? It may devolve into a distance based on tags. Therefore if at all, a distance matrix based on authors makes the most sense.
|
||||
|
||||
**This would require huge amounts of user input.** The work associated with tagging manga grows linearly with their number, the amount of work needed to maintain a distance matrix grows polynomially. It would never be complete.
|
||||
|
||||
**Authors' art style changes.** The problem with building the distance matrix on author level, is that the art styles of authors can change with time.
|
||||
|
|
@ -0,0 +1,19 @@
|
|||
# Exporting Metadata
|
||||
|
||||
This is important for [p2p sharing](p2p-sharing.md) and possibly backing up your library metadata. Metadata includes everything the user curated in regards to their library:
|
||||
|
||||
- rating
|
||||
- tags
|
||||
- names
|
||||
- authors
|
||||
- ...
|
||||
|
||||
One needs to define a format for exporting this metadata. I can think of the following requirements:
|
||||
|
||||
- readable by computer (to import and share with p2p)
|
||||
- being able to only export a part for p2p
|
||||
- a complete set of metadata might be too large
|
||||
|
||||
It will probably boil down to `xml` or `json`. I would choose `json` since that is easier to write and parse with Node.js.
|
||||
|
||||
Maybe the [JSON:API](https://jsonapi.org/) schema lends itself to the p2p application. For backing up locally it seems to be overkill. The local metadata backup might as well be just the database file.
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
# Greedy Loading of Manga Metadata
|
||||
|
||||
The idea here is to crawl all established hentai sites in the background all the time. Instead of the user importing manga explicitly, some automated migration (for each site) would load and save metadata for everything the app can reach.
|
||||
|
||||
**This is pretty much required for fast searching**. It would be unfeasible to request 5+ different websites with the same query, especially when none of them supply a robust API. The saved metadata would then include links to sources.
|
||||
|
||||
**This avoids multiple hits**. Most manga are available on multiple sites. The automated import process would combine them into one dataset.
|
||||
|
||||
**Should this be on by default?** With an empty library, this will probably take ages and unnecessarily overload the source websites. Depending on their infrastructure, they might even implement (or already have implemented) countermeasures. To alleviate this, the p2p-sharing system should be implemented first to take off some/most of the traffic.
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
# Importing and Saving Manga
|
||||
|
||||
Addresses user stories [#1], [#3], [#8].
|
||||
|
||||
## Import
|
||||
|
||||
There should be 2 ways to import manga: from files on the user's computer, and directly over the internet (see [p2p-sharing](p2p-sharing.md) and [site-crawling](site-crawling.md)). This feature document only describes the local import.
|
||||
|
||||
The user should be able to navigate to the files he would like to import, using their operating system's file explorer.
|
||||
|
||||
A point of discussion is how the application should interpret the selected files/folders:
|
||||
|
||||
- One folder is one work?
|
||||
- What about sub-folders?
|
||||
|
||||
I think a folder without sub-folders and files inside should be considered one work. If it has both, it is one work and the sub-folder should be explored further. If it has only sub-folders, it is not a work, and the sub-folders are to be explored. If it has no content, nothing happens.
|
||||
|
||||
Multiple folders should be selectable and importable at the same time.
|
||||
|
||||
## Saving
|
||||
|
||||
The manga should then be saved into a library folder defined by the user in the following format:\
|
||||
`<author>` > `<title>` > _\*files here\*_\
|
||||
This saving strategy should be extendable/configurable in the future.
|
||||
|
||||
## File Types
|
||||
|
||||
Since the application runs inside chromium, every file type supported by the browser should also be supported by the application: https://developer.mozilla.org/en-US/docs/Web/Media/Formats
|
||||
|
||||
This also includes video and audio files.
|
||||
|
||||
[#1]: ../user-stories.md#1
|
||||
[#3]: ../user-stories.md#3
|
||||
[#8]: ../user-stories.md#8
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
# Intellectual Properties and Parodies
|
||||
|
||||
Some works borrow the world from established sources (_Kantai Collection_, _The Idolmaster_), some build their own world.
|
||||
|
||||
Initially it is probably adequate to only implement the labelling of parodies.
|
||||
|
||||
## Parodies
|
||||
|
||||
An author's work can only have characters of an established world, can only take place in the established world, or both.
|
||||
|
||||
In each case the work needs to be findable by the name of the original source. **Would there be a point in allowing users to search for an established world but specifying that it doesn't involve its characters?**
|
||||
|
||||
## Intellectual Properties
|
||||
|
||||
Every original work is automatically a world in its own right. Therefore parodies of it are possible. I have yet to find an example.
|
||||
|
||||
Another case is an author who writes multiple works in their established world, without them being a series (e.g. _Michiking_ with his two _The Older Sister Experience_ Series). The user should still have a way to connect these two series.
|
||||
|
||||
What I am basically saying, is: **Should every work that is not a parody of an established world, be automatically an entrance in its own established world?**
|
||||
|
||||
This would require giving a name to the world and starting to label entrances into any established world as either parody or original.
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
# Manga Reviews
|
||||
|
||||
This idea makes only sense in a p2p world. It would take the place of comments on established sites.
|
||||
|
||||
**Cons**
|
||||
|
||||
- the user only sees the reviews of peers he connects with
|
||||
- not able to moderate
|
||||
|
||||
**Pros**
|
||||
|
||||
- ???
|
||||
- its fun i guess
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
# Multiple Stories Manga
|
||||
|
||||
Even if taggable units (see [tagging-system.md](tagging-system.md#tags-per-pagepanel)) are not implemented, the user should still be able to tag disconnected stories in a single manga differently.
|
||||
|
||||
There are manga which are a collection of different stories. Some of these stories do appear as separate manga as well. Other manga are a collection of parts of a series (see [multi-work-series.md](multi-work-series.md)).
|
||||
|
||||
So if the user searches for the name of a specific story, the collection manga in which it is contained should also be a result.
|
||||
|
||||
If the collection manga is the only result, the page interval of the contained story should also be defined. This way the user could directly navigate to the story.
|
||||
|
||||
If the the story is available as separate story as well, the tags should only have to be defined once.
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
# Multiple Manga Series
|
||||
|
||||
Some manga are part of a connected series. This document describes the feature of implementing this relationship.
|
||||
|
||||
**What would the user gain from this feature?**\
|
||||
If the user stumbles upon one work of the series, it is trivial to find preceding/succeeding parts.
|
||||
|
||||
In its simplest form, there would just be a collection of works belonging to one series defined somewhere.
|
||||
|
||||
The ordering could either be done by release date or some arbitrary user-defined order.
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
# Peer-To-Peer Tag/Source Sharing
|
||||
|
||||
The point of this application is for the user to have their library locally on their computer, not in the cloud ((a.k.a. on someone else's computer). With this they lose the power of collective tagging.
|
||||
|
||||
My vision is for a peer-to-peer (p2p) network is to be able to share metadata of hentai manga. There would be no central truth, but with enough users willing to share their manga metadata, the power of collective tagging is not lost.
|
||||
|
||||
The following features are a must:
|
||||
|
||||
- be able to operate out of a vpn
|
||||
- full anonymity (tor level), even without vpn
|
||||
|
||||
The [GNUnet](https://gnunet.org/en/) project ticks most boxes (javascript bindings are missing) but is in very early stages.
|
||||
|
||||
For alternatives, start reading https://en.wikipedia.org/wiki/Anonymous_P2P
|
||||
|
|
@ -0,0 +1,38 @@
|
|||
# Website Crawling
|
||||
|
||||
The application needs to be able to read data from established hentai sites. This includes the manga themselves, their metadata (tags, author, etc.), and user data (lists, rating, etc.).
|
||||
|
||||
This is derived from user stories [#4], [#5], [#6].
|
||||
|
||||
None of these sites have an official API. Which means data needs to be read from html.
|
||||
|
||||
Depending on whether the sites use captcha or not, authentication could also be difficult. It might be easier to leverage the chromium in electron to load the sites themselves and read their data from there. On the other hand, this might be vulnerable to dubious redirects to ad sites. Another idea is to use `<iframe>`.
|
||||
|
||||
| website | provides api | user data | metadata | torrent download |
|
||||
| ---------------------------------- | :----------: | :-------: | :------: | :--------------: |
|
||||
| [nhentai](https://nhentai.net) | - | ✓ | ✓ | ✓ |
|
||||
| [Tsumino](https://www.tsumino.com) | - | ✓ | ✓ | - |
|
||||
| [E-Hentai](https://e-hentai.org) | - | ✓ | ✓ | ✓ |
|
||||
| [Hentai Cafe](https://hentai.cafe) | - | - | ✓ | - |
|
||||
|
||||
**nhentai**
|
||||
|
||||
- probably most popular (either this or e-hentai)
|
||||
|
||||
**Tsumino**
|
||||
|
||||
- provides direct zip download, but locked behind Google's reCaptcha
|
||||
- the normal image view seems to has some kind of authentication key shenanigans as well
|
||||
|
||||
**E-Hentai**
|
||||
|
||||
- https://exhentai.org/
|
||||
- will probably be archived in the near future
|
||||
|
||||
**Hentai Cafe**
|
||||
|
||||
- the most bare functionality, probably easiest to crawl
|
||||
|
||||
[#4]: ../user-stories.md#4
|
||||
[#5]: ../user-stories.md#5
|
||||
[#6]: ../user-stories.md#6
|
||||
|
|
@ -0,0 +1,129 @@
|
|||
# Tagging System
|
||||
|
||||
Addresses user stories [#1], [#4], [#5].
|
||||
|
||||
The tagging system will be at the center of organizing the library. Therefore it needs to be done correctly. You are invited to question and discuss everything in this document.
|
||||
|
||||
A "[[...] tag is a keyword or term assigned to a piece of information](https://en.wikipedia.org/wiki/Tag_(metadata\))". Things which I do not consider tags:
|
||||
|
||||
- author/groups
|
||||
- source material of fan art
|
||||
- characters
|
||||
- language/translation
|
||||
- censorship
|
||||
|
||||
You should still be able to filter by these things, but their semantic difference disqualifies them as generic tags.
|
||||
|
||||
## Why?
|
||||
|
||||
**The purpose of tags is to find something specific**. A user should be able to put in what he wants and does not want, and the application should serve it.
|
||||
|
||||
## How?
|
||||
|
||||
A user should be able to add tags to works in their library. This addresses [#1], but does not enable the discovery of new manga. For this to be possible, existing sites need to be searched. This requires a way to unify the existing tagging systems.
|
||||
|
||||
Before a way to do that can be established, we first need to decide if users should be able to define their own tags or if tags are predefined.
|
||||
|
||||
**If tags are pre-defined**, they are static. Only new versions of the software would update tags.
|
||||
|
||||
**If tags are user-defined**, the user has all the responsibility. Tag migration gets harder.
|
||||
|
||||
I propose to provide a number of tags shipped with the application, preserving the ability of the user to define and curate tags.
|
||||
|
||||
Now to the more complicated part: **tag migration**. Tags coming from other sources (existing sites or p2p metadata) need to be migrated to the user library. Several things need to be implemented to make this possible:
|
||||
|
||||
- **tag aliases**\
|
||||
define different names for the same tag (_oppai_ = _big breasts_)
|
||||
- **migration rules**\
|
||||
depending on the complexity of the tagging system (discussed [below](tagging-system.md#possible-features)), different migration rules might be necessary
|
||||
|
||||
It might be necessary to define these things separately for each metadata source. Which is pretty much impossible for p2p.
|
||||
|
||||
## Possible Features
|
||||
|
||||
The following ideas are ordered by increasing complexity. The names for these features are not perfect but the best I could think of. The question for each one is if they have use cases?
|
||||
|
||||
### Tag Weight
|
||||
|
||||
**Tag weight is how prominent the tag is in the connected work**. This would be on a scale from 0% to 100% (0% excluded, since then the tag shouldn't be there), or (0,1].
|
||||
|
||||
- https://nhentai.net/g/205905/
|
||||
- _paizuri_, but only on 2 out of 20 pages, resulting in a weight of 0.1
|
||||
- _full color_, on every page. So this could be a new tag _color_ with weight 1, discussion see below
|
||||
|
||||
The point of tag weights is to enable sorting by relevance when searching for a group of tags. This also opens up advanced search queries if the user would like only results with a tag weight over/under a threshold.
|
||||
|
||||
As a work-around for tag weights, the user could just define a tag for each weight range they deem to be different enough to necessitate a distinction when doing a search.
|
||||
|
||||
Some tags change name with weight, maybe only on weight 1. A user should be able to put in the different name and then find works with the tag above that weight.
|
||||
|
||||
#### Tags per Page/Panel
|
||||
|
||||
This is a way to objectively (as possible) quantify the tag weight. Each manga would have a fixed number of taggable units (pages/panels). The user could then define if the tag is present in each unit or not. The weight can then be easily calculated.
|
||||
|
||||
The definition of taggable units would have to be done by the user if something more complex than just pages is chosen. With "just pages", the problem is that manga often have cover and end pages which do not contribute to taggable units, in my opinion.
|
||||
|
||||
#### Special Case: Multi-Story Manga
|
||||
|
||||
Some manga are a collection of multiple stories with each having different tags. In any case, the taggable unit should at least be the story. For single-story manga, the taggable unit would the complete manga if nothing else. Read more in [this document](multi-story-manga.md).
|
||||
|
||||
### Tag Relations
|
||||
|
||||
Some tags are connected to other tags. This is best explained with some examples.
|
||||
|
||||
**Category/Hierarchy**\
|
||||
If a tag is in a category, then the work is automatically tagged with the category, meaning searchable by it. This could apply to multiple levels.
|
||||
|
||||
- _kemonomimi_ ("animal ears") is category for _nekomimi_, _ookamimi_, etc.
|
||||
- _group_ is a category for _threesome_ is a category for _ffm threesome_
|
||||
|
||||
Question: What about multiple inheritance? Should it be implemented? With that i mean that with one tag curated, it would mean more than one parent tag automatically added. I can't think of a example.
|
||||
|
||||
**Opposition**\
|
||||
Tags related this way exclude each other (at least on the same taggable unit). There might also be a required choice between them.
|
||||
|
||||
- _monochrome_ | _color_ (one of them should always be present)
|
||||
- _rape_ | _consent_ (as long as there is sexual interaction, one of them is pretty much required)
|
||||
|
||||
In each of these cases, one of the tags could be the default and be omitted if the other is not given. As a work-around, the user could do a query with the non-default tag negated. A problem then arises with tag weight: Assume _monochrome_ is the default and omitted, since a manga can be partly in color and greyscale (and tagged _color_ with a weight), but when the user searches for _not color_ with the intent to find monochrome manga, the manga won't be in the result even though it is partly _monochrome_.
|
||||
|
||||
**More Complex Relations**\
|
||||
Some more complex relations are also thinkable:
|
||||
|
||||
- _nakadashi_, by definition, requires at least one penis and one vagina. Which means there should be a tag like _male_/_futanari_ and _female_ present. This would help in warning the user about invalid tag lists, but could not directly create tags for the user.
|
||||
|
||||
### Derived Tags from Tagged Character Network
|
||||
|
||||
**What do tags define?** In most cases they describe a character or the interaction between characters.
|
||||
|
||||
So this feature would not only have pages/panels as taggable units, but make characters taggable as well. Most interaction tags would also need a direction (who dicks down who), although obvious most of the time, and a reference to the actors. Defaults would make curating easier.
|
||||
|
||||
Example https://nhentai.net/g/271431/:
|
||||
|
||||
- character tags 1: Sumoto-senpai, _female_, _oppai_, _school uniform_, _unusual pupils_
|
||||
- character tags 2: Fudou-kun, _male_
|
||||
- interaction tags: _nakadashi_, _blowjob_, _defloration_, _paizuri_
|
||||
- normal tags: _condom_ (could be character tag of the guy, depending on what that tag is supposed to describe in the user's mind)
|
||||
|
||||
This would also enable searches like:
|
||||
|
||||
- "I want _virginity_, but only the guy and he has to be a shota."
|
||||
- "I want _dark skin_, but only the girl, she can have a dick tho."
|
||||
- "I want an anatomically _female_ _elf_, with _pointy ears_ and in a _maid costume_ who gets her _ear nibbled_ at some point by some other party."
|
||||
|
||||
This would also make some other tags purely emergent: _yuri_, _yaoi_, _group_, _vanilla_ (whatever that is for you). The emergence rules for these tags would need to be defined as well.
|
||||
|
||||
## Other
|
||||
|
||||
- tag description
|
||||
- could be curated for pre-defined tags
|
||||
- users could use it to remind themselves of what they envisioned with this tag
|
||||
- it could even help with p2p, explaining what the peer meant
|
||||
- pretend tags, should there be some kind of flag or just let the user implement 2 tags for each case?
|
||||
- real cat ears vs. a headband
|
||||
- maid costume vs. a real maid
|
||||
- role-play vs. the real thing
|
||||
|
||||
[#1]: ../user-stories.md#1
|
||||
[#4]: ../user-stories.md#4
|
||||
[#5]: ../user-stories.md#5
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
# Torrent
|
||||
|
||||
Question: **Should the app be a bitTorrent client as well?** There are npm packages that make it possible.
|
||||
|
||||
**As long as it can't guarantee anonymity, it should not be implemented.**
|
||||
|
||||
There should still be some way to use conventional torrent clients together with the application. This entails the building of a torrent file which fits the library directory structure. Also it must make use of already established torrents from the sites that support them.
|
||||
|
|
@ -0,0 +1,53 @@
|
|||
# User Stories
|
||||
|
||||
As a **_\<type of user>_**, I want **_\<some goal>_** so that **_\<some reason>_**.
|
||||
|
||||
[What is a user story?](https://www.google.com/search?q=what+is+a+user+story) These can be as specific or broad as you want. You don't have to use the template above. You don't have to be that type of user. Write them here, open an issue, or write an email. You can also discuss the existing user stories via the same channels.
|
||||
|
||||
## 1
|
||||
|
||||
> As a data hoarder, I want a way to categorize and store my library. That way I can find what I'm searching for locally.
|
||||
|
||||
---
|
||||
|
||||
## 2
|
||||
|
||||
> As a degenerate weeb, I just want to read some hentai. (for research purposes)
|
||||
|
||||
---
|
||||
|
||||
## 3
|
||||
|
||||
> As a paranoid person, I want to store my manga locally before Fakku (or the government) takes out another one.
|
||||
|
||||
---
|
||||
|
||||
## 4
|
||||
|
||||
> As a hentai connoisseur, I want some way to find new works so that I may enjoy them.
|
||||
|
||||
---
|
||||
|
||||
## 5
|
||||
|
||||
> As someone who doesn't like to put all his eggs into one basket, I'd like to be able to search multiple (if not all) established hentai sites.
|
||||
|
||||
---
|
||||
|
||||
## 6
|
||||
|
||||
> As an existing hentai site user, I would like to import my library, so that I don't have to do all the work again.
|
||||
|
||||
---
|
||||
|
||||
## 7
|
||||
|
||||
> As a conscious person with enough money, I'd like to know who created and/or translated a manga so I know where to put my money to get more.
|
||||
|
||||
---
|
||||
|
||||
## 8
|
||||
|
||||
> As someone who doesn't trust a single application, I would like my library saved in a sensible (human-navigable) folder structure.
|
||||
|
||||
---
|
||||
Loading…
Reference in New Issue