RenaiApp/workspace/ideas/greedy-loading.md

1005 B

Greedy Loading of Manga Metadata

The idea here is to crawl all established hentai sites in the background all the time. Instead of the user importing manga explicitly, some automated migration (for each site) would load and save metadata for everything the app can reach.

This is pretty much required for fast searching. It would be unfeasible to request 5+ different websites with the same query, especially when none of them supply a robust API. The saved metadata would then include links to sources.

This avoids multiple hits. Most manga are available on multiple sites. The automated import process would combine them into one dataset.

Should this be on by default? With an empty library, this will probably take ages and unnecessarily overload the source websites. Depending on their infrastructure, they might even implement (or already have implemented) countermeasures. To alleviate this, the p2p-sharing system should be implemented first to take off some/most of the traffic.