Merge branch 'dev' into dev-zephy

README.md

@@ -4,37 +4,35 @@
 [![Support me](https://img.shields.io/endpoint.svg?url=https%3A%2F%2Fshieldsio-patreon.herokuapp.com%2Fpitu&style=flat-square)](https://www.patreon.com/pitu)
 [![Support me](https://img.shields.io/badge/Support-Buy%20me%20a%20coffee-yellow.svg?style=flat-square)](https://www.buymeacoffee.com/kana)
 
+### Attention
+If you are upgrading from v3 to v4 (current release) and you want to keep your files and relations please read the [migration guide](docs/migrating.md).
+
 ### Pre-requisites
 This guide asumes a lot of things, including that you know your way around linux, nginx and internet in general.
 
 - Decently updated version of linux
-- `node` package installed and at least at version 10
-- `build-essential` package installed to build some dependencies
-- `ffmpeg` package installed if you want thumbnails
+- `node` version 12+
+- `build-essential` package installed to build dependencies
+- `ffmpeg` package installed if you want video thumbnails
 - `yarn` package installed. If you'd like to use npm instead change `package.json` accordingly
-- A database, postgresql preferably. You can also fall back to sqlite3 by default.
+- `pm2` globally installed (`npm i -g pm2`) to keep the service alive at all times.
+- A database, postgresql preferably. You can also fall back to sqlite3 which ships by default.
 
 ### Installing
 
 1. Clone the repository and `cd` into it
 2. Run `yarn install`
 3. Run `yarn setup`
-4. Run `yarn migrate`
-5. Run `yarn seed`
 
 Lolisafe is now installed, configured and ready. Now you need to serve it to the public by using a domain name.
 
 6. Check the [nginx](docs/nginx.md) file for a sample configuration that has every step to run lolisafe securely on production.
 
-After you finish setting up nginx, you need to start lolisafe by using pm2. If you want to use something else, figure out how. (More info on why pm2 [here](docs/pm2.md))
+After you finish setting up nginx, you need to start lolisafe by using pm2. If you want to use something else like forever, ensure that the process spawned from `npm run start` never dies.
 
 7. Run `pm2 start pm2.json`:
 8. Profit
 
-### Cloudflare
-
-If you want to run your site through CloudFlare because of the obvious advantages it has, lolisafe has your back. Unless you manually modify the `.env` file, uploads through the website will be uploaded in chunks thus bypassing CloudFlare's 100mb upload limit per file.
-
 ## Author
 
 **lolisafe** © [Pitu](https://github.com/Pitu), Released under the [MIT](https://github.com/WeebDev/lolisafe/blob/master/LICENSE) License.<br>

TODO

@@ -1,5 +1,4 @@
 - There's a vertical scrollbar on seemingly all pages even though there is no need for it
-- Passwords that contain special characters don't seem to work (refer to previous dms)
 
 - Uploaded text file (via ShareX) does not show up in the dashboard's "Files" area
 	* Currently only pictures show up on the dashboard due to having thumbs
@@ -9,3 +8,5 @@
 	- Think of a strategy to achieve this in a nice manner
 
 - Can't delete/rename albums when going into album view. Or ever.
+- Dark theme the annoying popups for deleting and other stuff
+- Logout button

docs/migrating.md

@@ -0,0 +1,17 @@
+### Migrate from v3 to v4
+This version introduces a few breaking changes and updating requires some manual work.
+For starters we recommend cloning the new version somewhere else instead of `git pull` on your v3 version.
+
+- After cloning move your `uploads` folder from the v3 folder to the new v4 folder.
+- Then copy your `database/db` file from your v3 folder to the root of your v4 folder.
+- You then need to run `yarn setup` or `npm start setup` from the v4 folder and finish the setup process.
+- Once that's done you need to manually run `node src/api/databaseMigration.js` from the root folder of v4.
+- This will migrate the v3 database to v4 and regenerate every single thumbnail in webp to save bandwidth.
+- After the migration finishes, the last step is to update your nginx config with the [newly provided script](./nginx.md).
+- Restart nginx with `sudo nginx -s reload`.
+- And lastly start your lolisafe instance with `pm2 start pm2.json`.
+
+### Breaking changes
+- If you are using the lolisafe extension from one of the stores, the new version has been submitted and could take up to a week to get approved. In the meantime you can load the unpacked extension by cloning [this repo](https://github.com/WeebDev/loli-safe-extension).
+- The lolisafe browser extension needs your new token. Instead of pasting your jwt token into it like before, you need to log in to lolisafe, go to your user settings and generate an `API KEY`, which you will use to access the service from 3rd party apps like the browser extension, ShareX, etc.
+- To upload a file to an album directly users used to use the endpoint `/api/upload/${albumId}`. This is no longer the case. To upload directly to an album now it's necessary to pass a header called `albumid` with an integer as the value of the album to which you want to upload the file to.

docs/pm2.md

@@ -1,11 +0,0 @@
-## Setting up PM2 to run lolisafe
-
-The best way to keep the service running in case of crashes or unexpected issues is to attach the process to PM2 and forget about it. This also gives you the ability to dettach the process from your terminal and run it in the background, which is a must since lolisafe now comes in 2 separate processes.
-The recommended way to set it up is to run the commands below, one for the API and the other for the site.
-
-```
-pm2 start npm --name "lolisafe-api" -- run api
-pm2 start npm --name "lolisafe-site" -- run site
-```
-
-All set, if you want to check the logs you can `pm2 logs lolisafe-api` or similar.

package.json

@@ -8,7 +8,6 @@
 		"email": "heyitspitu@gmail.com",
 		"url": "https://github.com/Pitu"
 	},
-	"main": "src/_scripts/start.js",
 	"scripts": {
 		"setup": "node src/setup.js && yarn build && yarn migrate && yarn seed",
 		"build": "nuxt build",
@@ -18,7 +17,7 @@
 		"seed": "yarn knex seed:run",
 		"api": "node src/api/structures/Server",
 		"update": "git pull && yarn install && yarn migrate && yarn build && yarn restart",
-		"restart": "pm2 restart lolisafe-api && pm2 restart lolisafe-website"
+		"restart": "pm2 restart lolisafe"
 	},
 	"repository": {
 		"type": "git",
@@ -28,7 +27,7 @@
 		"url": "https://github.com/WeebDev/lolisafe/issues"
 	},
 	"engines": {
-		"node": ">=8.0.0"
+		"node": ">=12.0.0"
 	},
 	"dependencies": {
 		"@mdi/font": "^5.3.45",

src/api/databaseMigration.js

@@ -3,6 +3,13 @@
 /* eslint-disable no-console */
 const nodePath = require('path');
 const moment = require('moment');
+const jetpack = require('fs-jetpack');
+const { path } = require('fs-jetpack');
+const sharp = require('sharp');
+const ffmpeg = require('fluent-ffmpeg');
+
+const imageExtensions = ['.jpg', '.jpeg', '.bmp', '.gif', '.png', '.webp'];
+const videoExtensions = ['.webm', '.mp4', '.wmv', '.avi', '.mov'];
 
 const oldDb = require('knex')({
 	client: 'sqlite3',
@@ -45,6 +52,10 @@ const start = async () => {
 	console.log('Starting migration, this may take a few minutes...'); // Because I half assed it
 	console.log('Please do NOT kill the process. Wait for it to finish.');
 
+	await jetpack.removeAsync(nodePath.join(__dirname, '..', '..', 'uploads', 'thumbs'));
+	await jetpack.dirAsync(nodePath.join(__dirname, '..', '..', 'uploads', 'thumbs', 'square'));
+	console.log('Finished deleting old thumbnails to create new ones');
+
 	const users = await oldDb.table('users').where('username', '<>', 'root');
 	for (const user of users) {
 		const now = moment.utc().toDate();
@@ -116,6 +127,13 @@ const start = async () => {
 			albumId: file.albumid,
 			fileId: file.id,
 		});
+
+		const filename = file.name;
+		if (!jetpack.exists(nodePath.join(__dirname, '..', '..', 'uploads', filename))) continue;
+		const ext = nodePath.extname(filename).toLowerCase();
+		const output = `${filename.slice(0, -ext.length)}.webp`;
+		if (imageExtensions.includes(ext)) await generateThumbnailForImage(filename, output);
+		if (videoExtensions.includes(ext)) generateThumbnailForVideo(filename);
 	}
 	await newDb.batchInsert('files', filesToInsert, 20);
 	await newDb.batchInsert('albumsFiles', albumsFilesToInsert, 20);
@@ -125,4 +143,45 @@ const start = async () => {
 	process.exit(0);
 };
 
+const generateThumbnailForImage = async (filename, output) => {
+	try {
+		const file = await jetpack.readAsync(nodePath.join(__dirname, '..', '..', 'uploads', filename), 'buffer');
+		await sharp(file)
+			.resize(64, 64)
+			.toFormat('webp')
+			.toFile(nodePath.join(__dirname, '..', '..', 'uploads', 'thumbs', 'square', output));
+		await sharp(file)
+			.resize(225, null)
+			.toFormat('webp')
+			.toFile(nodePath.join(__dirname, '..', '..', 'uploads', 'thumbs', output));
+		console.log('finished', filename);
+	} catch (error) {
+		console.log('error', filename);
+	}
+};
+
+const generateThumbnailForVideo = filename => {
+	try {
+		ffmpeg(nodePath.join(__dirname, '..', '..', 'uploads', filename))
+			.thumbnail({
+				timestamps: [0],
+				filename: '%b.png',
+				folder: nodePath.join(__dirname, '..', '..', 'uploads', 'thumbs', 'square'),
+				size: '64x64'
+			})
+			.on('error', error => console.error(error.message));
+		ffmpeg(nodePath.join(__dirname, '..', '..', 'uploads', filename))
+			.thumbnail({
+				timestamps: [0],
+				filename: '%b.png',
+				folder: nodePath.join(__dirname, '..', '..', 'uploads', 'thumbs'),
+				size: '150x?'
+			})
+			.on('error', error => console.error(error.message));
+		console.log('finished', filename);
+	} catch (error) {
+		console.log('error', filename);
+	}
+};
+
 start();

src/api/routes/service/versionGET.js

@@ -0,0 +1,15 @@
+const Route = require('../../structures/Route');
+
+class versionGET extends Route {
+	constructor() {
+		super('/version', 'get', { bypassAuth: true });
+	}
+
+	run(req, res) {
+		return res.json({
+			version: process.env.npm_package_version
+		});
+	}
+}
+
+module.exports = versionGET;

src/api/routes/uploads/uploadPOST.js

@@ -117,6 +117,7 @@ class uploadPOST extends Route {
 				this.saveFileToAlbum(db, albumId, insertedId);
 			}
 
+			uploadedFile = Util.constructFilePublicLink(uploadedFile);
 			return res.status(201).send({
 				message: 'Sucessfully uploaded the file.',
 				...uploadedFile,
@@ -125,6 +126,7 @@ class uploadPOST extends Route {
 	}
 
 	fileExists(res, exists, filename) {
+		exists = Util.constructFilePublicLink(exists);
 		res.json({
 			message: 'Successfully uploaded the file.',
 			name: exists.name,

src/api/structures/Server.js

@@ -103,9 +103,10 @@ class Server {
 		jetpack.dir('uploads/thumbs/square');
 		this.registerAllTheRoutes();
 		this.serveNuxt();
-		this.server.listen(this.port, () => {
+		const server = this.server.listen(this.port, () => {
 			log.success(`Backend ready and listening on port ${this.port}`);
 		});
+		server.setTimeout(600000);
 	}
 }
 

src/api/utils/Util.js

@@ -175,7 +175,13 @@ class Util {
 		}
 	}
 
-	static isAuthorized(req) {
+	static async isAuthorized(req) {
+		if (req.headers.token) {
+			const user = await db.table('users').where({ apiKey: req.headers.token }).first();
+			if (!user || !user.enabled) return false;
+			return user;
+		}
+
 		if (!req.headers.authorization) return false;
 		const token = req.headers.authorization.split(' ')[1];
 		if (!token) return false;

src/setup.js

@@ -16,14 +16,9 @@ async function start() {
 	const wizard = [
 		{
 			type: 'input',
-			query: 'Port to run the API in:',
+			query: 'Port to run lolisafe in:',
 			handle: 'SERVER_PORT',
 		},
-		{
-			type: 'input',
-			query: 'Port to run the Website in when in dev mode:',
-			handle: 'WEBSITE_PORT',
-		},
 		{
 			type: 'input',
 			query: 'Full domain this instance is gonna be running on (Ex: https://lolisafe.moe):',
@@ -39,13 +34,6 @@ async function start() {
 			query: 'Maximum allowed upload file size in MB (Ex: 100):',
 			handle: 'MAX_SIZE',
 		},
-		{
-			type: 'confirm',
-			query: 'Generate thumbnails for images/videos? (Requires ffmpeg installed and in your PATH)',
-			handle: 'GENERATE_THUMBNAILS',
-			accept: 'y',
-			deny: 'n',
-		},
 		{
 			type: 'confirm',
 			query: 'Allow users to download entire albums in ZIP format?',
@@ -55,31 +43,14 @@ async function start() {
 		},
 		{
 			type: 'confirm',
-			query: 'Serve files with node?',
-			handle: 'SERVE_WITH_NODE',
-			accept: 'y',
-			deny: 'n',
-		},
-		{
-			type: 'input',
-			query: 'Base number of characters for generated file URLs (12 should be good enough):',
-			handle: 'GENERATED_FILENAME_LENGTH',
-		},
-		{
-			type: 'input',
-			query: 'Base number of characters for generated album URLs (6 should be enough):',
-			handle: 'GENERATED_ALBUM_LENGTH',
-		},
-		{
-			type: 'confirm',
-			query: 'Run lolisafe in public mode? (People will be able to upload without an account)',
+			query: 'Allow people to upload files without an account?',
 			handle: 'PUBLIC_MODE',
 			accept: 'y',
 			deny: 'n',
 		},
 		{
 			type: 'confirm',
-			query: 'Enable user signup for new accounts?',
+			query: 'Allow people to create new accounts?',
 			handle: 'USER_ACCOUNTS',
 			accept: 'y',
 			deny: 'n',
@@ -131,6 +102,11 @@ async function start() {
 	let envfile = '';
 
 	const defaultSettings = {
+		GENERATED_FILENAME_LENGTH: 12,
+		GENERATED_ALBUM_LENGTH: 6,
+		WEBSITE_PORT: 5001,
+		SERVE_WITH_NODE: true,
+		GENERATE_THUMBNAILS: true,
 		CHUNK_SIZE: 90,
 		ROUTE_PREFIX: '/api',
 		RATE_LIMIT_WINDOW: 2,
@@ -158,8 +134,6 @@ async function start() {
 	console.log('=============================================');
 	console.log('==    .env file generated successfully.    ==');
 	console.log('=============================================');
-	console.log('== Run `yarn migrate` and `yarn seed` next ==');
-	console.log('=============================================');
 	console.log();
 }
 

src/site/components/grid/Grid.vue

@@ -152,6 +152,12 @@
 					</div>
 				</template>
 			</b-table>
+			<button
+				v-if="moreFiles"
+				class="button is-primary mt2"
+				@click="loadMoreFiles">
+				Load more
+			</button>
 		</div>
 		<b-modal :active.sync="isAlbumsModalActive" scroll="keep">
 			<ImageInfo :file="modalData.file" />

src/site/components/navbar/Navbar.vue

@@ -45,11 +45,12 @@
 					</router-link>
 				</b-navbar-item>
 				<b-navbar-item tag="div">
-					<a
-						class="navbar-item"
-						@click="logOut">
+					<router-link
+						to="/"
+						class="navbar-item no-active"
+						@click.native="logOut">
 						Logout
-					</a>
+					</router-link>
 				</b-navbar-item>
 			</template>
 			<template v-else>