Add a search utility

Add a search utility
Implement a dark theme
Add an FAQ page with some info
This commit is contained in:
Robin Malley 2020-12-28 04:01:07 +00:00
parent 8a60675236
commit 3a22c15dcd
12 changed files with 318 additions and 48 deletions

86
assets/faq.html Normal file
View File

@ -0,0 +1,86 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>&#x1f351;</title>
<link href="/_css/milligram.css" rel="stylesheet">
<link href="/_css/style.css" rel="stylesheet">
</head>
<body class="container">
<main class="wrapper">
<h1>FAQ</h1>
An attempt to answer frequently asked questions, and document
the detail of site features. As with all software documentation,
this is how the developer <b>thinks</b> these things work,
to find out how things actually work, see the source code.
<h2>How do a register an account?</h2>
You can go <a href="/_claim">here</a> to register an account,
although you can also post pastes and comments anonymously.
<h2>Parsers? What are those?</h2>
Parsers are the way you can style the text you post. While
this software started as a pastebin clone, the need to style
text better than pastebin.com quickly became apparent. Two
parsers are currently available:
<h3>The plain parser</h3>
<p>The goal of the plain parser is to format plain text exactly as you would expect it. It does not try to to anything fancy and tries to be faithful to the plaintext representation.
<h3>Imageboard parser</h3>
<p>The imageboard parser delivers an approximation of Infinity markup, and will be familiar to those who have used imageboards before. The imageboard parser supports the following markup:
<p>Surround text with double single-quotes(') to make text <i>italic</i>
<p>Surround text with triple single-quotes to make text <b>bold</b>
<p>Surround text with underscores(_) to make it <u>underlined</u>
<p>Surround text with double asterisks(*) to make it <span class="spoiler">spoilered</span>
<p>Surround text with tildes(~) to make it <s>strike through</s>
<p>Begin a line with a greater-than followed by a space to make it
<p><span class="greentext">&gt;greentext</span>
<p>Begin a line with a less-than followed by a space to make it
<p><span class="pinktext">&lt;pinktext</span>
<p>Surround text with forum-style [spoiler] and [/spoiler] tags as a second way to <span class="spoiler2">spoiler</span>
<p>Surround text with forum-style [code] and [/code] tags to make it
<pre><code>
Preformatted and monospace
</code></pre>
<p>If you have incomplete markup at the end, it shouldn't break anything, let me know if it does.
<h2>How does search work?</h2>
<p>The search utility searches for stories on the site.
At it's most simple, it searches stories based on tags,
but it can also filter stories based on: title, author,
date, and hits. In general, the syntax for search is
<code>+|-&lt;field&gt;&gt;operator&lt;&gt;value&lt;</code>
<p>The first <code>+</code> or <code>-</code> specifies
weather to include or exclude results based on this
search, the <code>&lt;field&gt;</code> specifies what
field to search for (or search based on tag if this is
missing), and <code>&lt;operator&gt;</code> specifies
how to search.
<p>For title, and author, the only allowed operator is
<code>=</code>. This operator will search for the text
appearing anywhere in the field, case insensitive. For
hits and time, the allowed operators are
<code>&gt;</code>,<code>&lt;</code>,<code>&gt;=</code>,
<code>&lt;=</code>,<code>=</code>, which searches for
greater than, less than, greater than or equal to, less
than or equal to, and strictly equal to, respectively.
Tag does not need a field or operator, and only allows
exact matches. As a quirk of this system, it is
impossible to search for the tags "author", "title",
"hits" or "date",
Examples:
<pre><code>+author=admin -meta</code></pre>
Will return all stories by the users "admin" and "badminton_enthusiast" that do not include the "meta" tag.
<pre><code>+hits&gt;20 -date&lt;=1609459201</code></pre>
Will return all stories with more than 20 hits that were posted before January 1, 2021 (unix timestamp 1609459201).<br/>
While the date field is a little hard to use for humans, it may be useful for robots.
<h2>How do I enable reader mode on my story?</h2>
Unfortunately, there is no web standard about how reader
modes for different browsers work, and no way to hint to
the browser that a page is readable. The site makes a
best effort to trigger browsers into thinking a post is
readable, but this is unreliable. In general, if reader mode
is broken you just need to make your post a little longer.
</main>
</body>
</html>

View File

@ -20,7 +20,19 @@ p,.tag-list{margin-bottom:0px}
padding: 0 1em 0 1em;
margin: 0 1px 0 1px;
}
.search{
display:flex !important;
flex:0.5 0.5 auto !important;
}
.search>.button{
flex:10 10 auto;
translate: -100%;
}
.column-0{margin-right:5px;}
@media (prefers-color-scheme: dark){
@import "css/style_dark.css";
body, form>*{
background: #1c1428;
color: #d0d4d8 !important;
}
}

View File

@ -36,6 +36,7 @@ domain * {
route /_css/milligram.css asset_serve_milligram_css
route /_css/milligram.min.css.map asset_serve_milligram_min_css_map
route /_css/style_dark.css asset_serve_style_dark_css
route /_faq asset_serve_faq_html
route /favicon.ico asset_serve_favicon_ico
route /_paste post_story
route /_edit edit_story
@ -77,7 +78,7 @@ domain * {
validate tags v_any
}
params get /_search {
validate tag v_any
validate q v_any
}
params get ^/[^_].* {
validate comments v_bool

View File

@ -25,7 +25,7 @@ local function download_get(req)
local err = util.do_sql(stmnt_download)
if err == sql.DONE then
--No rows, story not found
http_responose(req,404,pages.nostory{path=story})
http_response(req,404,pages.nostory{path=story})
stmnt_download:reset()
return
end

View File

@ -52,21 +52,17 @@ local function get_author_home(req)
stmnt_author_bio:bind_names{author=subdomain}
local err = util.do_sql(stmnt_author_bio)
if err == sql.DONE then
print("No such author")
stmnt_author_bio:reset()
return pages.noauthor{
author = subdomain
}
end
print("err:",err)
assert(err == sql.ROW,"failed to get author:" .. subdomain .. " error:" .. tostring(err))
local data = stmnt_author_bio:get_values()
local bio = data[1]
stmnt_author_bio:reset()
print("Getting author's stories")
stmnt_author:bind_names{author=subdomain}
err = util.do_sql(stmnt_author)
print("err:",err)
local stories = {}
while err == sql.ROW do
local data = stmnt_author:get_values()

View File

@ -6,6 +6,7 @@ local util = require("util")
local libtags = require("tags")
local pages = require("pages")
local config = require("config")
local search_parser = require("parser_search")
local stmnt_search
local oldconfigure = configure
@ -18,42 +19,40 @@ local function search_get(req)
local host = http_request_get_host(req)
local path = http_request_get_path(req)
http_request_populate_qs(req)
local tag = http_argument_get_string(req,"tag")
if tag then
stmnt_search:bind_names{
tag = tag
}
local results = {}
local err
repeat
err = stmnt_search:step()
if err == sql.BUSY then
coroutine.yield()
elseif err == sql.ROW then
local id, title, anon, time, author = unpack(stmnt_search:get_values())
local idp = util.encode_id(id)
local tags = libtags.get(id)
table.insert(results,{
id = idp,
title = title,
anon = anon,
time = os.date("%B %d %Y",tonumber(time)),
author = author,
tags = tags
})
elseif err == sql.DONE then
stmnt_search:reset()
else
error("Failed to search, sql error:" .. tostring(err))
local searchq = assert(http_argument_get_string(req,"q"))
log(LOG_DEBUG,string.format("search: %q",searchq))
local sqltxt, data = search_parser(searchq)
local stmnt = assert(db.conn:prepare(sqltxt), db.conn:errmsg())
local i = 1
for field,values in pairs(data) do
if field ~= "tags" then
for _, value in pairs(values) do
stmnt:bind(i,value[3])
i = i + 1
end
end
end
for _,value in pairs(data.tags) do
stmnt:bind(i,value[3])
i = i + 1
end
local results = {}
for row in stmnt:rows() do
table.insert(results,{
id = util.encode_id(row[1]),
title = row[2],
isanon = row[3] == 1,
author = row[4],
time = os.date("%B %d %Y",tonumber(row[5])),
tags = libtags.get(row[1])
})
end
until err == sql.DONE
local ret = pages.search{
domain = config.domain,
results = results,
tag = tag,
q = searchq,
}
http_response(req,200,ret)
end
end
return search_get

View File

@ -1,10 +1,84 @@
local lpeg = require('lpeg')
local etlua = require('etlua')
local args = {...}
lpeg.locale(lpeg)
local V,P,C,S,B,Cs = lpeg.V,lpeg.P,lpeg.C,lpeg.S,lpeg.B,lpeg.Cs
--Identity function
local ident = function(a) return a end
--Lowercase and capitalize first letter
local function capitalize(str)
return string.lower(str):gsub("^(.)",string.upper)
end
--Trim whitespace
local function trim(str)
return str:match("^%s*(.-)%s*$")
end
--SQL like match anywhere
local function like(str)
return "%" .. str .. "%"
end
--Tags are always trimed of whitepsace, lowercase with first letter capitalized
local function tag_fmt(str)
return capitalize(trim(str))
end
--Title match
local function title_fmt(str)
return like(trim(str))
end
--Author names are all lowercase alphanumeric, max 30 characters
local function author_fmt(str)
return like(string.lower(trim(str)))
end
local fieldnames = {
title = {name="title",type="string",fmt=title_fmt},
author = {name="author",type="string",fmt=author_fmt},
date = {name="date",type="number",fmt=tonumber},
hits = {name="hits",type="number",fmt=tonumber},
tags = {name="tag",type="string",fmt=tag_fmt},
}
local field_default = "tag"
local fields
local grammar = P{
"chunk";
whitespace = S" \t\n"^0,
itm = C(P(1-S"+-")^0), --go until the next '+' or '-'
likefield = C(P"title" + P"author") * V"whitespace" * C(P"=") * V"whitespace" * V"itm",
rangeop = P"<=" + P">=" + P">" + P"<" + P"=",
rangefield = C(P"date" + P"hits") * V"whitespace" * C(V"rangeop") * V"whitespace" * C(V"itm"),
field = C(S"+-") * (V"likefield" + V"rangefield" + V"itm") / function(pn,field,expr,value)
if expr and value then
fields[field] = fields[field] or {}
table.insert(fields[field],{pn,expr,value})
else
fields.tags = fields.tags or {}
table.insert(fields.tags,{pn,"=",field})
end
end,
chunk = V"field"^0
}
--Grammar
--Transpile a sting with + and - into an sql query that searches tags
local fname = "pages/search_sql.etlua"
local sqltmpl = assert(io.open(fname))
local c = etlua.compile(sqltmpl:read("*a"),fname)
sqltmpl:close()
local function transpile(str)
for chunk in str:gmatch("([+-])([^+-])") do
print("found chunk:",chunk)
str = string.lower(str)
fields = {tags={}}
table.concat({grammar:match(str)}," ")
--Sanity perform formatting on data
for field,values in pairs(fields) do
for _,value in pairs(values) do
local pn, expr, val = unpack(value)
local nval = fieldnames[field].fmt(val)
value[3] = nval
end
end
local ressql = c{
result = fields
}
return ressql, fields
end
return transpile

44
src/lua/search_sql.etlua Normal file
View File

@ -0,0 +1,44 @@
SELECT
posts.id,
posts.post_title,
authors.name,
posts.post_time
FROM
posts,authors
WHERE
authors.id = posts.authorid
<% print("reqult:",result) -%>
<% for field, values in pairs(result) do -%>
<% print("values:",values) -%>
<% for _,value in pairs(values) do -%>
<% local pn,expr,value = unpack(value) -%>
<% local n = (pn == "+" and "" or "NOT") -%>
<% if field == "title" then -%>
AND <%= n %> posts.post_title LIKE ?
<% elseif field == "author" then -%>
AND <%= n %> authors.name LIKE ?
<% elseif field == "date" then -%>
AND <%= n %> posts.post_time <%- expr %> ?
<% elseif field == "hits" then -%>
AND posts.views <%- expr -%> ?
<% end -%>
<% end -%>
<% end -%>
<% print("tags:",result.tags) %>
<% for _,tag in pairs(result.tags) do -%>
INTERSECT
SELECT
posts.id,
posts.post_title,
authors.name,
posts.post_time
FROM
posts,authors,tags
WHERE
posts.authorid = authors.id
AND tags.postid = posts.id
<% local n,v = unpack(tag) -%>
<% n = (n == "-" and "NOT" or "") -%>
AND <%= n %> tags.tag = <%= v %>
<% end -%>
;

View File

@ -45,7 +45,7 @@ function util.sqlbind(stmnt,call,position,data)
assert(call == "bind" or call == "bind_blob","Bad bind call, call was:" .. call)
local f = stmnt[call](stmnt,position,data)
if f ~= sql.OK then
error(string.format("Failed to %s at %d with %q: %s", call, position, data, db:errmsg()),2)
error(string.format("Failed to %s at %d with %q: %s", call, position, data, db.conn:errmsg()),2)
end
end

View File

@ -15,9 +15,15 @@
</h1>
<div class="container">
<a href="/_paste" class="button">New paste</a>
<a href="/_login" class="button">Log in</a>
<a href="/_claim" class="button">Register</a>
<div class="row">
<a href="/_paste" class="button column column-0">New paste</a>
<a href="/_login" class="button column column-0">Log in</a>
<a href="/_claim" class="button column column-0">Register</a>
<form action="https://<%= domain %>/_search" method="get" class="search column row">
<input class="column" type="text" name="q" placeholder="+greentext -dotr +hits>20"/>
<input class="column column-0 button button-clear" type="submit" value="&#x1F50E;"/>
</form>
</div>
<p>
Welcome to slash.monster, stories of fiction and fantasy<br/>
Not safe for work<br/>
@ -43,7 +49,7 @@
</td><td>
<ul class="row tag-list">
<% for i = 1,math.min(#v.tags, 5) do %>
<li><a class="tag button button-outline" href="https://<%= domain %>/_search?tag=<%= v.tags[i] %>"><%= v.tags[i] %></a></li>
<li><a class="tag button button-outline" href="https://<%= domain %>/_search?q=+<%= v.tags[i] %>"><%= v.tags[i] %></a></li>
<% end %>
<% if #v.tags > 5 then %>
<li>+<%= #v.tags - 5 %></li>

View File

@ -11,6 +11,12 @@
<h1 class="title">
<a href="https://<%= domain %>"><%= domain %></a>/
</h1>
<div class="row">
<form action="https://<%= domain %>/_search" method="get" class="search column row">
<input class="column" type="text" name="q" placeholder="+greentext -dotr +hits>20" value="<%= q %>"/>
<input class="column column-0 button button-clear" type="submit" value="&#x1F50E;"/>
</form>
</div>
<div class="content">
<% if #results == 0 then %>
No stories matched your search.
@ -32,6 +38,9 @@
<% for i = 1,math.min(#v.tags, 5) do %>
<li><a class="tag button button-outline" href="https://<%= domain %>/_search?tag=<%= v.tags[i] %>"><%= v.tags[i] %></a></li>
<% end %>
<% if #v.tags > 5 then %>
<li>+<%= #v.tags - 5%></li>
<% end %>
</ul>
</td><td>
<%= v.time %>

View File

@ -0,0 +1,43 @@
SELECT
posts.id,
posts.post_title,
posts.isanon,
authors.name,
posts.post_time
FROM
posts,authors
WHERE
authors.id = posts.authorid
<% for field, values in pairs(result) do -%>
<% for _,value in pairs(values) do -%>
<% local pn,expr,value = unpack(value) -%>
<% local n = (pn == "+" and "" or "NOT") -%>
<% if field == "title" then -%>
AND <%= n %> posts.post_title LIKE ?
<% elseif field == "author" then -%>
AND <%= n %> authors.name LIKE ?
<% elseif field == "date" then -%>
AND <%= n %> posts.post_time <%- expr %> ?
<% elseif field == "hits" then -%>
AND posts.views <%- expr -%> ?
<% end -%>
<% end -%>
<% end -%>
<% for _,tag in pairs(result.tags) do -%>
INTERSECT
SELECT
posts.id,
posts.post_title,
posts.isanon,
authors.name,
posts.post_time
FROM
posts,authors,tags
WHERE
posts.authorid = authors.id
AND tags.postid = posts.id
<% local n,v,t = unpack(tag) -%>
<% n = (n == "-" and "NOT" or "") -%>
AND <%= n %> tags.tag = ?
<% end -%>
;