haskal
/
chomp
2
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
chomp/NOTES.md

4.7 KiB
Raw Blame History

support tor and opennic sites reject brave browser

deps

  • crypto-lib
  • markdown
  • sass (submodule, needed modifications)
  • http

Disk structure

/cache/: cached remote objects /db/: generated info, eg pubkeys, followers, /public/: output of static generation. will contain all html files, activitypub json, and media /src/: source files that are used for static generation /taskq.sqlite3: sqlite database for naive persistent task queue stores the activitypub stuff to in a crash-resistant way

Building the static content

  1. checks posts that got posted but that have no corresponding source. deletes and sends out delete activities. if any posts got deleted it could touch a fake flag file that causes the next step to rebuild all indices even if there are no new or updated posts
  2. main pass that builds all the main activitypub objects and html pages and css. queues create or update activities if necessary
  3. second pass based on tag info extracted during previous step, regenerates tag pages and deletes ones that don't exist anymore. tag pages don't have corresponding AP objects so no activities are required here

User object

endpoint: https://my.instance.tld/

required info

  • name
  • preferredUsername
  • icon
  • summary (markdown format)

generated info

  • compiled summary:
    • mentions: class="u-url mention"
    • hashtagges: rel="tag"
  • type: person
  • pubkey (/db/actorkey.pub, privkey: /db/actorkey)
  • inbox/sharedInbox (POST endpoint)
  • outbox (generated based on src posts)
  • followers (/db/followers/)
  • following: null, liked: null, featured: null (for now)
  • contexts: activitystreams, security, as:Hashtag, as:sensitive, as:manuallyApprovesFollwers, as:movedTo, toot, toot:Emoji, toot:focalPoint, toot:featured

RSS

mvp will support rss. with full post contents

Cache

/cache/actors/ stores actor objects fetched from remote instances filename: .json, contents: the full activitypub json for the actor inotify will be used to detect completion of a queued request if a remote request fails after a reasonable retry interval, it may be aborted and the cache may be filled in with dummy data indicating the resource could not be fetched

/cache/posts/ stores intermediate HTML-rendered posts generated from markdown. this avoids needing to recompile posts from markdown multiple times because that's super unnecessary. it also contains front matter with the post metadata and parsed hashtags

/cache/tagindex.rktb stores an index of all hashtags and which posts contain them

Task queue

stores queued outgoing requests TODO

Procedures

  • compile: takes everything in src and db (and cache) and regenerates the contents of public compile may make queued requests and block until they are complete compile should be smart and only regenerate resources as necessary, based on file timestamps if new posts are present, compile must generate output products for the new posts, queue all activitypub outgoing requests, and then save the completion flag if a new post is present, the output product is also present, but the completion flag is not present, re-queue all outgoing requests then save the completion flag if an output product for a post is present, the completion flag is present, but the original post is not present, queue undo requests and then remove the output and flag
  • accept follow: fetch remote user, make all reasonable security checks, and accept follow request. store result in /cache/actors/ and /db/followers/. then call a compile
  • accept comment: fetch remote user, make all reasonable security checks, and save comment. store result in /cache/actors/ and /db/comments/. then call a compile

Followers/following lists

directory-based (/db/followers/) each file is a follower, filename is .json, file contents is the follow object received. corresponding cache entry for the remote actor

initially we can accept all follows. later a mechanism can be added for manual approval

Comments

/db/comments/ stores all comments received on posts filename: .json, contents: the contents of the comment json

Likes

will not be supported

Article object

/db/posted/: stores flags for completion for post generation and queuing of outgoing activitypub notifications. filename

todo

eventually this should support mathjax and highlighted code and stuff mathjax should be rendered as the mozilla native math thing with a javascript polyfill for gogle chrombe and others rss/atom gopher gemini

nginx caching - https://blog.theodo.com/2016/06/improve-the-performance-of-your-webapp-configure-nginx-to-cache/