Create init script

NOTES.md

@@ -1,22 +1,90 @@
+## Disk structure
+
+/cache/: cached remote objects
+/db/: generated info, eg pubkeys, followers,
+/public/: output of static generation. will contain all html files, activitypub json, and media
+/src/: source files that are used for static generation
+/taskq.sqlite3: sqlite database for naive persistent task queue
+    stores the activitypub stuff to in a crash-resistant way
+
 ## User object
 
 endpoint: https://my.instance.tld/
 
 required info
 
-- username
+- name
+- preferredUsername
 - icon
-- summary
+- summary (markdown format)
-    - mentions: class="u-url mention"
-    - hashtagges: rel="tag"
 
 generated info
+- compiled summary:
+    - mentions: class="u-url mention"
+    - hashtagges: rel="tag"
 - type: person
-- name, preferredUsername
+- pubkey (/db/actorkey.pub, privkey: /db/actorkey)
-- pubkey
+- inbox/sharedInbox (POST endpoint)
-- inbox/sharedInbox
+- outbox (generated based on src posts)
-- outbox
+- followers (/db/followers/)
-- followers, following
+- following: null, liked: null, featured: null (for now)
-- liked: null, featured: null
 - contexts: activitystreams, security, as:Hashtag, as:sensitive, as:manuallyApprovesFollwers,
     as:movedTo, toot, toot:Emoji, toot:focalPoint, toot:featured
+
+### RSS
+
+mvp will support rss. with full post contents
+
+## Cache
+
+/cache/actors/ stores actor objects fetched from remote instances
+filename: <sha256 of actor id>.json, contents: the full activitypub json for the actor
+inotify will be used to detect completion of a queued request
+if a remote request fails after a reasonable retry interval, it may be aborted and the cache may
+be filled in with dummy data indicating the resource could not be fetched
+
+## Task queue
+
+stores queued outgoing requests
+TODO
+
+## Procedures
+
+- compile: takes everything in src and db (and cache) and regenerates the contents of public
+           compile may make queued requests and block until they are complete
+           compile should be smart and only regenerate resources as necessary, based on file
+           timestamps
+           if new posts are present, compile must generate output products for the new posts,
+           queue all activitypub outgoing requests, and then save the completion flag
+           if a new post is present, the output product is also present, but the completion flag is
+           not present, re-queue all outgoing requests then save the completion flag
+           if an output product for a post is present, the completion flag is present, but the
+           original post is not present, queue undo requests and then remove the output and flag
+- accept follow: fetch remote user, make all reasonable security checks, and accept follow request.
+    store result in /cache/actors/ and /db/followers/. then call a compile
+- accept comment: fetch remote user, make all reasonable security checks, and save comment. store
+    result in /cache/actors/ and /db/comments/. then call a compile
+
+## Followers/following lists
+
+directory-based (/db/followers/)
+each file is a follower, filename is <sha256 of actor id>.json, file contents is the
+follow object received. corresponding cache entry for the remote actor
+
+initially we can accept all follows. later a mechanism can be added for manual approval
+
+## Comments
+
+/db/comments/ stores all comments received on posts
+filename: <sha256 of post id>.json, contents: the contents of the comment json
+
+## Likes
+
+will not be supported
+
+## Article object
+
+/db/posted/: stores flags for completion for post generation and queuing of outgoing activitypub
+notifications. filename <sha256 of post id>
+
+todo

scripts/init

@@ -0,0 +1,47 @@
+#!/usr/bin/env racket
+#lang racket
+; vim: ft=racket
+
+(require db
+         crypto
+         crypto/libcrypto
+         net/base64)
+
+; cache dirs
+(make-directory "cache")
+(make-directory "cache/actors")
+
+; db dirs
+(make-directory "db")
+(make-directory "db/followers")
+(make-directory "db/comments")
+(make-directory "db/posted")
+
+; actor keys
+(define PK-BEGIN "-----BEGIN PUBLIC KEY-----")
+(define PK-END "-----END PUBLIC KEY-----")
+(crypto-factories (list libcrypto-factory))
+(define actor-key (generate-private-key 'rsa '((nbits 4096))))
+(with-output-to-file
+  "db/actorkey"
+  (lambda ()
+    (write-bytes (pk-key->datum actor-key 'PrivateKeyInfo))
+    (void)))
+(with-output-to-file
+  "db/actorkey.pub"
+  (lambda ()
+    (write-bytes (string->bytes/utf-8 PK-BEGIN))
+    (write-bytes #"\n")
+    (define pem (pk-key->datum actor-key 'SubjectPublicKeyInfo))
+    (write-bytes (base64-encode pem))
+    (write-bytes (string->bytes/utf-8 PK-END))
+    (void)))
+
+; src and public
+(make-directory "src")
+(make-directory "public")
+
+; task queue
+(define c (sqlite3-connect #:database "taskq.sqlite3" #:mode 'create))
+(query-exec c "create table taskq (id blob(16) primary key, task blob, state integer)")
+(disconnect c)