crossfire/README.md

<!--
crossfire: distributed brute force infrastructure

Copyright (C) 2020 haskal

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License
along with this program.  If not, see <https://www.gnu.org/licenses/>.
-->
# crossfire

distributed brute force infrastructure

takes the difficulty out of creating custom brute force jobs

## getting started with the extremely basic but currently working MVP

`cd` to the repo directory. hopefully you're already here :P

add user-level racket dir to your PATH
```bash
export PATH=$HOME/.racket/$(racket -e "(displayln (version))")/bin:$PATH
```

install package from repo
```bash
raco pkg install ./crossfire
```

setup development environment
```bash
export DATABASE_URL="sqlite:lib/crossfire.sqlite"
mkdir -p lib/projects
```

initialize server with some temporary dev commands (this assumes your computer is x86_64 linux)  
the arch corresponds to the output of `gcc -dumpmachine` for the target
```bash
crossfire-server dev-new-agent meow x86_64-pc-linux-gnu  # exports lib/meow.rktd
crossfire-server dev-export-client  # exports lib/client0.rktd
```

start the server and a development version of the agent (WIP) using the exported agent config
```bash
crossfire-server
# somewhere else
racket crossfire/agent.rkt -c lib/meow.rktd
```

set up the client with access to the server using the previous client config
```bash
crossfire-client setup lib/client0.rktd
```

create a new project
```bash
crossfire-client new meow2
cd meow2
```

edit config `manifest.rktd`
 - change `(mode stdio)` to `(mode callback)` (stdio mode isn't working yet)
 - change `(command ...)` to `(command "./meow")`

generate files (Makefile, crossfire.c, crossfire.h -- these are support code for running your code
on agents. in callback mode, your code registers a callback with crossfire that gets called for each
possible input, and returns true or false)
```
crossfire-client generate
```

create a new source file `meow.c` with the following

```c
#include "crossfire.h"

bool my_callback( vartype a,vartype b,vartype c,vartype d,vartype e,vartype f) {
    return e == '4' && f == '2';
}

int main(int argc, char* argv[]) {
    return crossfire_main(argc, argv, my_callback);
}
```

update the makefile
```make
.PHONY: all clean

CC=gcc
CFLAGS=-std=c11 -O3
LDFLAGS=-static

all: meow

meow: meow.o crossfire.o

clean:
	$(RM) meow meow.o


# crossfire rules

### leave this section alone
```

build project
```bash
make
```

now we're ready to submit
```bash
crossfire-client submit
```

check status of projects
```bash
crossfire-client status
```

this one completes pretty fast, so you should see 100% progress and one match  
let's show the match (assuming the project ID from the status command is `1`)
```bash
crossfire-client show 1
```

aaaaand yeah that's all for now. smp is not supported yet. stdio mode is also not supported yet.
lots of stuff still probably broken tbh  
contributions welcome,,,,

## status

### base
- 🚧 input space manipulation functions
    - ✅ data types: using data/integer-set, pattern (vector of integer-set)
    - ✅ basic manipulation functions
    - ✅ representation of input space as a flat integer
- 🚧 #lang for configuration/definitions
    - (input) mode
        - stdio: user program gets input by stdio, integers separated by space, one per line
        - callback: input generator compiled into user program, user main calls `crossfire_main`
            with callback function that returns true or false
        - other modes??
    - SMP: performed by crossfire or performed by the user code
        - "performed by user code" can also mean GPU, for example
- 🚧 codegen for input generator (in C)
    - ✅ stdio mode
    - 🚧 callback mode
    - ✅ success reporting mechanism
    - low priority: configurable "character" type -- currently a "character" is a uint64\_t

### server: distribute jobs to workers
- ✅ base definitions of input classes and how to divide them
- ✅ dynamic slicing and scheduling based on agents' reported work rate
- low priority: randomized input space distribution
- low priority: store common configuration templates for clients
- low priority: track upload/download progress
- streaming interface for file transfers
- ✅ accept submitted projects (with client-compiled input generator) and distribute to agents
    - ✅ low priority: support for multiple architectures
- ✅ agent authentication
- ✅ client authentication

### agent: accept and run jobs
- ✅ securely connect to server
- ✅ retrieve assigned tasks
- handle smp correctly
- ✅ report completions
- 🚧 report errors
- ✅ report successes
- low priority: defer to external brute force program (eg, hashcat on GPU)
    - this could be implemented on top of the existing project format
- low priority: support finding _all_ matching inputs for a project, rather than just the first one
    - the architecture currently doesn't stop on the first match so it could be a thing

### client: submit jobs and view progress
- ✅securely connect to server
- command line interface
    - ✅ `crossfire new`: create new crossfire project
    - ✅ `crossfire generate`: updates autogenerated files in a project
    - ✅ `crossfire check`: checks some common misconfiguration issues
    - `crossfire test`: test project locally, replicates configuration of server with single local
        agent to debug issues
    - low priority: `crossfire node-test`: submit a mini-task to a node that has the necessary
        resources to debug issues
    - ✅ `crossfire submit`: submit task to server
    - ✅ `crossfire delete`: cancels/deletes submitted task
    - ✅ `crossfire status`: check status of server / task summary
    - ✅ `crossfire show`: shows task details
    - ✅ `crossfire setup`: sets up access to a server
- low priority: gui interface (racket/gui & framework time)

## misc

### porting

currently only linux is supported for the server and agent. the server might run on macOS but it's
not guaranteed. if you are interested in porting this to a new platform, take a look at
`agent-deployment` for the embedded build of racket for the all-in-one agent binary, and perhaps
create additional makefiles for other platforms. additionally, platform-specific racket code is
marked with an `XXX` comment