crossfire

distributed brute force infrastructure

takes the difficulty out of creating custom brute force jobs

getting started with the extremely basic but currently working MVP

cd to the repo directory. hopefully you're already here :P

add user-level racket dir to your PATH

export PATH=$HOME/.racket/$(racket -e "(displayln (version))")/bin:$PATH

install package from repo

raco pkg install ./crossfire

setup development environment

export DATABASE_URL="sqlite:lib/crossfire.sqlite"
mkdir -p lib/projects

initialize server with some temporary dev commands (this assumes your computer is x86_64 linux)
the arch corresponds to the output of gcc -dumpmachine for the target

crossfire-server dev-new-agent meow x86_64-pc-linux-gnu  # exports lib/meow.rktd
crossfire-server dev-export-client  # exports lib/client0.rktd

start the server and a development version of the agent (WIP) using the exported agent config

crossfire-server
# somewhere else
racket crossfire/agent.rkt -c lib/meow.rktd

set up the client with access to the server using the previous client config

crossfire-client setup lib/client0.rktd

create a new project

crossfire-client new meow2
cd meow2

edit config manifest.rktd

• change (mode stdio) to (mode callback) (stdio mode isn't working yet)
• change (command ...) to (command "./meow")

generate files (Makefile, crossfire.c, crossfire.h -- these are support code for running your code on agents. in callback mode, your code registers a callback with crossfire that gets called for each possible input, and returns true or false)

crossfire-client generate

create a new source file meow.c with the following

#include "crossfire.h"

bool my_callback( vartype a,vartype b,vartype c,vartype d,vartype e,vartype f) {
    return e == '4' && f == '2';
}

int main(int argc, char* argv[]) {
    return crossfire_main(argc, argv, my_callback);
}

update the makefile

.PHONY: all clean

CC=gcc
CFLAGS=-std=c11 -O3
LDFLAGS=-static

all: meow

meow: meow.o crossfire.o

clean:
	$(RM) meow meow.o


# crossfire rules

### leave this section alone

build project

make

now we're ready to submit

crossfire-client submit

check status of projects

crossfire-client status

this one completes pretty fast, so you should see 100% progress and one match
let's show the match (assuming the project ID from the status command is 1)

crossfire-client show 1

aaaaand yeah that's all for now. smp is not supported yet. stdio mode is also not supported yet. lots of stuff still probably broken tbh
contributions welcome,,,,

status

base

• 🚧 input space manipulation functions
  • ✅ data types: using data/integer-set, pattern (vector of integer-set)
  • ✅ basic manipulation functions
  • ✅ representation of input space as a flat integer
• 🚧 #lang for configuration/definitions
  • (input) mode
    • stdio: user program gets input by stdio, integers separated by space, one per line
    • callback: input generator compiled into user program, user main calls crossfire_main with callback function that returns true or false
    • other modes??
  • SMP: performed by crossfire or performed by the user code
    • "performed by user code" can also mean GPU, for example
• 🚧 codegen for input generator (in C)
  • ✅ stdio mode
  • 🚧 callback mode
  • ✅ success reporting mechanism
  • low priority: configurable "character" type -- currently a "character" is a uint64_t

server: distribute jobs to workers

• ✅ base definitions of input classes and how to divide them
• ✅ dynamic slicing and scheduling based on agents' reported work rate
• low priority: randomized input space distribution
• low priority: store common configuration templates for clients
• low priority: track upload/download progress
• streaming interface for file transfers
• ✅ accept submitted projects (with client-compiled input generator) and distribute to agents
  • ✅ low priority: support for multiple architectures
• ✅ agent authentication
• ✅ client authentication

agent: accept and run jobs

• ✅ securely connect to server
• ✅ retrieve assigned tasks
• handle smp correctly
• ✅ report completions
• 🚧 report errors
• ✅ report successes
• low priority: defer to external brute force program (eg, hashcat on GPU)
  • this could be implemented on top of the existing project format
• low priority: support finding all matching inputs for a project, rather than just the first one
  • the architecture currently doesn't stop on the first match so it could be a thing

client: submit jobs and view progress

• ✅securely connect to server
• command line interface
  • ✅ crossfire new: create new crossfire project
  • ✅ crossfire generate: updates autogenerated files in a project
  • ✅ crossfire check: checks some common misconfiguration issues
  • crossfire test: test project locally, replicates configuration of server with single local agent to debug issues
  • low priority: crossfire node-test: submit a mini-task to a node that has the necessary resources to debug issues
  • ✅ crossfire submit: submit task to server
  • ✅ crossfire delete: cancels/deletes submitted task
  • ✅ crossfire status: check status of server / task summary
  • ✅ crossfire show: shows task details
  • ✅ crossfire setup: sets up access to a server
  • ✅ crossfire agent: manage agents
• low priority: gui interface (racket/gui & framework time)

misc

porting

currently only linux is supported for the server and agent. the server might run on macOS but it's not guaranteed. if you are interested in porting this to a new platform, take a look at agent-deployment for the embedded build of racket for the all-in-one agent binary, and perhaps create additional makefiles for other platforms. additionally, platform-specific racket code is marked with an XXX comment