crossfire

distributed brute force infrastructure

takes the difficulty out of creating custom brute force jobs

getting started with the extremely basic but currently working MVP

cd to the repo directory. hopefully you're already here :P

add user-level racket dir to your PATH

export PATH=$HOME/.racket/$(racket -e "(displayln (version))")/bin:$PATH
export CROSSFIRE_DEVEL=1

install package from repo, and build the agent binary (and place it into the development state dir)

raco pkg install ./crossfire
mkdir -p lib/projects
make dev-make-agent
# this step could take a while, and needs an internet connection to download some additional sources

initialize server with some temporary dev commands

crossfire-server dev-export-client  # exports lib/client0.rktd

start the server

crossfire-server

set up the client with access to the server using the previous client config

crossfire-client setup lib/client0.rktd

register and start up one or more agents (this assumes your arch is x86_64-pc-linux-gnu, it's roughly the output of gcc -dumpmachine -- the arch is used to select an agent binary that was previously built)

crossfire-client agent -c -a x86_64-pc-linux-gnu -n myagent -r cpu
# this will output that it saved a binary to a certain file
# run the binary to start up the agent

check the status of agents

crossfire-client agent -l

create a new project

crossfire-client new meow2
cd meow2

edit config manifest.rktd

• change (mode stdio) to (mode callback)
• change (command ...) to (command "./meow")

generate files (Makefile, crossfire.c, crossfire.h -- these are support code for running your code on agents. in callback mode, your code registers a callback with crossfire that gets called for each possible input, and returns true or false)

crossfire-client generate

create a new source file meow.c with the following

#include "crossfire.h"

bool my_callback( vartype a,vartype b,vartype c,vartype d,vartype e,vartype f) {
    return e == '4' && f == '2';
}

int main(int argc, char* argv[]) {
    return crossfire_main(argc, argv, my_callback);
}

update the makefile

.PHONY: all clean

CC=gcc
CFLAGS=-std=c11 -O3
LDFLAGS=-static

all: meow

meow: meow.o crossfire.o

clean:
	$(RM) meow meow.o


# crossfire rules

### leave this section alone

build project

make

now we're ready to submit

crossfire-client submit

check status of projects

crossfire-client status

this one completes pretty fast, so you should see 100% progress and one match
let's show the match (assuming the project ID from the status command is 1)

crossfire-client show 1

aaaaand yeah that's all for now. lots of stuff still probably broken tbh
contributions welcome,,,,

status

base

• ✅ input space manipulation functions
  • ✅ data types: using data/integer-set, pattern (vector of integer-set)
  • ✅ basic manipulation functions
  • ✅ representation of input space as a flat integer
• ✅ #lang for configuration/definitions
  • ✅ (input) mode
    • ✅ stdio: user program gets input by stdio, integers separated by space, one per line
    • ✅ callback: input generator compiled into user program, user main calls crossfire_main with callback function that returns true or false
    • other modes??
  • ✅ SMP: performed by crossfire or performed by the user code
    • "performed by user code" can also mean GPU, for example
• ✅ codegen for input generator (in C)
  • ✅ stdio mode
  • ✅ callback mode
  • ✅ success reporting mechanism
  • low priority: configurable "character" type -- currently a "character" is a uint64_t

server: distribute jobs to workers

• ✅ base definitions of input classes and how to divide them
• ✅ dynamic slicing and scheduling based on agents' reported work rate
• low priority: randomized input space distribution
• low priority: store common configuration templates for clients
• ✅ low priority: track upload/download progress
• ✅ streaming interface for file transfers
• ✅ accept submitted projects (with client-compiled input generator) and distribute to agents
  • ✅ low priority: support for multiple architectures
• ✅ agent authentication
• ✅ client authentication

agent: accept and run jobs

• ✅ securely connect to server
• ✅ retrieve assigned tasks
• ✅ handle smp correctly
• ✅ report completions
• 🚧 report errors
  • errors are reported but detailed output is not currently uploaded
• ✅ report successes
• low priority: defer to external brute force program (eg, hashcat on GPU)
  • this could be implemented on top of the existing project format
• ✅ low priority: support finding all matching inputs for a project, rather than just the first one

client: submit jobs and view progress

• ✅ securely connect to server
• command line interface
  • ✅ crossfire new: create new crossfire project
  • ✅ crossfire generate: updates autogenerated files in a project
  • ✅ crossfire check: checks some common misconfiguration issues
  • crossfire test: test project locally, replicates configuration of server with single local agent to debug issues
  • low priority: crossfire node-test: submit a mini-task to a node that has the necessary resources to debug issues
  • ✅ crossfire submit: submit task to server
  • ✅ crossfire delete: cancels/deletes submitted task
  • ✅ crossfire status: check status of server / task summary
  • ✅ crossfire show: shows task details
  • ✅ crossfire setup: sets up access to a server
  • ✅ crossfire agent: manage agents
• low priority: gui interface (racket/gui & framework time)

misc

porting

currently only linux is supported for the server and agent. the server might run on macOS but it's not guaranteed. if you are interested in porting this to a new platform, take a look at agent-deployment for the embedded build of racket for the all-in-one agent binary, and perhaps create additional makefiles for other platforms. additionally, platform-specific racket code is marked with an XXX comment