Scratch Pad

Jump to bottom Edit New page

truepattern edited this page Jun 25, 2012 · 1 revision

Design

Terminology

Task - basic unit of work
Event - communication to tasks are via events
Job - collection of tasks to achieve a higher order goal

Modules

Distributed File System

Meta data to be stored on redis cluster and the actual data to be stored on ext4 filesystem
Provide a FUSE interface to operate on the file system

Job/Task Queue

Based off of redis

Scheduling

Pluggable modules, lets start with greedy

Names

cloudd - cluster of node.js hosts
mongol - unit of node (greedy task scheduler that runs within a host and pulls tasks)
scout - agent that collects data from each host
noyan - management node for cloudd