Projects

Cinched

Cinched is a microservice which provides key management and cryptographic operations. It’s main goal is to make it easier for application developers to implement encryption of data at rest.

Blargh post

Github

Riak Ensemble Demo

A small demonstration application on how to use the Riak Ensemble multi-paxos consensus application.

Github

Libocsp

A small C library for performing OCSP lookups.

Github

I have packages for CentOS 7 build in my repo. Click the Repo in the top menu above.

Passwderl

Erlang C NIF for miscellaneous Linux passwd/user related functions.

Github

MySQL binary log rotator

This is a tool to automatically rotate binary logs on high churn replication masters where using the built-in automatic rotation functionality might be dangerous (for example when slaves lose connectivity for extended periods of time).

It queries the state of slaves to ensure that the log purge is safe.

Github

OCSPerl

Erlang NIF driver to libocsp for performing OCSP lookups from within Erlang.

Github

Foldrerl

A folder replication tool for Erlang. Files are sent over TLS sockets, while folder metadata (file list and checksums) are sent over Erlang distribution.

Performance should be reasonable even for very large files withtout blowing up Erlang communication channels, however if you’re syncing folders with large numbers of files, Erlang might not be too happy with a gigantic file manifest.

Github

AMQP Viewer

A small perl script to bind to exchanges on an AMQP broker and output messages to STDOUT.

Github

Collectd Plugins

https://github.com/marksteele/collectd-plugins

These are a series of Perl plugins for Collectd. What I’ve implemented:

  • CPUSummary: Summarizes CPU utilization (the build-in plugin gives you metrics per core, which in my opinion isn’t terribly helpful)
  • HTTPCheck: A simple HTTP plugin that can apply a regular expression or parse a JSON object and test that a specific key in the object has a specific value. On success returns 1.
  • MySQL: Queries a MySQL instance and returns every single numeric variable that it can find (around 500 metrics?)
  • RabbitMQ: Monitors a RabbitMQ broker for queues and message rates.
  • Riak: Pulls all stats available from the stats HTTP endpoint in a Riak cluster
  • AMQPJson: Writes metrics gathered by a collectd instance to an AMQP broker in a JSON encoded payload
  • AMQPJsonUDP: Same as AMQPJson, but will send the data over UDP (assuming you have Tony Garrock-Jones’ UDP plugin for RabbitMQ setup)
  • AmqpGraphite: Same as above, but graphite format instead of JSON

Collectd Puppet

https://github.com/marksteele/collectd-puppet

A Puppet class to manage the Collectd service. Currently somewhat Ubuntu specific but I’ll be fixing that shortly.

Collectd-AMQP-OpenTSDB Integration

https://github.com/marksteele/collectd-amqp-opentsdb-gateway

A helper application which pulls Collectd metrics from AMQP and pushes them to an OpenTSDB cluster. Part of my perfect monitoring solution ™

Interruptus

https://github.com/interruptus/interruptus

Interruptus is the continuation of the idea I had been working on with new-hope. This time, it’s java through and through. Once again I’ve had the pleasure of collaborating with Fabio B. Silva on an interesting idea and his help has been tremendous. Interruptus has a greatly improved design, is clusterable with automatic master/slave failover based on Zookeeper, a REST API for managing the Esper runtime, on the fly metric cataloging and much more. Also part of my perfect monitoring solution ™.

New-Hope – retired

https://github.com/marksteele/new-hope

New hope uses jruby/ruby to embed the Esper complex event processing to a framework for reading and processing time series data published to AMQP. This project is dead in favor or Interruptus.