Introducing miniplumber

David Fritz

7 Feb 2017

Sandia number SAND2017-1449

Introduction

miniplumber provides uni- and multi-cast experiment communication to VMs, minimega instances, and external programs anywhere in an experiment.

Motivation

How do we plumb non-network based connectivity?

Plumbing the kitchen sink

Plumbing

Enter miniplumber

A quick example

# create a pipeline connecting foo and bar
plumb foo "sed -u s/foo/moo/" bar

plumb

# write something to foo
pipe foo "the cow says foo"

shell sleep .5

pipe

# works from the command line too
# echo "hello plumber!" | minimega -pipe foo

clear plumb
clear pipe

Plumbing objects

# plumbing productions are similar to unix pipelines
plumb "cat data.txt" "grep -i foo" lines_with_foo

# named pipes can attach directly
plumb this is a valid production

# pipelines live until any stage of the pipeline is destroyed
plumb foo "grep -i foo" bar

# kill the above pipeline by closing foo
clear pipe foo

Plumbing with CC

# miniccc programs can attach to pipes directly
# cat data.txt | miniccc -pipe foo
# miniccc -pipe bar | less

# or you can invoke pipes directly
cc exec stdin=foo some_program
cc background stdin=foo stdout=bar some_program

minimega distribution

Fully distributed via meshage

Out-of-band communication

Message scheduling

# create a pipe with round-robin scheduling
pipe foo mode round-robin

pipe

vias

# get a normal distributed value for a given mean
pipe foo via bin/normal -stddev 5.0

pipe foo 10.0

Use cases

Summary

Thank you

David Fritz

7 Feb 2017

Sandia number SAND2017-1449