-
Write opinionated workarounds
A few years ago, I decided that I should aim for my code to be as
portable as possible. This generally meant targeting
POSIX; in
some cases I required slightly more, e.g., "POSIX with OpenSSL installed
and cryptographic entropy available from /dev/urandom". This dedication
made me rather unusual among software developers; grepping the source
code for the software I have installed on my laptop, I cannot find any
other examples of code with strictly POSIX compliant Makefiles, for
example. (I did find one other Makefile which claimed to be
POSIX-compatible; but in actual fact it used a GNU extension.) As far
as I was concerned, strict POSIX compliance meant never having to say
you're sorry for portability problems; if someone ran into problems
with my standard-compliant code, well, they could fix their broken
operating system.
And some people did. Unfortunately, despite the promise of open source,
many users were unable to make such fixes themselves, and for a rather
large number of operating systems the principle of standards compliance
seems to be more aspirational than actual. Given the limits which would
otherwise be imposed on the user base of my software, I eventually
decided that it was necessary to add workarounds for some of the more
common bugs. That said, I decided upon two policies:
-
Workarounds should be disabled by default, and only enabled upon
detecting an afflicted system.
-
Users should be warned that a workaround is being applied.
-
FreeBSD on EdgeRouter Lite - no serial port required
I recently bought an
EdgeRouter Lite
to use as a network gateway; I had been using a cheap consumer wifi/NAT
device, but I wanted the extra control I could get by running FreeBSD
rather than whatever mangled version of Linux the device came with.
Someone wrote instructions on
installing FreeBSD onto the
EdgeRouter Lite two years ago, but they rely on using the serial port
to reconfigure the boot loader — perfectly straightforward if you have
a serial cable and know what you're doing, but I decided to take the
opportunity to provide a more streamlined process.
-
A challenge to startups
"From those unto whom much has been given, much shall be expected." In
various forms this sentiment has been expressed at least as far back as
the third century AD, via the Gospel of Luke; more recently it has been
cited frequently by US Presidents, and can be seen in modified form in
such places as Spider-Man ("With great power comes great
responsibility") and the demands of the Occupy movement that the "1%"
pay higher taxes. I started thinking about this a few days ago after
re-reading
an essay by Paul Graham
and thinking about how lucky I was to be running a startup company now
rather than two decades ago.
-
The design of my magic getopt
When I started writing the blog post announcing my
magic getopt, I was
intending to write about some of the design decisions which went
into it. I changed my mind partway through writing: My readers
probably cared about the functionality, but not about the ugly
implementation details. It turns out that my original plan was
the right one, as I've received questions about nearly every
design decision I made. Since this clearly is of interest
to my readers, here's the reasons behind some of the decisions I
made while writing that code.
-
A magic getopt
Parsing command lines in C is easy when all of the options are single
characters: You pass your command line to getopt along with
a string containing all the valid options; then you have a switch
statement with a case for each option you want to handle. It
looks something like this:
int ch;
while ((ch = getopt(argc, argv, ":af:")) != -1) {
switch (ch) {
case 'a':
aflag = 1;
break;
case 'f':
printf("foo: %s\n", optarg);
break;
case ':':
printf("missing argument to -%c\n", optopt);
/* FALLTHROUGH */
default:
usage();
}
}
Unfortunately if you want to add support for long options — say, to
accept a new --bar option — you need to switch to using
getopt_long and your list of options is no longer confined to the
options-processing loop:
enum options
{
OPTION_BAR
};
...
static struct option longopts[] =
{
{ "bar", required_argument, NULL, OPTION_BAR }
};
...
int ch;
while ((ch = getopt_long(argc, argv, ":af:", longopts, NULL)) != -1) {
switch (ch) {
case 'a':
aflag = 1;
break;
case OPTION_BAR:
printf("bar: %s\n", optarg);
break;
case 'f':
printf("foo: %s\n", optarg);
break;
case ':':
printf("missing argument to -%c\n", optopt);
/* FALLTHROUGH */
default:
usage();
}
}
Rather than adding a new option in one place (or two, if you count the
list of options at the top of the loop as being a separate place), new
long options require changes in three places — one of which (the
enum) is often placed in an entirely separate file. So much for keeping
code clean and free of duplication. There has got to be a better way,
right?
Enter
magic getopt. Via a little bit of macro magic, the above
options-handling code turns into this:
const char * ch;
while ((ch = GETOPT(argc, argv)) != NULL) {
GETOPT_SWITCH(ch) {
GETOPT_OPT("-a"):
aflag = 1;
break;
GETOPT_OPTARG("--bar"):
printf("bar: %s\n", optarg);
break;
GETOPT_OPTARG("-f"):
printf("foo: %s\n", optarg);
break;
GETOPT_MISSING_ARG:
printf("missing argument to %s\n", ch);
/* FALLTHROUGH */
GETOPT_DEFAULT:
usage();
}
}
with each option listed just once, at the point where it is handled.
-
The HTTP 500 Solution
My online backup service offers
bug bounties for
mistakes in the client software; while I explicitly exclude bugs in
the Tarsnap website (with the exception of cosmetic errors) in an
attempt to discourage people who blindly run vulnerability scanners
against websites, I still get a lot of bogus "bug reports". One of
the more common reports is "I managed to trigger an HTTP 500 Internal
Server Error" response; what people don't realize is that this is in
fact entirely deliberate, as part of what I call the "HTTP 500 Solution".
-
A FreeBSD AMI Builder AMI
I've been working on the
FreeBSD/EC2
platform for a long time; five years ago I finally had it running, and for
the past few years it has provided the behaviour FreeBSD users expect —
stability and high performance — across all EC2 instance types. Making
the platform work was just the first step though; next comes making it
usable.
Some people are happy with simply having a virtual machine which runs the
base FreeBSD system; for them, the
published
FreeBSD/EC2 images (which, as of FreeBSD 10.2-RELEASE, are built by
the FreeBSD Release Engineer) will be sufficient. For users who want to
use "stock" FreeBSD but would like to have some extra setup performed when
the instance launches — say, to install some packages, edit some
configuration files, and enable some services — I wrote the
configinit
tool. And for users who need to make changes to FreeBSD itself, I added
code for building AMIs into the FreeBSD source tree, so you can take a
modified FreeBSD tree and run
make ec2ami
to generate a reusable image.
There was one group for whom I didn't have a
good solution yet, however: Users who want to create FreeBSD AMIs with minor
changes, without wanting to go to the effort of performing a complete FreeBSD
release build. Ironically, I am exactly such a user: All of the EC2 instances I
use for my online backup service make
use of spiped to protect
sshd and provide encrypted and authenticated tunnels to my mailserver and
package server; and so having spiped preinstalled with the appropriate keys would
significantly streamline my deployment process.
While it's possible to launch a FreeBSD
EC2 instance, make some changes, and then ask EC2 to create a new AMI out of
it, this rarely produces a "clean" AMI: A lot of code runs when an EC2
instance first launches — creating the ec2-user user,
installing the appropriate SSH public key, creating SSH host keys, growing
the root filesystem if launched with a larger root disk, downloading and
installing updates to FreeBSD, downloading and installing packages... —
and much of this needs to be manually
reverted before a reusable AMI can be created; not to mention command
histories and log files written during the configuration process, which the
more fastidious among us may wish to avoid publishing. To solve
this problem, I present the FreeBSD AMI Builder, now available as
ami-28682f42 in the EC2 US-East-1 region.
-
Tarsnap email confirmation bypass
Over the past four years,
Tarsnap's bug bounties
have received quite a bit of attention. Most of it has been very useful
— almost 400 mistakes (most either cosmetic or harmless, but some
of them significant) have been
reported and fixed — but it does also get some unwanted attention:
Despite my clear statement that Tarsnap's bug bounties are for problems
in tarsnap code, not for problems in the website, I regularly see people
running automated vulnerability scanners... which invariably yield a
selection of absurd non-vulnerability "vulnerabilities".
One consequence of these unsolicited security scans is that —
since they feed a variety of inputs to forms, including the account
creation form — I see a lot of obviously fake signup attempts
(alas, none yet from
the world's most obviously fake domain
name). These are harmless, since the signup code sends out a
confirmation email and the account isn't actually created until the
alleged registrant follows a link in that email; so I wasn't
concerned when I received an email last week telling me that someone
was trying to create an account as admin@tarsnap.com .
Five minutes later, I was very concerned upon receiving an email
telling me that the registration for admin@tarsnap.com
had been confirmed and the account created.
-
Safe from what?
I woke up this morning to a headline news story on CBC's website:
Is
your baby monitor safe?
According to a literal reading of
Betteridge's
law of headlines, the answer to this question should be "no", although if
you consider the spirit of the law — as a commentary on sensationalist
journalism — then the answer should probably be "yes".
To me, neither answer makes sense, because the question itself doesn't
make sense.
-
Tarsnap $1000 exploit bounty
For somewhat over four years,
Tarsnap has been offering
bounties for bugs found in the Tarsnap code. Two thirds of the
bounties Tarsnap has paid out have been $1 each for cosmetic bugs (e.g.,
typos in source code comments), and a quarter of the bugs have been $10 each
for harmless bugs — mostly memory leaks in error paths where the tarsnap
client is about to exit anyway — but there have also been some more
serious bugs: Several build-breakage bugs ($20 each); a variety of cases
where tarsnap behaviour is wrong in a user-visible — but generally
very obscure — way ($50 each); a few crashes ($100); and of course the
critical crypto
bug which first convinced me to offer bounties.
Most bugs are straightforward, but occasionally one comes up which is not so
clear in its impact. Such is the case with
a
bug which is fixed in tarsnap 1.0.36. This bug causes the NUL string
termination byte to overflow the heap-allocated buffer used for paths
of objects examined as tarsnap traverses a directory tree; such
one-byte heap overflows
have
been shown to be exploitable in the past. In the case of tarsnap, I will
be very surprised if it turns out to be possible to cause anything worse than
a crash, but I can't absolutely rule out the possibility.
In light of this, Tarsnap is offering a
$1000 exploit bounty: The first person before the end of 2015 who can
convincingly demonstrate a serious exploitation of this bug will receive $1000.
While there are many organizations which pay more than this for exploits, I
think this is a reasonable prize: After all, I'm already telling you what the
bug is which you need to exploit!
Fine print: No bounty if you're in Iran, North
Korea, or some other problem countries. Bounties are awarded at my sole
discretion; in particular, I get to decide whether the "convincingly
demonstrate" and "serious exploitation" conditions are satisfied. Payment by
US dollar check or paypal. To avoid races, contact me before publishing
anything. If you can't accept cash prizes, the bounty can be donated to a
mutually-acceptable charity of your choice.
|