« May 2006 | Main | September 2006 »

August 29, 2006

Software specs are not the enemy.

When it comes to discussions about writing software specs, I really like to quote Sen-Rikyu, the master of the modern Japanese tea ceremony: "Tea is naught but this. First you make the water boil. Then infuse the tea. Then you drink it properly. That is all you need to know." But if you know anything about the actual practice of the tea ceremony, there is -- for laypeople who have not mastered the Zen of simply boiling the water and infusing the tea -- an immensely complex protocool, replete with symbolism that must be adhered to.

Writing specs is simple. All you do is document what has been decided and capture known uncertainties, and underlie everything with a giant disclaimer that until the code is written you won't know exactly how it's going to work.

In fact, perhaps we shouldn't think of specs as specifications of software. They are specifications of goals for the software.

I'm about to start working on a UI spec for some upcoming changes to the Google Base UI. I'm always looking to improve, and this is a pretty complex series of workflows that we have to get just right. So I want to make sure that I capture and communicate all of the decision points that I've traversed up until now, and that I specify behavior that may not be fully obvious in the prototype that I built.

Let me highlight some key concepts from that last paragraph: Capture. Communicate. Spell out corner cases.

So I searched for write UI specs. The first result is 37 signals explaining why specs don't work. I read this type of article -- along with some Agile and XP techniques -- as a radical overreaction to the other extreme (of overplanning and presuming you can actually precisely specify software on a piece of paper before anything is written). In that sense, the 37 signals article is irresponsible. It's speaking to a particular historical context of software engineering without acknowledging that there are a lot of ways that specs can be written, usefully, in the context of more modern software engineering practices. Moreover, when you're not working at a small startup, it is irresponsible to not write specs. Specs help other teams anticipate and plan for what's happening down the line, and when you don't have the same 10 people doing everything -- when you have a marketing department, a customer service team, writers who have to do help documentation -- you need to let them know as early as possible what you think is coming down the pipe, so they have maximum time to plan their own resources and schedules.

1. Software specs can create a false sense of security.

And as we all know, a false sense of security is worse than no sense of security. If you get signoff on a spec -- whether early- or late-stage -- and the signers-off don't understand that some things are subject to change, your budget and schedule will be in for some very nasty surprises.

One function of a spec is to document and decide on everything that it's possible to decide, and to minimize (not eliminate) uncertainty. Sometimes you just have to put a stake in the ground so you can make a decision and get something done, even if you know it's probably going to be wrong down the road. Decide what you're going to do and do it, and when you need to fix it, fix it.

The fact that a lot of organizations cram unreasonable, un-vetted features into the spec and call it a "spec" is not a problem with specs. It is an organizational problem. Blaming the existence of specs for problems caused by bad management is not fair to the spec.

2. A software spec cannot overspecify or underspecify. To be useful, it has to specify just right.

This is a rhetorical task. When I work with an expert front-end coder, there's a lot that I don't need to specify. For example, I'm currently working with a great front end engineer, DT. DT knows that things need to line up on exact vertical margins (he's possibly more OCD about this than I am) and knows that when he does a walkthrough of mockups with me, there are probably a bunch of error cases that I'm not specifying, and that when he runs into those in the code, he'll do something reasonable. (Like use our standard error framework.) On the other extreme, when I work with people who are newer coders or who don't usually do front end stuff, I'll write the production javascript so I can "specify" the exact time delay for the mouseover effect.

Any overspecification wastes time because it could have just been implicitly specified in the code (or otherwise downstream). Underspecification wastes time because downstream people have to wonder how the hell it's supposed to work, or what the goal is.

3. Specs can indeed be written in the face of uncertainty, and they can be useful in the face of uncertainty.

For example, you can double-spec a feature: "If it turns out that it's cheap and easy to hook up to the live WhizBangerNiftySearch engine, then do (A). The fallback is (B)." Then everyone sits down and someone says "If we can't get (A) working in two weeks, then we'll fallback to (B)." You document this in the spec, so the team remembers what the decision was, and you stick with a plan long enough to see things through to fruition.

4. Voice-of-God specs are generally disconnected from reality.

You cannot write a spec without input from every department: you can't make estimates about how hard it'll be to write that new piece of functionality without some sort of estimate from your engineers, and without cost estimates, you can't make decisions about whether or not the functionality is actually worth its expense for the current release. But after enough back-and-forth, you somehow have to say "we agree that this is the minimum functionality that we need for this release, this other stuff is nice to have, this other stuff is version nn+1."

The goal with the spec is to iterate *quickly* on paper -- or any other medium that is faster and less expensive than code -- to get buyin and agreement from all relevant parties. If you know on Day 2 that senior management isn't going to let you menion "commerce" in the product, you can build around that without much hassle. If they tell you on Day 180 that they didn't want you to refer to "commerce" or commerce-like activities, and the entire thing is a product feed engine, you're screwed, and you've just wasted a lot of time.

Paper is cheap. Code is absurdly expensive. Prototype-based specifications help people know what is being built before it actually gets built. It takes me 10 minutes to do a paper flow sketch and maybe a day to build out a lightweight dynamic PHP prototype of say, 5 HTML pages. It'll take engineering 4 weeks plus two weeks to send all the strings to translation. Every change in code is exponentially more expensive than changes in prototypes.

5. Dynamic prototypes can't capture everything and communicate it as fast as a spec.

Yeah, I could make my engineers click on my dynamic prototype until they run into every single error case, or I could give them a screenshot and a tabular view of error conditions and associated error messages.

You prototype until you start writing the details that don't need to be tested in a prototype -- like form validation, error messages, and corner cases. You can leave those to the actual code. Write a spec for how they should be handled and forget about it. Use the prototype to capture the dominant 80% use cases and leave the rest to paper.

22 minutes later: Of course, now I find out that Joel on Software said it better.

August 24, 2006

Electronic voting systems are corrupt, and paper is not the answer.

Testimony of Clinton Eugene Curtis, a programmer, under oath before the U.S. House Judiciary Members.

Electronic voting systems are corrupt. Current systems are closed, uninspectable, and their processes are so complex and obscured that there is almost no way to ensure that they accurately tally the input of the voter. Not only does source code have to be open source, but the compilers and probably the chip design should be open source too. Everything must be generic, inspectable, and transparent.

A friend asked last month, about an article explaining Diebold's corrupt electronic voting machines:

can someone explain what the anti-paper record argument is about? wouldn't that make everyone happy? it can't cost much per unit. The unwillingness to implement some sort of non-digitial backup trail just seems so odd to me. it's difficult to avoid getting all conspiracy theory-y.

Paper isn't enough. You need totally open-source code so any engineer or organization can audit the code. The article criticizes Diebold's engineering on the grounds of "this model does not produce a voter verified paper trail so there is no way to check if the voter’s choices are accurately reflected in the tabulation." But simply outputting a paper trail means nothing.

If I were designing one of these systems to flip votes, I would program it so that all reflections of the tally accurately represent the voter's choices. Printouts - yes, have them so everyone feels good: "You voted for Al Gore." But somewhere in the back, at the single point where votes are recorded to be read back later, that's where I'd flip votes. Only here and there; only if the candidate of choice is losing; weighted by how many votes the candidate is actually getting so I don't skew the results unbelievably from exit polls. Without open source code, paper receipts will only provide a false sense of security. And a false sense of security is worse than no sense of security.

If paper is going to be at all useful you'd have to print out a human/machine readable copy, let the voter verify that it is correct, and then have the voter turn the form over to the election agency. Then, if there is a challenge/recount, the collected paper receipts can be matched to the machine-reported counts.

Still, if I were writing evil code, I would flip bits within the margin of error of human recounting. You couldn't necessarily guarantee any one particular election outcome, but over time you would skew power to the party of your choice in a much less obvious way than blatanly stealing a single election. So paper receipts should be printed in a machine-scannable way, and the code that reads the scanners would have to be outcome-neutral: so the Scantron wouldn't have the capacity to know whether choice A or B were associated with one or the other parties / candidates; it would be a dumb counting machine that would only spit back tallies. It would have to be a general-purpose counting machine -- in fact, the receipt printer should print out in the format that is readable by currently on-the-market scanner machines that are used to grade multiple-choice tests in schools. (Are they still using those things?)

The other huge problem with voting software that's not open source is that it only takes one intelligent and malicious person [with access to the source code] to control very very many voting systems (assuming that most precincts use one of a couple available systems). With old fashioned paper ballot systems, the most you can do is fuck with one or two precincts -- after that, there is too much manpower required, and too many logistical issues, to effect much change without getting caught. If you wanted to stuff ballot boxes in the Gilded Age, you had to physically access hundreds of boxes, and that's going to take a lot of collusion to pull off: only one person (of 5, 10, 20) has to spill the beans. But with closed-code systems, you can have just one engineer in the code: you have to pay off one person, you have one person mysteriously disappear, and the code is deployed to countless systems, affecting millions of votes.

Referencing the FOR property of the HTML LABEL element using JavaScript

In javascript, I needed to compare the "for" attribute of a LABEL with the ID of an input object to see if I'm looking at the LABEL associated with that radio button / checkbox.

Programmatically, it doesn't work to say something like

if( theLabel.for == theRadio.id ) 

because for is a reserved word in javascript.

So after some investigation (spitting out every single property of the LABEL element and looking to see how the "for" is referred to programmatically), I learned that the way to refer to the FOR attribute with JavaScript is with the htmlFor property:

if( theLabel.htmlFor == theRadio.id ) 

August 19, 2006

Reaction to "Homeward Bound" in American Prospect

In December 2005, American Prospect published Homeward Bound, one of the best articles I've ever read on feminism and the women's movement outside of The Feminine Mystique. The article discusses the "choice" movement and the opt-out "revolution" and generally takes the perspective that it's not much of a choice for women to be choosing the same undervalued sphere that they've been forced into for millennia, against the spheres of external power, intellectual pursuits, and wage-earning that men have traditionally claimed.

Great as liberal feminism was, once it retreated to choice the movement had no language to use on the gendered ideology of the family. Feminists could not say, “Housekeeping and child-rearing in the nuclear family is not interesting and not socially validated. Justice requires that it not be assigned to women on the basis of their gender and at the sacrifice of their access to money, power, and honor.”


Here’s the feminist moral analysis that choice avoided: The family -- with its repetitious, socially invisible, physical tasks -- is a necessary part of life, but it allows fewer opportunities for full human flourishing than public spheres like the market or the government. This less-flourishing sphere is not the natural or moral responsibility only of women. Therefore, assigning it to women is unjust. Women assigning it to themselves is equally unjust. To paraphrase, as Mark Twain said, “A man who chooses not to read is just as ignorant as a man who cannot read.”

One of my female friends responded:

who defines the word "flourish" here? and by whose definition do we judge whether or not someone is "flourishing"? and why on earth would the home be called "this less-flourishing sphere"? who allowed the family to be so devalued and debased that it less valuable than other realms of activity (um... The Man)? surely that is not every one's definition: not everyone's value. i do not allow anyone to tell me that the family is a less valuable sphere, and therefor my choice to assign it to myself makes me just as unable to acheive "full human flourishing" as women who have it put upon them. that's crap. we've let this society's values dictate for us the value of the family sphere. THAT was a mistake in old school feminism. (in my humble opinion).

I think you can judge whether someone is flourishing or not by whether they're left with feelings of hopelessness, of worthlessness... of getting divorced after 30 years of service to spouse and children... by being neurotic, by being on atavan and prozac and zoloft and whatever else. The author, along with Betty Fredian and similar, talks about "flourishing" as having the latitude to exercise one's mind and one's talents to the fullest extent of one's capacities. They make the judgement -- in its capitalistic, western-individual style of morality -- that the physical work of homelife and the mental exercise of raising children, day in and day out, in the modern American nuclear family, for ~15 years, is not enough to fulfill adult mental and social capacities, as evidenced by 60 years of watching women do it and then become psychotic. Because then the kids leave and you are left with 20 years remaining in life, having spent the previous 20 doing not much more than reasoning with toddlers. They're saying that the evidence points to everyone talking about how fulfilling of a choice this is -- and it just turns out not to be after 15 or 20 years. And they're trying to give women the language and the moral authority to avoid the same old trap.

Isn't there something funny going on when we have (1) a many-thousands-year-old tradition of telling women that their only acceptable sphere is the home; (2) a lot of prejudices against women taking the helm in the valued spheres; (3) almost zero men ever actually making the decision to handicap their wage-earning ability to raise children or take care of housework?

How often is it that the woman (versus the man) hamstrings her earning potential and her ability to support herself, by quitting her job in order to raise children? Or, forget the capitalist earning potential -- how often does she give up a passion or a dream because she's not getting 8 hours of sleep tonight and has to toilet train the toddler and is presumed to manage the household? Is it not almost exclusively women who can mentally infantilize themselves because somewhere, in the back of their mind, they always have the socially acceptable recourse of marriage? Isn't it still mostly men who grow up with the mental gravitas that they will always have to look to themselves and be their own last recourse? I'm not saying that this is the idealized state of the world -- to be able to only count on yourself, without trust or fallback when shit happens. But I'm saying it's gendered and I'm saying it's bad that it's gendered. Why is it still moderately radical and weird to choose not to have kids -- it is -- or for the man to work part time while the woman is a fulltime career behemoth?

The "choice" is still only really left to women, therefore gendered, therefore suspect -- the "choice" isn't one that men get to make. (Hell, maybe if you're both in icky deadend jobs, you each want to cut back because family life is more interesting and fulfilling than whatever you're doing.)