Sunday, September 27, 2015

Programmers who just don't care

I'm trying to import a tab-delimited quiz file into Blackboard. I get this message:

"A parse error occurred on line 1. Processing of other questions continued uninterrupted."

The first problem with this message is that it is a lie: Blackboard failed to process any of the questions at all and produced no quiz.

The second problem is: "A parse error occurred..." A parse error? He couldn't be bothered telling me exactly which parse error, information that might help me to fix the parse error? And the programmer who was parsing my file almost certainly knew exactly what the error was at that time he detected it. For instance, perhaps he found a tab instead of a question type at the beginning of a line. In that case, he should have given the user feedback like, "Question type missing at beginning of line 1."

Very simple. Very helpful. But this joker couldn't be bothered.

And then he delivers the coup de grĂ¢ce: "Review logs for details."

Without a hint about where these "logs" are, or how one goes about reviewing them.


  1. A large organization I recently worked for had an internal search engine for their corporate intranet. It would often report. 'Search failed due to no result or too many results'!

  2. Programmer here. It's not necessarily the case that if you catch an error, you know what the error is.

    First of all, they could be using a library to do the parsing, and it could be that the library doesn't do error reporting/propagation.

    Secondly, when you write code (especially under time constraints), you often write code that will produce the correct result given valid input and that will give generally reasonable error messages when the input is invalid. However, these error messages won't necessarily be detailed or fine grained, because the code is written most efficiently when it's written with a single purpose - to process correct input correctly - the more fine-grained the error detection becomes, the more complex the code often becomes - yes, even if just having to give a broad error like "missing question type at beginning of line."

    Let's give an example: Say each line in the file to parse can be either an instruction, a question, or an informational box.

    Here is how an instruction should be written:

    Instr Please hand in your papers when you are done.

    Here is how a question should be written:

    Who won World War 2? A) The Allies B) The Axis C) Stalin D) The Grinch

    Here is how an informational box should be written:

    -- Many students have failed the following section because they have not answered the central question concretely --

    Now, suppose a line in the file I submit to parse is as follows:

    Instructors in Prussia pioneered what teaching tactic?

    What error should my parser output? Is it a missing space after the beginning "instr" (which would imply the line is a malformed instruction)? Is it missing answer choices (implying it's a malformed question)? Or did the author forget the "--" demarcation for informational boxes? We have no idea.

    Programming even simple user-facing programs is surprisingly difficult to do when taking into account the myriad of things programmers must keep in mind: correctness, speed, error reporting, security, maintainability of code, and so on.

    Programming *parsers* is notoriously annoying and difficult. In fact, when you're programming more complex parsers, it might be impossible to know why parsing failed. For example, when we parse English in a natural language processing program, we often use language models that boil down to "which set of rules does this sentence follow?" - which reveals information about the structure of the sentence in question. However, when you start checking thousands of rules and the sentence fails some completely, passes some completely, and then fails the rest only *partially*, we can't really say which rule being broken was the cause of the overall parse failure. For example, given the sentence "John jumped over big bush", is the mistake here the omission of "the" (as in John jumped over the big bush) or is it maybe the omission of "es" at the end of "bush" (as in John jumped over big bushes)? We can't know. All we know is the sentence fails parsing.

    Without knowing more details about the specific code optimizations the programmer is doing behind the scenes - and without knowing the difficulty of the overall parsing problem - it's difficult to say the programmer was being lazy or "doesn't care."

    Imagine having an assistant that can only do literally what you tell him. A single mistake or a single scenario you failed to think of, and what he ends up doing will be a deviation from your intent. It's hard to write instructions for brain dead assistants - i.e., computers. And the more you try to cover all your bases, the longer that post-it note you leave on the fridge for him will have to be. So sometimes we're content with telling him "if any steps fail, just give us a ring at the office and tell us that we won't be getting a sandwich today."

    1. Michael: Programmer here who was programming when you were nursing at your mama's teet.

      They were just parsing a tab-delimited file. The overall parsing problem was trivial. Since I *did* know the details of what was going on, it was not difficult to say the programmer screwed up.

    2. Oh, and if you have a parsing library that returns nothing more than "Parse error": time for a new parsing library!

      Look, really complicated parsers like the Python interpreter return MUCH better errors than "Parse error", so if you're doing a tab-delimited file, you can pretty easily do better than that.

    3. I just generated an error in Python, which accepts input way, way more complex than this little tab-delimited file. The error I got back was:
      "TypeError: respond_to_cond() got an unexpected keyword argument 'env_vars'"
      This is a quite reasonable attempt to tell the user just what went wrong. And if the programmers working *for free* on Python can do this, I would hope the *paid* programmers at Blackboard, working with a much simpler parsing problem, could at least do better than "Parse error."


Career Fair Organization

A dark thought came into my mind. I imagined the hosts of career fairs plan eight hours worth of events for their participant companies. ...