Bad Engineering Idea of the Week

I'm struggling to remember to post new stuff here, or maybe I'm struggling to find something new to say. Either way, I'm going to try a different strategy: post about software engineering things from my job--lessons learned, handy tips, interesting bugs, hard problems, etc.

Today's tidbit is a bad idea for fixing a bug. Our architecture has a primary server and a secondary server for backup purposes, both of which must be kept in sync to guarantee correct backup behavior. One new feature attempts to provide better error detection and feedback, a key part of which is determining whether the backup process is running.

For a little more context, the primary server already will not allow clients to connect until it handshakes with the backup process and verifies a synchronized starting point. There is a simple socket connection and protocol to determine if the backup process is listening and do the handshake. If the backup process is not available, the primary server polls the socket occasionally and waits forever.

The objective of the new feature is to watch the startup of the primary server and send status to a separate application that monitors the status of all servers and clients in the network. How would you solve this problem?

Well, one of our engineers decided to modify the startup batch script to log in to the secondary server, get a task list and see if the backup process was running. If it's not, it fails immediately and stops the startup process. Why is this a bad idea? Let me count the ways...

• It introduces a new interface between the two servers that didn't exist before, which adds complexity to the model.
  
• It introduces technology not used elsewhere in the product, namely logging in to other servers and using non-portable command line tools.
  
• It adds its own failure mode for incorrect login/passwords on the secondary server.
  
• It couples the two servers in the startup sequence, as opposed to letting both start independently.
  
• It requires starting over completely to recover from the identified failure mode.
  
• It relies on the primary server configuration knowing the name of the backup process on a different machine.
  
• The condition it detects is different from the information you want. It detects if a process is running or not, as opposed to detecting if the backup process is accepting connections and is ready to handshake (the process might be running, but more subtly broken, which this will not detect).
  

I'm sure there are more, but you get the idea. So what is the better solution? How about using the existing socket connection information instead of adding a redundant channel? Provide the status to the administration tool and wait for the backup to start up. An admin can easily diagnose the situation and get the backup process running if needed. You could add an optional timeout if the backup hasn't started in 20 minutes or so, but that's not really necessary.

There are a couple of key lessons/principles at work here. One is loose coupling, which is almost always a win, generally without introducing much complexity. The other is reuse: don't add new stuff unless you really need it.

Java Fork/Join

Does anyone else find it ironic that Doug Lea's Fork/Join for Java reinvents green threads? Yeah, I'm a little late reading this, but there is other more recent good commentary on Fork/Join (the last paragraph says it all). Don't get me wrong, I think Doug Lea is one of the smartest people working on concurrency problems these days, but I can't help feeling that most of the work on Java concurrency these days is a big bandage on top of the gaping chest wound of the shared-state model.

PLT Scheme Turns 13

I'm glad to see PLT Scheme continue to thrive and strike out on its own in a way that both supports the R6RS standard and continues to innovate. Relegating mutable pairs to second class status last year was a delightful move. Now if only they would get rid of shared state concurrency I'd be a total convert.

Gartner Top 10 "Technologies" For Next 4 Years

Gartner includes Semantics in it's list of the top 10 disruptive technologies for the next 4 years, as item number 10. Some commenters on the blog link are saying it should be higher (assuming an ordered list), while calling the virtualization and multicore "boring trends".

It should be no surprise that I think semantics should be taken off the list. It's not a next-4-year big thing, if any big thing at all. Multicore and virtualization is riding an exponential curve, which is the only real way to be disruptive. I believe semantics is not only difficult, but also linear. Thus, our progress in the semantic space will be far outpaced by exponential trends. I do believe we'll make progress, but my prediction is that it will be in the brute force space, aided by Moore's law over time.

Multicore isn't exciting per se, but the disruption it will drive in the software space is already visible. Concurrency is already huge if you look in the right places, but the its increasing ubiquity will start to sink in very soon.

Interoperability is Hard

True interoperability between independent software products is hard. There are multiple levels on which you need to guarantee compatibility (read: agreement among all parties) in order for any deep interoperability to work correctly, including: transport, schema, semantics, ownership, and identity.

A lot of so-called interoperability works fairly well by severely limiting one or more of these aspects (most often identity and ownership), which is fine. However, most of the conversations I hear tend to revolve around the transport, paying minimal attention to schema, and almost none to semantics.

I think this is because 95% of your implementation time tends to get eaten up by the transport, which fools you into thinking it's the hardest part. In fact, semantics is often the hardest part, but that's all done in design, which can easily get ignored. The problem is, a transport defect can be found and fixed in one product, whereas a semantic defect usually causes you to need to change the schema, which affects all products (and therefore usually doesn't get fixed, leading to poor interoperability).

Therefore good interoperability needs the meaning of your data understood and agreed upon by all parties before you settle on the schema. This turns out to be quite hard to do. It's pretty difficult even if you're in control of all the moving pieces.

I'd like to think there's a way to decouple parts of interoperability to be able to iterate on standards after they are entrenched, but I haven't found it yet.

Adding meta-data to search

Fred Wilson covered an interesting search innovation by one of his portfolio companies, Indeed, in which they interpolate salaries for job postings that don't include a salary.

Fred categorizes this clever trick as adding "intelligence" to search, but it's really an example of semantic extraction combined with search. Now this is a perfect example of how aspects of the semantic web will emerge. It's shallow, but very useful, and there's no requirement for exhaustive human meta-data entry or conformity to a standard.

Attractive but flawed

The Economist chimes in on the Semantic Web as well. I quote: "It sounds a mess and it is" ... reviews of Twine "have been mixed". But of course if people are investing millions is must be a good idea, right?

Listen, the idea of a semantic web is alluring, but it's just not going to happen. We can't agree on semantics in real life except in small groups or in very shallow ways. Computers just aren't going to be any better at it until we create something smarter than ourselves. These technologies are all parlor tricks when compared to the grand vision espoused by semantic web evangelists.

I'm not saying some of the automated semantic extraction technologies are useless. Some of them are very cool, and it is absolutely the way we should be heading (waiting for humans to tag everything is a waste of time). However, we need to recognize while this path is taking us somewhere good it will ultimately fall short of the vision--much in the same way that the AI field has given us some great improvements without approaching a true artificial general intelligence.