What follows is a true story although the names and places have been changed.

At 13:45 a developer execute the following query

DELETE FROM table
return 159 rows effected


He intended to execute a SELECT * FROM table, and just remove the WHERE clause from a previous query but it turned out to be a DELETE FROM table WHERE ID = #.

As soon as the trigger was pulled he realized his mistake. I was brought in to resurrect the data because I previously performed all backups.

Off we go.

1. Stop replication
2. Flush logs
3. if you can do the processing on the server wonderful, other wise copy the binlogs to a recovery system.
4. mysqlbinlog -d mydb mysql-binlog.[0-9]* | sed ‘/table/,/;/!d’ > recovery_file
5. start replication

From the recovery_file we had a list of every statement issued against the table and we could replay all transactions to return the table to it’s original state with a little editing.

We are now reviewing our backup and recovery practices. I hope this was educational for you as it was for me.

I found the key point for the sed command in this document, and this tutorial helped greatly.

Since HTTP is a stateless protocol it runs into problems when data changes occur. With HTTP the client must poll for new data, over, and over, and over, and over … Flickr threw up some numbers at the session that nailed home the challenges large web apps face. July alone Flickr had nearly 3 million requests from FriendFeed (a social content aggregator for your online presence), for 6000 users. Clearly this sort of traffic won’t fly as more people climb aboard.

The presenters, Evan Henshaw-Plath and Kellan Elliot proposed repurposing XMPP (typically an IM protocol) for the purposes of creating data streams for more efficient exchange. While REST is a constant request for new data, XMPP really is a subscription model, where a client establishes a session and listens for requested data. I don’t have their metrics handy, but the difference was remarkable, effectively shrinking the whole thing down to about 1% of the original traffic.

Their reason were:

• persistent connections
• stateful - less ID on each connection
• designed to be event streaming protocol
• natively frederated and asychronous
• identity, security, and presence built in
• jabber servers are built to do this

They also used PubSub (http://www.xmpp.org/extensions/xep-0060.html), a protocol extension, to achieve the subscription model, and OAuth for authentication.

While this had little to do with my work, it was fascinating to hear how larger enterprising handle the scaling issues that inevitably plague any popular web app. We have tried to do data streams ourselves in a couple of situations using Flash’s XML Sockets, which work well except 1. it’s in Flash and 2. it doesn’t route through firewalls easily. I don’t necessarily think Flash is bad, but it is difficult to support at times. I thoroughly enjoyed the talk. Evan and Kellan have a great presentation style that makes them easy to relate to.

A couple of miscellaneous notes:
The jid per user oauth authentication handles the identification of users.

XMPP Data service example

1. hello world
• jabber:simple
• xmpphp for php
• smack javae
2. jabber account
3. connect and send a message

Servers

• ejabberd
• djabberd
• openfire/wildfire java server
• tigase - java server

When setting up a server, turn off lots of features, for example turn off open subscribing. For this sort of data streaming there is little need for the multitude of features that XMPP offers to IM clients.

Build a component example

• use a compnent XEP-0114
• persistent talks over local socket
• YAGNI: rosters. presence
• Load balance

Starling - no idea what this was about

OAuth over XMPP - delgation of identification

Enter Laika (http://laikaproject.org); a test suite for Electronic Heath Records (EHR). Laika’s purpose is to support the accessibility of data for providers and patients. Publishers of EHRs using the C32 specification must have their data exports certified by the CCHIT (http://en.wikipedia.org/wiki/CCHIT), which uses Laika for that exact purpose. Laika’s architecture consists of a Rails app up front with Java engines behind for the XML parsing. At this time they only support C32, but with other players entering the market they want to support the CCR standard as well and transforming between the two specifications.

The C32 specification is a combination of of both the CDA (Clinical Document Architecture) spec and CCD (Clinical Care Data). These standards are governed by HL7/ATSM (http://en.wikipedia.org/wiki/HL7), the industry gorilla when it comes to EHRs. A more recent cousin of the C32, the CCR (or Continuity of Care Record, http://en.wikipedia.org/wiki/Continuity_of_Care_Record), has struck the fancy of software behemoths Google in their Google Health (https://www.google.com/health/html/about/index.html) system, and Microsoft’s Health Vault (http://en.wikipedia.org/wiki/HealthVault). The main differences between the two are the document size and detail. As one would expect, the software companies prefer the lighter CCR standard while the industry backs the dense but more verbose C32.

This means that for my projects that as we continue to look at the clinical space we will have to eventually either consume or publish resident information in one of these two standards.

Additional Notes:
would be interesting to see what avenues are open to the consumer market. Google’s plan

NIST provides transform tools
HL7 has a style sheet

RewriteCond %{REQUEST_URI} ^/user
RewriteCond %{SERVER_PORT} ^80$
RewriteCond ^(.*)$ https://example.com/$1 [L,R]

Since Drupal uses the same login section of the site, I was able to push the user under SSL when they went to log in. This doesn’t necessarily prevent them from dropping out of SSL when theyenter the site, but it does mean that they will end up in the right place at the outset.

Most sites want users to use the nearly ubiquitous “www” prefix. As Jeff Atwood mentioned in his post on the topic. It has become so pervasive in the web sector that it is nearly a non-entity; if you mention it you are being redundant. As Jeff argues, with the power of mod_rewrite penetrating even the ivory towers of IIS with add-ons like ISAPI Rewrite, we as developers must shoulder the responsibility to conform to our users behavior. It is so simple to gracefully handle the requests that not being flexible for our users is the ultimate crime.

In my experiment I had to tackle removing the “dub-dub-dub” prefix and sending the user the the stripped URL without. In most circumstances, the site maintainer will add the following to either the httpd.conf or to an .htaccess

RewriteCond %{HTTP_HOST} ^www\.(.*)$ [NC]
RewriteRule ^(.*)$ http://%1/$1 [L,R=301]

The RewriteCond is effectively our if statement. If the requested host begins (^) with the world famous “www.”, followed by any number of characters “(.*)”, without regard to character case (”[NC]“), then go to our RewriteRule.

The RewriteRule then changes the whole mess into the previous argument without the “www” mess and informs the browser that the request is permanently redirected.

These two short lines change bizarre “http://www.kire.notneb.net” into the familiar “http://kire.notneb.net” once again righting the wrongs of “www” misuse.

I also found that even great sites like del.icio.us don’t permit the misdirection from “www”, which makes me wonder how far we have to go in our quest for better URL management.

To at least give me some sanity while I worked I wrote this little Greasemonkey script that blocks Greader during my productive hours, 9-12 and 13-17. When I try to go to the site I get redirected to a site that reminds me what I’m working on, in this case the Java Language reference. You can easily add sites to black list through the Included Pages in the Greasemonkey management window. And the script is so simple that anyone can modify it to fit their needs.

deathtogreader

First up there is of course the iPod. Everyone loves it, it’s everywhere, hacked and back, and has a great little interface.

Next we have the newest player on the block, Microsofts Zune. With WiFi, FM, and a giant screen this is a likely contender, except for the fact this is MSs first attempt in the DAP market, and they notoriously have problems with first generation hardware.

Then there is the Creative Zen Vision:M, similar to the Zune without the WiFi. Supports tons of formats and supposedly has a great screen.

Outside of the individual features my other requirements are something that will work under linux - preferably ubuntu, something I can use as a UMS drive.

Any thoughts?