alpinegizmo

recent history

$ history|awk ‘{a[$2]++} END{for(i in a){printf “%5d\t%s \n”,a[i],i}}’|sort -rn|head
68 ls
48 cd
39 jruby
31 git
30 less
24 ssh
22 rm
21 ping
17 cat
15 grep

credit goes to tim bray

(im)pureXML

Rather than producing a native XML database like eXist or MarkLogic, IBM has chosen to add XML features to DB2, creating an interesting – and convoluted – hybrid that IBM has branded “pureXML”.

If you’ve drunk the XML database koolaid, you may be thinking about how going down this path is going to (1) give you a toolchain where web app development becomes a much more declarative process, with better separation of concerns; (2) give you more flexibility than a relational database; (3) fit more naturally with a web-services model; and/or (4) get you moving toward the semantic web.

For me, one of the attractive prospects would be to clean up the process of generating HTML. If the data coming out of your database is already XML, then its not much of a leap to use an XSLT to transform that XML into HTML. Traditional view templating schemes (such as erb) can be a mess, so maybe this world of XML databases offers a cleaner solution.

Architecturally you’ve got three choices of where to apply an XSL transform – in the database, in your application server, or in the browser. Each of these has advantages and disadvantages, but as a first pass we’ve been exploring having DB2 do the job. The results have been somewhat disappointing.

To get DB2 to apply an XSL stylesheet to some XML, you have to use an SQL function called XSLTRANSFORM. The XSL stylesheet has to be provided as the result of an SQL expression, which generally means that it is stored in an XML column in the database. The XML document to be transformed might also come from an XML column, or it might be the result of an XQUERY embedded in the SQL you’ve been forced to use as the overall structure for this operation. Given this structure it’s not surprising that xsl:include declarations are not supported, but this is definitely problematic. Also disappointing is that stylesheets incorporating XQUERY statements are not supported.

A simple example in which the XML to be transformed is coming from the xmldata column in the employees table:

SELECT XSLTRANSFORM(xmldata 
  USING (SELECT xsldata FROM xslts WHERE name = 'employee.xsl')) AS html 
FROM employees WHERE id = 833038373

It’s a little more fun when the source of the XML is an embedded xquery:

SELECT XSLTRANSFORM (
  (SELECT XMLCAST(XMLQUERY('$data//Phones' PASSING xmldata AS "data") AS XML) 
    FROM employees WHERE id = 833038373) 
  USING (SELECT xsldata FROM xslts WHERE name = 'phones.xsl')) AS html 
FROM employees WHERE id = 833038373

It may look a bit strange that the WHERE id = 833038373 clause occurs twice. The second instance could be pretty much anything that returns one row. While many SQL databases will accept a simple “select 1”, DB2 insists on a FROM clause, such as “select 1 from employees fetch first 1 rows only”. The only really important part of the last line above (i.e. “FROM employees WHERE id = 833038373”) is that it return exactly one row.

XSLTRANSFORM isn’t the only case where DB2’s pureXML forces you to use an unholy mixture of SQL and XQUERY. As IBM explains, “With plain XQuery you can not exploit full-text search capabilities provided by the DB2 Net Search Extender (NSE). You need to involve SQL for full-text search.” This means doing something like this:

xquery 
  <Employees> { 
    for $e in db2-fn:sqlquery(
      "SELECT xmldata FROM EMPLOYEES.EMPLOYEES 
       WHERE CONTAINS(xmldata, 
         'SECTION(""/Employee/Info/Address/LocationInfo"") ""madrid"" ') > 0"
    ) 
    let $ei := $e/Employee/Info 
    return 
      <EmployeeInfo> 
        { $ei/Name, <Departments> {$ei/DepartmentInfo/Name} </Departments> } 
      </EmployeeInfo> 
  } </Employees>

For clarity, this example has been broken across muliple lines. Don’t expect DB2 to be happy with input this easy to read.

This whole business of XQUERY embedded in SQL, and vice versa, can get amazingly messy. We’ve spent quite a bit of time figuring out how to do quoting and casting in order to keep both sides happy. And although DB2 supports SQL sub-queries within XQUERY, so far we’ve found it impossible to figure out how to quote SQL sub-queries within an XQUERY that in turn is embedded in SQL, which is necessary if you want to use XSLTRANSFORM on the result of an xquery like the one above.

Resources:

pureXML in DB2 9: Which way to query your XML data?
DB2 9.5 documentation
my DB2 links on del.icio.us
my XML links on del.icio.us
on the prospects for transforming web app development with XForms, REST and XQuery
short reviews of various XML databases

7 weeks on DB2

For a new project we’ve been using IBM’s DB2 database and its pureXML features. This has been my first serious exposure to the brave new world of building web applications on an XML platform. Now that I’ve been doing battle with DB2 for seven weeks, I thought I’d give a progress report. In subsequent posts I’ll try to share what we’re learning about how well this all works.

In the past I’ve used SQLite, MySQL and PostgreSQL (usually underneath Rails) for web development. Compared to these open source databases, my first impression of DB2 has been – to be blunt – that it sucks. Now I’m aware that proponents of DB2 [see ZenAndTheArtOfRubyProgramming (link currently down – try google cache), for example] will tell you that because Rails is database agnostic (with a mysql bias) it doesn’t show off the strengths of DB2’s advanced database features. Apparently, DB2’s strengths include “utmost speed, pureXML, compression, replication, high availability, [and] affordable 24/7 support.” That may be true. I feel a bit like I work up one morning, and found an 18-wheeler in my garage. It may be extremely capable and powerful, but so far, I’m just feeling proud that I’ve managed to get it down the driveway and onto the street without hitting anything or killing anyone. I still miss all the things that made driving my car a comfortable experience.

IBM, if you’re listening, there are a few simple things you could do to make the transition to DB2 less painful:

• Put some resources into improving the usability of the tools. For starters, put readline support into the command line processor, and modify the command editor in db2cc to wrap lines within a fixed-width textarea. (Yes, I’m aware of the history and edit features in CLP.) Figuring out how to do really complex queries with the current tools has been a seriously frustrating experience.
• Get rid of silly limits. For example, the shell interface truncates XML results to 4000 bytes. What’s the point of offering a shell-based command line processor if you can’t use it in shell scripts?
• Find some way to make routine administrative tasks, including backup and restore, managing text and xml indexes, and doing server restarts, less of a problem. We’re spending way too much time fighting with the basics.
• The ibm_db ruby on rails adapter is half-finished. Please get us to the point where “rake test” works.

We’re getting past these issues by building in-house tools to smooth over the rough edges. Why bother? pureXML is the answer. In the future I hope to let you know if it proved worth the bother.

the E-111 is dead, long live the E-111

Just to clarify for folks ... the old E-111 form has been replaced by the new European Health Insurance Card. Since Jan 2006 the E-111 is invalid, and you need to get one of these cards, which look like this:

I think this is actually easier to deal with, but, now that there's a better system in place, it would appear that the hospitals are insisting on folks having these cards.

a medico-linguistic adventure

This is the fifth morning I have woken up in a hospital in Madrid. Unless something unforeseen changes the plan, I will be leaving the hospital sometime today, and going home to France tomorrow.

This has been a rather interesting experience, both medically and linguistically. I've learned that it is surprisingly difficult for me to brush my teeth with my left hand; that Spanish nurses are, as a whole, very warm and friendly (and some are very pretty); and that self-medicating minor problems for decades might be a bad idea. I've also used these few days as an intensive language learning experience, and I'm quite pleased with myself on that account.

Perhaps I shouldn't jump to too many conclusions from a single data point, but the socialized health care system in Spain would appear to be working quite well. Admittedly I am in the hospital for central Madrid, near the royal palace, but the level of care provided here and the professionalism of the staff are excellent.

Having a medical emergency in a country where you don't speak the language adds an extra level of scariness to the whole experience. Thanks to my knowledge of French I can understand some fraction of what people say, if they are speaking slowly and clearly, but when I arrived here at the hospital (by ambulance) I could communicate very, very little in Spanish. Fortunately, right after arranging for the ambulance I had the presence of mind to call someone I know locally who was able to quickly join me and translate for the EMTs and ride with me in ambulance. It was reassuring to have Minoru there during that initial period of engagement with the Spanish medical system.

Initially most everyone I encountered spoke either no English, or a teeny bit (and no French, either) -- but in the final tally what I've seen is that many (perhaps most) of the doctors here speak English, some extraordinarily well, but almost no one else in the system does (a few of the nurses speak French pretty well). Meanwhile I've learned enough Spanish to say things like "my arms hurt," "my head is congested," and "I need something for my fever."

A few thoughts for others who might find themselves hospitalized while traveling in a foreign country:

1. You are going to need your passport. The ambulance drivers insisted on it, and waited for someone to fetch it from my hotel room (the whole adventure started in the hotel, so that didn't take long). I have no idea what would've happened had I not had it handy, but I gather it would've been complicated. The hospital also insisted on seeing my passport, presumably in order to later deal with extracting payment from someone. I don't normally keep my passport with me when I'm traveling, and I probably still won't, but it's something to think about.
2. If you are a european resident, travel with a valid european health insurance card (model E-111). I didn't have one; I just had the French Carte Vitale (the social security system card), and this caused some concern at the hospital here for a few days until my wife was able to get the French to produce some appropriate paperwork. If I had been in and out of the hospital over the weekend, I think they might've tried to force me to pay them somehow.
3. If you regularly take any drugs (even things like Tylenol), learn their non-branded names (eg Tylenol is paracetamol). You can't expect foreign doctors to know all of our US marketing jargon -- things like Sudafed and Pepcid-AC may just leave them scratching their heads.
4. Try to get a roommate who speaks a little English. Despite being hard of hearing and having somewhat broken English, Eusebio was really helpful to me on several occasions. Given how few adults in Spain speak English, I give the hospital credit for finding me a Spenglish speaker to be my roommate.
5. At least in France and Spain, the hospitals don't provide anything for the patients to wear. Patients' families provide them with clean underwear and pajamas. If you are traveling alone I don't know how you'd solve this problem. "Put on clean underwear every day, in case you wind up in the hospital, and put a few extra pairs and some PJs in your laptop bag in case you have to stay a few days. And maybe your toothbrush and some slippers." Not very practical. Fortunately I had a friend who could bring me the things I needed from my hotel room. Thanks, Larry.
6. The Lonely Planet Spanish Phrasebook is terrific. I have several other books for learning Spanish here with me that I've used to good effect, but that alone would've been enough to get me past the worst of the language barrier.
7. And my final piece of advice for travelers: try to get sick in a country where the people are naturally outgoing, like Spain or Italy.

And the bad? In a couple of ways the experience has been diabolical. What put me in the hospital was problems with my esophagus, and my digestion, following a history of problems with choking on things (well, ok, not "things", but food and pills). So after 3 days of a liquid diet what am I served for my first solid meal? A terrifying piece of very bony fish. And now part of my prescription involves some horse pills that there's no way I can bring myself to swallow. I'm grinding them up and hoping that's not going to cause me some other problem. Besides the fish (which was pretty good, I'll have to admit), the rest of the food was only scary because of massive quantities of indigestible grease -- just what I'd serve if I ran the gastro-enterology ward of a major hospital.

Update later the same day: yes, I have indeed left the hospital. And many thanks to all those who've sent their well-wishes.

Invalid certificates and GMail Notifier

I've been using Google Mail as my primary email solution for some time now (I do send a copy of my business email to another server, just in case). One of the things I like about Google Mail is their notifier for OS X. But until yesterday I've been using a version from 2005, because all newer versions have had a big problem: at least for me, modern versions of the gmail notifier pop-up a dialog every 10 minutes promping me to signin again with my Google email address and password. This goes beyond annoying into the realm of totally unusable. (I also preferred the old version because it was small and simple. The newer versions use a lot more memory to provide features I don't want.)

Yesterday my old, cherished gmail notifier stopped working. After going all day with a silent gray M in my mac's menubar, I decided to try the latest version, which, sure enough, gave me trouble. Convinced that google had finally turned off some service the older version depends upon, I was motivated to find a solution -- which I seem to have found. If you're having this problem, here's what to do:

1. Open Safari.
2. Browse to https://google.com
3. Accept the invalid certificate that's causing the trouble. In my case the certificate for google.com came from google.es.
4. That's it.

Seems like google could make this a lot easier.

gandi.net and their new hosting service -- first impressions

For many years now I’ve been a fan and small-time user of gandi.net’s domain registry services. Initially drawn in by their attractive pricing and non-evil policies (in contrast to some of their larger competitors), I’ve been mildly disappointed that their pricing has remained constant (in euros, thus rising considerably in dollars), but have stuck around because they seem committed to competence and ethical behavior. Over the past year I’ve watched with interest as they’ve added new (free) services – IMAP mailboxes, blogs, etc. Over the past couple of days I’ve begun to experiment with their latest, more ambitious offering: virtualized hosting.

Cowboys and Waterfalls

Why research labs don’t produce good software

I spent the first twenty-plus years of my career working as a researcher in university and corporate research labs – at Carnegie Mellon, Mitsubishi Electric (MERL), and Sun Labs. It has been my privilege to work on some great projects with some amazingly talented people. But I have to tell you that by and large, the quality of the software coming out of research labs isn’t very high. I blame the cowboys and the waterfalls.