Friday, March 26, 2010

Database Specification FAIL

I just booked a flight with a company that until today I considered a major Canadian airline. Their online booking system has changed slightly since they switched to SABRE and now they have input constraints that I hope would result in serious marks being docked in an elementary programming course. The company serves all regions of Canada and also offers international flights. But they come up with this:

Please spell your first and last names as they are on your government issued photo ID that you use for check-in. For example if your name is on your passport as Thomas Jones, please enter it as Thomas Jones, and not Tom Jones. Information entered into the name, address, password and cardholder name fields must be entered without special characters. Special characters such as apostrophes, hyphens, periods, symbols or accents (e.g. ^ à é # ) are not accepted in these fields by our system. (e.g. Enter Stéphanie Ducharme-L'Heureux as Stephanie Ducharme LHeureux, St. John's as St Johns, and Apt. # 123 as Apt 123).

I find that unprofessional to the point of insulting. If the purpose of the data is to collect the customer's name and home town, then collect that, not some garbled bastardization thereof. If the purpose is to compare the data with a straight ASCII database, then it is the duty of the programmer to strip or convert unwanted characters. I fly airplanes for a living and I think I could write a function that cleanly replaced accented characters with their unaccented equivalents and removed other special characters from a line of input, in not much more time than it would take me to write and format the dialogue telling the user to. But I'd be more likely to work a little harder to allow my database to spell my customers' names correctly. Asking a person to corrupt her own name for the purpose of buying an airline ticket is requiring a person to lie about her identity and herself. If an airline can't hire a programming staff that can deal with the concept of special characters, it makes me wonder if they can hack the complexities of time zones and confidential information. This would make me laugh if it didn't disgust me.

Update: I wrote them an e-mail to tell them that myself and I promptly received an autoreply beginning ...

Dear Guest,

If your issue is of an urgent nature (you’re traveling within the next 72 hours and/or needing a change or cancellation to an existing booking), please contact the Sales Super Center at 1 888 WestJet (937 8538).

That's right, travelling is misspelled and there are nine spurious characters replacing the apostrophe in you're. If you work at Westjet in Calgary, do me a favour. Find the person responsible for approving that garbage and slap them upside the head for me.

21 comments:

Geoff Arnold said...

Yet another example of the mindless behemoth of US-centric software stomping all over the English-speaking world. You realize, of course, that "traveling" is the correct spelling south of the border.

I remember the first time that I encountered this nonsense, at the UK Atomic Energy Research Establishment at Harwell in England in 1968. I had to sign up for an account on the IBM 360 mainframe, and I filled out the IBM-supplied form. They wanted my first name, last name, and middle initial. But I had two "middle" initials: I was Geoffrey M. B. Arnold. I gritted my teeth, and consigned the "B." to oblivion....

Sarah said...

Funny. There is a degree of incompetence revealed in customer facing web forms that boggles the mind sometimes.

They may as well just come out and ask it. "Are you on the do-not-fly-list, answer 'yes' or 'no'."

Rhonda said...

I tried to request a quote from a vendor once, and their web form (the only way to request a quote, because they were in a different time zone and their office was already closed for the day) kept telling me that I had entered illegal characters. I slowly stripped out obvious things that might cause problems like microsoft "smart quotes", dashes, colons, etc. Nothing worked.

In frustration, I loaded the source of the page and found the javascript that determined what characters were legal.

Whoever had programmed that page had spent a great deal of time allowing all kinds of obscure accented letters, but did not include in the list of allowed characters such rarely used things as commas, periods, or apostrophes.

Fortunately, the incompetence extended far enough that the email address the form was submitted to was also in the source of the page, so I just emailed them directly.

dpierce said...

Sometimes, in legal/security contexts, you can't take liberties with the encoding of a person's name. You (the developer) can tell the *user* to bastardize their name, but you may not alter it yourself. I would think if that's the legal situation in Canada, though, you would be seeing this behavior all over the place, not just at WestJet.

A likely situation is that testing (such as it was) discovered too late in the process that special characters "don't work". While it's expensive to get a developer to apply a quick fix to the code, it's very easy for their "HTML guy" (or girl) to add a silly note to the appropriate pages explaining the problem to the user.

The biggest crime is that a modern software implementation can't handle characters beyond the basics. The platform I work on has supported unicode (covering everything from Chinese to Klingon) since '93.

@Geoff In the US, you can spell the word travelling or traveling ('canceling' is another one with variable L's). Those mail templates are just blobs of text that are expected to be customized at implementation. Judging by the poor grammar in general (both "you'res" are screwed up), I'd say whoever did their website didn't focus on the details regardless of the country of origin.

GPS_Direct said...

I think part of it is incompetence - like the way that the Facebook programmers can't figure how to write a cookie so that it remembers I prefer to see updates ordered by date - and the rest is laziness or dependence on technology.

Has anybody noticed how many egregious typos show up in AP articles? These folks make their living via the printed word!

I was recently at a conference at ERAU (that 800lb. gorilla of aviation training the "airline way") and a student was attempting to type in our group's comments to an FAA NPRM. Many were astounded to see how poorly he spelled.

What I noticed however, was his technique... He knew the word he wanted, and he would "sound it out" via typing. MS Word would of course tag it, and he would very quickly right-click and select the corrected spelling of the word he was looking for. So, he "knew" how to spell and was just letting the technology work for him...

Progress?!?!

Aluwings said...

I've groaned at similarly inadequate input routines that can't strip dashs out of phone numbers or credit card numbers, etc.. That's a first year programming exercise for goodness sakes!

Likewise I've hit frustratoin index TEN when uploading websites created on my Mac and then discovering that the host site cannot handle file names with long names or special characters! Arrrgh.

belyins said...

With things like these, it's not really surprising that people may get arrested not only because they have the same name as somebody else in a "bad guy" database, but also because they have a similar name.

If it's just booking information, it's annoying. If it's also used to filter pax lists upon immigration from other countries, I don't even want to know what to call it.

zb said...

ah, these forms on the internets... just noticed I was stupid enough to type the verification word of the above comment into the field for my nickname.

townmouse said...

I've spent far too much of my working life wrangling accented text into ascii text so the user wouldn't have to bastardise their spellings, but also allow them to search by the bastardised version in case they were second guessing whether we'd stripped the accents off or not. Decent databases of course will all use unicode now, and do most of this for you, but there's always software that hasn't quite got with the program and can't handle the more exotic characters. And you always hit a wall of imcomprehension when talking to American programmers about the complexities of it: 'Oh we just strip the accents off, it's much simpler...'

rw2 said...

Ok, I don't typically feel a need to defend the US. But in this case, the issue is more about the age of sabre than anything else. Sabre was first built in, approximately, the mid-1800's by a single airline for a single purpose. It grew and become more important beyond that. Because of that growth it's now an absolutely giant system. Retrofitting it as @geoff implies would likely cost millions of dollars.

So, the point is, this isn't really about US centricity as much as the age of the software and the state of the art when it was written. Even a US company would, by simply virtue of the technology they would use by default, do a better job if it were written today.

Aviatrix said...

The Mid-1800s date is hyperbole of course, but it would explain why it's SABRE and not the US spelling. I once came across a "the South will rise again" website that urged true patriots to eschew the spelling corruptions introduced by Daniel Webster. Apparently Webster was a Yankee. Maybe Sabre was named by a die-hard US Southerner.

I can confirm that anglophone North Americans think of accents as things that can be stripped off, leaving the ordinary letters. There is no understanding of the difference between the é in French cassé and the é in Spanish qué (or that without the accents, each is a different word). The idea that n and ñ are different letters may be filtering through, but any letter looks vaguely like a latin one it is presumed to be equivalent. I wonder if it's the computer age that has deprived Spanish ll of its letter status. In old dictionaries ll words are in a different alphabetical sequence than l, but in new ones the ll words are all together between the li words and the lo words. And then we have the movie that appears on the box to be titled VOYADT.

dpierce said...

Many countries that use accented characters strip them off themselves when capitalizing them.

SABRE is an acronym for Semi Automated Business Research Environment. Like many software projects, it was just a testbed to see if electronic reservations were feasible, and, like many things only half-finished, it took off as it was and grew into established infrastructure.

Critical Alpha said...

I have two pet hates in this area: Web forms are BAD unless you need very structured information like a booking form. If it's a customer contact however then give me your EMAIL address and assume that I have the intelligence to communicate with you. Most web forms are so simplistic that they never have my use case in their form or logic.
The other pet hate is when the form asks me what country I live in and then insists that I choose the state I live in...from a list of American states. If I live in NZ I don't have a state and if I live in Australia, Vermont is not a state it's a suburb of Melbourne in the state of Victoria which BTW is shortened to a 3 letter abbreviation not a 2 letter one. If you want to do business with the world then realise that it's a diverse world.
Grump, grump, grump...

Christopher said...

None of the three major CRS providers allow special characters, even hyphens, in the name fields.

However the phone and address fields will allow hyphens.

zb said...

@Critical Alpha: Yeah, it's very weird when you are a non-US citizen and the form insists you select a state (and Bavaria is not in the list, even if you wanted to choose it...).

@Geoff Arnold: The thing about middle names is also funny. I don't have one, so I prefer the forms that don't have the field for the middle name tagged as mandatory.

The most funny part is the one about European (or other non-US based) companies who use US software for their websites demanding a bunch of information that's clearly specific to the US; ranging from phone numbers (x-xxx-xxxx) to middle initials ("this field is mandatory. please do not leave it blank") and states.

Nevertheless, I feel like pointing out that bad websites are nothing one may only find in the US and I don't want my comments to sound like I'm bashing US companies for their bad websites. It's a bit off-topic with regard to the original post, but here are two examples:

Deutsche Bahn tries to help me so much it tends to paralyze my computer: Their forms auto-guess my entries in a way that just takes ages and there are huge delays because much data obviously is transferred to a third-party contractor that collects user data (etracker), delaying everything else in the process and, of course, not asking for permission.

Last time I tried, Air France used fancy bling bling features when showing seat maps and stuff that just won't work on standard browsers (note that standard browsers include more than Internet Explorer with a particular set of pulg-ins available for x86 architectures only).

D.B. Cooper said...

The funny thing is, that Webster's Dictionary was not written by Daniel Webster, the orator from NH, who has a well respected aviation college named after him in Nashua. It was written by Noah Webster. He was, however, a Connecticut Yankee.

Aviatrix said...

D.B. Cooper, that's entirely on me. Daniel Webster was also a guy in my grade one class who was nice to me, even though I was a shy kid and had just changed schools, and apparently my brain and fingers prefer him to Noah when I write a quick note.

Curt Sampson said...

If it's expensive to get a developer to fix a problem, you have serious (albeit common) problems with your software development process.

That said, we still to this day live in a world where we're requested to enter our credit card or bank account numbers without they hyphens or spaces with which they're delivered to us. If modern developers can't even figure out how to remove the spaces from a string, how can you expect them to be so sophisticated as to be able to remove anything else?

(And as for searching: FoxPro did Soundex in the mid-90s; sheesh.)

Anonymous said...

It has nothing to do with Sabre or WestJet. It has everything to do with DHS and the e-APIS submission that is required for cross border flights. The e-APIS system doesn't grok accents or anything that ain't vanilla ascii. I know because I have to handbomb the e-APIS for GA flights for the charter company I work for.

Best Regards,
Tom Servo

Aviatrix said...

So in 2009 someone developed an application for the purpose of processing the exact passport names of international travellers and they couldn't wrap their minds around the fact that some people's names and hometowns have hyphens, dashes and accented letters?!

I was willing to cut the aged SABRE system some slack, hailing as it does from the days before computers knew about lower case, but for e-APIS, it's unforgivable. Can the DHS do anything right? I mean it: can someone give me an example of a streamlined, well-implemented, cost-effective procedure designed and managed by the US Department of Homeland security?

dpierce said...

Maybe this new feature will make up for the crummy web site.