Tuesday, 30 August 2011

What I Did On My Hols

Yes, I know it's nothing to do with DBA Tasks, but you don't have to read it if you don't want to!

I went to the Eifel National Park.  No relation to the famous tower – two effs - but a landscape of steep wooded hills and lakes just over the Belgian border in Western Germany, about four hours drive east of Calais.  Imagine the Lake District with far more trees and far fewer tourists, and little medieval villages like Monschau full of half-timbered houses.  The warm sun shines on open air cafés serving beer with Wienerschnitzel (not every stereotype is false). 

Put on your walking boots or get on a bike and wander round the hills and lakes - this is what people do here.  Take a boat trip on the lake and walk back.  Cycle lanes and footpaths abound, making for lots of white paint at road junctions.  (Remember to wait for your pedestrian traffic light, even if it’s 3am and there’s nothing coming - the cops enforce jaywalking laws with 100 euro on the spot fines.  Ditto cyclists).  Once you get away from the main routes, a detailed map would be handy - I wasn’t impressed by the signposting on the footpaths.  Those of you adept at reading between the lines will realise that this means I got lost.  

The village is Gemünd, which is close to Schleiden.  There’s another Gemünd near Bitburg somewhere, and lots more places called Gemunden, so be careful programming the satnav.  Nearest big towns are Aachen to the north, Köln to the east. 

Accommodation is in a series of apartment blocks.  All the flats have a balcony I think,  but sadly although there are great views to east and especially west, our block faced south so we gazed at the closed shutters of the empty block next door.  Good sized living room with sofa bed, with galley kitchen and small verandah; smallish double bedroom; shower/wc. All fine for a week.  There’s one flat for sale with an asking price of 60,000 euros - you wouldn’t get a flat of this quality for that kind of money in the UK. 

What it doesn’t have on site is a restaurant or bar; nearest is the Salzberg 500m down the hill.  We found Aristotle’s rather smart Greek restaurant at the bottom of the footpath from the top of the complex down to the valley; a steep climb home...  Apart from that you need to  go down into Gemünd itself about 1.5 km away down a steep hill, where there are probably a dozen quiet bars and restaurants, so it’s probably not a resort suitable for party animals.  Although I admit I didn’t try the sauna.

Speaking of party animals having to make their own entertainment with their kit off - dominating the skyline to the west is the tower of Ordensburg Vogelsang (look it up on Wikipedia).  The National Socialists used to bring the best and brightest hunky blond young Aryan men to this training establishment and inducted them into a secret cult which involved them going into a secret room in the tower decorated with a big mural of the perfect German man. Who just happens to be naked.  You may wish to roll your eyes at this point.  After the war the place was used briefly by the British and then the Belgian army, who built a huge barracks block that Albert Speer would have admired, and replaced all the German eagles with Belgian heraldic lions. 

Off to Köln for the day, starting with a boat trip up the Rhine from the cathedral, commentary in German and English - oldest, longest, busiest, highest etc.  2000 years of history, check out the cathedral and the adjacent Roman museum, plus it has a chocolate museum, so pretty much the perfect weekend break.  Students of urban planning should come and see how to do it.  Cheap to park as well, 1€20 for an hour right by the cathedral. 

A month or two back I visited the Museum of Welsh Life in Cardiff (Amgueddfa Werin Cymru), and just down the road from Gemünd near Kommern is a very similar German version.  Park at the bottom and walk up the hill, and there among the trees are little clusters of half-timbered wattle and daub houses from up and down the Rhine, preserved as they were before the coming of the 18th century..  They have a working wood-fired bakery where they bake the bread they sell on site.  Chickens and geese wander round the villages, but the pigs are confined to their sty which is probably just as well.  The occasional member of staff wanders round in historical costume, the effect spoilt only by the occasional squawk of the two way radio. They are building a new village covering the period from the 1950s to the 1970s, which makes me feel very old.  You can eat on site, but we went down the hill to Kommern, a living village of, er, half-timbered wattle and daub houses, and had a very nice Greek meal in the restaurant’s tiny shady courtyard.

Drove back to France via Maastricht in the Netherlands; lots of clothes shops for those that like them.  But the main square was blocked off as they were setting up stalls for a forthcoming antiques fair, so the pavement café atmosphere was spoiled for me.  I was surprised that the tourist information office actually charged for a map of the town; everyone else in Europe gives them away for free.  And the swish new Vrijthof underground car park beneath the main square charged more for an hour than Köln did for four.

Tuesday, 9 August 2011

Data Quality - What could possibly go wrong?

Well, pretty much everything.  If you have enough data, there will be mistakes in it.  Try writing your own name down on a piece of paper.  In CAPITALS.  There's quite a good chance that you will spell your own name wrong - something about the unfamiliar capitals trips up the link between your brain and your fingers.  Now imagine that you are a data entry clerk typing in hundreds of names every day.  Some of them are Arabic, others Polish, some even Welsh.  Can you guarantee that every name is correct?

There are ways to help, of course.  Take addresses, now - most countries have a postal code system which can be used in automatic sorting machinery to help your letter get to the right place.  Most countries use a 4 or 5 digit numeric code - 90210 famously defines Beverly Hills in California, so typing an easy number into your address software automatically fills in the town (and it isn't Beverley Hills) and state.  And by reading the relevant postal address file, the software can check the spelling of the streets.  The postcode systems in the UK and the Netherlands can go right down to street, even sometimes premises level.  But these things have to be kept up to date - and periodically the postal authorities make changes in order to take on board changes out in the wild, even sometimes to fix mistakes.

So your name and address data may contain errors - perhaps your older data even preceded the current computer system and was transcribed from a Rolodex.  Check it.

Check the names and genders and titles as well, while you are at it.  Where you find blanks, there are a few things you can do:

  • Where there is a gender, you can derive a title (if Gender = F, then Title = Ms).  Note that while this is fairly risk-free for men, some women dislike being called Ms.  
  • And vice versa - where there is a title, you can derive a gender (if Title = Mr, then Gender = M).  
  • Where there is a forename, you can - usually - derive a title and gender (if Forename = Andrew, then Title = Mr and Gender = M).  This will still leave a number of names (e.g. Hilary) where it is not possible to determine the gender.  

Maybe you don't have a first name at all, just the initials.  Is that a man or a woman?  If you don't know, how can you address a letter?  Suppose it says Mr J Smith, but then states the gender as F?  Something wrong, but what?   You could of course classify such cases as "Unknown", but that might screw up your letters completely -  “Dear Unknown Smith” is not going to win you much business for your widget factory.  “Dear Customer” might be acceptable.

Suppose you have a list of email addresses - you need to make sure that they are valid before sending off your mailshot.  Lots of things you can check here:

  • Check for spaces in field
  • Check that name is present
  • Check that address does not end in a full stop     
  • Check that suffix e.g. .co.uk is not missing
  • Check that suffix is not truncated
  • Check that @ symbol is present     
  • Check that @ symbol is not duplicated      
  • Check that there is no spurious full stop in the address
I found some SQL code on the internet to do this - you don't expect me to do any actual WORK, do you? (After writing that sentence, the boss made me rewrite my code in PL/SQL in order to do that useful job for an Oracle database - I should have kept my mouth shut).  If you want a copy, let me know and I'll pass it on.  One big problem, of course - you can check all these things and have the most valid email address ever, but it's no good if your customer has changed to a new ISP last month.  

And while I think about it, don't forget that you need to comply with data protection legislation to hold all this stuff.  UK readers can find a handy checklist here, but there are similar rules for most countries.  You can get your data checked for people who have moved house, or people who have died (if you want to upset a bereaved relative, send a cheery letter to the recently deceased).  And if you know that someone has died, make sure that you don't contact them, especially not if you are going to write to Dear Mr Smith (Deceased).