Your Mobile Phone Number is not Safe with your Mobile Operator

There has been a lot of fuss today about mobile network O2 handing out the mobile numbers of its customers to websites that they visit.

That's clearly an unfortunate occurrence, and O2 users as well as the web in general were understandably outraged. But what surprised me was not that it happened at all, it was the fact that the public really don't understand quite how widely it was happening already.

MongoDB Support for Tera-WURFL

This is just a quick heads up to point out that Tera-WURFL now has support for using MongoDB as its data store. This is particularly exciting for me, since I contributed the original code. I've been benefiting from Tera-WURFL for years now, so it's pleasing that our ongoing migration to MongoDB at work provided an opportunity to give something back.

Making Phone Calls and Sending SMS with HTML

Okay, so you can't really make phone calls and send SMS messages using only HTML; that would be silly. However, if you are developing web sites and web applications for mobile handsets, you can take advantage of some features in XHTML Mobile Profile which make it easy for a user to call a number without typing that number in. You can also use the same mechanism to trigger - on the user's handset - an SMS or MMS dialog with the intended recipient's number and the message content prepopulated.

That this can all be done by creating a specially-formatted HTML link on a web page is one of the most convenient, if occasionally misunderstood features of XHTML-MP. This post will tell you everything you need to know in order to take advantage of this feature.

Mobile-Friendly Short URLs with Wag.gd

A few weeks ago I quietly launched Wag.gd, which is what I like to think of as a mobile-friendly URL shortening service. It's "mobile-friendly" in the sense that the URLs which it generates are designed to be exceptionally convenient to type on a standard mobile handset keypad. It's time to introduce the service to the wider world, and look in a bit more depth at what I've tried to achieve with the site and how I've gone about it.

A First Look at the New WURFL API for PHP

About a month ago, the New WURFL API for PHP was officially released. While the code had been available in one form or another for some time, the official release coincided nicely with the early stages of a new project at work, so it seemed like an appropriate time to have a look at the API and see if it was something we wanted to use.

By way of a refresher, WURFL is a "Device Description Repository" - a huge open-source XML-based database of information regarding mobile handsets and their capabilities. I've discussed WURFL in the past, for example here. Prior to this release, the only practical method of querying WURFL in real time from PHP was via a library named Tera-WURFL, which I blogged about here. In fact, both WURFL and Tera-WURFL were covered in an article I wrote for php|architect magazine last year.

We've generally been very happy with Tera-WURFL, but it's always worth considering one's options, so what follows is an overview of my experiences with, and first impressions of the New WURFL API.

"PHP Tools for Mobile Web Development" Published

This is just a quick heads up to say that my article, "PHP Tools for Mobile Web Development" has today been published, and is currently gracing the cover of July's php|architect magazine.

Of course, I jinxed things a little by blogging that it would be published in June, but never mind, we got there.

Big thanks must go to Ciaran for giving the initial draft the once over (on a related note, check out Ciaran's post about web development for the iPhone). Thanks also to my occasional colleague Gerard for clueing me in to the fact that the damn thing had been published.

For what it's worth, php|architect is recommended reading even when I'm not in it, so get yourself over there and get subscribed!

Ok...now to crack on with that second article...

Mobilising a Website, Part 2: Strategies

In Mobilising a Website, Part 1: The Problem I noted that this site is practically unusable when viewed using the browser on a mobile handset, and that I'd like to do something about that.

This time around, I'd like to size up some of the approaches and strategies that developers can take in order to make an existing website mobile-friendly.

Mobilising a Website, Part 1: The Problem

It hasn't escaped my notice that if one happens to visit Pointbeing.net - this very site - using the browser on a mobile phone, the experience is more than a little painful. In fact, more often than not, the site is simply unusable.

The reason for this is that the site does not adapt itself in any way to the smaller screens, slower connection speeds, and idiosyncratic navigation methods found in mobile devices.

In my defence, this is not unusual right now: many, many sites are in the same position (have you ever tried to visit LinkedIn on a mobile?). However, given my faith in the future of the mobile web, and also given what I do for a living [1], this is something of an embarassment. The time has come to mobilise Pointbeing.net.

My php|architect Article to be Published in June

I purposefully didn't mention this here before now, as I didn't want to jinx anything.

But the time has come, and I'm pleasantly surprised to be able to report that my article - named something along the lines of "PHP Tools for Mobile Web Development" - is to be published in the June edition of php|architect magazine.

This will be my first ever contribution to php|architect, so it's a huge compliment that as well as being published somewhat sooner than expected, it looks like becoming the cover feature for June.

We're currently in the final stages of editing, and I'm really enjoying working with editor Steph Fox to turn this into something worth publishing. Stay tuned for further news.

The End of Mowser is Not the End of the Mobile Web

In the last few days there has been a certain amount of rather sensationalistic and poorly informed commentary floating around on tech sites and blogs, predicting the immediate death of the mobile web. For example, this piece on CNET, and The Register's dramatically titled A Requiem for the Mobile Web.

And what is the basis for this doom-and-gloom mongering? Well, it turns out that a poorly-marketed twelve-month-old startup, named Mowser, which has never been able to attract VC, and seemingly staked its future entirely on its ability to attract VC, has called it a day.

That's it.

New Pitch Website

I'm pleased to learn that Pitch's corporate website has today been relaunched.

Sure, it's a corporate website with the traditional combination of slick agency design, press releases and mugshots of the usual suspects. But have a dig around and I think you'll see that we're up to some pretty cool stuff.

I find the work section particularly exciting, of course. It mentions a few of the things which we're doing right now, including the recent work with Red Dwarf, Warner Brothers and a new project with legendary Thin Lizzy guitarist Scott Gorham.

Update 01/06/2008: This post has quickly become entirely irrelevant! We've now joined the PlayPhone family and will be rebranding accordingly. Watch this space.

An Introduction to Fire Eagle

A definite highlight of Over the Air 2008 was London-based Yahoo Steve Marshall's introduction to Fire Eagle. For those not in the loop (which, to be fair, is most people: Fire Eagle is currently only open to a limited number of invited developers) Fire Eagle is Yahoo!'s brand new API for location-based services.

An Introduction to Tera-WURFL

I recently added a post about Wurfl, a comprehensive open-source XML database of mobile device capabilities. I noted that actually querying Wurfl in a performant manner:

is going to be a non-trivial task, and is perhaps a topic for a further article.

Well, I guess this is that article. It's time to have a look at Tera-WURFL, which is perhaps the most popular tool for querying Wurfl programmatically - from PHP, at least.

A Brief Glossary of Mobile Jargon

A few terms that seem to get bandied about in the industry. I'll probably add to this over time.

Managing Mobile and Non-mobile Versions of a Site Using Tera-WURFL and Zend Framework

This is a quick proof-of-concept I put together after a discussion on how to deal with running a mobile site and a 'full' web site on the same hostname in a sane way, and to transparently route user agents to the appropriate site.

So strict...

Quoted directly from the rules of a rather well-known mobile developer list:

Extremelly poor knowledge of the english language is also likely to put you on the fast track out of the list.

Quite.

Mobile Device Detection: WURFL and UAProf

If you do any kind of development for mobile devices, you'll soon stagger into the minefield of browser and device detection.

Now, this is quite a different sort of challenge to that faced on the desktop web. On the desktop we have maybe one browser worth using, plus a whole lot of people using Internet Explorer, along with a handful of computer programmers who, bewilderingly, persist in using Firefox.

Ok, light-hearted browser snobbery and a few CSS hacks aside, the point is that these days, you basically know where you stand with desktop browsers.

By contrast, when we turn to mobile development, we find we're up against quite literally thousands of subtly and not-so-subtly different devices, and countless combinations of device, firmware and operating system versions. How can we get around this staggering complexity in order to find out the user's screen size? What markup can they handle? What ringtone formats can they accept? How about Java applications and games?

This is where WURFL comes in. WURFL is a free, open-source database of mobile devices and their specifications and capabilities. It is as close to a "standard" solution to this problem as currently exists.

WURFL

The WURFL database is essentially a large (just over 6 Megabytes at the time of writing) XML file. The data is hierarchical and is ultimately keyed by user agent string (as typically supplied in an HTTP request's User-Agent header). The key is mapped to a specific device, and to one or more families of devices, via the "fall_back" mechanism. This allows us, with a bit of work, to build up a list of the device's "capabilities".

If it helps, you can think of this in OO terms as a big stack of classes extending other classes, overriding fields as they go.

Let's try a concrete example. Imagine we come across an agent which identifies itself as "SEC-SGHE950/1.0 NetFront/3.4 Profile/MIDP-2.0 Configuration/CLDC-1.1". Querying WURFL, we find:

;

This tells us...well, nothing! Nothing, beyond the fact that this user agent string is known to WURFL, and thanks to the crucial "fall_back" attribute, we know that it's one of an unspecified number of user agent strings which identify a device or family of devices known internally to WURFL as "samsung_sgh_e950_ver1". So we need to query WURFL again to find the details of this item. Here it is:

This is more like it. We've identified the device (actual_device_root="true"), we have its model number (E950) and its screen size (240 x 320), and we've learned that, ideally, it would like to be served web pages as XHTML-MP. Cool. But there's more: the device falls back further, to "sec_e900_ver1":

Jackpot! We've found out a lot about the device now, in particular the content types it supports. (Note that the model name here is specified as "E900". We can disregard that, as we know it to be overridden one step further down the hierarchy). We can keep going in this manner, successively falling back to "netfront_ver3", "generic_xhtml", "generic" and finally "root". At each stage we pick up a little more information about the device (and at each stage we'll need to disregard a whole bunch of default information too).

Now, I'm sure you're as excited by all of this as I am, but before we get carried away, let's bear in mind that WURFL itself currently weighs in at roughly 127,000 lines of XML. Evidently, recursively querying WURFL directly, in real time, is not a practical option. We'll most likely want to import the data into a relational database, and schedule regular updates. This is going to be a non-trivial task, and is perhaps a topic for a further article.

Alternatives to WURFL

So WURFL's pretty awesome, but it's going to involve some effort on our part to press it into service. Since we're professionals, we'll want to consider whether there are any alternatives which provide better value-for-effort. Well, it turns out that there are alternatives, but they're few and far between. Perhaps the most notable of these few is a mechanism known as "UAProf".

UAProf

UAProf is a W3C initiative, the idea behind which is as follows: the user agent supplies a specific HTTP request header (of which, more in a moment). This header contains the URL of an XML file which describes the capabilities and specifications of the device. The developer programmatically retrieves the XML file, parses it and responds appropriately, typically caching the derived data somehow.

This approach is considered to be attractive, for reasons including:

• We don't need to do a lot of work up front. When an unfamiliar handset hits our site we simply retrieve the UAProf reactively. Theoretically, maintenance is minimal.
• UAProf data is supposed to be supplied by the handset manufacturer, thus ensuring it to be correct and up-to-date.

The reality is a little different. I'll loosely quote from Wikipedia here:

• Not all devices have UAProfs (including many new Windows Mobile devices, iDen handsets, or legacy handsets)
• Not all advertised UAProfs are available (about 20% of links supplied by handsets are dead or unavailable, according to figures from UAProfile.com)
• UAProf can contain schema or data errors which can cause parsing to fail
• Retrieving and parsing UAProfs in real-time is slow and can add substantial overhead to a web request
• There is no industry-wide data quality standard for the data within each field in a UAProf
• UAProf headers can often be plain wrong (i.e. for a completely different device)

Add to this that even the name of the HTTP header itself is not clearly defined. Typically, it is "x-wap-profile", but there are several alternatives in the wild right now.

In any case, there's little motivation for manufacturers to provide accurate data. To quote Michael Kaye on the wmlprogramming Yahoo tech group:

We cannot rely on the manufacturers to provide correct information: no company I know of would provide an accurate list of their bugs and misfeatures for the entire world to see.

Quite. All in all, UAProf is considered neither mature nor stable enough to be relied upon in production systems. Moreover, since UAProf is merely one of the sources of data integrated into WURFL in the first place, there's little benefit in developers reinventing the wheel by working with the UAProf mechanism directly.

Other Alternatives

There's a certain amount of talk about how nice it would be if there were an official, centralised, open "Device Description Repository". To this end, the W3C's Mobile Web Initiative has spawned a Device Description Working Group to continue talking - albeit in a more focused manner - about just how nice it would be.

There are commercial offerings too, many of which either layer a Web Services API over WURFL, or attempt to maintain their own database of mobile devices. Inexplicably, dotMobi themselves have hired one of WURFL's original developers and taken the first steps towards setting up their own competing device database.

There's also some activity around automated adaptive rendering technologies. For the .NET developer there's ASP.NET Mobile Controls, but I don't hear a lot of happy noises coming from that direction. For Java and - allegedly - PHP, there's WALL, a tag library somewhat based on WURFL. These may approach some of the minor issues around fragmentation of markup standards.

I have little to no experience with these so I won't be commenting (and I'd be interested to hear from developers with experience of any of the above). Suffice to say, your mileage may vary, and for the time being, WURFL remains a powerful tool in many cases.