Travis Roth’s Weblog

BlackBerry users have long had the bragging rights of push email – the ability to receive new email notifications on their mobile device nearly in real-time wherever they were in range of a signal. Not everyone has BlackBerry though, but would like push email. It has been possible to have push email on Windows Mobile if you had access to an Exchange server, but many of us do not have this either. This left ActiveSync polling, which could be set to poll as often as 5 minutes, but hey this isn’t quite real-time! For more on push email see this Wikipedia article.

My Requirements

I needed to add email syncing to my Windows Mobile device in terms of my current email solution as I did not want to change email providers, nor get a new email address. And I did not want to have two different mailboxes with duplicate copies of my mail. With the latter requirement in mind this ruled out the possibility of forwarding mail to Windows Live Hotmail or using a Mail2web hosted Exchange account. My current email setup uses Google Apps to host email for my won domain—essentially Gmail and my own domain name. I use IMAP to connect Microsoft Outlook and it is also possible to use IMAP to link Windows Mobile and keep data synchronized, just not in real-time on the Windows Mobile device as discussed above.

Possible Solutions

As I researched alternative solutions to getting push email onto my Windows Mobile device without purchasing an Exchange server or using a hosted Exchange service I came across two developments, one new and one that has been around a few years but hasn’t gotten much press. The first solution I looked at is the open source Z-push by Zarafa.

Z-Push Open Source

Z-Push claims to be an open source implementation of the ActiveSync protocol, which is built into Windows Mobile device (and others) for syncing data including email, contacts and calendars. It is used in the Zarafa mail server, but has been separated into a stand alone package. Seeing it was open source and I could run it on almost any web server that supported PHP I was interested. According to the install guide it was possible to have Z-Push monitor an IMAP server connection and push changes to a mobile device. This could be perfect! I have an IMAP server courtesy of Google Apps after all…

A quick Google search indicated that a few others had came up with this idea. For example the forum post “Push email no forwarding required” explained that the author had it working. A few other posts also indicated that it worked including with the Gmail instead of Zarafa as the backend.

I threw up a Debian 5.0 server on my Linode VPS, installed Apache and PHP 5 and then set about testing Z-Push. The current stable release was Z-Push 1.2.2 and the latest beta was 1.3beta2. I opted to try the 1.2.2 version first as I hoped to avoid any beta bugs. Installation is not difficult as long as you are used to installing open source packages like this under Linux with some editing of PHP settings and configuring Apache. All is explained in the install guide and is not covered here. I edited Z-Push’s config.php file to use IMAP as the backend instead of Zarafa, also explained briefly in the install guide and in config.php. (I will note here as I was doing a test, I did not set up an SSL connection. For using this in production a SSL connection is recommended between Z-Push and the ActiveSync compatible device.)

This is when I encountered my first problem. Using the stock Debian 5 installation, with Apache and PHP5 packages installed using apt-get install, Z-Push would not run. After some investigation I found that a “canary mismatch on efree() - heap overflow detected” error was being thrown as shown in the Apache error log (/var/logs/apache2/error.log). This turns out to be caused by the suhosin patch compiled into the default Debian PHP 5 package (and it is included in other flavors as well). This seemed a strange error to be getting, and I did not find any useful resolutions to the problem on the Z-Push forums. Short of removing the suhosin patch as the post “Fixing Mismatched Canaries – How to Remove suhosin from Debian/Ubuntu Packages” describes, which seemed like a lot of work and possibly cause even more issues in the process, this seemed like a no-go.

I uninstalled Z-Push 1.2.2 (by deleting the z-push directory under the web server root) and installed Z-Push 1.3beta2. Upon launching this setup, the canary mismatch was resolved and there were no more errors in Apache’s error log. This was a positive step. (It is interesting that the Z-Push change log does not mention any fixes for this between 1.2.2 and 1.3beta2 that I could find, so I do not know what was causing the issue or what resolved it code-wise.)

Testing it further I opened the URL in the web browser and completed the authentication dialog using my Google Apps credentials. The browser reported that I was accessing it with an unsupported device, which was correct and passed the test that the install guide recommended.

Next it was time to try ActiveSync on Windows Mobile. For this I used my HTC 6800 running Windows Mobile 6.1. Using the New Account wizard on the Smartphone I configured an Exchange account, using my Z-Push server as the server and setting the login credentials for the Google Apps account. I set ActiveSync to sync email only, as other data types are not supported for this setup with Z-Push. Then I tried to sync.

The Smartphone tried to sync, then reported there was an error and to try later. No matter the cause this is the only error I got—not real useful. Back to the server again. I set Z-Push to keep a log of activity and tried the sync again. The log showed that the session was initiated, then Z-Push communicated successfully with the IMAP server, reporting there were 20 folder changes. Then the IMAP session was closed, and the log ended. No errors were shown. Also I found no new errors in the Apache error log. Yet, there was no email on my phone and the phone reported the sync operation failed.

TO ensure that it was not my Smartphone and/or Windows Mobile 6.1 I decided to try another account. For this I set up a trial account at Mail2web which is a hosted Microsoft Exchange provider, presumably using the full flavored Exchange server. I configured the Smartphone to sync email with Mail2web, and all went fine. The sync happened. Push email happened, when I sent the Mail2web account an email it showed up on the phone in just a few moments. Clearly, the Smartphone and ActiveSync were capable of syncing. Yet I could not get this syncing to occur with Z-Push.

And this is where the Z-Push project stands at present. Others claim to have it working. It looks like it should work. And it does not. I am hopeful to find a solution at some point in the future, perhaps an update to Z-Push, or there is some setting in Z-Push, Apache, or Windows Mobile that needs to be tweaked and I just have not found it?

New NuevaSync Service

NuevaSync is a relatively new sync service which offers calendar and contact syncing with Google over the air for free with ActiveSync compatible devices, including Windows Mobile and the iPhone. Recently NuevaSync added email syncing with IMAP servers including Gmail. When I read this it sounded very similar to what I was trying to do with Z-Push. And an added benefit is it can sync calendars and contacts. This was not a priority for me, but more features are better right?… While the calendar. Contact syncing is free; e-mail is a premium service currently costing $25 USD/year.

Following the advice on NuevaSync’s site, I first set up a free account to sync contacts with Google Contacts. Ensuring this works is recommended before plunking down $25 for the premium service. This will save you the hassle of trying to get a refund—which appears may not be possible. Since you can try before you buy of sorts this should not be a hang-up. The contact sync did work for me, although it did seem slow to update. This may be due to Google Contacts, or perhaps the free service is gaining popularity. Also most of the time it probably isn’t’ necessary to have instant contact updates, so this did not cause me much concern after I was able to determine that the sync connection did work.

I then upgraded to the premium email service and set it to retrieve my mail from Google Apps. I told ActiveSync to start syncing mail, and to schedule to receive items “as they arrive.” And… It worked! I am currently receiving mail on my Smartphone in near real-time, they often arrive in the same minute that they are reported being sent. Success!

It is nice to have found a solution. I would still like to figure out what is going wrong with Z-Push, in the meantime NuevaSync is providing a nice stand-in option.

ActiveSync and MobileSpeak

A brief note on configuring ActiveSync using MobileSpeak Pocket 2.1.212. Using the new account wizard, and using the Server configuration screens in ActiveSync, worked fine with MobileSpeak 2.1.212 Pocket PC edition, except for the last screen of configuring which items to sync. After an Exchange partnership is set up, the items to sync screen can also be found under Options in ActiveSync. The dialog is a tree of items to sync, and each item can be checked and unchecked. MobileSpeak was not announcing the state of the checkboxes, for either state. This made it difficult to tell what was to be synced and not. In my case it appears that by default the items are unchecked when a new Exchange partnership is created. However, I do not know if this is always the case, or if different phone models will behave differently. It is possible to use Windows Mobile Device Center to check and uncheck items, so there is a workaround…