Sunday, January 29, 2006

Palm OS Development Suite (PODS) in Anger

First post, I guess. I've decided to create a separate blog for my random technical musings.

Today, I wrassled with the Palm OS Developers Suite (PODS). Getting it installed was a real challenge, so here's the short version of what I learned today:

If you already have Cygwin installed:

  1. Make sure you already have all of Cygwin's relevant "devel" packages.
  2. Download PODS from PalmSource. (Create a free login.)
  3. Run the PODS installer, but don't let it install its own Cygwin.
  4. Download the Cygwin bz2 files from the PRC-Tools download page
  5. Create a dummy setup.ini file and put it in the same directory as the bz2 files
  6. Point Cygwin at your local directory to make it install the bz2s.
  7. Make sure Cygwin and the other Palm resources are added to your path
  8. Run "palmdev-prep" to inform it where you installed Garnet
  9. Launch Eclipse, create a 68k project with the sample application, and try to build it. It should build successfully.
  10. Create a new Debug configuration. (You can't just "Run As" or "Debug As" the way you normally would debug in Eclipse.) "Run -> Debug..." Click on "Palm OS Application" in the "Configurations" panel, and click the "New" button at the bottom there. Click on "Target" and select the "Palm OS Garnet Simulator 5.4" (not the "Palm OS Emulator"; they're different). Click Debug.
  11. You'll get a pop-up saying "The selected Target Environment is being launched. Press OK when it is done launching, or Cancel to stop the debug session." The Palm Simulator should load up, and ask you for a ROM file. For the Simulator (not the Emulator), the ROM files are in sdk-5r4\tools\Palm_OS_54_Simulator\debug. You can use the enUS or EFIGS (English/French/Italian/German/Spanish) ROM; I used "Simulator_Full_EFIGS_Debug.rom". The Simulator should start.
  12. Back in Eclipse, click "OK". Your application should be paused on its first line of code in the "PilotMain" function, ready for you to begin debugging it.

Installing Pre-Requisite Software for PODS

It used to be that compiling Palm software wasn't possible without shelling out big bucks for Metrowerks CodeWarrior. Times have changed, and it is now possible to compile Palm software using GCC and PRC-Tools. But configuring these tools on Windows is challenging to say the least. At a minimum, it requires Cygwin, the Linux-like environment for Windows, whose setup tool is notoriously finicky. Moreover, PRC-Tools doesn't come with the "base" Cygwin packages, and PRC-Tools' automated Cygwin-ified installer is busted.

So the good folks at PalmSource decided that it would be a good idea to package up a known-good version of Cygwin + GCC + PRC-Tools, along with the Eclipse C Development Toolkit (CDT), and make an installer out of it.

If you like, you can go grab PODS for free from PalmSource. You have to create a free login, but then you can download the installer and get to work.

All fine and dandy, but the version of Cygwin they packaged up was rather out of date. Worse, if you read the README instructions carefully, you'll find that you must not install their Cygwin on top of your existing Cygwin, lest they undergo Total Protonic Reversal.

NOTE: If you choose to install the Palm OS Developer Suite provided Cygwin, you should remove all previous Cygwin installations since Cygwin does not support multiple installations of that product to coexist on one machine. Palm OS Developer Suite installer modifies the Cygwin registry settings as part of the installation process and therefore may negatively affect any previous installations.

If you're like me and you already have Cygwin installed, you'll need to follow their readme and, using the Cygwin setup tool, carefully ensure that you have every package on the list that they demand. In my case, I already had all of them just by installing the basic "Devel" package for Cygwin, except for, of course, PRC-Tools.

I did mention that PRC-Tools' Cygwin installer is busted, didn't I? Well, it is. The PRC-Tools Installation Instructions for Windows/Cygwin suggest that you just point Cygwin at "http://prc-tools.sourceforge.net/install" and let 'er rip. Bad idea. Cygwin will actually use that path to look at this file: http://prc-tools.sourceforge.net/install/setup.ini. That, in turn, is supposed to tell Cygwin where to download all of the packages referenced in the setup.ini file.

But, of course, it can't. Go on, try browsing to http://prc-tools.sourceforge.net/install/prc-tools/prc-tools-2.3-cygwin.tar.bz2. It won't work, because that's not really a bz2 file. Instead, it's a redirect to a SourceForge.net HTML page, telling you about the site from which it will *really* do the download. Your browser can handle these circuitous redirects, but Cygwin's setup tool definitely can't.

Instead, I had to download the binaries myself from the PRC-Tools Download page, and then hacking up my own setup.ini, that looks more like this:

# Cygwin-setup.exe config file for prc-tools et al.
# DO NOT EDIT! This file was generated by gensetup.
#
setup-timestamp: 1090157492
setup-version: 2.427

@ prc-tools
sdesc: "GCC for Palm OS and related tools"
category: Base Devel
requires: cygwin
version: 2.3
install: prc-tools-2.3-cygwin.tar.bz2 4285677 fb7bef9542d3ce6957535fd8315fcf0f

@ prc-tools-arm
sdesc: "GCC for Palm OS on ARM (armlets)"
category: Devel
requires: cygwin prc-tools
version: 2.3
install: prc-tools-arm-2.3-cygwin.tar.bz2 4459228 3bc19ac45a9aaee9f790b67f63f8cf97

@ prc-tools-htmldocs
sdesc: "Documentation for prc-tools, GCC, etc as HTML"
category: Devel Doc
version: 2.3
install: prc-tools-htmldocs-2.3-cygwin.tar.bz2 854795 bfe8f60a69b5ccfa3d247567150d463a
I put that file, and all of the bz2 files, into one directory, and then used Cygwin to install everything from a "Local Directory". That finally got prc-tools installed, apparently successfully. (But wait, there's more.)


Selecting the Correct Project Type: 68k, PNO, Protein

Once I was done with that, I tried running the PODS installer to get PalmSource's old version of Eclipse (3.0, missing the cool CDT enhancements available in 3.1, sadly), and told it not to install their version of Cygwin. That seemed to work OK, until it finally came time to try building a sample Project.

I was attempting to follow the instructions from Professional Palm OS Programming, a book which seems like it has most of the information you want, even if it isn't laid out particularly well. (And, sadly, the almost total lack of information available on Google make this book an unfortunately good value.)

As you attempt to create a new Project in PODS, you are immediately given the choice between three types of Projects: "68k", "68k PNO (PACE Native Objects)", or "Protein". The difference here is that Palms used to run on Motorola 68k processors, but most of the modern ones now run on ARM. In order to ensure backwards compatibility, the ARM-based Palms emulate 68k processors, in what they call "Palm Application Compatibility Enviroment" or PACE . (68k used to be big-endian, ARM is little-endian, and PACE handles that sort of thing for you.) If you want to write a teensy bit of native code for the ARM, you can write PNOs, which are ARM-native objects that can be used by 68k applications. And if you want to go whole-hog and write your entire application ARM-native, that's called "Protein."

But there's a catch. (Isn't there always?) Even if you DID want to write your entire application in Protein (the latest and greatest), you wouldn't be able to run it on anything except a Palm device that runs Palm OS 6.0 "Cobalt". But, as of January 2006, there are no Cobalt devices available for sale, even though Cobalt was released in early 2004! PNO will still work for Palm OS 5.4 Garnet applications, if you insist on going ARM-native, but you'll still have to write your application in 68k and then link to your "real" code in PNO.

So, as of January 2006, it's still best to just write your application in 68k... forget about crazy native ARM stuff for now.

Compling 68k applications: Configuring palmdev-prep

OK, so, back to PODS. If you try to create a Protein Project, (let's say you decide to create the Sample Application, which is a MineHunt game,) it will build and debug just fine. But if, realizing that no one can run anything written in Protein, you decide to create a 68k Project and then try to build that, you'll find that it won't compile. Instead, you'll get an error from prc-tools, suggesting that you try running "palmdev-prep".

Early versions of prc-tools contained a hard-coded list of directories to be searched. Unfortunately, more recently there has been a tendency for each new Palm OS SDK to introduce new directories unpredictably and sometimes even to rearrange the old ones. Hence the hard-coded list approach is no longer practical; instead, you can use palmdev-prep to generate a list of directories tailored to the SDKs and other Palm OS development material you actually have installed.

The palmdev-prep utility scans the standard PalmDev directory and any extra directories listed on its command line, and generates the options required to make GCC search as appropriate each of the subdirectories found under each of the root directories given. The "standard PalmDev location" is determined when prc-tools is configured; typically on Unix it is `/opt/palmdev' and on Cygwin it is typically `/PalmDev', which typically corresponds to the Windows directory `C:\PalmDev'.

Pay attention here, kids, because unless you had 20/20 foresight, you probably didn't install PODS into C:\PalmDev. Instead, you probably installed it into the default location, "C:\Program Files\PalmSource\Palm OS Developer Suite".

That means, to get your 68k project to compile, you'll need to make a symlink, in Cygwin. Launch your Cygwin bash shell and then do this:

> cd /
> ln -s "C:/Program Files/PalmSource/Palm OS Developer Suite" PalmDev
> palmdev-prep
"C:/Program Files/PalmSource/Palm OS Developer Suite"

Even once I did this, I still got a warning saying: "palmdev-prep: warning: can't open '/PalmDev/sdk-5r4/include/Core/CoreTraps.h': No such file or directory", but it didn't appear to do any damage. At that point, my Palm 68k application compiled just fine.

Debugging your Application

Debugging Palm Applications is a bit awkward relative to debugging standard C applications or Java applications in Eclipse, because even though techinically you are creating what might be called an "application", the output of the compiler is a .prc file, not something that's actually executable on your operating system. The applications you actually use for debugging are the Simulators/Emulators that get installed with PODS.

Since you won't be using Cobalt until somebody finally gets around to creating a device for it, you'll probably mostly want to use the Garnet Simulator (Palm OS 5.4) or the Palm OS Emulator (POSE), which emulates devices up to Palm OS 4.1.2. To start up a Simulator/Emulator and use it, you'll need to select a ROM file... note that Simulator ROMs don't work with the Emulator, and vice versa. All of the ROMs you need should be included with PODS. POSE ROMs live in "PalmOSTools\Palm OS Emulator\ROMS"; Garnet ROMs live in "sdk-5r4\tools\Palm_OS_54_Simulator\debug". There are several Garnet ROMs, meant to support numerous languages. I tend to prefer the EFIGS ROM (English/French/Italian/German/Spanish), but you may prefer just the plain-vanilla enUS ROM. Regardless, always make sure you use the "Debug" ROMs instead of the "Release" ROMs, because they're the ones with the handy debugging facilities you'll use to debug your Palm application.

Conveniently, Eclipse/PODS can be configured to automatically launch the Simulator of your choice, though it is a little bit screwy if you're familar with Eclipse. Instead of clicking on "Run As..." you need to click on Run -> "Debug..." and click on "Palm OS Application", then finally click on the "New" button to create a new configuration. In the "Target" tab, you're allowed to pick your own Simulator/Emulator. (If you're clever, you can even use this to debug a real live running Palm plugged into your computer over a serial port (COM1/COM2) or over USB, but I haven't tried it yet.) I recommend the Garnet Simulator for now.

At that point, you should be able to click the "Debug" button on your configuration to launch the Garnet Simulator. Once it starts, the Garnet Simulator will open up a port (port 2000 by default) and wait for Eclipse to connect to it. Eclipse will give you a pop-up saying "The selected Target Environment is being launched. Press OK when it is done launching, or Cancel to stop the debug session." Once Garnet looks like it's good and started, click OK, and Eclipse should pause on your first line of code.

I had a very hard time getting this to work on my first try, though. Instead, it at first told me: "Launching (Error: Target request failed: Target is not responding (timed out).)" Apparently I'm not the only one to have this problem, but Google provided very little in the way of helpful workaround information. A big part of the problem is that all the magic is happening by opening ports. Anything could go wrong in this picture... a common thing that can happen is that another application may be holding the port, so PODS can no longer use it. Another common problem is that a software firewall like Symantec, McAfee or ZoneAlarm may be blocking port-to-port connections, or even the built-in firewall in Windows XP SP2. So let's consider some common tricks you can use to test this.

First, if you're having trouble, I highly recommend that you download a copy of Process Explorer from Sysinternals. (The download links are at the bottom of the page.) Process Explorer is a really awesome piece of software; I can't recommend it highly enough. In this case, you'll use it by looking for any PalmSim* processes that may be running. Make sure you start with no PalmSim* processes before you begin debugging, and that the right one starts up when you push Debug. Get the Properties of the PalmSim process by right-clicking on it. There, you should be able to see the command line which was used to start it, which may tell you something useful. You can also click on the TCP/IP tab on the Properties dialog to see what ports the process is listening on and/or using.

One problem I had was that Eclipse accidentally "orphaned" my Simulator, so I had one invisible copy running in the background, hogging the ports. Once I killed that, Eclipse worked fine. Another problem I had was a software firewall blocking the port. An easy way to test this is to try to telnet to the port in question. So if on the TCP/IP tab I saw that PalmSim was "LISTENING" on Local Address "mymachine:2000", I could try "telnet mymachine 2000". If it worked, I'd expect to see a blank screen. (Press Ctrl-] to pause telnet, then type "quit" to exit.) If it didn't work, I'd expect to see something like: "Could not open connection to the host, on port 59939: Connect failed". Note that you should expect this if you telnet to the wrong port.

If you're still uncertain as to what's going on, try rebooting, and make sure all software firewalls (including the Windows XP Firewall) are disabled. Good luck!