Exporting Data to iCal, Outlook, Google Calendar, etc.

Demo file: 2010-12-13-export to ical

The other day I needed to export some appointments from FileMaker to iCal. I’d never done this before, but I did a bit of reading on Wikipedia and elsewhere, and it turns out to be fairly straight forward. I don’t claim that what follows is in any way authoritative, just that it works… and not just with iCal, but with Outlook, Google Calendar and any other program that recognizes the iCalendar format.

Let’s start with a basic table of appointments, like this.

We’re going to create an “ics” file, which is a text file with three distinct elements. At the beginning of the file is the header, which looks like this:

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//FileMaker Pro//NONSGML yourSolutionNameHere//EN

Next we have one or more body entries that look like this:

BEGIN:VEVENT
SUMMARY:Marketing Meeting
UID:00188BD54F5D-63427112000-1000028
DTSTAMP:20101206T180000Z
DTSTART:20101206T180000Z
DTEND:20101206T193000Z
END:VEVENT

And at the very end of the file comes the footer, which looks like this:

END:VCALENDAR

Let’s take a closer look at the body section. The BEGIN, SUMMARY and END entries seem self-explanatory, but what’s going on with those “DT” entries? Well, as you’ve probably guessed, they’re timestamps. First we have the date formatted as YYYYMMDD, followed by a T, followed by the time in 24hour format, hhmmss, followed by a Z.

One important observation about the time portion: it is represented as UTC time, not local time. So the first row in our appointments table, with a starting time of 10:00 Am, will be converted to the UTC equivalent of 180000 when we export it, because the database is located in the Pacific time zone, which is 8 hours offset, i.e., UTC-08. During Daylight Savings Time, the offset value would be -7, but we’re on Standard Time now, so the correct value is -8.

Clearly DTSTART is the beginning time for the appointment, and DTEND is the ending time, but what’s DTSTAMP? That is the “creation time” of the appointment, and apparently is required. Since I don’t care about that, I’m just making it the same as DTSTART.

And finally we come to the UID entry, which is simply a unique serial number. OK, I lied. Ideally it’s some form of “UUID” which means that heroic measures have been taken to attempt to guarantee its absolute uniqueness. But as long as it’s unique within your calendar, you’ll be fine.

The UID is very important, because if you import the same appointments more than once, the calendar will be smart enough to update existing entries where there is a matching UID, rather than allowing them to appear twice. That’s right… if an appointment changes in your database, you can export it, and reimport it into your calendar without having to worry about duplicates.

So, let’s add a UID field (auto-enter, text) and a UTC offset field (global, number).

Now that we understand the design of a minimal “ics” file, we can define a calculation field, ical_basis, in our database to assemble and format the data accordingly. Reminder: the first thing we do when we define a calculation field is, of course, make sure the result type is correct: in this case we want the result type to be text. Also, let’s set the storage type to “unstored”.

This calculation is the heart of the whole operation. Its main purpose is to generate one body segment per appointment record. Additionally, it will create the header at the beginning of the first record, and the footer at the end of the last record with the help of the “rn” and “fc” variables.

Note how the conversion to UTC is handled:

1. transform a date and time into a timestamp
2. add 8 hours (8 * 3600 seconds)
3. break it into individual components (year, month, etc.)
4. pad the components with leading zeros if necessary
5. reassemble them as YYYYMMDDThhmmssZ

Let’s reduce our found set to three records, go into preview mode and see how that calc evaluates. Looks like we’re home free doesn’t it?

We simply export our found set of appointments to a text file with an “ics” extension and declare victory, right? Well, not quite. If we try that, here’s what we get:

…a text file with three lines, each a mile long. While inconvenient, this makes sense if you think about it. When you export to a text file, FileMaker uses ASCII 13 as the record delimiter, i.e, puts a hard return between the data from each record. To avoid confusion, any existing hard returns from within the source data are converted to a “vertical tab” character (ASCII 11).

But we’re not going to let a puny ASCII character defeat us, are we? (Yes, I know, these are Unicode, not ASCII, codes… but the values are the same either way, and it’s easier to refer to ASCII codes than Unicode “code points”.)

Broadly speaking, there are two different approaches we can take to solve this problem:

A. Generate the text file, and then post process the file using some sort of external technology.

B. Parse each row of ical_basis into a separate record, and then export those records.

Let’s look at option A first. There are various ways the post-processing can be accomplished, including AppleScript on the Mac, and VB Script or PowerShell on the PC, but it can also be done via a 3rd-party FileMaker plug-in, Troi File. (There may be other plug-ins that can accomplish this as well, but Troi File is the one I’m familiar with, having used it since 1998.)

What I like about the TroiFile method is, a) it works the same regardless of whether you’re on a Mac or PC, and b) how incredibly easy it is. One little command…

TrFile_Substitute("-IgnoreCase";$fileSpec;"";Char(11);Char(13))

…and no more pesky problem.

Method B involves creating a special table to facilitate the generation of the ics file. As with method A, there are any number of ways to accomplish this, but the one I want to look at here is Bruce Robertson’s Virtual List technique. It’s a highly versatile tool to add to your box of tricks, and it also happens to score a perfect 10 on the coolness scale.

Here’s a basic explanation of how Virtual List can be implemented to solve this particular problem, and don’t worry if it doesn’t make immediate sense. You don’t have to fully understand it to reap the benefits, and the act of implementing it in one of your solutions will help you understand it better.

1. Create a new table in your solution. Name the table VL_Utility, and don’t create any records yet.
2. Define a number field, serial_number, as an auto-enter serial number with an initial value of 1.
3. Define a calculated text field, virtual_list, as
   GetValue ($$virtual_list;serial_number)
   …and set the storage type to unstored.
4. Create “more records than you’ll ever need” in this table. If that sounds unhelpfully vague, why not start with 10,000? You can always add more later.
5. Back in your Appointment table, locate the found set you wish to export.
6. Using a looping script, walk the records and populate the $$virtual_list variable.
7. Define a variable, $vc, as ValueCount($$virtual_list) .
8. Locate records in VL_Utility where serial_number <= $vc, and make sure these records are unsorted.
9. Export these records to a text file with an “ics” extension, e.g., appointments.ics

The result looks like this and is ready to import into your calendar program.

For Google Calendar, you can import entries by clicking the “Add” button at the left of the calendar. On the Mac, you can import an “ics” file to iCal by double clicking it, and for Outlook choose Import and Export from the File menu.

Here’s what we see after importing.

Really we’ve just scratched the surface of what can be accomplished using the iCalendar format. There are many optional properties that can be included, for example “description” (a.k.a. note). If we want to include notes in our export, we can modify our ical_basis calculation as follows:

And here’s how an appointment with a note looks in iCal: