Exporting an Events Calendar to iCalendar and Google Calendar

In this section, I’ll show you how to use what you’ve learned so far to solve a specific problem. After you have used event aggregators such as Upcoming.yahoo.com and Eventful.com, you’ll get used to the idea of having a single (or at least a small number) of places to see all your events. ­iCalendar-­savvy calendars (such as Google Calendar, Apple iCal, and Microsoft Outlook 2007) have also become unifying interfaces by letting you subscribe to iCalendar feeds containing events that might be of interest to you. As extensive as Upcoming.yahoo.com, Eventful.com, and Google Calendar (which has been a marketplace of events by letting users author publicly available calendars) might be, there are still many sources of events that are not covered by such services. This section teaches you how to turn ­event-­related information toward destinations where you might like to see them.

Specifically, I will work through the following example: converting events listed under the Critic’s Choice section of UC Berkeley’s online event calendar (http://events.berkeley.edu) into two different formats:

I use this example to demonstrate how to use Python and PHP libraries to parse and write iCalendar feeds and to write to a Google calendar. I’ve chosen the UC Berkeley event calendar because it already has calendaring information in a structured form (XML and iCalendar), but as of the time of writing, it’s not quite in the configuration that I create here. You can generalize this example to the event calendars that you might be interested in, some with more structured information than others. Moreover, instead of writing to Google Calendar, you can use the techniques I showed earlier in the chapter to write the events to Upcoming.yahoo.com or Eventful.com.

The Source: UC Berkeley Event Calendars

The Critic’s Choice section of the UC Berkeley event calendar highlights some of the many events that happen on the campus:

http://events.berkeley.edu/

As documented here:

http://events.berkeley.edu/documentation/user/rss.html

the calendar provides feeds in three formats: RSS 2.0, a live_export XML format, and iCalendar. Of particular interest is that every event in the calendar, which is referenced by an event ID (for example, 3950), is accessible in a number of representations:

  • As HTML:

    http://events.berkeley.edu/?event_ID={event_ID}

  • As RSS 2.0:

    http://events.berkeley.edu/index.php/rss/sn/pubaff/?event_ID={event_ID}

  • As iCalendar:

    http://events.berkeley.edu/index.php/ical/event_ID/{event_ID}/.ics

  • As live_export XML:

    http://events.berkeley.edu/index.php/live_export/sn/pubaff/?event_ID={event_ID}

You can get feeds for many parts of the event calendar (including feeds for events for today, this week, or this month), but there is currently no Critic’s Choice iCalendar feed. Having such a feed would enable one to track Critic’s Choice events in Google Calendar or Apple iCal. The Critic’s Choice is, however, available as an RSS 2.0 feed here:

http://events.berkeley.edu/index.php/critics_choice_rss.html

The following two sections show you how to extract the event ID for each of the events listed as part of Critic’s Choice, read the iCalendar instance for an event to create a synthesized iCalendar feed, and write those events to Google Calendar.

Creating an iCalendar Feed of Critic’s Choice Using Python

The following code, written in Python, knits together the iCalendar entries for each of the Critic’s Choice events into a single iCalendar feed through the following steps:

  1. Parsing the list event_ID from here:

    http://events.berkeley.edu/index.php/critics_choice_rss.html

  2. Reading the individual iCalendar entries and adding it to the one for the Critic’s Choice

Note that this code treats iCalendar essentially as a black box. In the next section, we’ll parse data from iCalendar and rewrite it in a format demanded of Google Calendar:

            """
            generate iCalendar feed out of the UC Berkeley events calendar
            """
            
            import sys
            try:
                from xml.etree import ElementTree
            except:
                from elementtree import ElementTree
            
            import httplib2
            client = httplib2.Http(".cache")
            
            import vobject
            
            # a function to get individual iCalendar feeds for each event.
            # http://events.berkeley.edu/index.php/ical/event_ID/3950/.ics
            
            def retrieve_ical(event_id):
                ical_url = "http://events.berkeley.edu/index.php/ical/event_ID/%s/.ics" % (event_id)
                response, body = client.request(ical_url)
                return body
            
            # read the RSS 2.0 feed for the Critic's Choice
            
            from elementtree import ElementTree
            
            cc_RSS = "http://events.berkeley.edu/index.php/critics_choice_rss.html"
            response, xml = client.request(cc_RSS)
            doc = ElementTree.fromstring(xml)
            
            from pprint import pprint
            import urlparse
            
            # create a blank iCalendar
            ical = vobject.iCalendar()
            
            for item in doc.findall('.//item'):
                # extract the anchor to get the elementID
                # http://events.berkeley.edu/index.php/critics_choice.html#2875
                ev_url = item.find('link').text
                # grab the anchor of the URL, which is the event_ID
                event_id = urlparse.urlparse(ev_url)[5]
                print event_id
                s = retrieve_ical(event_id)
                try:
                    ev0 = vobject.readOne(s).vevent
                    ical.add(ev0)
                except:
                    print "problem in generating iCalendar for event # %s " % (event_id)
                    
            
            ical_fname = r'D:\Document\PersonalInfoRemixBook\examples\ch15\critics_choice.ics'
            f = open(ical_fname, "wb")
            f.write(ical.serialize())
            f.close()
            
            # upload my feed to the server
            # http://examples.mashupguide.net/ch15/critics_choice.ics
            
            import os
            os.popen('scp2 critics_choice.ics Â
            "rdhyee@pepsi.dreamhost.com:/home/rdhyee/examples.mashupguide.net/ch15')
         

By automatically running this script every day, whenever the RSS for the Critic’s Choice is regenerated, the resulting iCalendar feed will be kept ­up-­to-date:

http://examples.mashupguide.net/ch15/critics_choice.ics

Writing the Events to Google Calendar

In this section, instead of generating an iCalendar feed directly, I will instead write the events to Google Calendar using the PHP Zend Calendar API library. I created a new calendar for this purpose, whose user ID is as follows:

n7irauk3nns30fuku1anh43j5s@group.calendar.google.com

Hence, the public calendar is viewable here:

            http://www.google.com/calendar/embed?src=n7irauk3nns30fuku1anh43j5s@group.calendar.Â
            google.com
         

The following code loops through the events listed in the Critic’s Choice RSS feed, extracts all the corresponding iCalendar entries, and then writes those events to the Google Calendar. The code first clears out the old events in the calendar before writing new events.

Perhaps the trickiest part of this code is handling recurring events. The relevant documentation in the Google Calendar API on recurring events includes the following:

The Google Calendar API expresses recurrence using the syntax and data model of recurring events in iCalendar, which you can learn about in the following sections of the iCalendar specification (section 4.3.10 on RECUR, section 4.8.5.1 on EXDATE [exception dates/times], and section 4.8.5.4 on the Recurrence Rule):

More to the point, the following code captures information about recurring events by using regular expressions to extract occurrences of the DTSTART, DTEND, RRULE, RDATE, EXDATE, and EXRULE statements to pass to the Google Calendar API as recurrence data. (Remember to substitute your own Google username and password and the user ID for a Google Calendar for which you have write permission.)

            <?php
            
            /*
             *
             * ucb_critics_gcal.php
             */
            
            require_once 'Zend/Loader.php';
            Zend_Loader::loadClass('Zend_Gdata');
            Zend_Loader::loadClass('Zend_Gdata_ClientLogin');
            Zend_Loader::loadClass('Zend_Gdata_Calendar');
            
            require_once 'iCalcreator/iCalcreator.class.php';
            
            function getResource($url){
              $chandle = curl_init();
              curl_setopt($chandle, CURLOPT_URL, $url);
              curl_setopt($chandle, CURLOPT_RETURNTRANSFER, 1);
              $result = curl_exec($chandle);
              curl_close($chandle);
            
              return $result;
            }
            
            // UCB events calendar
            
            # gets all relevant rules for the first VEVENT in $ical_string
            function extract_recurrence($ical_string) {
            
              $vevent_rawstr = "/(?ims)BEGIN:VEVENT(.*)END:VEVENT/";
              preg_match($vevent_rawstr, $ical_string, $matches);
            
              $vevent_str = $matches[1];
            
              # now look for DTSTART, DTEND, RRULE, RDATE, EXDATE, and EXRULE
            
              $rep_tags = array('DTSTART', 'DTEND', 'RRULE', 'RDATE', 'EXDATE', 'EXRULE');
            
              $recur_list = array();
            
              foreach ($rep_tags as $rep) {
            
                $rep_regexp = "/({$rep}(.*))/i";
                if (preg_match_all($rep_regexp, $vevent_str, $rmatches)) {
                  foreach ($rmatches[0] as $match) {
                     $recur_list[]= $match;
                  }
                }
            
              } //foreach $rep
            
              return implode($recur_list,"\r\n");
            
            }
            
            function parse_UCB_Event($event_id) {
            
              $ical_url = "http://events.berkeley.edu/index.php/ical/event_ID/{$event_id}/.ics";
              $rsp = getResource($ical_url);
            
              # write out the file
              $tempfile = "temp.ics";
              $fh = fopen($tempfile,"wb");
              $numbytes = fwrite($fh, $rsp);
              fclose($fh);
            
              $v = new vcalendar(); // initiate new CALENDAR
              $v->parse($tempfile);
            
              # how to get to the prelude to the vevent? (timezone)
            
              #echo $v->getProperty("prodid");
            
              # get first vevent
              $comp = $v->getComponent("VEVENT");
            
              #print_r($comp);
            
              $event = array();
            
              $event["summary"] = $comp->getProperty("summary");
              $event["description"] = $comp->getProperty("description");
            
            # optional -- but once and only once if these elements are here:
            # dtstart, description,summary, url
            
              $dtstart = $comp->getProperty("dtstart", 1, TRUE);
              $event["dtstart"] = $dtstart;
            
            # assume that dtend is used and not duration
            
              $event["dtend"] = $comp->getProperty("dtend", 1, TRUE);
            
              $event["location"] = $comp->getProperty("location");
              $event["url"] = $comp->getProperty("url");
            
            # check for recurrence -- RRULE, RDATE, EXDATE, EXRULE
            
              $recurrence = extract_recurrence($rsp);
            
              $event_data = array();
              $event_data['event'] = $event;
              $event_data['recurrence'] = $recurrence;
              return $event_data;
            
            } // parse_calendar
            
            function extract_eventIDs($xml)
            {
            
             $ev_list = array();
            
             foreach ($xml->channel->item as $item) {
            
               $link = $item->link;
               $k = parse_url($link);
               $ev_list[] = $k['fragment'];
             }
             return $ev_list;
            }
            
            // Google Calendar facade
            
            function getClientLoginHttpClient($user, $pass)
            {
              $service = Zend_Gdata_Calendar::AUTH_SERVICE_NAME;
            
              $client = Zend_Gdata_ClientLogin::getHttpClient($user, $pass, $service);
              return $client;
            }
            
            // code adapted from the Google documentation
            // this posts to the DEFAULT calendar -- how do I change to post elsewhere?
            
            function createGCalEvent ($client, $title, $desc, $where, $startDate = '2008-01-20', 
                 $startTime = '10:00:00',
                 $endDate = '2008-01-20', $endTime = '11:00:00', $tzOffset = '-08', 
                 $recurrence=null, $calendar_uri=null)
            {
              $gdataCal = new Zend_Gdata_Calendar($client);
              $newEvent = $gdataCal->newEventEntry();
            
              $newEvent->title = $gdataCal->newTitle($title);
              $newEvent->where = array($gdataCal->newWhere($where));
              $newEvent->content = $gdataCal->newContent("$desc");
            
            # if $recurrence is not null then set recurrence -- else set the start and enddate:
            
              if ($recurrence) {
                $newEvent->recurrence = $gdataCal->newRecurrence($recurrence);
              } else {
                $when = $gdataCal->newWhen();
                $when->startTime = "{$startDate}T{$startTime}{$tzOffset}:00";
                $when->endTime = "{$endDate}T{$endTime}{$tzOffset}:00";
                $newEvent->when = array($when);
              } //if recurrence
            
            // Upload the event to the calendar server
            // A copy of the event as it is recorded on the server is returned
            
                $createdEvent = $gdataCal->insertEvent($newEvent,$calendar_uri);
                return $createdEvent;
            }
            
            function listEventsForCalendar($client,$calendar_uri=null) {
            
              $gdataCal = new Zend_Gdata_Calendar($client);
            
              $eventFeed = $gdataCal->getCalendarEventFeed($calendar_uri);
              foreach ($eventFeed as $event) {
                echo $event->title->text, "\t", $event->id->text, "\n";
                foreach ($event->when as $when) {
                  echo "Starts: " . $when->startTime . "\n";
                }
              }
              echo "\n";
            }
            
            function clearAllEventsForCalendar($client, $calendar_uri=null) {
            
              $gdataCal = new Zend_Gdata_Calendar($client);
            
              $eventFeed = $gdataCal->getCalendarEventFeed($calendar_uri);
              foreach ($eventFeed as $event) {
                $event->delete();
              }
            
            }
            
            // bridge between UCB events calendar and GCal
            
            function postUCBEventToGCal($client,$event_id, $calendar_uri=null) {
            
              $event_data = parse_UCB_Event($event_id);
            
              $event = $event_data['event'];
              $recurrence = $event_data['recurrence'];
            
              #print_r($event);
              #echo $recurrence;
            
              $title = $event["summary"];
              $description = $event["description"];
              $where = $event["location"];
            
            # there is a possible parameter that might have TZ info. Ignore for now.
              $dtstart = $event["dtstart"]["value"];
              $startDate = "{$dtstart["year"]}-{$dtstart["month"]}-{$dtstart["day"]}";
              $startTime = "{$dtstart["hour"]}:{$dtstart["min"]}:{$dtstart["sec"]}";
            
            # there is a possible parameter that might have TZ info. Ignore for now.
              $dtend = $event["dtend"]["value"];
              $endDate = "{$dtend["year"]}-{$dtend["month"]}-{$dtend["day"]}";
              $endTime = "{$dtend["hour"]}:{$dtend["min"]}:{$dtend["sec"]}";
            
              # explicitly set for now instead of calculating.
              $tzOffset = '-07';
            
              # I might want to do something with the url
              $description .= "\n" . $event["url"];
            
              echo "Event: ", $title,$description, $where, $startDate, $startTime, $endDate, 
                   $endTime, $tzOffset, $recurrence, "\n";
            
              $new_event = createGCalEvent($client,$title,$description, $where, $startDate, 
                   $startTime, $endDate, $endTime, $tzOffset,$recurrence, $calendar_uri);
            
            }
            
            # credentials for Google calendar
            
            $USER = "[USER]";
            $PASSWORD = "[PASSWORD]";
            
            # the calendar to write to has a userID of 
            # n7irauk3nns30fuku1anh43j5s@group.calendar.google.com
            # substitute the userID of your own calendar
            $userID = urlencode("[USERID]");
            $calendar_uri = "http://www.google.com/calendar/feeds/{$userID}/private/full";
            
            $client = getClientLoginHttpClient($USER, $PASSWORD);
            
            # get UCB events list
            
            $cc_RSS = "http://events.berkeley.edu/index.php/critics_choice_rss.html";
            $rsp = getResource($cc_RSS);
            
            # for now, read the cached file
            #$fname = "D:\Document\PersonalInfoRemixBook\examples\ch15\cc_RSS.xml";
            #$fh = fopen($fname, "r");
            
            #$rsp = fread($fh, filesize($fname));
            #fclose($fh);
            
            $xml = simplexml_load_string($rsp);
            $ev_list = extract_eventIDs($xml);
            
            echo "list of events to add:";
            print_r($ev_list);
            
            # loop through events list
            
            # limit the number of events to do
            $maxevent = 200;
            $count = 0;
            
            # clear the existing calendar
            
            echo "Deleting existing events....";
            clearAllEventsForCalendar($client,$calendar_uri);
            
            # Add the events
            foreach ($ev_list as $event_id) {
            
              $count +=1;
              if ($count > $maxevent) {
                break;
              }
              echo "Adding event: {$event_id}", "\n";
              postUCBEventToGCal($client,$event_id,$calendar_uri);
            
            }
            
            # list the events on the calendar
            listEventsForCalendar($client,$calendar_uri);
            ?>