Why don’t calendars do Time Zones?

Jim Phelps asks (2006-01-18) "Why don’t calendars do Time Zones?"

He laments "My laptop and my palm both understand Time Zones. I understand Time Zones. Why don’t my calendar applications (Oracle’s Calendar and the Palm Calendar in the handheld) understand Time Zones?"

Jim has the following "simple" requests:

1. When I create an appointment I should be able to mark the Time Zone for the appointment.


2. I should also be able to make appointments that are Time Zone neutral - not tied to a Time Zone.


3. My clients (Palm and Desktop and Web) should all understand Time Zones and should be able to shift the alarms to compensate for my changes in Time Zone.

Here's how Jim would use such functionality, were it only available:

• If I get an invite to join a conference call at 11AM EST, I could just enter that time without adjusting for my local time zone. Same with UTC time.


• If I changed time zones (say fly from Madison to San Francisco), my alarms would change to be appropriate. As it is now, if I fly to California, I need to keep track of the fact that the 10AM meeting is really a[t] 10AM Central time so I need to set an alarm 2 hours earlier. But I can’t do a global change because some things might be dinners in California so they need to stay on Pacific time.

Of course, Jim's not the only one whose Use Cases aren't well satisfied by most current applications and/or date time libraries. For example, a few years ago, some date/time Use Cases were posted on Zope.org, because Zope's date/time functionality was seen as unsatisfactory. And on BoingBoing, there was the rant "Extended iCal rant from a timezone warrior," whose basic complaint resembles Jim's. There was also recently a discussion on Bugzilla where some of the same issues and concerns were raised.

Here are the key requirements that can be abstracted from the aforementioned discussions/rants:

• It must be possible to associate each point-in-time value with a time zone that may or may not be the same as the time zone associated with other point-in-time values.
• It must be possible to specify a point-in-time without reference or association to any particular time zone. (Examples: The point-in-time that specifies the first moment of 2008 (for any and all time zones,) the annual date on which a person's birthday is celebrated, the list of non-business days of the US Federal government, the date on which a person must have been born in order to legally purchase alcohol today.)
• Point-in-time values must either know, or be able to derive, both their local time and their Universal Time.
• It must be possible to convert point-in-time values from one time zone to another.
• The same point-in-time value must be able to act as though it is associated with ("local to") a different time zone in different contexts (e.g., the possibly-different local time zone of different users at the same time, and also the local time zone of the same user at different times, even when the user's local time zone changes as he travels from one location to another.)
• Point-in-time values should provide both sufficient sub-second resolution and sufficient range (from earliest to latest representable value.)
• Point-in-time values should provide appropriate granularities--timestamps should span a reasonably small sub-second duration, but dates should span a calendar day (and not be sub-second-resolution timestamps that happen to specify the first moment of the day.)
• Point-in-time values must be immutable so that changes to their value don't invalidate the logic that uses them (e.g., they must be useable as dictionary keys.)
• Adding any combination of some number of days, months and years to a point-in-time value should result in a new point-in-time value that number of days/months/years distant--but having the same civil time of day, regardless of any DST transitions that may have occurred.
• It must be possible specify partial/recurring dates and times (e.g., "every/any day at 5pm," "the last Sunday of October every/any year.")

I was especially interested in what one of the Zope developers had to say in response to the Zope Use Cases posted by the general public (spelling and punctuation as in the original):

"I find all of that very allarming.

Do people want the datetime object solve all this use cases out of the box? I hope that they are only samples of applications and that people agree that they will have to do a little work by theimself.

It is already difficult to find a straightforward interface for basic operations, if we wait to have an universal datetime type, we will never have it before Python 48.12b1 release.

So please, go for a minimal API, and let people play with the type and discover the better way to do elaborated stuff.

By the way from the tz database README file at: http://www.twinsun.com/tz/tz-link.htm you can't guess a time zone from a time-zone offset and a daylight saving flag.

So I think that a full and cross-platform timezone support is just a wonderful dream. But if you can prove me that I'm false, with an actual implementation ..."

I'm quite sure that many programmers would have a similar reaction: They'd be highly alarmed. Well, tough. People don't care how you solve their problems, they just want them solved. A solution that doesn't work right can be worse (or no better than) not having any solution at all.

We've had "a minimal API" for date/time computations available in just about every programming language and operating system for decades now--but the requests, complaints and rants shown above are from recent times, not from decades past. It's almost universally true that programming languages, operating systems and applications either don't handle dates/times at all, or don't do so correctly and consistently, and don't satisfy most of the requirements discussed above.

"Do the simplest thing that could possibly work" is not a license to provide incomplete and/or incorrect functionality. Real use cases that are commonly encountered are not being satisfied. The first step to a cure is to admit there's a problem.

Dates and times are not simple. Not at all. They're a relatively hard problem. The truth is, dates and times are too hard for most programmers to handle correctly on their own--which is precisely why they don't do it, and instead attempt to justify their failure by misapplying design principles. And that's why the "provide the simple stuff, and let the application coders solve the hard problems" approach won't work. It doesn't work in other domains, either--not in the past, not now, and probably not ever.

The "let them do it themselves" approach is about as wise as letting each application programmer code his own number classes, his own collection classes, his own filesystem, his own garbage collection engine, his own virtual memory subsystem, his own graphics library and GUI framework, his own web/application server--and in his spare time, develop his own programming language and compiler. If you really feel that a date/time library that actually gets the job done would be just too complex, then to be consistent, you would have to raise the same objection to all the other tools that programmers don't do themselves anymore (because the problems are hard, and most programmers will take unwise shortcuts, or make serious errors, if they have to do it themselves.)

As time has gone by, we've gone from machine language, to assembly language, to procedural languages, to functional languages, to object-oriented languages. We've gone from no operating system, to reusable, statically-linked code libraries, to dynamically-linked code libraries, to run time systems, to basic operating systems, to operating systems with virtual memory, to operating systems with GUIs. We've gone from punched cards to Microsoft Word, and from the abacus to Microsoft Excel.

Does anyone see a pattern here? Yes, you have to walk before you can run. But we've been crawling around trying to avoid actually dealing with dates and times in all their glory for far too long now. It's time to advance to the next level. So, to the programmers of the world, I say: Admit it. Admit that you're not satisfying the real date/time use cases.

As for an actual implementation of a date/time library that satisfies the requirements we've been discussing, it happens that there is at least one: The Chronos Date/Time Library. Of course, it's a date/time library, not a calendaring/scheduling application, so it would be more accurate to say it makes it much easier to write applications with just about all of the requested features. Yes, applications do need to take some responsibility for their own particular use cases.

What are the features of Chronos that satisfy (or greatly help to satisfy) the requirements discussed above? They are as follows:

1. Chronos uses the Olson Time Zone Database, so it knows about all the world's time zones--including historical information. For example, Chronos knows, and is able to correctly handle, all the following facts:
   • Most (but not all!) locations in North America have been switching from standard time to daylight saving time on the first Sunday of April each year, from 1987 through 2006--but starting in 2007, most North American locations will switch from standard time to daylight saving time on the second Sunday of March.
   • All the countries in the European Union modernly transition to and from daylight saving time at the same moment in Universal Time--unlike the case in North America, where all zones with the same offset transition to/from daylight saving time at the same local time, but at a different moment in Universal Time.
   • All of India uses the same time zone offset, does not observe daylight saving time, and uses an offset of 5.5 hours east of Universal Time.
   • In Australia, South America, New Zealand and Africa, daylight saving time, if it is observed at all, is observed during the Southern Hemisphere's summer (for example, from September to April.)
   • During World War II, several countries were on "daylight saving time" all year round for several years in a row (in the US, this was called "War Time," and was abbreviated as "EWT," "CWT," "MWT" and "PWT.")
   • For the years 1921 and 1922 only, Moscow started the year with an offset of 3 hours, transitioned to an offset of 4 hours on 1920-02-24T23:00 and 1921-02-24T23:00, transitioned to an offset of 5 hours on the 79th day of the year (different dates and day-of-the-week each year,) transitioned back to an offset of 4 hours on the 244th day of the year, and then transitioned back to an offset of 3 hours on the 274th day of the year (so there were 4 offset transitions and 3 different offsets each year.)


2. In addition to the predefined time zones that come from the information provided by the Olson Time Zone Database, Chronos permits you to define your own time zones--which can be as simple as "-5 hours asTimezone" or as complex as any of the cases described in the preceding bullet item. Or you can easily construct a Chronos time zone from either a POSIX time zone rule literal or from the information in the Windows registry--and it will behave identically to its UNIX or Windows counterpart.


3. The Chronos Time Zone Repository (which is generated from the Olson Time Zone Database) is deployed separately and independently from the Chronos codebase. Either can be changed without any need to redeploy the other--and even already-running applications can see and use a newly-deployed version of the time zone repository. And both Chronos and the Chronos Time Zone Repository are available for Windows, Mac, Unix and Linux--so your applications will not only run unchanged in all those environments, but will exhibit identical date/time behavior.


4. Chronos point-in-time values can be associated with ("bound to" in Chronos terminology) a particular time zone. Or, they can be bound to a proxy time zone that dynamically refers to a different time zone at different times or in different contexts (so that the associated point-in-time's local time changes dynamically whenever the proxy time zone is changed to have a different time zone as its referent.) Or, Chronos point-in-time values can be bound to a special time zone that Chronos refers to as "nominal time." Nominal time represents the concept of "any time zone," or "local time in some unspecified context, or in any and all contexts simultaneously."


5. Chronos point-in-time values that are bound to a specific time zone or to a proxy time zone (and so aren't bound to nominal time) are said to be invariant to Universal Time. Chronos point-in-time values that are bound to nominal time are said to be invariant to nominal time. Point-in-time values that are invariant to Universal Time keep the moment in time that they represent in Unversal Time as their semantic invariant. Point-in-time values that are invariant to nominal time keep the moment in time that they represent in nominal time (their "local time") as their semantic invariant. In other words, two points-in-time that are invariant to Universal Time are equal when they represent the same moment in Universal Time, regardless of the moment they nominally specify in their local time zone (the one to which they are bound.) But a point-in-time that is invariant to nominal time is equal to any other point-in-time that specifies the same nominal (local) time. So the following expressions both evaluate to true:

   • (Timepoint 
         year: 2007 month: 2 day: 3 
         hour: 10 minute: 0 second: 0 
         timeZone: -8 hours)
     = (Timepoint 
           year: 2007 month: 2 day: 3 
           hour: 13 minute: 0 second: 0 
           timeZone: -5 hours)
     "10 am in California is the same moment 
     in Universal Time as is 1pm in New York"

   • (Timepoint 
         year: 2007 month: 2 day: 3 
         hour: 13 minute: 0 second: 0 
         timeZone: Timezone nominal)
     = (Timepoint 
           year: 2007 month: 2 day: 3 
           hour: 13 minute: 0 second: 0 
           timeZone: -5 hours)
     "1 pm nominal time is 1pm local time, 
     regardless of time zone"



6. Chronos point-in-time values that are invariant to Universal-Time internally store their value in Universal Time, but report their year, month, day, hour, minute and second according to the equivalent local time in the time zone with which they are associated. Also, the local time values for their year, month and day-of-month are cached (lazily computed when needed--it's an expensive computation.)


7. Chronos point-in-time values can easily (and efficiently) be converted from any time zone to any other:
   

   (Timepoint 
       year: 2007 month: 2 day: 3 
       hour: 13 minute: 0 second: 0 
       timeZone: 'Pacific/Honolulu') >> #'Asia/Katmandu'

   results in 2007-02-04T04:45:00+05:45 (and yes, Asia/Katmandu actually does have an offset of 5 hours 45 minutes.)


8. Chronos point-in-time values have no limits (other than available memory on your computer) with respect to the range of representable dates--a feature that derives mostly from the fact that Chronos is implemened in Smalltalk--although, even if the implementation language had limited the design to using nothing but 32-bit signed integers, the way Chronos internally represents point-in-time values would still have supported a range of +/- 6 million years for timestamps (and +/- 2 billion years for dates.) Chronos also provides a special value that represents the infinite past, and another that represents the infinite future.


9. Chronos point-in-time values are actually time intervals, as explained in the Chronos blog entry Chronos 101: "Points" in Time. Chronos points-in-time that are meant to represent dates have a temporal extent (and hence resolution) of one calendar day. Chronos points-in-time that are meant to be used as timestamps have a temporal extent (and hence resolution) of one nanosecond. In addition to these two common cases (one optimized for dates, the other for timestamps,) Chronos provides general Timeperiod values, which can start at any point in time, and can have any temporal extent/duration.


10. Chronos point-in-time, durational and time interval values are all immutable.


11. Chronos handles date/time arithmetic correctly, satisfying both civil and scientific/technical use cases. Adding a day results in the same civil time of day on the next calendar day.


12. Chronos provides a variety of classes for representing partial/recurring dates--including the time-of-day without any associated date, and the month/day without any specified year. You can even construct a value that specifies "the first Tuesday of November, every fourth year modulo 4."

And there's yet more. If you're interested, you can peruse the Chronos web site, browse the Chronos blog, or read the "Chronos 101" tutorials, of which the following have been published so far:

• Chronos 101: "Points" in Time


• Chronos 101: Durational Values


• Chronos 101: Intervals of Time

So, to answer Jim's original question, calendar applications don't understand time zones because most people, including software developers, programming language library creators and programming language designers don't understand time zones. Or in some cases, they understand them well enough, but decide (for whatever reason) that full and correct time zone behavior is more work than they want to take on. Fortunately, that unwillingness to tackle the date/time domain comprehensively is not universal.

---

• Chronos blog home page
• Chronos Date/Time Library web site

Static typing is so last century

In the article Schizoid Classes published on the ACM Queue, Rodney Bates (of Wichita State University) does a good job of explaining what's wrong with the class models of the curly-braced mainstream langauges.

However, in his last paragraph, he says:

"Smalltalk pays a high price elsewhere for taking object orientation to the extreme, notably in complete loss of static typing and serious runtime efficiency penalties. Special, one-instance forms of classes are, for many programming problems, not as good a conceptual match as modules. But at least it provides a single, consistent, and syntactically explicit call mechanism."

He's right to say that "special, one-instance forms of classes are, for many programming problems, not as good a conceptual match as modules." But he's demonstrably wrong to say that "Smalltalk pays a high price elsewhere for taking object orientation to the extreme, notably in complete loss of static typing and serious runtime efficiency penalties."

Firstly, it is only sometimes true that Smalltalk pays a measurable performance penalty due to the lack of primitive types. In fact, it is very often the case that Smalltalk programs benchmark as faster than equivalent programs written in C++, C# or Java--for any of a variety of reasons I won't go into here.

Secondly, Smalltalk pays no net penalty for its lack of static typing--commonly-believed myths to the contrary notwithstanding. In fact, it receives the net benefit of making developers more productive and making code more generic and reusable.

Static typing is so last century. Interest in dynamic languages is growing. A Kuhnian revolution may be brewing. I sense that it won't be long now before the general community of programmers wakes up to reality, and stops believing the myths of the past.

---

• Chronos blog home page
• Chronos Date/Time Library web site

Taking Exception To Smalltalk Exceptions

About 3 weeks ago, Cincom's James Robertson posted a blog entry that showcased Smalltalk's exception handling capabilities.

About three days later, some posters at Lambda The Ultimate took exception to the power of Smalltalk's exception handling capabilities [fixed URL]:

"So what you're saying is that you want to be able to bind exception throw behaviour dynamically, rather than lexically. The end result being that a caller can bind handler code into any (uncaught) exception throw, without having any way of knowing the source(s) of the possible exceptions, or their root cause.

How, exactly, is one supposed to document the exceptional behaviour of a library class in that case? What possible security gaurantees could one give, if a calling library can inject arbitrary code on any error condition, and ignore or mask the error as it will? How would you even exhaustively test such a thing, other than by ignoring the horrible stuff that your caller could have done to you and pretending you're in some some language?" -- Dave Griffith [I assume that "some some language" was intended to be "some sane language."]

I have news for Mr. Griffith: Programs Are Data. That truth is one of the pillars of computer science. Deal with it.

If programmers can't be trusted with the stack, they can't be trusted to assign the right value to a variable, nor to pass the right value as a parameter to a function. Compilers are tools, not hall monitors.

---

• Chronos blog home page
• Chronos Date/Time Library web site

Time Semantics: Getting it wrong is easy

Outlook, Appointments and Time Zones:

"To many people, Outlook really seems really bad at handling time zone changes, and that's being kind. This is in part because people don't understand that Outlook uses UTC time for appointments and adjusts the time using the time zone offset configured on the computer."

Translation: You're stupid. Learn to think like your computer!

"Note that this is not "an Outlook problem", as this is how computers, email clients and mail servers handle time zones."

Translation: Other programs work just like Outlook, and the programmers can't all be wrong. [Or can they?]

"Outlook does not support an absolute time option for the calendar, which would permit you to enter 2 PM and the appointment would always stay on 2 PM, no matter how many times you changed the time zone. It also doesn't have an option for ‘in what time zone?’ so that you could make an appointment for 2 PM and select Pacific time zone and it would show up as 5 PM in your calendar when the computer is using the Eastern time zone."

Translation: Ok, we fess up: The problem really does lie in the fact that Outlook uses the wrong time semantics.

The Moral of the Story: Even Billion-Dollar Corporations can get time semantics wrong.

Sometimes, you want timestamps that are invariant to nominal time, so that 3pm = 3pm, with the time not bound to any time zone at all. Other times, you want timestamps that are invariant to Universal Time--so that 2pm in San Francicso will be equal to 5pm in New York.

Sometimes, you want timestamps statically bound to a specified time zone, so that 2pm in San Franciso always is displayed as 2pm (even though the computer treats it as equal to 5pm in New York.) Other times, you want timestamps whose "local time" changes dynamically to match whatever time zone the user specifies as the "local time" of the computer--so that the time displays as 2pm when you've set the computer's time zone to San Francisco time, but displays as 5pm when you've set the computer's time zone to New York time.

Unfortunately, not only do most applications get this wrong, most date/time libraries don't support both nominal-time invariance and Unviersal Time invariance, nor do they support both timestamps statically-bound to a specified time zone and also timestamps bound dynamically to the current system time zone.

Chronos does it all.

---

• Chronos blog home page
• Chronos Date/Time Library web site

A Strategic Opportunity for Smalltalk

Mark Shuttleworth:

"Instead of fighting over turf or syntax, I sensed a genuine willingness to synthesize the best work from both camps into something that could have both Python's pop-culture widespread appeal, and pedagogical foundations that build on years of Alan's experience in the Squeak world. The mouse might yet become the snake's strongest ally."

The computational model (which depends heavily on the VM) is strategic.

Syntax (which is just a matter of which compiler one happens to use) is tactical.

Avi Bryant had the right idea: Strategic thinkers in the Smalltalk community should create strong, compelling implementations of Python, Ruby and Perl, hosted on Smalltalk VMs--such as Squeak's.

Adding Erlang, Haskell, Io and Ocaml wouldn't hurt, either.

When it comes to advanced, dynamic programming languages, it's "United we stand, divided we fall."

---

• Chronos blog home page
• Chronos Date/Time Library web site

Architecture is not inflexible dogma

Chris Petrilli goes off on a great rant against the typical stupidities of corporate IT:

"People obsess over the choice of tools, but the true talent, and the true skill, is the understanding of business problems that are trying to be solved, including the window available. If my tool—whatever it is—can help me focus and be more collaborative with the people whose problems I’m trying to solve, then it’s a better tool."

Coffeehouse chess players have often memorized all the moves for all the variations for all the standard chess openings--as recorded in books. But when a Grandmaster presents them with a move not in the books, they're lost. After all, it's the grandmasters who write those books.

True artists create great works of art using their native talent. Others paint by the numbers. Of course, true artists specify those numbers.

True software architects know when to apply architectural/design patterns elucidated by others, and when and how to invent their own if that's appropriate. Enterprisey architects follow the rules popularized a decade (or more) ago by someone they ignored at the time, but whose methodology and thinkng they now apply dogmatically without fully understanding the system's limitations.

Gifted sofware engineers know when counting the number of classes, or the number of methods, or the number of instance variables is a valid measure of code quality--and they know when such "painting by the numbers" is simply invalid to the case at hand.

Great architects judiciously break the mold, creating new rules that work better than the old ones. Dogmatic architects defend the old rules by force of habit, because that's the "safe" thing to do.

Don't be caught fiddling while the enterprise burns.

---

• Chronos blog home page
• Chronos Date/Time Library web site

Why Can’t The Windows Registry Timezones be used?

A poster with the handle iwonder started a thread on the ASP.net forum, titled On Time Zones and Daylight Saving Time (Summer Time).. (I've added the whole thread to the "Cautionary Tales" section of the Chronos web site.) In a subsequent post to the thread, he has a subsection titled "Why Can’t The Windows Registry Timezones be used?", where he states:

First, and foremost the 75 windows registry timezones only are setup to support current year. Meaning that there is no historical reference. In fact, when a given locale changes their time zone daylight saving (summer time) rules, these rules need to be manually ‘fixed’ in the registry. These types of corrections are not distributed by Microsoft as updates to anyone. The corrections appear in KB articles, but the corrections could easily be missed by folks that are not in the locale affected. As each year starts, new rules are in place, which may not calculate the correct date time adjustments for any past years. Actually, any date time adjustment based on the registry alone results in a date time factored with the current years’ rules. Not exactly the result time sensitive sites would require.

Secondly, the windows registry is not totally accurate. Independent tests show it to be around 75-80% accurate at any point in time. Also, the time zones do not represent every time zone on the globe, just the more commonly used zones are included. The UNIX / Linux TZInfo database includes many more time zone descriptions, but there is a question of just how many require support. Obviously, MS made the determination that only 75 of the more than 108 zones in the TZInfo database really require support. It’s a question of judgement.

Lastly, web applications are not really meant to have access to the windows registry. It’s really a question of security, but I can’t imagine any host provider wanting to allow a web application to manipulate the registry. In fact, I’m not sure if the Code Access Security model will even allow the type of registry read and write permission required to effectively use and maintain a registry timezone information.

Correct answers in date/time computations are just as important as correct answers in issuing payroll checks, crediting interest earned, sending out invoices, computing work schedules, sheduling airline flights, administering the right medications to the right patients at the right time, getting a spaceship, rocket or missile to its intended destination at the right time, or even just dividing one number by another.

When I hear people say otherwise, there's no doubt in my mind that what I'm really hearing is an emotional reaction against the unwanted complexity of the date/time domain. But neither reality nor complexity goes away just because you wish things were otherwise.

Time is complex. Deal with it. The Universe doesn't care about your preferences, and will happily bite you in a private part of your anatomy if you can't bring yourself to accept and handle the world as it is.

---

• Chronos blog home page
• Chronos Date/Time Library web site

AJAX × Date × Time × Time zones - best practices

ecmanaut offers a great rant!

Quote:

Are you developing a site or application which relies on javascript, which somewhere on some page or in some view shows some time, or a date, or even a weekday to the visitor? Registration time, for instance. Or an advance notice of your next scheduled downtime? Anything at all! Do you also let visitors from other cities, and maybe even from other countries, visit your web site?

Great! Then this article is for you.

---

• Chronos blog home page
• Chronos Date/Time Library web site

Ignoring Time Zone Issues Can Have Serious Consequences

According to the bug report in DateTime Difference Bug Makes PayPal_Framework Impractical for Real-Time Transactions, the fact that different components of a particular software system reside in different time zones--and use their own local time for timestamps--resulted in a critical error.

Timestamps should be recorded in UT (Universal Time, which most people incorrectly refer to as either GMT or UTC,) especially in situations where system components may reside in different time zones.

Even when all server and client machines will be in the same time zone, recording timestamps in local time can cause bugs.

For example, in most of the United States, whether 2005-10-30T01:50:42 is greater than or less than 2005-10-30T01:10:17 (both local time) is not well defined, because both timestamps occur during the ambigous hour T01:00:00/T02:00:00 (a left closed, right open interval) which occurs twice in local time on days where a transition from DST to standard time occurs. Most of the United States transitioned from DST to standard time at T02:00:00 local time on 30 October 2005--which resulted in local time in San Franciso (for example) jumping from 2005-10-30T01:59:59-07:00 to 2005-10-30T01:00:00-08:00.

Such discontinuities and ambiguities do not occur when timestamps are recorded in UT. For example, the UT timestamp that corresponds to 2005-10-30T01:59:59-07:00 is 2005-10-30T08:59:59Z, and the UT timestamp that corresponds to 2005-10-30T01:00:00-08:00 is 2005-10-30T09:00:00Z.

Don't add to the bug list.

---

• Chronos blog home page
• Chronos Date/Time Library web site