Times and Dates

Time is an essential component of nearly all geoscience data. Timescales span orders of magnitude from microseconds for lightning, hours for a supercell thunderstorm, days for a global weather model, millenia and beyond for the earth’s climate. To properly analyze geoscience data, you must have a firm understanding of how to handle time in Python. In this notebook, we will examine the Python Standard Library for handling dates and times. We will also briefly make use of the pytz module to handle some thorny time zone issues in Python.

Time Versus Datetime Modules and Some Core Concepts

Python comes with time and datetime modules. Unfortunately, Python can be initially disorienting because of the heavily overlapping terminology concerning dates and times:

  • datetime module has a datetime class
  • datetime module has a time class
  • datetime module has a date class
  • time module has a time function which returns (almost always) Unix time
  • datetime class has a date method which returns a date object
  • datetime class has a time method which returns a time object

This confusion can be partially alleviated by aliasing our imported modules:

import datetime as dt
# we can now reference the datetime module (alaised to 'dt') and datetime
# object unambiguously
pisecond = dt.datetime(2016, 3, 14, 15, 9, 26)
print(pisecond)

2016-03-14 15:09:26

import time as tm
now = tm.time()
print(now)

1559320921.335239

time module

The time module is well-suited for measuring Unix time. For example, when you are calculating how long it takes a Python function to run (so-called “benchmarking”), you can employ the time() function from the time module to obtain Unix time before and after the function completes and take the difference of those two times.

import time as tm
start = tm.time()
tm.sleep(1)  # The sleep function will stop the program for n seconds
end = tm.time()
diff = end - start
print(f"The benchmark took {diff} seconds")

The benchmark took 1.0002760887145996 seconds

(For more accurate benchmarking, see the timeit module.)

datetime module

The datetime module handles time with the Gregorian calendar (the calendar we are all familiar with) and is independent of Unix time. The datetime module has an object-oriented approach with the date, time, datetime, timedelta, and tzinfo classes.

  • date class represents the day, month and year
  • time class represents the time of day
  • datetime class is a combination of the date and time classes
  • timedelta class represents a time duration
  • tzinfo (abstract) class represents time zones

The datetime module is effective for:

  • performing date and time arithmetic and calculating time duration
  • reading and writing date and time strings in a particular format
  • handling time zones (with the help of third-party libraries)

The time and datetime modules overlap in functionality, but in your geoscientific work, you will probably be using the datetime module more than the time module.

What is Unix Time?

Unix time is an example of system time which is the computer’s notion of passing time. It is measured in seconds from the the start of the epoch which is January 1, 1970 00:00 UTC. It is represented “under the hood” as a floating point number which is how computers represent real (ℝ) numbers .

The Thirty Second Introduction to Object-Oriented Programming

We have been talking about object-oriented (OO) programming by mentioning terms like “class”, “object”, and “method”, but never really explaining what they mean. A class is a collection of related variables, similar to a struct, in the C programming language or even a tuple in Python) coupled with functions, or “methods” in OO parlance, that can act on those variables. An object is a concrete example of a class.

For example, if you have a Coord class that represents an earth location with latitude, and longitude, you may have a method that returns the distance between two locations, distancekm() in this example.

import math
class Coord:
    """Earth location identified by (latitude, longitude) coordinates.
    distancekm  -- distance between two points in kilometers
    """

    def __init__(self, latitude=0.0, longitude=0.0):
        self.lat = latitude
        self.lon = longitude

    def distancekm(self, p):
        """Distance between two points in kilometers."""
        DEGREES_TO_RADIANS = math.pi / 180.0
        EARTH_RADIUS = 6373  # in KMs
        phi1 = (90.0 - self.lat) * DEGREES_TO_RADIANS
        phi2 = (90.0 - p.lat) * DEGREES_TO_RADIANS
        theta1 = self.lon * DEGREES_TO_RADIANS
        theta2 = p.lon * DEGREES_TO_RADIANS
        cos = (math.sin(phi1) * math.sin(phi2) *
               math.cos(theta1 - theta2) + math.cos(phi1) * math.cos(phi2))
        arc = math.acos(cos)
        return arc * EARTH_RADIUS

To create a concrete example of a class, also known as an object, initialize the object with data:

timbuktu = Coord(16.77, 3.00)

Here, timbuktu is an object of the class Coord initialized with a latitude of 16.77 and a longitude of 3.00. Next, we create two Coord objects: ny and paris. We will invoke the distancekm() method on the ny object and pass the paris object as an argument to determine the distance between New York and Paris in kilometers.

ny = Coord(40.71, 74.01)
paris = Coord(48.86, 2.35)
distance = ny.distancekm(paris)
print(f"The distance from New York to Paris is {distance:.1f} kilometers.")

The distance from New York to Paris is 5517.0 kilometers.

The old joke about OO programming is that they simply moved the struct that the function takes as an argument and put it first because it is special. So instead of having distancekm(ny, paris), you have ny.distancekm(paris). We have not talked about inheritance or polymorphism but that is OO in a nutshell.

Reading and Writing Dates and Times

Parsing Lightning Data Timestamps with the datetime.strptime Method

Suppose you want to analyze US NLDN lightning data. Here is a sample row of data:

06/27/07 16:18:21.898 18.739 -88.184 0.0 kA 0 1.0 0.4 2.5 8 1.2 13 G

Part of the task involves parsing the 06/27/07 16:18:21.898 time string into a datetime object. (The full description of the data are described here.) In order to parse this string or others that follow the same format, you will employ the datetime.strptime() method from the datetime module. This method takes two arguments: the first is the date time string you wish to parse, the second is the format which describes exactly how the date and time are arranged. The full range of format options is described in the Python documentation. In reality, the format will take some degree of experimentation to get right. This is a situation where Python shines as you can quickly try out different solutions in the IPython interpreter. Beyond the official documentation, Google and Stack Overflow are your friends in this process. Eventually, after some trial and error, you will find the '%m/%d/%y %H:%M:%S.%f' format will properly parse the date and time.

import datetime as dt
strike_time = dt.datetime.strptime('06/27/07 16:18:21.898',
                                   '%m/%d/%y %H:%M:%S.%f')
# print strike_time to see if we have properly parsed our time
print(strike_time)

2007-06-27 16:18:21.898000

Going Further