Motivation for the course

The main part of the first half of this course is to learn to program in Python. However, in addition to learning to program, we hope to help you learn a number of other skills related to open science. These include:

  1. Writing programs that are easy to understand and share
  2. Keeping a log of the changes you make to your programs
  3. Using programs that ensure your science is reproducible
  4. Producing simple, effective data visualizations that make your results easy to understand

To help clarify our goals, consider the example below.

Effective data visualization

One of the things we will learn in this course is how to use Python to plot data. As you well know, raw data itself is often not particularly useful in helping you understand what the data shows. Let’s look at an example that might be familiar to you, global temperature anomalies. And we can start with the data.

-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   -84   -42   -27   -10     6    -5     8    55   163   330   525   731   957  1191  1429  1669  1883  2044  2155  2227  2262  2266  2250  2232  2228  2247  2289  2355  2440  2535  2626  2693  2726  2748  2789  2819  2808  2791 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  1351  1312 -9999 -9999 -9999 -9999 -9999   852 -9999   736   764   777   743   643   528   439   312   138    19     3   -15  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -123   -82   -39   -25   -12   -13   -12    -5    47   156   323   518   726   952  1188  1430  1673  1885  2039  2139  2199  2224  2225  2214  2205  2212  2241  2293  2368  2464  2571  2668  2734  2761  2777  2811  2834  2822  2803 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  1376  1333 -9999 -9999 -9999 -9999 -9999   745   712   700   722   741   734   672   569   472   342   184    65    50    42  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -131   -83   -38   -25   -16   -23   -20   -12    41   148   310   500   707   936  1179  1429  1677  1886  2033  2122  2169  2184  2182  2175  2177  2193  2231  2293  2382  2494  2611  2710  2776  2798  2806  2829  2850  2841  2830 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  1423  1368  1297  1262 -9999 -9999 -9999   693   669   667   680   699   699   660   592   498   374   241   132   116   106  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -132   -85   -39   -25   -18   -34   -30   -19    34   146   299   483   685   914  1167  1430  1682  1887  2025  2102  2136  2143  2139  2139  2148  2166  2208  2283  2394  2528  2655  2752  2813  2830  2828  2847  2856  2851  2841 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  1462  1406  1332  1294 -9999 -9999 -9999 -9999   604   617   635   662   670   645   602   523   414   306   208   174   150   -91  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -137   -92   -43   -25   -19   -42   -38   -31    24   141   291   473   674   902  1162  1438  1695  1894  2018  2081  2105  2108  2104  2104  2108  2115  2154  2252  2402  2564  2696  2785  2833  2843  2844  2848  2853  2849 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  1485  1439  1373  1332 -9999 -9999 -9999 -9999   450   489   525 -9999 -9999   621   602   544   455   348   237   173   150   -19  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -148  -102   -47   -25   -19   -50   -51   -42    12   123   275   470   681   915  1183  1468  1726  1914  2021  2067  2081  2075  2058  2043  2032  2027  2074  2212  2412  2597  2721  2794  2826  2828  2837  2844  2850  2848 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  1516  1467  1417  1373 -9999 -9999 -9999 -9999   295   319   387 -9999 -9999   599   590   554   480   376   228   152   125   -51  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -163  -112   -51   -26   -20   -58   -58   -45     0    98   262   469   696   949  1238  1536  1787  1953  2035  2061  2055  2019  1964  1923  1905  1914  2000  2187  2435  2626  2721  2780  2803  2822 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  1691  1631  1554  1502  1450 -9999 -9999 -9999 -9999 -9999 -9999   245   303 -9999 -9999 -9999   557   538   483   408   261   168   113   -92  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -175  -117   -51   -26   -19   -62   -59   -45     0    86   253   468   710   994  1322  1638  1872  2001  2044  2036  1998  1939  1861  1801  1791  1828 -9999 -9999 -9999  2650  2704 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  1726  1659  1581  1531  1457  1369  1315 -9999 -9999 -9999 -9999   248 -9999 -9999 -9999 -9999 -9999   481   475   422   300   189   110  -109  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -168  -113   -50   -25   -12   -57   -55   -35     0    83   245   461   717  1035  1406  1743  1955  2034  2025  1973  1916  1882 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  1798  1753  1677  1597  1537 -9999  1401 -9999 -9999 -9999 -9999 -9999   267 -9999 -9999 -9999 -9999 -9999 -9999   473   427   291   146    51  -129  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   -98   -49   -26    -1   -52   -52   -35    -8    73   230   446   713  1064  1472  1827  2023  2061  2012  1927  1870 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  1808  1755  1682  1611  1549  1497 -9999 -9999 -9999 -9999 -9999 -9999   275   217    82    21 -9999 -9999 -9999   460   410   245    47  -180  -165  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   -95   -51   -30     1   -49   -53   -47   -26    51   206   427   710  1083  1509  1873  2070  2102  2060 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  1801  1753  1692  1638  1572  1520 -9999 -9999 -9999 -9999 -9999 -9999   233   205   118    48    -4 -9999 -9999 -9999   401   199  -138  -179  -179  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -160  -111   -58   -36     8   -45   -52   -53   -41    32   187   413   709  1090  1512  1871  2083  2149  2141 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  1742  1693  1639 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   213   184   134    65   -18   -94 -9999 -9999   366   178  -144  -180  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -180  -131   -68   -45    15   -27   -41   -47   -36    34   188   410   706  1078  1481  1825  2055  2172  2210 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  1752  1695  1619  1543 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   131   106 -9999 -9999  -117 -9999 -9999   364   200  -104  -180  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -180  -147   -79   -48    14   -15   -25   -30   -11    58   204   421   710  1066  1450  1778  2015  2172  2256 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  1833  1782  1717  1636  1552 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999    44 -9999 -9999 -9999 -9999 -9999   371   232   -53  -180  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -153   -87   -50     5   -18   -11   -12    15    91   231   445   726  1063  1431  1754  1989  2160  2279 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  1846  1797  1747  1690 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999    22 -9999 -9999 -9999 -9999 -9999   359   246    15  -180  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -159   -92   -51     4   -22    -7    -4    33   116   257   470   740  1055  1409  1733  1969  2137  2279  2382 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  1856  1817  1785 -9999 -9999   890   834   780 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   310   326   241    39  -180  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -168   -96   -49    10   -21    -7     0    44   133   272   478   733  1031  1380  1713  1952  2115  2259  2396  2490 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  1869  1837  1813 -9999 -9999   894   854   798 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   280   289   216    30  -180  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -176   -97   -43    18   -17    -6     4    54   142   272   461   701   991  1348  1698  1944  2101  2240  2383  2506  2601  2657 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  2319 -9999  1877  1859  1845 -9999 -9999 -9999   853 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   249   246   174     6  -180  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -180   -93   -35    25   -12    -4     9    60   142   258   432   657   947  1315  1680  1931  2088  2226  2370  2499  2595  2665  2724  2762  2775 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  2501  2406  2333 -9999 -9999  1862  1858 -9999 -9999   887   840   783 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -180 -9999   223   206   130   -20  -180  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -180   -90   -30    29    -7    -6     6    57   134   246   412   628   914  1285  1651  1905  2063  2206  2354  2488  2586  2663  2732  2778  2797  2806 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  2718  2632  2542 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   912   864   796 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -180 -9999   196   172    96   -37  -180  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -180   -96   -33    26    -2   -10    -6    42   124   242   407   613   888  1256  1623  1874  2031  2176  2333  2477  2582  2667  2748  2808  2835  2835  2834 -9999 -9999 -9999  2819  2797  2750 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  2707  2740  2740 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   959   921 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -174 -9999   166   153    83   -41  -180  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -180  -101   -39    22    -1   -11   -18    26   120   248   410   599   862  1237  1613  1860  2007  2149  2311  2465  2583  2682  2774  2843  2870  2870  2861  2871  2885  2876  2831  2796  2726  2654 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  2681  2726 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   982 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999    44   126   144    85   -35  -180  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   -96   -45    22     4    -5   -18    23   125   256   410   578   831  1217  1609  1855  1998  2139  2301  2458  2585  2687  2785  2864  2894  2894  2890  2894  2888  2879  2840  2796  2715  2631  2583 -9999 -9999 -9999 -9999 -9999  2568  2572 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999    22    89   135   104   -15  -180  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   -97   -50    15    10     7    -4    39   142   268   402   553   800  1190  1591  1845  1997  2145  2304  2458  2585  2681 -9999 -9999 -9999 -9999 -9999  2906  2888  2879  2862  2806  2732  2648  2584  2558 -9999 -9999 -9999 -9999  2561  2564 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   -92    60   131   125   -30  -180  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -105   -54    11    14    19    15    69   172   285   399   540   785  1173  1570  1831  1997  2152  2310  2461  2588  2685  2753 -9999 -9999 -9999 -9999  2902  2879  2874  2856  2827  2765  2683  2598  2563  2558 -9999 -9999 -9999  2538  2545 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   448   373 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -180    35   117   129   -48  -180  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   -94   -54     5    15    27    34    99   204   307   407   546   801  1189  1574  1826  1992  2149  2310  2466  2597  2698  2779  2841  2878  2884 -9999 -9999  2865  2862  2851  2823  2771  2698  2626  2589  2584  2590  2596 -9999  2534  2526 -9999 -9999 -9999 -9999 -9999 -9999  1982  1942 -9999 -9999 -9999  1055   932   719   483   309 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -180    17   110   131    28  -180  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   -41     1    18    34    54   125   226   327   428   582   858  1248  1608  1834  1986  2140  2307  2471  2602  2706  2784  2841  2872  2883  2885  2884  2873  2853  2849  2819  2777  2716  2651  2613  2609  2611  2604  2580  2549  2525  2507 -9999 -9999 -9999 -9999  2121  2058 -9999 -9999 -9999 -9999  1103   989   811   520    25 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999    13    57   109   -59  -180  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   -32     0    23    43    74   147   244   349   469   654   955  1336  1653  1846  1985  2136  2303  2463  2601  2705  2782  2833  2862  2877  2886  2887  2880  2853  2840  2811  2781  2735  2680  2648  2642  2642  2632  2608  2574  2532  2500 -9999 -9999 -9999 -9999  2192 -9999 -9999 -9999 -9999 -9999  1114  1033   909 -9999   259 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -120    24   -16  -144  -180  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   -31     2    30    55    93   169   264   376   524   752  1075  1425  1688  1851  1988  2138  2298  2456  2592  2697  2771  2819  2848  2867  2880  2887  2879  2854  2817  2795  2779  2751  2709  2684  2682  2681  2665  2634  2591  2541  2499  2477 -9999 -9999 -9999  2269 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -180  -180  -141  -155  -180  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   -63   -27     5    39    67   112   192   284   400   575   846  1184  1492  1701  1847  1992  2142  2298  2449  2582  2686  2755  2796  2824  2846  2862  2872  2872  2850  2818  2793  2779  2763  2737  2721  2718  2715  2697  2660  2609  2557  2512  2476  2447 -9999 -9999  2290 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -180  -180  -180  -175  -180  -180  -180  -180  -180  -180
-9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999   -62   -18     9    48    78   131   215   301   411   596   897  1245  1520  1698  1841  1994  2148  2300  2443  2570  2669  2734  2770  2797  2821  2840  2855  2862  2845  2814  2793  2779  2778  2762  2749  2745  2740  2719  2679  2627  2577  2532  2488  2447  2396  2348  2316 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999 -9999  -180  -180  -180  -180  -180  -180  -180  -180  -180  -180
...

Not that exciting, right? There is an interesting story here, but we need some way to illustrate the power of this data.

One option is to use an x-y plot of temperature anomalies versus time.

Global mean temperature anomalies

Global mean temperature anomalies from 1880-2011. Source: https://www.ncdc.noaa.gov/sotc/global/201113

This is obviously much better, showing clearly how temperatures have changed with time and how global temperatures have increased significantly since 1970. Now we see a clear step toward making the data easier to understand. However, this is global data and we are missing something important about the data, its connection to geographical locations.

Let’s consider another option, plotting temperature anomalies on a map.

Global mean temperature anomaly map

Global temperature anomalies for March 2016. Source: https://www.ncdc.noaa.gov/sotc/global/201603

And yet again, this helps us understand the data further. Not only do we see the changed in temperature, but now we see how temperatures vary across the globe. The drawback here is that we only see a single time snapshot, rather than a time series. To see both will require a truly remarkable visualization.

So, let’s look now at an excellent example of data visualization with Python. We have essentially the same data plotted above, but now we can see how temperatures vary in space and time,

Global temperature anomalies by country from 1900-2017. Visualization by Antti Lipponen (@anttilip). Source: https://t.co/ZdGPVTM5yO

This animated “pill packet” plot of temperature anomalies conveys a huge amount of information in a simple form. People can immediately understand what is plotted, and the combination of the plot format, colors and animation are very effective. What even better is the fact that this animation was made using Python!

For the rest of the first part of this course, plots like that above can be our inspiration. In fact, we will be working with similar data throughout this part of the course and may even end up producing similar plots in by the end of this teaching period.