Data Set Descriptors Title: National Oceanic and Atmospheric Administration nClimDiv Statewide Regional Modified Palmer Drought Severity Index (PMDI) from Jan-1895 to Dec-2023 Accession Number: Originator(s): Organization: National Oceanic and Atmospheric Administration Address: 1401 Constitution Avenue NW, Room 5128 City: Washington State: District of Columbia Postal Code: 20230 Abstract: This data set contains the Modified Palmer Drought Severity Index (PMDI). This is a modification of the Palmer Drought Severity Index. The modification was made by the National Weather Service Climate Analysis Center for operational meteorological purposes. The Palmer drought program calculates three intermediate parallel index values each month. Only one value is selected as the PDSI drought index for the month. This selection is made internally by the program on the basis of probabilities. If the probability that a drought is over is 100%, then one index is used. If the probability that a wet spell is over is 100%, then another index is used. If the probability is between 0% and 100%, the third index is assigned to the PDSI. The modification (PMDI) incorporates a weighted average of the wet and dry index terms, using the probability as the weighting factor. (Thomas R. Heddinghause and Paul Sabol, 1991; "A Review of the Palmer Drought Severity Index and Where Do We Go From Here?," Proceedings of the Seventh Conference on Applied Climatology, pp. 242-246, American Meteorological Society, Boston, MA). The PMDI and PDSI will have the same value during an established drought or wet spell (i.e., when the probability is 100%), but they will have different values during transition periods. Study Type: Synthesis Study Themes: Climatology Georeferences: none Submission Date: Keywords: drought, index, climate, precipitation, temperature, Palmer Drought Severity Index, PSDI Study Sites Study Sites: Geographic Coordinates: Site History: Study Description Experimental or Sampling Design Design Characteristics: The major parameters in this file are sequential climatic divisional, statewide, regional and national monthly Standardized Precipitation Index (SPI), and Palmer Drought Indices (PDSI, PHDI, PMDI, and ZNDX). Period of record is 1895 through latest month available, updated monthly. Values from the most recent two calendar years will be updated on a monthly basis. Period of record updates will occur when the underlying data set undergoes a version change. Permanent Plots: Sampling Design: not applicable Beginning of Observations: 01-Jan-1895 End of Observations: 01-Dec-2023 Research Methods Field and Laboratory Methods: Overview - Divisional, statewide, regional and national values in nClimDiv were derived from area-weighted averages of grid-point estimates interpolated from station data. A nominal grid resolution of 5 km was used to ensure that all divisions had sufficient spatial sampling (only four small divisions had less than 100 points) and because the impact of elevation on precipitation is minimal below 5 km. Station data were gridded via climatologically aided interpolation to minimize biases from topographic and network variability. Data Acquisition - The Global Historical Climatology Network (GHCN) Daily dataset is the source of station data for nClimDiv. GHCN-Daily contains several major observing networks in North America, five of which are used here. The primary network is the National Weather Service (NWS) Cooperative Observing (COOP) program, which consists of stations operated by volunteers as well as by agencies such as the Federal Aviation Administration. To improve coverage in western states and along international borders, nClimDiv also includes the National Interagency Fire Center (NIFC) Remote Automatic Weather Station (RAWS) network, the USDA Snow Telemetry (SNOTEL) network, the Environment Canada (EC) network (south of 52°N), and part of Mexicos Servicio Meteorologico Nacional (SMN) network (north of 24°N). Note that nClimDiv does not incorporate precipitation data from RAWS because that networks tipping-bucket gauges are unheated, leading to suspect cold-weather data. Quality Control -- All GHCN-Daily stations are routinely processed through a suite of logical, serial, and spatial quality assurance reviews to identify erroneous observations. For nClimDiv, all such data were set to missing before computing monthly values, which in turn were subjected to additional serial and spatial checks to eliminate residual outliers. Overall, the quality assurance reviews deemed less than 0.25% of the monthly data as being erroneous. Stations having at least 10 years of valid monthly data since 1950 were used in nClimDiv. Bias Adjustments - For temperature, bias adjustments were computed to account for historical changes in observation time, station location, temperature instrumentation, and siting conditions. Changes in observation time are only problematic for the COOP network whereas changes in station location and instrumentation occur in almost all surface networks. As in the U.S. Historical Climatology Network version 2.5, the method of Karl et al. (1986) was applied to remove the observation time bias from the COOP network, and the pairwise method of Menne and Williams (2009) was used to address changes in station location and instrumentation in all networks. Because the pairwise method also largely accounts for local, unrepresentative trends that arise from changes in siting conditions, nClimDiv contains no separate adjustment in that regard. For additional information on how nClimDiv is constructed, please see: http://journals.ametsoc.org/doi/abs/10.1175/JAMC-D-13-0248.1 Historical Data - Historical drought data have been added to this file for the period 1895 to present. The file is updated monthly. All drought data are calibrated using the period 1931-1990 (cf. Karl, 1986; Journal of Climate and Applied Meteorology, Vol. 25, No. 1, January 1986). Protocols: For additional information on how nClimDiv is constructed, please see: http://journals.ametsoc.org/doi/abs/10.1175/JAMC-D-13-0248.1 Instrumentation: not applicable Taxonomy and Systematics: not applicable Species List: not applicable Personnel: Affiliations: Data Set Accessibility Storage Location: Distribution Formats: Contact Person: Position: GCE-LTER Information Manager Address: 130 Marine Sciences, University of Georgia City: Athens State: Georgia Postal Code: 30602-3636 Phone: (706) 542-5955 Email: gcelter@uga.edu Copyright Restrictions: Restrictions: Release Date: Affiliates: , Public: Citation: Disclaimer: Data Structural Descriptors Data Set File File Name: climdiv-pmdidv-v1.0.0-20240105.csv Size: 532512 records File Format: ASCII text (comma-separated value format) Delimiters: single comma Header Information: 5 lines of ASCII text Quality Control Flag Codes: Authentication Procedures: Processing History: Software version: GCE Data Toolbox Version 3.9.10 (23-May-2022) Data structure version: GCE Data Structure 1.1 (29-Mar-2001) Original data file processed: climdiv-pmdidv-v1.0.0-20240105_filtered.txt (44376 records) Data processing history: 11-Jan-2024: new GCE Data Structure 1.1 created ('newstruct') 11-Jan-2024: 44376 rows imported from ASCII data file 'climdiv-pmdidv-v1.0.0-20240105_filtered.txt' ('imp_ascii') 11-Jan-2024: 5 metadata fields in file header parsed ('parse_header') 11-Jan-2024: data descriptor metadata updated based on the template 'NCDC_nClimDiv_PMDI' ('meta_template') 11-Jan-2024: data structure validated ('gce_valid') 11-Jan-2024: automatically assigned study date metadata descriptors based on the range of date values in date/time columns (add_studydates) 11-Jan-2024: updated title ('newtitle') 11-Jan-2024: updated 1 metadata fields in the Dataset section(s) ('addmeta') 11-Jan-2024: Q/C flagging criteria applied for column(s) Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov and Dec, 'flags' field updated ('dataflag') 11-Jan-2024: created normalized data set by combining values in columns Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov and Dec to generate categorical column Month and value column IndexValue, converting values in IndexValue to floating point data type, replicating values in column(s) StateCode, Region, IndexCode and Year for each distinct Month ('normalize_cols') 11-Jan-2024: updated attribute metadata descriptors ('update_attributes'): updated units for column Month from none to MMM; updated description for column Month from Categorical values from normalization of data columns Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov and Dec to Calendar month of calculation; updated variabletype for column Month from nominal to datetime 11-Jan-2024: updated attribute metadata descriptors ('update_attributes'): updated units for column IndexValue from none to none; updated description for column IndexValue from Combined values from normalization of data columns Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov and Dec to Calculated Drought Index 11-Jan-2024: added column 'Date' at position 4 ('addcol') 11-Jan-2024: sorted by columns StateCode(ascending), Region(ascending), IndexCode(ascending), Date(ascending) ('sortdata') 11-Jan-2024: decoded values in column StateCode based on code values in the metadata, and added column State as a categorical text column ('decodecols') 11-Jan-2024: added column 'State' at position 2 ('addcol') 11-Jan-2024: updated attribute metadata descriptors ('update_attributes'): updated units for column State from none to none; updated description for column State from NOAA state code to State name; updated variabletype for column State from nominal to nominal 11-Jan-2024: automatically assigned study date metadata descriptors based on the range of date values in date/time columns (add_studydates) 11-Jan-2024: updated title ('newtitle') 11-Jan-2024: updated 1 metadata fields in the Dataset section(s) ('addmeta') 11-Jan-2024: converted dates in column(s) Date to format yyyy-mm-dd HH:MM:SS 11-Jan-2024: flags for column IndexValue converted to data columns, flag codes updated in metadata ('flags2cols') 11-Jan-2024: updated 6 metadata fields in the Data section(s) ('addmeta') 11-Jan-2024: updated 15 metadata fields in the Status, Data sections to reflect attribute metadata ('updatecols') 11-Jan-2024: parsed and formatted metadata ('listmeta') Variable Information Calculations: Data Anomalies: Variable Codes: IndexCode: 05 = Palmer Drought Severity Index StateCode: 01 = Alabama, 02 = Arizona, 03 = Arkansas, 04 = California, 05 = Colorado, 06 = Connecticut, 07 = Delaware, 08 = Florida, 09 = Georgia, 10 = Idaho, 11 = Illinois, 12 = Indiana, 13 = Iowa, 14 = Kansas, 15 = Kentucky, 16 = Louisiana, 17 = Maine, 18 = Maryland, 19 = Massachusetts, 20 = Michigan, 21 = Minnesota, 22 = Mississippi, 23 = Missouri, 24 = Montana, 25 = Nebraska, 26 = Nevada, 27 = New Hampshire, 28 = New Jersey, 29 = New Mexico, 30 = New York, 31 = North Carolina, 32 = North Dakota, 33 = Ohio, 34 = Oklahoma, 35 = Oregon, 36 = Pennsylvania, 37 = Rhode Island, 38 = South Carolina, 39 = South Dakota, 40 = Tennessee, 41 = Texas, 42 = Utah, 43 = Vermont, 44 = Virginia, 45 = Washington, 46 = West Virginia, 47 = Wisconsin, 48 = Wyoming, 66 = Guam Missing Value Code: Number of Columns: 9 Variable Attributes: Name Units Data type Variable type Number type Precision QC Criteria 1. StateCode none string coded value none 0 none 2. State none string nominal none 0 none 3. Region none string nominal none 0 none 4. IndexCode none string coded value none 0 none 5. Date yyyy-mm-dd HH:MM:SS string datetime none 0 none 6. Year YYYY integer datetime discrete 0 none 7. Month MMM string datetime none 0 none 8. IndexValue none floating-point data continuous 2 none 9. Flag_IndexValue none string coded value none 0 none Variable Definitions: 1. NOAA state code 2. State name 3. NOAA nominal climate region 4. Climate index code 5. Starting date of monthly calculation 6. Year 7. Calendar month of calculation 8. Calculated Drought Index 9. QA/QC flags for Calculated Drought Index (flagging criteria, where "x" is IndexValue: ) Supplemental Descriptors Quality Assurance/Quality Control Procedures: Data Validation Procedures or Notes: History of Data Set Usage Publications: Data Request History: Data Set Update History: Reviewer Comments: Questions and Comments from Users: