README.md

   1 InfoEx AutoWx (IEAW)
   2 =============
   3
   4 This program fetches data from an NRCS SNOTEL site and pushes it into
   5 the InfoEx system using the new automated weather system implementation.
   6
   7 License under the ISC license (see file: LICENSE).
   8
   9 Disclaimer
  10 ----------
  11
  12 Your usage of the NRCS and InfoEx systems is bound by their respective
  13 terms and this software makes no attempt or claim to abide by any such
  14 terms.
  15
  16 Installation
  17 ------------
  18
  19 It's recommended to use venv and pip with this program. Here's a typical
  20 sequence of commands for a new setup:
  21
  22 `$ cd /path/to/src`
  23 `$ python3 -m venv env`
  24 `$ . env/bin/activate`
  25 `$ pip install -r requirements.txt`
  26
  27 How to use it
  28 -------------
  29
  30 This program is designed to be run from the command line (or via
  31 cron(8)) and administered via a simple, concise configuration file.
  32
  33 This design allows you to run a separate program instance for each NRCS
  34 weather station from which you'd like to automate the importation of
  35 data into your InfoEx subscriber account.
  36
  37 To run ad-hoc (be sure to activate the virtual environment, as detailed in the
  38 Installation section):
  39
  40 `./infoex-autowx.py --config [path/to/config-file.ini] [--dry-run]`
  41
  42 **NOTE: Specifying --dry-run will also not clean up the generated CSV
  43 file.** This is so that you can debug any issues more easily.
  44
  45 Here's an example of a crontab(5) with two SNOTEL sites, each of which
  46 will run once per hour (note that this will activate the virtual environment
  47 created earlier):
  48
  49 `2 * * * * /usr/bin/env bash -c 'cd /home/user/infoex-autowx && source env/bin/activate && ./infoex-autowx.py --config laurance-lake.ini'`
  50 `4 * * * * /usr/bin/env bash -c 'cd /home/user/infoex-autowx && source env/bin/activate && ./infoex-autowx.py --config mud-ridge.ini'`
  51
  52 Configuration File
  53 ------------------
  54
  55 The configuration file is separated into two parts, the [station]
  56 portion, and the [infoex] portion.
  57
  58 The [station] values describe which weather station's data you're after.
  59 See the next section in this README for instructions on obtaining these
  60 values.
  61
  62 The [infoex] values describe your credentials for the InfoEx automated
  63 weather station FTP server and other InfoEx-related configuration
  64 options.
  65
  66 `[station]`
  67 `type = # either mesowest or nrcs #`
  68 `token = # MesoWest API token -- ignored when type is nrcs #`
  69 `station_id = # the NRCS/MesoWest identifier for a particular station #`
  70 `desired_data = # a comma-delimited list of fields you're interested in #`
  71 `units = # either english or metric -- ignored when type is nrcs #`
  72
  73 `[infoex]`
  74 `host = # InfoEx FTP host address #`
  75 `uuid = # InfoEx-supplied UUID #`
  76 `api_key = # InfoEx-supplied API Key #`
  77 `csv_filename = # arbitrary name of the file that will be uploaded to InfoEx #`
  78 `location_uuid = # the UUID used by InfoEx to identify your automated Wx site #`
  79
  80 Finding Your NRCS `station` values
  81 ----------------------------------
  82
  83 To complete the [station] configuration section for an NRCS station, you
  84 must fill in the attributes of the NRCS SNOTEL site from which you want
  85 to import data.
  86
  87 Here are the steps to do that:
  88
  89 1. Find your station by clicking through this website:
  90
  91    https://www.wcc.nrcs.usda.gov/snow/sntllist.html
  92
  93    Or by using the interactive map:
  94
  95    https://www.nrcs.usda.gov/wps/portal/wcc/home/quicklinks/imap
  96
  97 2. Once you've found your station, look for the "Station ID" (a 3- or
  98    4-digit number).
  99
 100 3. Combine your Station ID, state abbreviation, and the network type
 101    "SNTL" to get your NRCS station triplet (`station_id`, in the
 102    configuration file). For example:
 103
 104    655:OR:SNTL
 105
 106    would represent the Mud Ridge station (Station ID 655) in the state
 107    of Oregon (OR). SNTL just represents that the station is in the
 108    SNOTEL network and is used internally by NRCS.
 109
 110 Once you have your station ID, fill in the field in your configuration
 111 file. Now you must select which data you'd like to pull from NRCS to
 112 push into InfoEx.
 113
 114 For that, visit the NRCS web service:
 115
 116 https://wcc.sc.egov.usda.gov/awdbWebService/webservice/testwebservice.jsf?webserviceName=/awdbWebService
 117
 118 Click "getElements" on the left, and then click, "Test Operation." This
 119 will return a long list of elements to your web browser from which you
 120 can choose. Each returned element has its identifier and a description.
 121
 122 Once you've chosen your elements, combine all of their respective
 123 "elementCd" values into a comma-delimited string and put that into your
 124 configuration file as the `desired_data` value.
 125
 126 A complete example:
 127
 128 `[station]`
 129 `type = nrcs`
 130 `station_id = 655:OR:SNTL`
 131 `desired_data = TOBS,PREC`
 132
 133 indicates that I'd like to import "AIR TEMPERATURE OBSERVED" and
 134 "PRECIPITATION ACCUMULATION" from the NRCS SNOTEL site at Mud Ridge, OR,
 135 into InfoEx.
 136
 137 Finding your MesoWest `station` values
 138 --------------------------------------
 139
 140 MesoWest has great documentation which can be found here:
 141
 142 https://developers.synopticdata.com/mesonet/v2/getting-started/
 143
 144 To complete the [station] configuration section for a MesoWest station,
 145 you must fill in the attributes of the MesoWest station ID from which
 146 you want to import data. Here are the steps to do that:
 147
 148 1. Firstly, get set up with MesoWest's API by going to the above
 149    'Getting Started' link. Once you're set up, you can copy a token from
 150    the MesoWest web portal into your configuration file's `token` value.
 151
 152 2. Next, you will want to find the Station ID for the MesoWest weather
 153    station of interest and copy it to the `station_id` value.
 154
 155 3. Finally, you must choose what data types you want to push into
 156    InfoEx and compile them into a comma-separated list. MesoWest refers
 157    to these as 'field names' or 'station variables' and a list is
 158    available here:
 159
 160 https://developers.synopticdata.com/about/station-variables/
 161
 162 The MesoWest API supports on-the-fly unit conversion, so that can be
 163 specified to infoex-autowx via the configuration option `units`. This
 164 can be either 'english' or 'metric', with 'english' meaning imperial
 165 units as used in the United States.
 166
 167 A complete example:
 168
 169 `[station]`
 170 `type = mesowest`
 171 `token = # token id copied from MesoWest web account #`
 172 `stid = OD110`
 173 `desired_data = air_temp,snow_depth`
 174 `units = english`
 175
 176 indicates that I'd like to import "Temperature" and "Precipitation
 177 accumulated" from the MesoWest station at Santiam Pass, OR, into InfoEx,
 178 and that I want that data in imperial units.
 179
 180 Version History
 181 ---------------
 182
 183 - 2.0.0 (Jul 2020)
 184
 185   Implement MesoWest integration.
 186
 187   This release also makes significant changes to the configuration file,
 188   hence the major version bump. Such changes are not taken lightly but
 189   given the desire to support multiple data sources, were necessary.
 190
 191 - 1.0.0 (Jun 2020)
 192
 193   First released version. Cleaned up the program and design.
 194   Implemented configuration file, added LICENSE, README, docs, etc.
 195
 196 - 0.8.0 (Apr 2020)
 197
 198   First finished (unreleased) version.
 199
 200 - pre-0.8.0 (Apr 2020)
 201
 202   First (private) finished implementation with successful importation of
 203   NRCS data into InfoEx.