README.md

   1 InfoEx AutoWx (IEAW)
   2 =============
   3
   4 This program fetches data from an NRCS SNOTEL or MesoWest station and
   5 pushes it into the InfoEx system using the new automated weather system
   6 implementation.
   7
   8 License under the ISC license (see file: LICENSE).
   9
  10 Disclaimer
  11 ----------
  12
  13 Your usage of the NRCS, MesoWest, and/or InfoEx systems is bound by
  14 their respective terms and this software makes no attempt or claim to
  15 abide by any such terms.
  16
  17 Installation
  18 ------------
  19
  20 It's recommended to use venv and pip with this program. Here's a typical
  21 sequence of commands for a new setup:
  22
  23 `$ cd /path/to/src`
  24 `$ python3 -m venv env`
  25 `$ . env/bin/activate`
  26 `$ pip install -r requirements.txt`
  27
  28 How to use it
  29 -------------
  30
  31 This program is designed to be run from the command line (or via
  32 cron(8)) and administered via a simple, concise configuration file.
  33
  34 This design allows you to run a separate program instance for each NRCS
  35 or MesoWest weather station from which you'd like to automate the
  36 importation of data into your InfoEx subscriber account.
  37
  38 To run ad-hoc (be sure to activate the virtual environment, as detailed in the
  39 Installation section):
  40
  41 `./infoex-autowx.py --config [path/to/config-file.ini] [--dry-run]`
  42
  43 **NOTE: Specifying --dry-run will also not clean up the generated CSV
  44 file.** This is so that you can debug any issues more easily.
  45
  46 Here's an example of a crontab(5) with two SNOTEL sites, each of which
  47 will run once per hour (note that this will activate the virtual environment
  48 created earlier):
  49
  50 `2 * * * * /usr/bin/env bash -c 'cd /home/user/infoex-autowx && source env/bin/activate && ./infoex-autowx.py --config laurance-lake.ini'`
  51 `4 * * * * /usr/bin/env bash -c 'cd /home/user/infoex-autowx && source env/bin/activate && ./infoex-autowx.py --config mud-ridge.ini'`
  52
  53 Configuration File
  54 ------------------
  55
  56 The configuration file is separated into two parts, the [station]
  57 portion, and the [infoex] portion.
  58
  59 The [station] values describe which weather station's data you're after.
  60 See the next section in this README for instructions on obtaining these
  61 values.
  62
  63 The [infoex] values describe your credentials for the InfoEx automated
  64 weather station FTP server and other InfoEx-related configuration
  65 options.
  66
  67 `[station]`
  68 `type = # either mesowest or nrcs #`
  69 `token = # MesoWest API token -- ignored when type is nrcs #`
  70 `station_id = # the NRCS/MesoWest identifier for a particular station #`
  71 `desired_data = # a comma-delimited list of fields you're interested in #`
  72 `units = # either english or metric -- ignored when type is nrcs #`
  73
  74 `[infoex]`
  75 `host = # InfoEx FTP host address #`
  76 `uuid = # InfoEx-supplied UUID #`
  77 `api_key = # InfoEx-supplied API Key #`
  78 `csv_filename = # arbitrary name of the file that will be uploaded to InfoEx #`
  79 `location_uuid = # the UUID used by InfoEx to identify your automated Wx site #`
  80
  81 Finding your NRCS `station` values
  82 ----------------------------------
  83
  84 To complete the [station] configuration section for an NRCS station, you
  85 must fill in the attributes of the NRCS SNOTEL site from which you want
  86 to import data.
  87
  88 Here are the steps to do that:
  89
  90 1. Find your station by clicking through this website:
  91
  92    https://www.wcc.nrcs.usda.gov/snow/sntllist.html
  93
  94    Or by using the interactive map:
  95
  96    https://www.nrcs.usda.gov/wps/portal/wcc/home/quicklinks/imap
  97
  98 2. Once you've found your station, look for the "Station ID" (a 3- or
  99    4-digit number).
 100
 101 3. Combine your Station ID, state abbreviation, and the network type
 102    "SNTL" to get your NRCS station triplet (`station_id`, in the
 103    configuration file). For example:
 104
 105    655:OR:SNTL
 106
 107    would represent the Mud Ridge station (Station ID 655) in the state
 108    of Oregon (OR). SNTL just represents that the station is in the
 109    SNOTEL network and is used internally by NRCS.
 110
 111 Once you have your station ID, fill in the field in your configuration
 112 file. Now you must select which data you'd like to pull from NRCS to
 113 push into InfoEx.
 114
 115 For that, visit the NRCS web service:
 116
 117 https://wcc.sc.egov.usda.gov/awdbWebService/webservice/testwebservice.jsf?webserviceName=/awdbWebService
 118
 119 Click "getElements" on the left, and then click, "Test Operation." This
 120 will return a long list of elements to your web browser from which you
 121 can choose. Each returned element has its identifier and a description.
 122
 123 Once you've chosen your elements, combine all of their respective
 124 "elementCd" values into a comma-delimited string and put that into your
 125 configuration file as the `desired_data` value.
 126
 127 A complete example:
 128
 129 `[station]`
 130 `type = nrcs`
 131 `station_id = 655:OR:SNTL`
 132 `desired_data = TOBS,PREC`
 133
 134 indicates that I'd like to import "AIR TEMPERATURE OBSERVED" and
 135 "PRECIPITATION ACCUMULATION" from the NRCS SNOTEL site at Mud Ridge, OR,
 136 into InfoEx.
 137
 138 Finding your MesoWest `station` values
 139 --------------------------------------
 140
 141 MesoWest has great documentation which can be found here:
 142
 143 https://developers.synopticdata.com/mesonet/v2/getting-started/
 144
 145 To complete the [station] configuration section for a MesoWest station,
 146 you must fill in the attributes of the MesoWest station ID from which
 147 you want to import data. Here are the steps to do that:
 148
 149 1. Firstly, get set up with MesoWest's API by going to the above
 150    'Getting Started' link. Once you're set up, you can copy a token from
 151    the MesoWest web portal into your configuration file's `token` value.
 152
 153 2. Next, you will want to find the Station ID for the MesoWest weather
 154    station of interest and copy it to the `station_id` value.
 155
 156 3. Finally, you must choose what data types you want to push into
 157    InfoEx and compile them into a comma-separated list. MesoWest refers
 158    to these as 'field names' or 'station variables' and a list is
 159    available here:
 160
 161 https://developers.synopticdata.com/about/station-variables/
 162
 163 The MesoWest API supports on-the-fly unit conversion, so that can be
 164 specified to infoex-autowx via the configuration option `units`. This
 165 can be either 'english' or 'metric', with 'english' meaning imperial
 166 units as used in the United States.
 167
 168 A complete example:
 169
 170 `[station]`
 171 `type = mesowest`
 172 `token = # token id copied from MesoWest web account #`
 173 `station_id = OD110`
 174 `desired_data = air_temp,snow_depth`
 175 `units = english`
 176
 177 indicates that I'd like to import "Temperature" and "Precipitation
 178 accumulated" from the MesoWest station at Santiam Pass, OR, into InfoEx,
 179 and that I want that data in imperial units.
 180
 181 A note on supported measurements
 182 --------------------------------
 183
 184 While this program supports several measurements, and will faithfully
 185 request all of the ones you specify (provided they're supported), the
 186 weather station may not record them. In this case, the data will simply
 187 be ignored (i.e. it will NOT log "0" when there's no measurement
 188 available).
 189
 190
 191 InfoEx provides a mechanism for inspecting your automated weather
 192 station data, so use that after setting this program up and compare it
 193 with the data you see in your web browser.
 194
 195 Here's the list of measurements currently supported:
 196
 197 **NRCS:**
 198 PREC
 199 TOBS
 200 SNWD
 201 PRES
 202 RHUM
 203 WSPD
 204 WDIR
 205
 206 **MesoWest:**
 207 precip\_accum
 208 air\_temp
 209 snow\_depth
 210 pressure
 211 relative\_humidity
 212 wind\_speed
 213 wind\_direction
 214 wind\_gust
 215
 216 Future plans
 217 ------------
 218
 219 - Improve the documentation
 220 - Implement unit conversion for NRCS stations
 221
 222 Version history
 223 ---------------
 224
 225 - 2.0.0 (Jul 2020)
 226
 227   Implement MesoWest integration.
 228
 229   This release also makes significant changes to the configuration file,
 230   hence the major version bump. Such changes are not taken lightly but
 231   given the desire to support multiple data sources, were necessary.
 232
 233 - 1.0.0 (Jun 2020)
 234
 235   First released version. Cleaned up the program and design.
 236   Implemented configuration file, added LICENSE, README, docs, etc.
 237
 238 - 0.8.0 (Apr 2020)
 239
 240   First finished (unreleased) version.
 241
 242 - pre-0.8.0 (Apr 2020)
 243
 244   First (private) finished implementation with successful importation of
 245   NRCS data into InfoEx.