README.md

   1 InfoEx AutoWx (IEAW)
   2 =============
   3
   4 This program fetches data from an NRCS SNOTEL site and pushes it into
   5 the InfoEx system using the new automated weather system implementation.
   6
   7 License under the MIT license (see file: LICENSE).
   8
   9 Disclaimer
  10 ----------
  11
  12 Your usage of the NRCS and InfoEx systems is bound by their respective
  13 terms and this software makes no attempt or claim to abide by any such
  14 terms.
  15
  16 Installation
  17 ------------
  18
  19 It's recommended to use venv and pip with this program. Here's a typical
  20 sequence of commands for a new setup:
  21
  22 `$ cd /path/to/src`
  23 `$ python3 -m venv env`
  24 `$ . env/bin/activate`
  25 `$ pip install -r requirements.txt`
  26
  27 How to use it
  28 -------------
  29
  30 This program is designed to be run from the command line (or via
  31 cron(8)) and administered via a simple, concise configuration file.
  32
  33 This design allows you to run a separate program instance for each NRCS
  34 weather station from which you'd like to automate the importation of
  35 data into your InfoEx subscriber account.
  36
  37 To run ad-hoc (be sure to activate the virtual environment, as detailed in the
  38 Installation section):
  39
  40 `./infoex-autowx.py --config [path/to/config-file.ini] [--dry-run]`
  41
  42 **NOTE: Specifying --dry-run will also not clean up the generated CSV
  43 file.** This is so that you can debug any issues more easily.
  44
  45 Here's an example of a crontab(5) with two SNOTEL sites, each of which
  46 will run once per hour (note that this will activate the virtual environment
  47 created earlier):
  48
  49 `2 * * * * /usr/bin/env bash -c 'cd /home/user/infoex-autowx && source env/bin/activate && ./infoex-autowx.py --config laurance-lake.ini'`
  50 `4 * * * * /usr/bin/env bash -c 'cd /home/user/infoex-autowx && source env/bin/activate && ./infoex-autowx.py --config mud-ridge.ini'`
  51
  52 Configuration File
  53 ------------------
  54
  55 The configuration file is separated into two parts, the
  56 [nrcs]/[mesowest] portion, and the [infoex] portion.
  57
  58 The [nrcs]/[mesowest] values describe which NRCS SNOTEL site's data
  59 you're after.  See the next section in this README for instructions on
  60 obtaining these values.
  61
  62 The [infoex] values describe your credentials for the InfoEx automated
  63 weather station FTP server and other InfoEx-related configuration
  64 options.
  65
  66 `[nrcs]`
  67 `station_triplet = [The NRCS identifier for a particular SNOTEL site]`
  68 `desired_data = [A comma-delimited list of NRCS elements you're interested in]`
  69
  70 `[infoex]`
  71 `host = [InfoEx FTP host address]`
  72 `uuid = [InfoEx-supplied UUID]`
  73 `api_key = [InfoEx-supplied API Key]`
  74 `csv_filename = [Arbitrary name of the file that will be uploaded to InfoEx]`
  75 `location_uuid = [The UUID used by InfoEx to identify your automated Wx site]`
  76
  77 Finding Your NRCS values
  78 ------------------------
  79
  80 To complete the [nrcs] configuration section, you must fill in the
  81 attributes of the NRCS SNOTEL site from which you want to import data.
  82
  83 Here are the steps to do that:
  84
  85 1. Find your station by clicking through this website:
  86
  87    https://www.wcc.nrcs.usda.gov/snow/sntllist.html
  88
  89    Or by using the interactive map:
  90
  91    https://www.nrcs.usda.gov/wps/portal/wcc/home/quicklinks/imap
  92
  93 2. Once you've found your station, look for the "Station ID" (a 3- or
  94    4-digit number).
  95
  96 3. Combine your Station ID, state abbreviation, and the network type
  97    "SNTL" to get your station triplet (`station_triplet`, in the
  98    configuration file). For example:
  99
 100    655:OR:SNTL
 101
 102    would represent the Mud Ridge station (Station ID 655) in the state
 103    of Oregon (OR). SNTL just represents that the station is in the
 104    SNOTEL network and is used internally by NRCS.
 105
 106 Once you have your station triplet, fill in the field in your
 107 configuration file. Now you must select which data you'd like to pull
 108 from NRCS to push into InfoEx.
 109
 110 For that, visit the NRCS web service:
 111
 112 https://wcc.sc.egov.usda.gov/awdbWebService/webservice/testwebservice.jsf?webserviceName=/awdbWebService
 113
 114 Click "getElements" on the left, and then click, "Test Operation." This
 115 will return a long list of elements to your web browser from which you
 116 can choose. Each returned element has its identifier and a description.
 117
 118 Once you've chosen your elements, combine all of their respective
 119 "elementCd" values into a comma-delimited string and put that into your
 120 configuration file as the `desired_data` value.
 121
 122 For example:
 123
 124 `station_triplet = 655:OR:SNTL`
 125 `desired_data = TOBS,PREC`
 126
 127 indicates that I'd like to import "AIR TEMPERATURE OBSERVED" and
 128 "PRECIPITATION ACCUMULATION" from the NRCS SNOTEL site at Mud Ridge, OR,
 129 into InfoEx.
 130
 131 Finding your MesoWest values
 132 ----------------------------
 133
 134 To complete the [mesowest] configuration section, you must fill in the
 135 attributes of the MesoWest station ID from which you want to import
 136 data.
 137
 138 Here are the steps to do that:
 139
 140 Field names are listed here:
 141 https://developers.synopticdata.com/about/station-variables/
 142
 143 The MesoWest API supports on-the-fly unit conversion, so that can be
 144 specified to infoex-autowx via the configuration option `units`. This
 145 can be either 'english' or 'metric', with 'english' meaning imperial
 146 units as used in the United States.
 147
 148 Version History
 149 ---------------
 150
 151 - 2.0.0 (Jul 2020)
 152
 153   Implement MesoWest integration.
 154
 155   This release also makes significant changes to the configuration file,
 156   hence the major version bump. Such changes are not taken lightly but
 157   given the desire to support multiple data sources, were necessary.
 158
 159 - 1.0.0 (Jun 2020)
 160
 161   First released version. Cleaned up the program and design.
 162   Implemented configuration file, added LICENSE, README, docs, etc.
 163
 164 - 0.8.0 (Apr 2020)
 165
 166   First finished (unreleased) version.
 167
 168 - pre-0.8.0 (Apr 2020)
 169
 170   First (private) finished implementation with successful importation of
 171   NRCS data into InfoEx.