InfoEx AutoWx (IEAW)
=============
-This program fetches data from an NRCS SNOTEL or MesoWest station and
-pushes it into the InfoEx system using the new automated weather system
-implementation.
+This program fetches data from an NRCS SNOTEL or MesoWest station, or
+your own custom data source, and pushes it into the InfoEx system using
+the new automated weather system implementation.
-License under the ISC license (see file: LICENSE).
+Licensed under the ISC license (see file: LICENSE).
Disclaimer
----------
or MesoWest weather station from which you'd like to automate the
importation of data into your InfoEx subscriber account.
+To get started, copy the included example config file
+(`config.ini.example` in the root source directoy) and modify the values
+for your own use.
+
To run ad-hoc (be sure to activate the virtual environment, as detailed in the
Installation section):
-`./infoex-autowx.py --config [path/to/config-file.ini] [--dry-run]`
+`./infoex-autowx.py --config path/to/config-file.ini [--dry-run] [--log-level debug|info|warning]`
+
+**NOTE: Specifying --dry-run will not clean up the generated CSV file.**
+This is so that you can more easily debug any issues that arise in the
+setup process.
+
+You can also specify `--log-level` as debug, info, warning. The
+log messages produced by the program will try to be logged to journald,
+but if that's not available, they will be printed to stdout. This output
+can be helpful early on in the setup process.
-**NOTE: Specifying --dry-run will also not clean up the generated CSV
-file.** This is so that you can debug any issues more easily.
+Automation
+----------
Here's an example of a crontab(5) with two SNOTEL sites, each of which
will run once per hour (note that this will activate the virtual environment
options.
`[station]`
-`type = # either mesowest or nrcs #`
-`token = # MesoWest API token -- ignored when type is nrcs #`
+`type = # mesowest, nrcs, or python #`
+`token = # MesoWest API token -- only applies when type is mesowest #`
`station_id = # the NRCS/MesoWest identifier for a particular station #`
`desired_data = # a comma-delimited list of fields you're interested in #`
-`units = # either english or metric -- ignored when type is nrcs #`
+`units = # either english or metric -- only applies when type is mesowest #`
+`tz = # any entry from the Olson tz database e.g. America/Denver #`
+`path = # the filesystem path to the Python program -- only applies when type is python #`
`[infoex]`
`host = # InfoEx FTP host address #`
"elementCd" values into a comma-delimited string and put that into your
configuration file as the `desired_data` value.
-A complete example:
+A complete [station] section example:
`[station]`
`type = nrcs`
https://developers.synopticdata.com/about/station-variables/
-The MesoWest API supports on-the-fly unit conversion, so that can be
-specified to infoex-autowx via the configuration option `units`. This
-can be either 'english' or 'metric', with 'english' meaning imperial
-units as used in the United States.
+The MesoWest API supports on-the-fly unit conversion. If desired, that
+can be specified to infoex-autowx via the configuration option `units`.
+This can be either 'english' or 'metric', with 'english' meaning
+imperial units as used in the United States.
-A complete example:
+A complete [station] section example:
`[station]`
`type = mesowest`
accumulated" from the MesoWest station at Santiam Pass, OR, into InfoEx,
and that I want that data in imperial units.
+Custom weather station support
+------------------------------
+
+This program supports custom weather station data by allowing the user
+to specify the path to an external Python program. The external Python
+program should emit its data in the form expected by infoex-autowx.
+
+This is a powerful feature which enables the user to upload data from
+any source imaginable into InfoEx. Common examples are a local database
+or a remote web page which requires some custom parsing.
+
+Please see the program located at examples/custom-wx.example.py for a
+complete description of what's required.
+
+A note on time zones
+--------------------
+
+This program is aware of time zones via the pytz library. The way in
+which NRCS and MesoWest deal with time zones differs as follows:
+
+NRCS expects the request to come in the appropriate time zone, and the
+data retrieved will be in the same time zone (no transformation
+required before sending to InfoEx).
+
+MesoWest expects the request to come in UTC, and the data retrieved will
+be in the same time zone (transformation from UTC to the desired time
+zone is required before sending to InfoEx).
+
+As long as you specify the correct timezone in your configuration file,
+all will be handled correctly. The list of time zones comes from the
+Olson tz database. See that for more information.
+
+If you specify an invalid time zone, the program will exit and inform
+you of such.
+
+Lastly, InfoEx itself is timezone aware. If you notice that the data
+which makes it into your operation is inaccurate, start your
+investigation with time zone-related issues and move on only once you've
+ruled this out as a cause of the inaccuracy.
+
+Unit conversions
+----------------
+
+Desired units may be specified in the configuration file.
+
+For MesoWest, the desired unit will be passed along in the API request
+and the conversion will take place through the MesoWest/Synoptic API.
+
+For NRCS, this program will do the conversion manually, as NRCS does not
+permit specifying the desired unit.
+
+A note on supported measurements
+--------------------------------
+
+While this program supports several measurements, and will faithfully
+request all of the ones you specify (provided they're supported), the
+weather station may not record them. In this case, the data will simply
+be ignored (i.e. it will NOT log "0" when there's no measurement
+available).
+
+InfoEx provides a mechanism for inspecting your automated weather
+station data, so use that after setting this program up and compare it
+with the data you see in your web browser.
+
+Here's the list of measurements currently supported:
+
+**NRCS:**
+PREC
+TOBS
+SNWD
+PRES
+RHUM
+WSPD
+WDIR
+
+**MesoWest:**
+precip\_accum
+air\_temp
+snow\_depth
+pressure
+relative\_humidity
+wind\_speed
+wind\_direction
+wind\_gust
+
+**Custom Wx program**
+*infoex-autowx expects a custom Wx data provider to provide at least one
+of the following:*
+precipitationGauge
+tempPres
+tempMaxHour
+tempMinHour
+hS
+baro
+rH
+windSpeedNum
+windDirectionNum
+windGustSpeedNum
+
Version history
---------------
+- 3.2.4 (Mar 2021)
+
+ Fix a small bug that allowed MesoWest HS values to flow through in
+ millimeters when metric was the specified unit. MesoWest metric HS
+ values are now correctly in centimeters.
+
+- 3.2.3 (Feb 2021)
+
+ Fix a small bug that allowed a TypeError to be raised with some
+ regularity.
+
+- 3.2.2 (Feb 2021)
+
+ Various small fixes.
+
+ - Round precipitation accumulation values to 2 decimal places.
+ - Catch requests' ConnectionException.
+ - Improve logging output when using stdout.
+
+- 3.2.1 (Feb 2021)
+
+ Fix config validation bug with units and custom Python program.
+
+- 3.2.0 (Feb 2021)
+
+ Implement NRCS unit conversion.
+
+- 3.1.1 (Feb 2021)
+
+ Fix relative humidity rounding.
+
+- 3.1.0 (Jan 2021)
+
+ Implement time zone support.
+
+- 3.0.2 (Jan 2021)
+
+ Use UTC time when asking MesoWest for data.
+
+- 3.0.1 (Jan 2021)
+
+ General fixes.
+
+ - MesoWest wind data (speed and gust speed) units are now transformed
+ from their origin unit (meters per second) to the unit expected by
+ InfoEx (miles per hour).
+
+ - Relative humidity is now rounded to one decimal place, preventing
+ InfoEx from reddening the auto-filled value.
+
+- 3.0.0 (Nov 2020)
+
+ Implement Custom Wx data providers.
+
+ This release enables the user to write their own Python programs and
+ specify them to infoex-autowx as a data provider.
+
+ This in turn enables the user to pull data from e.g. a local database
+ or an HTML page and push it into their InfoEx auto station data,
+ limited only by the imagination.
+
+- 2.2.0 (Nov 2020)
+
+ Add support for Tmin/Tmax values (directly from MesoWest/NRCS).
+
+- 2.1.0 (Nov 2020)
+
+ Adjust precision of certain values before sending them to InfoEx.
+
+- 2.0.2 (Jul 2020)
+
+ Fix issues shown by pylint(1).
+
+- 2.0.1 (Jul 2020)
+
+ Major restructuring, but nothing which should impact the end user.
+
+ - Took the monolithic main () routine and broke it out into logical
+ components.
+ - Improved the names of variables.
+
- 2.0.0 (Jul 2020)
Implement MesoWest integration.
hence the major version bump. Such changes are not taken lightly but
given the desire to support multiple data sources, were necessary.
+ Other minor changes include:
+
+ - New switches: --log-level and --version.
+ - Better documentation.
+ - Expanded supported measurement types (from three to eight, in number).
+
- 1.0.0 (Jun 2020)
First released version. Cleaned up the program and design.
wylark / infoex-autowx.git / blobdiff