InfoEx AutoWx (IEAW)
=============
-This program fetches data from an NRCS SNOTEL or MesoWest station and
-pushes it into the InfoEx system using the new automated weather system
-implementation.
+This program fetches data from an NRCS SNOTEL or MesoWest station, or
+your own custom data source, and pushes it into the InfoEx system using
+the new automated weather system implementation.
-License under the ISC license (see file: LICENSE).
+Licensed under the ISC license (see file: LICENSE).
Disclaimer
----------
or MesoWest weather station from which you'd like to automate the
importation of data into your InfoEx subscriber account.
+To get started, copy the included example config file
+(`config.ini.example` in the root source directoy) and modify the values
+for your own use.
+
To run ad-hoc (be sure to activate the virtual environment, as detailed in the
Installation section):
-`./infoex-autowx.py --config [path/to/config-file.ini] [--dry-run]`
+`./infoex-autowx.py --config path/to/config-file.ini [--dry-run] [--log-level debug|info|warning]`
+
+**NOTE: Specifying --dry-run will not clean up the generated CSV file.**
+This is so that you can more easily debug any issues that arise in the
+setup process.
-**NOTE: Specifying --dry-run will also not clean up the generated CSV
-file.** This is so that you can debug any issues more easily.
+You can also specify `--log-level` as debug, info, warning. The
+log messages produced by the program will try to be logged to journald,
+but if that's not available, they will be printed to stdout. This output
+can be helpful early on in the setup process.
+
+Automation
+----------
Here's an example of a crontab(5) with two SNOTEL sites, each of which
will run once per hour (note that this will activate the virtual environment
options.
`[station]`
-`type = # either mesowest or nrcs #`
-`token = # MesoWest API token -- ignored when type is nrcs #`
+`type = # mesowest, nrcs, or python #`
+`token = # MesoWest API token -- only applies when type is mesowest #`
`station_id = # the NRCS/MesoWest identifier for a particular station #`
`desired_data = # a comma-delimited list of fields you're interested in #`
-`units = # either english or metric -- ignored when type is nrcs #`
+`units = # either english or metric -- only applies when type is mesowest #`
+`tz = # any entry from the Olson tz database e.g. America/Denver #`
+`path = # the filesystem path to the Python program -- only applies when type is python #`
`[infoex]`
`host = # InfoEx FTP host address #`
"elementCd" values into a comma-delimited string and put that into your
configuration file as the `desired_data` value.
-A complete example:
+A complete [station] section example:
`[station]`
`type = nrcs`
https://developers.synopticdata.com/about/station-variables/
-The MesoWest API supports on-the-fly unit conversion, so that can be
-specified to infoex-autowx via the configuration option `units`. This
-can be either 'english' or 'metric', with 'english' meaning imperial
-units as used in the United States.
+The MesoWest API supports on-the-fly unit conversion. If desired, that
+can be specified to infoex-autowx via the configuration option `units`.
+This can be either 'english' or 'metric', with 'english' meaning
+imperial units as used in the United States.
-A complete example:
+A complete [station] section example:
`[station]`
`type = mesowest`
accumulated" from the MesoWest station at Santiam Pass, OR, into InfoEx,
and that I want that data in imperial units.
+Custom weather station support
+------------------------------
+
+This program supports custom weather station data by allowing the user
+to specify the path to an external Python program. The external Python
+program should emit its data in the form expected by infoex-autowx.
+
+This is a powerful feature which enables the user to upload data from
+any source imaginable into InfoEx. Common examples are a local database
+or a remote web page which requires some custom parsing.
+
+Please see the program located at examples/custom-wx.example.py for a
+complete description of what's required.
+
+A note on time zones
+--------------------
+
+This program is aware of time zones via the pytz library. The way in
+which NRCS and MesoWest deal with time zones differs as follows:
+
+NRCS expects the request to come in the appropriate time zone, and the
+data retrieved will be in the same time zone (no transformation
+required before sending to InfoEx).
+
+MesoWest expects the request to come in UTC, and the data retrieved will
+be in the same time zone (transformation from UTC to the desired time
+zone is required before sending to InfoEx).
+
+As long as you specify the correct timezone in your configuration file,
+all will be handled correctly. The list of time zones comes from the
+Olson tz database. See that for more information.
+
+If you specify an invalid time zone, the program will exit and inform
+you of such.
+
+Lastly, InfoEx itself is timezone aware. If you notice that the data
+which makes it into your operation is inaccurate, start your
+investigation with time zone-related issues and move on only once you've
+ruled this out as a cause of the inaccuracy.
+
A note on supported measurements
--------------------------------
be ignored (i.e. it will NOT log "0" when there's no measurement
available).
-
InfoEx provides a mechanism for inspecting your automated weather
station data, so use that after setting this program up and compare it
with the data you see in your web browser.
wind\_direction
wind\_gust
+**Custom Wx program**
+*infoex-autowx expects a custom Wx data provider to provide at least one
+of the following:*
+precipitationGauge
+tempPres
+tempMaxHour
+tempMinHour
+hS
+baro
+rH
+windSpeedNum
+windDirectionNum
+windGustSpeedNum
+
Future plans
------------
-- Improve the documentation
- Implement unit conversion for NRCS stations
Version history
---------------
+- 3.1.0 (Jan 2021)
+
+ Implement time zone support.
+
+- 3.0.2 (Jan 2021)
+
+ Use UTC time when asking MesoWest for data.
+
+- 3.0.1 (Jan 2021)
+
+ General fixes.
+
+ - MesoWest wind data (speed and gust speed) units are now transformed
+ from their origin unit (meters per second) to the unit expected by
+ InfoEx (miles per hour).
+
+ - Relative humidity is now rounded to one decimal place, preventing
+ InfoEx from reddening the auto-filled value.
+
+- 3.0.0 (Nov 2020)
+
+ Implement Custom Wx data providers.
+
+ This release enables the user to write their own Python programs and
+ specify them to infoex-autowx as a data provider.
+
+ This in turn enables the user to pull data from e.g. a local database
+ or an HTML page and push it into their InfoEx auto station data,
+ limited only by the imagination.
+
+- 2.2.0 (Nov 2020)
+
+ Add support for Tmin/Tmax values (directly from MesoWest/NRCS).
+
+- 2.1.0 (Nov 2020)
+
+ Adjust precision of certain values before sending them to InfoEx.
+
+- 2.0.2 (Jul 2020)
+
+ Fix issues shown by pylint(1).
+
+- 2.0.1 (Jul 2020)
+
+ Major restructuring, but nothing which should impact the end user.
+
+ - Took the monolithic main () routine and broke it out into logical
+ components.
+ - Improved the names of variables.
+
- 2.0.0 (Jul 2020)
Implement MesoWest integration.
hence the major version bump. Such changes are not taken lightly but
given the desire to support multiple data sources, were necessary.
+ Other minor changes include:
+
+ - New switches: --log-level and --version.
+ - Better documentation.
+ - Expanded supported measurement types (from three to eight, in number).
+
- 1.0.0 (Jun 2020)
First released version. Cleaned up the program and design.
wylark / infoex-autowx.git / blobdiff